Automation Test Frameworks

A test automation framework is a collection of re-usable methods which helps in automating test process.

Automation of test process include automation of following modules:

Test Execution: This is the place where a tool is used to execute the scripts.

Report Generation: This module concentrates on taking test execution results as input and will generate the customized reports of different levels.
It can be summary report which provides details like PASS/FAIL status and Time taken to execute the test case.
In detailed reports, it can list the number of exceptions cases handled, Start time and End time for each test case, test cases passed, failed status and .

Defect Logging: This module concentrates on automatic defect logging of defects into bug tracking tool and taking the screen shots.

Test Case writing: Automation of this part includes providing help for writing manual test cases as well as generating corresponding keywords into automation steps.

Framework Methodologies

There are four popular test automation methodologies :

1. Modular
2. Keyword
3. Datadriven
4. Hybrid.

Modular Framework:

In this methodology, is one of the most basic type of automation framework. In this framework, a test scripts are written to match a functionality that represent modules of the application-under-test. These modules in turn are used in a hierarchical fashion to build large test cases.

Advantages:

Modular division of scripts leads to easier maintenance and also the scalability of the automated test Scripts are independent to write.

Disadvantages:

Data Driven Framework:

In this methodology, test data is separated from test scripts and results are returned against the test data. And finally if all the test data combinations are pass, then only the test case is treated as "PASS". If any one of the test data combination is failed, then the entire test case will be treated as "FAIL".

Advantages:

Disadvantages:

Keyword Driven :

In this methodology, keywords are developed which are equal to a unit level functionality. It is an application independent framework utilizing data tables methods and keywords to perform the actions

Advantages:

Disadvantages:

Hybrid Framework:

This framework is the combination of both data-driven and keyword driven testing frameworks. It allows data driven scripts to take advantage of the powerful libraries and utilities in a keyword based approach.

Advantages:

The Hybrid framework is build with a number of reusable modules / function libraries that are developed with the following features in mind:

The main problem with modular frameworks is that the test script have test data embedded in them, which will become problem when updating the code /script.

It is also difficult when ever a test step fails which has to be find out by debugging where the test case failed.

It is suitable for test cases with multiple test data combinations.

It reduces the number of test scripts needed to implement all the test cases.

The test data can be identified / prepared before test implementation is ready.

It is not suitable for test cases having very simple actions which doesn't include any test data.

High re-usability can be achieved by re-using across multiple test cases.

Maintainability is easy which doesn't require any Automation expertise.

Debugging is easy.

Requires a complicated framework than the data driven framework.

Test cases grow longer and complex and this is due to the greater flexibility

Takes time to build and stabilize the framework.

Maintainability – Hybrid framework significantly reduces maintenance effort

Re-usability – It allows to reuse test cases and library functions

Manageability - effective test design, execution, and traceability

Accessibility – easy to design, develop, modify and debug test cases while executing

Availability – Allows to schedule automation execution

Reliability – due to advanced error handling and scenario recovery

Flexibility – framework independent of system or environment under test

Measurability – customizable reporting of test results ensure the quality output