Earlier in the Test Automation series, we looked at dispelling test automation myths, identified how to optimize ROI of test automation, outlined how to begin a successful test automation implementation, and addressed concerns about test automation that all QA professionals and product owners have.
In this blog entry, we get ‘down and dirty’ with a component that is core to all test automation programs – the test automation framework. Specifically, we review:
- Benefits of a Test Automation Framework
- What Makes up a Test Automation Framework
- How to Choose a Test Automation Framework
Benefits of a Test Automation Framework
Test automation frameworks provide tangible process benefits that ultimately save time. Specifically, these frameworks:
- Improve efficiency and reduce overall project costs by eliminating duplication or redundancy of multiple test cases that are ‘run across’ an application; and
- Streamline testing projects that involve a group of testers working on multiple features of an application.
What Makes up a Test Automation Framework
Since test automation frameworks are central to the success of all test automation initiatives, you need to ‘have a handle’ on what such frameworks should actually look like.
The first thing to keep in mind is that a test automation framework is a ‘blueprint’ for implementing, managing, and documenting the test automation process. Put another way, it is a set of assumptions, concepts, guidelines and practices – including elements such as coding standards and test-data handling procedures – that are followed throughout automation scripting.
To ensure that the test automation framework you choose is practical, at a minimum, it should:
- Outline a specific methodology for creating, managing, and executing tests
- Identify how to deal with unusual test scenarios
- Define the structure and content of test reports
How to Choose a Test Automation Framework
Every test automation framework has its pros and cons. For example:
- The Record & Playback Testing Framework can be executed without automation expertise and can produce scripts quickly. However, maintenance is difficult and there is little opportunity to reuse scripts.
- The Structured Scripting Testing Framework does produce codes that can be reused. However, technical expertise is necessary to write scripts.
- The Data-Driven Testing Framework allows for test cases to be executed with multiple sets of data. However, time is required to plan and prepare test scripts and test data.
- The Keyword-Driven or Table-Driven Testing Framework creates codes that are highly re-usable. However, to do leverage this framework, you will need to draw upon automation expertise.
- The Hybrid Test Automation Framework draws on any of the above frameworks – attempting to optimize their strengths and minimize their weaknesses. In practice, whatever framework you begin with will evolve into some kind of hybrid. In some instances, that means drawing on Behaviour-Driven Development (BDD) processes and/or Test-Driven Development (TDD) processes.
Having several frameworks to choose from is a blessing and a curse. The truth is that selecting a framework that is appropriate for your organization, your project, and your team depends on several key factors, specifically:
- The coding expertise that you can draw on
- The type of tests that are being conducted
- The number of tests being conducted
- The degree to which you need code to be reused
- Data storage and maintenance requirements
- The product/version release schedules that you must follow
Whichever framework you choose, you end up ‘walking a tightrope’. That’s because your framework must be flexible enough to allow for new testing technologies, new testing requirements, and/or management’s evolving demands for reporting. At the same time, if you rely on a framework that is limited in scope, you will be tempted to switch to another during test execution. Unfortunately, doing so reduces your project’s success by creating costly bottlenecks, wasted effort, and time-consuming duplications that erode the benefits of implementing a test automation framework.
Choosing a Test Automation Framework
Finally, when choosing a test automation framework, keep in mind that it should be constructed so that it is completely independent of any specific software application. That helps to ‘bake in’ the flexibility you will need as your projects, and project testing needs evolve.