Automation testing can deliver significant value in software quality assurance testing including lower cost, greater accuracy and faster cycle times. However, it is not a panacea for every task or project. Using test automation is not always cheaper than manual testing nor is it well suited for every testing activity. The following best practices can help CIOs and testing managers understand where and how automation testing can generate the best results and ROI.
What to automate
Automating every test case is neither possible nor practical. So, test managers need to first identify cases that best play to automation’s strengths. Ideal test cases:
- Run frequently on multiple builds
- Involve copious amounts of data performing the same action
- Are prone to human error
- Feature high risk conditions for commonly used functionality
- Are difficult to perform manually
- Run on numerous hardware and software platforms & configurations
- Applications without a graphical user interface
- Composite and distributed applications
- Test cycles executed outside normal business hours
Create a plan
Once the test cases are identified, they need to be incorporated into a comprehensive test strategy and plan. This plan should spell out the specific testing objectives as well as quality assurance goals.
Effective testing plans should break test cases down into logical and executable components focused on a single task, as opposed to application functionalities, which create large and complex test cases that are difficult to edit and debug. Focusing on small, bite-size activities enables you to maintain test scripts, test data and processes thereby improving productivity and collaboration. As well, it is easier to combine these simple automated tests into a larger automated test by application functional areas, common areas or a base set of data.
Choose the optimal tool
The selection of a particular automation testing tool can be the difference between success and failure. The choice of the tool will be dependent on many business and technical factors. In general, the testing team should look for a tool that:
- Supports your hardware, operating systems and development environment
- Enables testing by all skill and experience levels
- Is easy to use but does not lack key functionality
- Provides for the creation of reusable, maintainable and robust automated test assets.
Test early and often
To maximize bug identification, automation testing should be undertaken as early as possible and ran as often as needed. Fixing bugs earlier in the SDLC is always less expensive than addressing them later in production or deployment. The optimum method would be to utilize a risk based assessment to prioritize the automation effort early towards the highest risk test cases.
Organize the testing
Success with automated testing depends on getting the people equation right. For example, putting the right people in the right roles will help maximize testing productivity. The average quality assurance engineer will not be good at everything, such as writing both test scripts and test cases. Furthermore, getting the testers to collaborate with each other will improve testing productivity and problem-solving.
Develop scalable automated tests
It is common for the UI to change between builds, especially early in the process. These changes may affect test results. Moreover, your automated tests may no longer work with future versions of the application. One of the UI challenges centers around how the testing tool identifies and locates an object on the screen (e.g., buttons, text fields, windows … etc.). When the UI changes the automated test script may be unable to locate the original object causing the test to fail. Quality assurance engineers would then need to update test scripts, before running the test against the latest version of the application. A better approach is using unique identifiers that can make the automated tests resistant to UI changes and makes the tests themselves more reusable.
Generate quality test data
Creating test data for your automated tests can be cumbersome, time-consuming and boring. However, it is an essential task. The payoff is a much more streamlined testing process as well as the ability to easily maintain and reuse the automated tests. Furthermore, high quality data allows you to easily extend the automated test to different testing scenarios within the application’s development.
Like many other technologies, automation will deliver compelling value when effectively specified and deployed. This requires a team that is versant with testing best practices, a systematic quality assurance process and a practical understanding of automation testing tools. For organizations inexperienced with automation, we recommend starting with a clearly defined project and a limited number of test cases.
Consider advanced automation tools
Companies may want to leverage Business Process Testing tools, which convert business processes into modularized testing components. Testing teams may also want to explore leveraging keyword-driven scripts and automated object recognition technologies that reduce the amount of rework that needs to be done.
Conclusion
Based on our experience with dozens of automation projects, Automated Testing can powerfully improve testing outcomes – when deployed intelligently. The best way to exploit its benefits is by identifying the right testing activities for automation and then choosing the rights tools, people and structural model to support it. QA Consultants has extensive experience in designing automation strategies, deploying automation tools and building automation frameworks, either as a stand-alone initiative or part of an end-to-end quality assurance initiative. Our automation solutions are designed to speed up the testing effort, improve quality and reduce the cost of testing. For example, our Turn-Key automation product allows customers to only pay for the use of the automation results plus an annual maintenance fee.