Test Automation: The Reality About Getting Started

How to Get Started

In this blog post, we address the question of how you should get started with test automation. That means, we address the following issues common to all test automation projects – regardless of your industry, your project scope, or the application(s) you are testing:

1. When to automate
2. What needs to be in place before adopting test automation
3. The measures of success that you should use to evaluate your test automation
4. Elements you should include in developing the business case for pursuing test automation.

When to Introduce Test Automation

Before you choose test automation, make sure that you have a good grasp of the following project-realities:

• Hard-dollar costs of executing required tests
• The volume of tests that need to be conducted (having identified all of the platforms and operating systems that need to be considered)
• The time period during which the tests need to be completed
• Risks to the project/business of completing tests later than required

Having said that, test automation is recommended when:

1. test cases will be used multiple times and will require minimal updating
2. tests need to be conducted simultaneously on multiple machines
3. tests require multiple data sets
4. tests need to be performed across multiple technologies (e.g. mobile, browser) and/or multiple operating systems.

Furthermore, you should consider leveraging test automation when complex systems require ‘patches’ and upgrades that impact core features/functionalities. Finally, test automation should be included in your overall testing strategy when you need to free up the time of highly-skilled (and/or stressed) testers so that they focus on more strategic activities.

What Needs to Be In Place for Test Automation

At the core of test automation is a basic assumption: conducting more tests faster is a good thing. But, what if the testing processes currently underway are well-intentioned but ill-conceived? Then you run the real risk of increasing the speed at which poor processes are conducted.

So, to achieve the full range of benefits that accompany automated testing, product owners and QA professionals need to have mapped out, or implemented, the following project elements:

• Current/relevant QA testing team job responsibilities (including key skills) documented
• QA testing workflows that clearly articulate responsibilities (including overlaps of deliverables and ‘hand-offs’)
• Test cases (including requirements, use cases, expected outcomes, storyboards)
• Exposure to, and experience with, chosen test automation tools
• A budget to purchase and or license relevant test automation tools
• A dedicated test environment
• A plan for collecting and/or simulating test data
• Defined project metrics (that you can actually capture)
• A budget for on-going test automation activities

While getting all of these elements in place takes time, by doing so, you will effectively be building a work culture that is truly committed to test automation.

Test Automation Metrics

To ensure that your test automation initiatives are successful, you need to come to an agreement on key metrics with which you and your team will measure success. According to a recent international survey, 60% of respondents judged test automation success as ‘reducing the time to execute tests’ – thereby getting products to market faster. The second most commonly referenced measure of success was cost reduction (13%). Another vital test automation metric is being able to demonstrate an increase in ‘test coverage’ over a reduced span of time.

Building the Business Case for Test Automation

The business case for test automation serves as a ‘blueprint’ for all the key stakeholders. So, draw upon the insights from people who have actually planned, implemented, and evaluated several test automation programs. They will be able to quantify the costs/benefits/time required so that you can better manage expectations within your organization. Here are a few things to consider:

• Go through the same cost justification process as you would with any other project. This will serve to validate the need (and scope) of test automation.
• Test automation costs include: test automation software; a test automation environment; efforts to automate and run test cases/test scenarios; efforts to review and interpret results; maintenance of test automation tools, data, and scripts; and increased/different headcount for test automation skills (unless outsourced).
• Expected benefits from test automation include: time saved from manual testing; more effective use of manual testing; costs saved by finding issues earlier; and reduced risk of flaws making their way into production and/or into the market.