Agile development has become commonplace in many companies. However, many quality assurance and testing leaders are unclear about how testing – still a vital activity in the software development life cycle – is undertaken. In particular, there are questions around the role of testing, the best approach to test and what makes an ideal tester.
For example, Agile can create discomfort for a tester who specializes in Waterfall methodologies – in the ideal world they would have a finished product to verify against a finished specification. To be asked to validate a moving code target against changing requirements is counter-intuitive. It also means the use of many technologies and automation is more difficult, requiring the testing team to pragmatically apply each tool.
6 Key Success Factors
The following are 6 key success factors that must be taken into account when testing in Agile environments:
Align the testing strategy to the development environment
Most Agile environments today are actually hybrids, coexisting with waterfall practices and having their own specific practices and policies. This brings unique testing challenges around how and when to test. As such, testing decisions should be made according to pragmatic principles based on the existing business and technical requirements, current processes and structure. Regardless of the approach, what’s critical to remember is that the value of Agile lies in providing a continuous feedback loop to the development team.
Agile environments require a well thought out test strategy and supporting capabilities. The approach should take into account what is being tested and how it is being coded. Moreover, test managers need to consider the skill and maturity of the testers as well as enabling resources like QA labs, infrastructure and security teams to ensure it supports the project deliverables. Finally, managers need to make sure the test strategy considers how the end of release activities fit with current resourcing, test data readiness and the reuse of test assets.
Understanding the testing mandate
In Agile, the focus of the effort is on building working software based on what’s important to the user as opposed to what is a procedural or a documented requirement. As a result, the tester needs to be engaged in not only evaluating the code but also deciding on how the code will be evaluated. It is also not uncommon for acceptance testing to be the responsibility of the end-user. Testers must be cognizant that they may not have sole or even primary responsibility on deciding what works and what doesn’t.
Recruit the right people
Testing in Agile is about the 3 Cs – curiosity, collaboration and communication – rather than slavishly and independently following a blueprint. Agile testers need to be able to work with software products and applications directly, exploring a system, programming test scripts and analyzing a large range of outputs. At the same time, Agile testers need to interface directly with architects, software developers and even product managers – in scrums – to understand the business and technical requirements and restrictions that impact the software and its testability. Agile testers should be comfortable in a world of multiple blurred roles: between developers, testers and customers, between managers and workers; or between users and software designers. The most successful testers tend to be generalists, possess a full range of technical and soft skills.
Into the right roles
It is not always easy to have role definitions within Agile development, but it is highly recommended to avoid project delays. A lack of clarity around roles increases the potential for conflict, duplication and missed testing opportunities. In general, the job of a tester encompasses two key tasks:
Optimizing the confirmation tests at component and acceptance levels. This may include bringing automation to part of their acceptance testing as well as introducing new tools or practices across various outputs.
Identifying, diagnosing and exploiting unexpected actions (i.e. not ‘happy path’ interactions) that are typical of customers or users. In essence, the tester is acting as a proxy for a series of behaviors. Of particular importance is how the code responds to bad user behaviors, such as:
- Empty sets, null inputs
- Invalid parts of input – characters, values, combinations
- Going too fast
- Stop halfway / Jump in halfway
- Time changes
There sometimes is an inclination to have developers do their own testing. This is not recommended due to their lack of objectivity, testing skill and interest. Experienced Agile testers can usually see beyond specific functions to look at “the big picture”, ask detached questions like “what if…?” and possess the requisite testing skills and experience with automation. In reality, the majority of developers do not want to spend their time writing tests and then developing code to prove the tests work.
With the right habits
Agile workflows require testers to be fully embedded within a collaborative team, or a scrum. Unlike waterfall processes, testers cannot function as an independent group. However, the same testers will need to maintain objectivity in their outlook and consider how to retain testing independence while still being an engaged team player. One way to do this is for the tester to play the role of a customer advocate, in some cases acting as a ‘bad customer.’
In addition, testers need to adjust their work styles to an Agile environment. Comfort within a rapidly changing and potentially uncertain process (e.g., there is no requirements freeze before coding starts) is important; testers will function in a development process that is more an iterative exercise than a negotiation over deliverables or discussion over bugs.
Consider multiple testing approaches
Confirmation testing is typical in Agile projects. However, there is also a role for risk-based, black box or investigative testing. This type of testing allows for business or technical context (not rigid test plans) to drive test decision-making and enables the code to be evaluated based on non-catastrophic changes in direction. Black-box testing can improve the number, speed and quality of bugs found and reduce the reliance on standard processes and documentation.
Despite workflow-based challenges, it is still possible, practical and desirable to leverage automation in Agile projects. The early stages of Agile code are often unstable requiring the system to be manually tested. Eventually, the system will evolve and settle, becoming automation-ready. The key is capturing and re-using the manual testing and design information and then using Automation technology that proactively supports early manual testing but provides a path to integrating automation later into the testing effort.
Conclusion
Given their unique characteristics, Agile projects usually demand even better, more well-rounded testers than in waterfall development methodologies. To be successful, Agile testing teams need to intimately engage in the project “what” and “how” while ensuring the system meets the business needs. To do this, the testing process and team need to be agile and open-minded themselves, discarding previous paradigms while focusing on new testing techniques that enable better, and faster development. In fact, Agile testers are ideally positioned to bridge the technical and requirements gaps between product developers, coders and customers.