Many companies are coping with a faster pace of infrastructure, application and product change by adopting Agile development methodologies. There are many flavours of Agile but all of them employ iterative, dynamic team-based development, and QA. The Agile approach differs significantly from the traditional, linear Waterfall development method. Given the differences, many IT managers are unsure of where and how critical testing activities fit in. Specifically, what is the role of testing in Agile methodologies? How do you test mission-critical software within this approach?
Our considerable experience with hundreds of Agile projects reinforces the importance of properly planned and executed testing, whether in pure Agile or hybrid development environments. The key success factor for IT managers is to understand how critical testing actions are executed and by whom.
In Waterfall development, testing activities are defined early and executed at the end of the development. There is role clarity, execution certainty, and (hopefully) committed resources. On the other hand, Agile development employs testing but in a totally different way. Testing activities are executed more iteratively, in real-time, and with less documentation. The difference is in style and approach but not importance and intent. Simply put, ensuring proper testing is just as critical in Agile as it is in Waterfall.
Given the relative newness of Agile development, most CIOs remain understandably and appropriately cautious about how they can leverage it. They want the flexibility and time-to-value benefits of Agile without incurring extra risk, rework and cost. As well, our clients regularly ask how we can leverage our proven Waterfall methodologies, testers, and automated tools into their emerging Agile environment
Transition to Agile Development Methodology
QAC has worked with clients such as Toyota Canada and Canadian Tire to help with their Agile QA Transformation journey, and to provide input and perspective on the implications and changes for QA/Testing teams. QAC recommends that clients transition to an Agile Software Development Methodology for both software and content development in a phased approach. There are several advantages to adopting an Agile Development & QA Methodology:
- The Agile methodology allows for changes to be made after the initial planning. Re-writes to the program, as the client decides to make changes, are expected
- At the end of each sprint, project priorities are evaluated. This allows clients to add their feedback to ultimately get the product they desire
- The testing at the end of each sprint ensures that the bugs are caught and taken care of in the development cycle. In contrast, in a Waterfall Development Methodology, testing is an after through process and the late stage in which defects are found makes it much costlier to resolve, or even worse, leave them unaddressed as the project runs out of time
- Since the products are tested so thoroughly with Agile, the product could be launched at the end of any cycle. As a result, it’s more likely to reach its launch date
As with any transformation, QAC experience shows us that an Agile Transformation QA strategy is developed and executed against. From a risk mitigation perspective, it is recommended that as clients continue their Agile Transformation journey, the transformation is rolled out in a phased approach. It is highly recommended that a small-scale project/program is used for piloting the transition to an Agile QA Methodology. An enterprise-wide Agile transformation is a complex process and should be guided a certified Agile Transformation Coach.
Using QA Consultants’ Agile Project Deployment Approach
Our approach to delivering QA projects under an Agile development process is shown in the figure below. Our assumptions regarding the breakdown of a client versus QAC project responsibilities are also depicted:
Figure 1 – QAC Agile Project Deployment Approach
Key lessons from our Agile QA experience
Define the role of QA and overall strategy upfront
Goal definition, planning and defining testing strategy are vital in Agile but are undertaken differently. The reality is that defining the goals and what the desired “end state” looks like is not always simple in an Agile world.
QAC ensures at the outset of the project that testing needs are properly documented and instilled within the daily testing activities. For example, we help the client understand the role of testing and planning in Agile. We define the technical, business and project goals upfront including coverage and resource requirements. Proactively, we support the identification of goals and what “done” looks like. The iterative and sprinting nature of Agile delivery relies heavily on everyone having a deep and common understanding of what success looks like and how they will get there. Following goal definition, we help Agile teams ensure they have the right resource allocation and a complete understanding of all the requirements and risks.
We work with project owners to determine the testing strategy as well as identify possible pitfalls and enablers. If Agile is to be successful, testing must be core and embedded within the development process. Furthermore, special attention must be paid to information management. We determine which information needs to be captured, reported, and archived. This is a vital task given that many Agile environments purposely generate less documentation to go faster. In some cases, our clients have asked us to continue to provide Waterfall type services (e.g., documentation, archiving) as a back-up and to capture our best practices.
Following project is kicked off, we provide regular feedback and reporting on how the project team and the code is tracking against the objectives. Since the end goal is likely to change in an Agile world, our experts can continue to play a vital role by re-focusing the testing effort to ensure the latest business and technical priorities are being met.
Deploy a skilled Agile QA team
Great talent is at the heart of every Agile project. Successful Agile projects require a team of technical experts who possess a unique set of soft skills such as effective communications, coordination, and problem-solving. Not surprisingly, assembling a high-performance Agile team is not easy or inexpensive to create or maintain. Many companies don’t want the cost, time lag, or hassle of recruiting or having expensive resources sit idle on a bench. Such tasks can be easily and economically outsource to objective, Agile-savvy testing firms like QA Consultants’ ‘on demand.’
A typical Agile QA delivery team has the following roles:
- QA Manager
- Location: Primarily onsite at clients, but may go to TestFactory™ as needed
- Strong BI Architect background and expertise
- Focus: Overall QA Strategy, Architecture, and Direction; QA Delivery
Management
- Line of Business QA Leads
- Location: Primarily onsite at clients, but may go to TestFactory™ as needed
- Strong client LOB SME knowledge, Strong BI expertise (SQL, ETL, etc.)
- Leads have cross-LOB experience to enable movement between LOB
workstreams - Focus: In-Sprint Test Strategy & Approach; Test Design, Test Case &
Expected Results Definition; Test Script Development & Execution (as needed in support of Dev Testing); Test Data Development/Sourcing; Support for Development and BA activities
- QA CORE TestFactory™ Team
- Location: Primarily in the TestFactory™ (up to 1 day onsite per week)
- Strong SQL Background, Knowledge/Experience with Netezza and
Hadoop - Focus: Test script execution under guidance and direction from LOB QA Lead
- QA FLEX OnDemand™ TestFactory™ Team
- Guaranteed and Dedicated Bench for the client
- No Cost Retainer
- 48-Hours’ Notice for Engagement Lead Time
- 48-Hours’ Notice for Engagement Ramp-Down
- Team will have screening and sanctioning Completed for Quick Engagement
- Team will perform shadowing of an existing team to enable minimal ramp-up Time
- Location: Primarily in the TestFactory™ (up to 1 day onsite per week)
- Strong SQL Background, Knowledge/Experience with Netezza and Hadoop
- Focus: Test script execution under guidance and direction from LOB QA Lead
In general, our approach is to meld Agile-ready resources quickly and seamlessly within the client’s dynamic Agile environment. Expertise, flexibility, and agility are our bywords. Our talent can provide real-time testing, consulting or facilitation support ‘as needed when needed,’ to suit the rhythms and pace of the Agile project. Since every project need and environment is different, our testing resources can be mix n’ matched within multiple scrums or as consultants/governance experts within a project management office.
The high tempo nature of Agile requires getting the right people in the right places at the right time. We actively recruit, vet and maintain a bench of Agile-ready testers with the right package of soft skills. Moreover, we regularly train these resources to fit seamlessly within an Agile process. This way, we can provide expert testing along with supporting scrum coordination, integration, and reporting. Finally, many of our Agile team members have the capability to concurrently interact with Agile and non-Agile teams to ensure coordination, scalability and the sharing of best practices.