(416) 238-5333

Transitioning to an Agile QA Delivery Model

Many companies are coping with a faster pace of infrastructure, application and product change by adopting Agile development methodologies. There are many flavours of Agile but all of them employ iterative, dynamic team-based development, and QA. The Agile approach differs significantly from the traditional, linear Waterfall development method. Given the differences, many IT managers are unsure of where and how critical testing activities fit in. Specifically, what is the role of testing in Agile methodologies? How do you test mission-critical software within this approach?

Our considerable experience with hundreds of Agile projects reinforces the importance of properly planned and executed testing, whether in pure Agile or hybrid development environments. The key success factor for IT managers is to understand how critical testing actions are executed and by whom.

In Waterfall development, testing activities are defined early and executed at the end of the development. There is role clarity, execution certainty, and (hopefully) committed resources. On the other hand, Agile development employs testing but in a totally different way. Testing activities are executed more iteratively, in real-time, and with less documentation. The difference is in style and approach but not importance and intent. Simply put, ensuring proper testing is just as critical in Agile as it is in Waterfall.

Given the relative newness of Agile development, most CIOs remain understandably and appropriately cautious about how they can leverage it. They want the flexibility and time-to-value benefits of Agile without incurring extra risk, rework and cost. As well, our clients regularly ask how we can leverage our proven Waterfall methodologies, testers, and automated tools into their emerging Agile environment

Transition to Agile Development Methodology

QAC has worked with clients such as Toyota Canada and Canadian Tire to help with their Agile QA Transformation journey, and to provide input and perspective on the implications and changes for QA/Testing teams. QAC recommends that clients transition to an Agile Software Development Methodology for both software and content development in a phased approach. There are several advantages to adopting an Agile Development & QA Methodology:

  • The Agile methodology allows for changes to be made after the initial planning. Re-writes to the program, as the client decides to make changes, are expected
  • At the end of each sprint, project priorities are evaluated. This allows clients to add their feedback to ultimately get the product they desire
  • The testing at the end of each sprint ensures that the bugs are caught and taken care of in the development cycle. In contrast, in a Waterfall Development Methodology, testing is an after through process and the late stage in which defects are found makes it much costlier to resolve, or even worse, leave them unaddressed as the project runs out of time
  • Since the products are tested so thoroughly with Agile, the product could be launched at the end of any cycle. As a result, it’s more likely to reach its launch date

As with any transformation, QAC experience shows us that an Agile Transformation QA strategy is developed and executed against. From a risk mitigation perspective, it is recommended that as clients continue their Agile Transformation journey, the transformation is rolled out in a phased approach. It is highly recommended that a small-scale project/program is used for piloting the transition to an Agile QA Methodology. An enterprise-wide Agile transformation is a complex process and should be guided a certified Agile Transformation Coach.

Using QA Consultants’ Agile Project Deployment Approach

Our approach to delivering QA projects under an Agile development process is shown in the figure below.  Our assumptions regarding the breakdown of a client versus QAC project responsibilities are also depicted:

Figure 1 – QAC Agile Project Deployment Approach


Key lessons from our Agile QA experience

Define the role of QA and overall strategy upfront

Goal definition, planning and defining testing strategy are vital in Agile but are undertaken differently. The reality is that defining the goals and what the desired “end state” looks like is not always simple in an Agile world.

QAC ensures at the outset of the project that testing needs are properly documented and instilled within the daily testing activities. For example, we help the client understand the role of testing and planning in Agile. We define the technical, business and project goals upfront including coverage and resource requirements. Proactively, we support the identification of goals and what “done” looks like.  The iterative and sprinting nature of Agile delivery relies heavily on everyone having a deep and common understanding of what success looks like and how they will get there. Following goal definition, we help Agile teams ensure they have the right resource allocation and a complete understanding of all the requirements and risks.

We work with project owners to determine the testing strategy as well as identify possible pitfalls and enablers.  If Agile is to be successful, testing must be core and embedded within the development process.  Furthermore, special attention must be paid to information management.  We determine which information needs to be captured, reported, and archived. This is a vital task given that many Agile environments purposely generate less documentation to go faster. In some cases, our clients have asked us to continue to provide Waterfall type services (e.g., documentation, archiving) as a back-up and to capture our best practices.

Following project is kicked off, we provide regular feedback and reporting on how the project team and the code is tracking against the objectives.  Since the end goal is likely to change in an Agile world, our experts can continue to play a vital role by re-focusing the testing effort to ensure the latest business and technical priorities are being met.

Deploy a skilled Agile QA team

Great talent is at the heart of every Agile project. Successful Agile projects require a team of technical experts who possess a unique set of soft skills such as effective communications, coordination, and problem-solving.  Not surprisingly, assembling a high-performance Agile team is not easy or inexpensive to create or maintain.  Many companies don’t want the cost, time lag, or hassle of recruiting or having expensive resources sit idle on a bench. Such tasks can be easily and economically outsource to objective, Agile-savvy testing firms like QA Consultants’ ‘on demand.’

In general, our approach is to meld Agile-ready resources quickly and seamlessly within the client’s dynamic Agile environment.  Expertise, flexibility and agility are our bywords. Our talent can provide real-time testing, consulting or facilitation support ‘as needed when needed,’ to suit the rhythms and pace of the Agile project. Since every project need and environment is different, our testing resources can be mix n’ matched within multiple scrums or as consultants/governance experts within a project management office.

The high tempo nature of Agile requires getting the right people in the right places at the right time. We actively recruit, vet and maintain a bench of Agile-ready testers with the right package of soft skills.  Moreover, we regularly train these resources to fit seamlessly within an Agile process. This way, we can provide expert testing along with supporting scrum coordination, integration, and reporting. Finally, many of our Agile team members have the capability to concurrently interact with Agile and non-Agile teams to ensure coordination, scalability and the sharing of best practices.

A typical Agile QA delivery team has the following roles:

  • QA Manager
    • Location: Primarily onsite at clients, but may go to TestFactory as needed
    • Strong BI Architect background and expertise
    • Focus: Overall QA Strategy, Architecture, and Direction; QA Delivery
  • Line of Business QA Leads
    • Location: Primarily onsite at clients, but may go to TestFactory as needed
    • Strong client LOB SME knowledge, Strong BI expertise (SQL,
      ETL, etc.)
    • Leads have cross-LOB experience to enable movement between LOB
    • Focus: In-Sprint Test Strategy & Approach; Test Design, Test Case &
      Expected Results Definition; Test Script Development & Execution (as needed in support of Dev Testing); Test Data Development/Sourcing; Support for Development and BA activities
  • QA CORE TestFactory Team
    • Location: Primarily in the TestFactory (up to 1 day onsite per week)
    • Strong SQL Background, Knowledge/Experience with Netezza and
    • Focus: Test script execution under guidance and direction from LOB QA Lead
  • QA FLEX OnDemand TestFactory Team
    • Guaranteed and Dedicated Bench for the client
    • No Cost Retainer
    • 48-Hours’ Notice for Engagement Lead Time
    • 48-Hours’ Notice for Engagement Ramp-Down
    • Team will have screening and sanctioning Completed for Quick Engagement
    • Team will perform shadowing of an existing team to enable minimal ramp-up Time
    • Location: Primarily in the TestFactory (up to 1 day onsite per week)
    • Strong SQL Background, Knowledge/Experience with Netezza and Hadoop
    • Focus: Test script execution under guidance and direction from LOB QA Lead

For example, at a recent project for Canadian Tire, QAC deployed the following Agile delivery team in a Managed Service model:

Figure 2 – QAC’s Agile Rapid Delivery Model

Choosing tools & methodologies

QAC supercharges Agile development by leveraging and tailoring our proven Waterfall testing approach to the unique demands of Agile projects. In Agile environments, all types of testing tools and methodologies are used, depending on the technical requirements.  For example, we have undertaken regression testing, load testing, and security testing to name but three use cases.  Additionally, we made our proprietary and production-proven methodology, Quality Point ™, Agile-ready.  Its practical and risk-based approach is ideal for dynamic and iterative testing styles as well as minimizing project cost and risk. The use of an Agile scrum methodology in a multi-release/project-based environment requires a specialized testing framework. QA Consultants’ Agile testing methodology include processes and standards for Agile in addition to more traditional development methods such as waterfall. We are currently supporting many clients with both waterfall and agile projects.

Figure 3 – QAC Agile Scrum Methodology


Our approach to supporting our client’s Agile transformation

QAC can utilize our Discovery Portfolio Assessment (DPA) methodology as part of the kickoff and pre-launch program for our engagement responsibilities and client objectives. Such methodologies provide a steady state QA service capable of supporting Agile QA methods.

The QAC DPA program is designed to provide a full review of Agile QA parameters of a client’s applications, current test strategies, quality assurance philosophy, incumbent protocols, business requirements, all for the purposes of transitioning to the soon to be established MTS. A test assessment can be conducted which defines an implementation roadmap for all of a client’s QA service requirements:

QAC can utilize a proprietary DPA methodology to conduct a comprehensive analysis of the current infrastructure, stakeholders and existing ecosystem that will include people, process, and technology. The DPA is designed to accurately assess the complexity of the client’s Development & QA environment, and provide a real-world assessment of what will be required to achieve the business, financial, and quality objectives of the Agile QA service.

This effort is critical and necessary to deliver the project requirements, identifying the potential risks that may impact the expectations, timelines, and most importantly, outcomes.

Figure 4 – Discovery Portfolio Assessment (DPA) Process

DPA process

The QAC Team members can conduct an in-depth discovery assessment and review both business and IT metrics for success. This includes the objective of understanding the client’s current program delivery capabilities and environments, supporting technologies, business processes, integration services and existing API’s, interfacing with existing applications, technologies, and environments that will form the basis of the client’s MTS platform.

The QAC DPA is a comprehensive assessment of the client’s applications portfolio/environment. The DPA will identify all factors required to include project transition risk and will identify the current and future stakeholder actors, required SME disciplines, project management metrics, KPIs, and timelines. The program will outline the dimension of work effort required to move the environment from the current (in house state) to the new MTS. This effort will establish the required governance, compliances, and risk factors associated with customers who transition to a new world. 

Our approach to mitigating risks in an Agile transformation

Key to a successful MTS and supporting Agile Testing processes is the understanding that it is a transformational initiative and requires a considerable amount of time and a series of phases. It is important to acknowledge that “change is a process, not an event”. Our experience with implementing an MTS has shown that managers are pressured to show results fast, as a result of which process adherence is compromised, and steps are skipped. Skipping steps in any of the phases only create an illusion of speed and never produces satisfying results. Chances for a successful MTS implementation are increased not only when there is an adherence to the process, but also when there is an understanding of the challenges unique these to each stage.

To mitigate these challenges, QAC has adopted a change management framework that focuses on the following:

  • Establish a sense of urgency. Starting a transformation effort without the cooperation of all stakeholders involved does not produce results. One of the common mistakes in this stage is underestimating the difficulty of driving leaders out of their comfort zones. Another common pitfall for this stage is when executives become paralyzed by the downside of risk, either fearing that there will be push back from employees with more seniority or that short-term business results will be jeopardized by the change initiative.
  • Form a powerful guiding coalition. Getting executive sponsorship and support for the change is not sufficient. Successful transformations require a critical mass, otherwise, progress is slow. It is essential to form a leadership coalition with a shared commitment to lead the change effort. It is important to empower individuals and encourage them to work outside their normal hierarchy. Two of the common pitfalls for this stage of the change process are: 1) no prior experience in teamwork at the top and 2) relegating team leadership to an HR, quality, or strategic-planning executive rather than a senior line manager.
  • Create a vision. Once the change coalition has developed a shared understanding of the company’s problems and opportunities, it is essential to create a vision that directs the change efforts. The scope of the vision should be beyond specific numbers tied to three and five-year plans. The vision should provide a clear direction in which the organization needs to move. Once the vision has been through several revisions and is clear and concise, the next step is to develop the strategies required to realize the vision. A common mistake for this stage is presenting a vision that is too vague or too complicated to be communicated in five minutes. If the vision cannot be communicated in five minutes or less and create a reaction that indicates both understanding and interest, then the vision requires further fine-tuning
  • Communicate the vision. Transformation is impossible without getting the buy-in and the willingness of employees to help to the point that they are ok with making short-term sacrifices if required. To achieve this stage of employee, buy-in and commitment, it is essential for executives to “walk the talk” and to become a living symbol of the new corporate culture. A common pitfall for this stage of the change process is under communicating the vision. Communication on its own alone is not sufficient. It is important for executives to exhibit actions and behaviors consistent with the vision.
  • Empower others to act on the vision. The success of this stage relies on removing or altering systems or structures that undermine the vision. It is also imperative to encourage risk-taking, non-traditional ideas, activities, and actions. By its nature, change can often be met with resistance and reluctance. A common mistake for this stage of the change process is a failure to eliminate obstacles early on. Often, the obstacle could be a powerful individual who resists the change effort. Failure to eliminate obstacles becomes antithetical to the vision and the commitment for change and renewal, resulting in growing cynicism and collapse of efforts.
  • Plan for and create short term wins. To keep momentum, it is important to define and engineer visible performance improvements. Very important for this stage is also to recognize and reward employees who contribute to the improvement efforts. One of the common mistakes for this stage of the process is leaving short-term successes up to chance. Another common mistake is a failure to score successes early enough in the change process.
  • Consolidate improvements and produce more change. The success of this stage is characterized by hiring, promoting and developing employees who can implement the vision. Using increased visibility from early wins is a powerful tool to change systems, structures, and policies, undermining the vision. It is essential to consistently invigorate the change process with new projects and change agents.
  • Critical mistakes during any stage of the change process can have a devastating impact on the overall initiative. One of the common pitfalls for this stage is managers declaring success too soon, resulting in loss of momentum and quick reversals to doing business the traditional “old way”. It is important to realize that new approaches are fragile and subject to regression and can take several months until changes sink deeply into a company’s culture.
  • Institutionalize new approaches. If there has been an intense level of urgency, a powerful coalition and a clear vision for the change initiative to succeed, important in this stage of the change initiative is to institutionalize the new approaches. To achieve this goal, it is essential to articulate the connections between new behaviors and corporate success.


For further information on this topic and performance testing please contact:

Todd Merritt
Senior Director, Client Service Organization
(416) 580-0590