The pursuit of high quality is at the top of most corporate agendas. Quality is the end result of an organization’s activities and products around meeting consumer or user expectations in terms of reliability performance, cost, service and delivery. Software, in particular plays a critical role. In many companies, improving the quality of software applications and products will have the biggest overall impact on their business and financial performance. This begins with fixing how they organize and implement some of their quality-focused functions.
Delivering over 5000 testing and consulting projects over the last 20 years has taught us many lessons about producing quality software. One common area for improvement is how companies conduct quality assurance (QA) and testing within the software development lifecycle (SDLC). Very often, organizations confuse the two activities and end up doing a poor job in both. To overcome this, CIOs and IT managers need to understand the differences between QA and testing and then ensure each activity is performing its desired function, effectively and efficiently.
There is a strong business case to get the QA and testing mix right. For one thing, quality is an organizational imperative; its elevation requires an enterprise-wide commitment and solution. Secondly, it is much more expensive to find and fix problems during the testing phase than it is to catch bugs earlier at the development stage. Moreover, waiting till the testing stage to undertake quality control increases the chances of poorly–performing products and applications slipping through, and increases the likelihood of the project running late (e.g., due to re-coding). Finally, process confusion can often produce ancillary problems such as misplaced effort and poor resource allocation.
Testing and QA are different concepts with unique objectives: QA is a set of company-wide processes, systems and policies that ensures all applications or products are built and maintained to corporate and/or independent quality benchmarks. Quality assurance processes incorporate some key principles, tools and methods such as: good governance, independent verification and validation, standards, audits and reporting.
Two key prerequisites for maximizing QA capabilities are having a good testing strategy and maintaining internal testing competencies. Testing is a planned set of specific activities to identify bugs, based on the completeness, security and quality of a new product, system or application – as compared to the user and technical requirements. Testing can be performed in different companies, by different kinds of teams (e.g., Agile versus Waterfall developers), using different kinds of testing methodologies.
The differences between QA and testing is summarized in the table below:
Comparing QA and Testing
| Variable | QA | Testing |
|---|---|---|
| Mandate | Build in quality throughout the SDLC. | Executing with excellence the final step of the QA process (quality control). |
| Focus | Proactively get the organization’s capabilities right – structure, process, people, tools, culture. | Reactively finding bugs in code before users discover them; suggesting code improvements. |
| Scope | The entire SDLC: Requirements, Planning, Development, Documentation, Reporting, Testing. | Product or application-based, at the quality control stage. |
| Value | Bug prevention through repeatable, high performance processes. | Quickly and efficiently find bugs. |
| Team | Duty of everyone including the Business. | Testers and developers only (typically). |
- The economic and technological environment is fast paced, increasing complexity yet compressing project lead times.
- There are often competing internal demands due to shifting priorities and limited resources.
- Firms have weak compliance around QA processes – or no processes at all.
- Organizations lack the skills or experience to get the QA process right.
Going forward
Meeting software quality expectations is often easier said than done given the above realities. As indicated above, step one is about clarifying the different roles and resourcing between QA and testing. Secondly, we recommend firms follow some software quality ‘best practices’ to enhance their capabilities.
To Improve QA
- Process map, analyze and retool (if necessary) the entire SDLC to ensure quality is baked in to all activities.
- Create quality ‘gates’ that enable developers, architects and business owners to review and send code back for rework – before it gets to testing.
- Align all technical and business stakeholders in advance of development.
- Validate all requirements before coding begins.
- Utilize proven implementation methodologies like Lean and Six Sigma.
To Improve Testing
- Prioritize tester retention and training to capture key technical and business knowledge.
- Use different but specialized vendors for testing and development.
- One size does not fit all, so consider multiple testing strategies and approaches based on needs and type of development (e.g., Agile versus Waterfall).
- Use automation, where and when it makes financial and project sense.
- If the project is running late or over-budget, fight the urge to shortcut testing time or deploying the necessary resources.