White Paper: Next generation Test Factory
In 2008, QA Consultants launched the most disruptive innovation to hit the software quality assurance industry since the rise of off-shoring. The Test Factory, a Canadian-based testing facility, was designed to be a compelling alternative to remote offshore testing and expensive in-house efforts. The value was immediately recognized; this unique business model was a success right out of the gate garnering dozens of mission critical projects and Fortune 500 clients. Not surprisingly, a number of other “Test Factories” were introduced a few years later hoping to capitalize on the popularity of QA Consultant’s innovation. However, these were me-too offerings with no unique value. Meanwhile, QA Consultants continuously refined their Test Factory model. In 2012, they launched the industry’s first Test Factory Centre of Excellence (CoE).
The Test Factory CoE is designed to meet today’s client and market demands for superior value & time to market, high quality, cost certainty & transparency and lower business & project risk. A CoE pushes the capabilities and value envelope far beyond the first generation Test Factory by leveraging the synergies between three quality assurance pillars: 1) a Toronto-based, highly scalable & secure facility; 2) technically expert and industry savvy testers; and 3) extensive use of proprietary automation, streamlined operating processes and risk-based methodologies.
Traditional approaches to testing, through in-house resources, staff augmentation or off-shored to India, have not kept pace with many of today’s business and technical requirements and challenges. Specifically, maintaining a static and expensive internal testing group does not make sense when testing and skill requirements are dynamic and uncertain, and margins are compressing. Staff augmentation can be an effective testing approach but it is expensive to manage and does little to capture testing best practices or application knowledge. Furthermore, staff augmentation is more costly than other testing approaches in projects requiring over 100 full time testers; that need unique technical or industry knowledge or; that run over six months in duration.
Off shoring presents a host of other issues. There are significant challenges and costs doing business in a distant geography like India, who have a completely different culture, language and business norms. For example, it is not uncommon for Indian providers to experience 80 percent employee turnover. These factors create systematic delivery and quality issues which cannot be rectified in the medium term. Furthermore, India’s labour cost advantage has shrunk dramatically in the last five years reducing offshore’s original appeal – especially after transaction and administrative costs are factored in. Moreover, many companies remain concerned about the security of their intellectual property and customer data residing half away around the world. Finally, organizations are increasingly focusing on proper project governance; they are questioning the potential conflict of interest at generalist IT services firms when developers test their own code.
A new model emerges
The original Test Factory generated compelling customer value. It has successfully delivered over 720 projects for more than two-dozen market leaders such as Aviva, Bank of America, Loblaw, Praxair and Canadian Tire. The Test Factory CoE has enhanced capabilities and capacity to handle larger, more complex projects in dynamic and regulatory-intensive sectors like financial services, government, energy and manufacturing. Migrating to a CoE operating model enables QA Consultants to handle projects requiring up to 1000 testers, delivering a myriad of testing services, with full cost and resource transparency.
Below are some of the unique capabilities that make the new Test Factory a superior alternative to offshore or in-house testing schemes:
The Test Factory’s CoE unique Shared Services Model (Figure 1.) features streamlined and track-able workflows, processes and policies that maximize resource productivity and flexibility. Where possible, automation is used to reduce cost and accelerate project delivery. As well, clients enjoy full visibility around how their project is progressing, which resources are being deployed and when, and how, their fees are being spent.
Flexibility, to handle unique and dynamic client wishes, is baked right into the operational model. Test Factory services can be easily combined with other services like consulting and staff augmentation to deliver a bespoke testing solution. Furthermore, our delivery model is easily integrated into the client’s workflows and teams, effectively turning the Test Factory into an extension of the client’s organization.
The Test Factory maintains over 600 dedicated testers and business resources ‘ on-demand.’ This deep bench of technical, industry and project management experts can scale quickly to handle any project for any industry. Our team has extensive experience with a variety of applications including SAP, Manhattan, Oracle/Siebel, Salesforce.com, and PeopleSoft, using industry standard testing tools (we are an HP reseller!). In fact every Test Factory CoE tester is a computer science graduate. Our testing teams are managed within a custom designed resource pool, allowing for quick resource allocation and tracking.
Maintaining a roster of experts is not enough; their skills and practices need to be continuously renewed to reflect the latest technology and business practices. To stay current, we spend over a million dollars per year upgrading the technical and soft (e.g., communications and team work) skills of all Test Factory CoE employees and managers. This investment has helped reduce employee turnover to less than 7% annually.
TRACK RECORD OF INNOVATION
Spending millions of dollars in R&D has helped the Test Factory CoE develop advanced levels of automation and patented, risk-based methodologies that drive higher testing quality, reduced cost and accelerated time to market/value. Ongoing innovation investments also ensure that processes, methodologies and tools are continuously improved and deployed.
Innovation is not limited to the testing discipline. The Test Factory CoE utilizes a proprietary knowledge management system that captures all client and industry data, gleans best practices and key market developments and archives critical project information like test cases.
A people-driven, ‘on demand’ business requires a large, modern facility to maximize productivity. The Test Factory CoE is housed in a modern 30,000 sq. ft. facility on the 14th floor of the North American Center at Yonge and Finch in Toronto. The infrastructure features all the latest network, security and communication advances, with the ability to accommodate a large amounts of testers under one roof.
These days, every organization is concerned about the security of their intellectual property and brand assets as well as the privacy of the consumer data they manage. The Test Factory CoE provides bank-level security for every project and client.
The above capabilities work seamlessly and symbiotically to produce superior client value, as demonstrated by:
LOWERING TOTAL COST OF TESTING
The Test Factory has delivered a 30% total cost savings and a 20% improvement in testing completion time versus traditional approaches like off-shoring, in-house staffing and staff augmentation. This cost advantage becomes even more pronounced when offshore’s higher error and duplication rates are factored in.
Lower costs come from higher labour productivity, using risk-based methodologies and leveraging extensive amounts of testing and process innovation. Figure 2. illustrates the cost and delivery advantages from the Test Factory operating model.
The Test Factory CoE’s lower total cost also boosts project ROI by accelerating time to market or value – leading to higher product revenues or operating efficiencies sooner – and by minimizing cost through testing tools re-use, experience effects (more experience leads to higher productivity, fewer errors etc.) and scale economies. Longer term, our higher quality standards will support a client’s margins by improving application and/or product performance, and enhancing their brand image and competencies.
Finally, we give clients the ability to closely monitor and limit their tester and tools spending so there are no budget surprises and projects have a better chance to hit their financial goals.
A core mission of the Test Factory CoE is to minimize business, brand and technical risk. All of the facilities, resources and projects are fully visible to clients at any time. All IP and brand assets are safe and secure, often within a 30-minute drive from the client’s office.
Since the Test Factory CoE is focused only on testing, clients can rest easy knowing that they are getting objective and bias-free work, unencumbered by any conflicts when developers test their own code. And, we conduct business the same way our Fortune 500 clients do; the Test Factory is fully compliant with workplace norms, standards and laws.
HIGHER QUALITY PERFORMANCE
Poor software quality reduces product or application performance, upsets users and can be a hassle and expensive to fix. Since it is virtually impossible and impractical to write perfect code, it comes down to the testing effort to identify all bugs and software architecture issues. Figure 3. illustrates the prohibitive cost and business risk of not catching errors during testing.
The Test Factory CoE is designed with quality in mind. Being a mature business unit of North America’s largest dedicated tester is a good start. A track record of successfully delivering 5000+ projects for many Fortune 500 companies demonstrates that we are experts in all facets of testing and quality assurance. Moreover, quality outcomes arise naturally from a leveraging a highly trained, more stable roster of testers and managers as well as high levels of automation and quality-focused workflows. Finally, our ‘can-do’ culture emphasizes continuous improvement, creative troubleshooting and ‘personal touch’ service.
All Rights Reserved ©.