Over the past few years, we have seen IT departments (and their business sponsors) demand better performance at lower cost from their testing vendors. This higher bar is being driven in large part by strategic imperatives to reduce time to market, enhance application & service quality and improve the ROI of IT activities. Testing has come under particular scrutiny, since it is fundamental to quality goals and a major cost center.
Not surprisingly, conventional methods to testing are being reexamined. Offshoring testing to India is risky, a hassle and not the financial bargain it once was. Using local contractors can work – if you can find the right people at the right price. In-sourcing all testing activities may be ideal in theory but is not a practical option for the vast majority of firms given the difficulty of managing fluctuating testing demand with static resources. Moving forward, CIOs and business sponsors need innovative testing solutions that align with their firm’s strategic goals and support the entire software development lifecycle in an efficient, flexible and expert manner. Fortunately, an innovative testing model – the Test Center of Excellence (TCoE) – is now deployment-ready for most medium-to-large enterprises. Moving to the TCoE model has proven in many large organizations to deliver higher quality testing, locally, at less cost and risk.
The bar has been raised
Most organizations we deal with are grappling with significant IT challenges such as the need to get to value quicker with new technologies and upgrades, dealing with IT complexity and better managing their steadily increasing costs and risks. In our experience, testing activities could consume upwards of 40% of the total cost of some software development projects. These issues have pushed testing – long an ignored area of IT – to the top of corporate agendas. Recent problems with Obamacare’s healthcare.gov and Apple’s iPhone map launch are tip of the iceberg examples of the failures of proper testing to support business and financial objectives. These expensive and public embarrassments are helping spark frank organizational conversations around the importance of testing; the true cost of poor quality and how testing can be best delivered.
Tatyana Dovga, AVP of Aviva, knows all about testing innovation. She helped implement one of the first TCoEs in the Canadian insurance industry. Says Dovga, “In today’s world of globalization and extreme competition the business demands speed, cost effectiveness and agility – being able to quickly deliver applications to support new technologies, run complex functionality, bring new products to market and take advantage of new opportunities in order to stay competitive.”
Many CIOs are beginning to explore innovative ways to get more and better testing, at less cost and risk. A TCoE is a compelling solution that is finding its way at the top of many lists.
TCoE: enterprise – wide value
The ROI for a properly designed and implemented TCoE is impressive. Significant improvements in application and product quality are just the beginning. We have seen many companies reduce total testing costs (people, tools, overhead) by an average of 42%, improve tester retention by 70% and accelerate time to value (product launch or application deployment) by an average of 30%. Our clients have discovered other important organizational, brand and process benefits:
Enhance quality
- Design in higher quality by bringing testing considerations further upstream in the software development cycle
- Bring new disciplines, best practices and independence to testing practices
- Increase the awareness of good test requirements across the enterprise
Avoid risk
- Avoid incidence of public failures
- Move away from a slipshod, ‘hacker’ approach with customers serving as testers
Drive efficiencies
- Improve build quality so defects are ‘designed out’
- Ensure proper resourcing and time is available for testing
- Promote the ‘right’ use of automation
- Increase resource (people, tools, facilities) utilization
A unique delivery model
A TCoE is a next generation professional services group that aims to improve the delivery and efficiency of end-to-end software testing services. A formal, centralized structure within the IT organization, a TCoE consolidates all testers, tools and facilities within one operating unit with a codified set of processes, templates, practices, and metrics. The TCoE is responsible for all aspects of testing within a company, including:
- Strategy, design and planning
- Tool configuration and automation scripts
- Test preparation
- Execution, defect tracking and reporting
Ideal deployments
A TCoE is ideal for any medium to large size organization that views IT strategically and expends a significant amount of capital and effort on testing. TCoEs have been successfully deployed in many sectors including banking, insurance, IT and communications. These companies usually face considerable integration; security and application upgrade challenges and typically must grapple with heterogeneous enterprise and line of business infrastructures.
Key operational pieces
The secret sauce of TCoE is its ability to leverage knowledge, tools and talent in a symbiotic way. To do this, it combines three enablers that include management, technology, culture and process elements:
1. People & Habits
- Consolidate a team of expert, professional testers with industry experience
- Foster close partnerships with testing outsourcers and contractors
- Freely share knowledge and best practices
- Provide regular training and career planning
2. Tools & Infrastructure
- Standardize on the right testing tools
- Closely track tool utilization and purchases
- Centralize testing activities in one location that is accessible to business sponsors
3. Governance & Process
- Establish a common mission, plus uniform processes and practices
- Ability to accommodate multiple development methodologies (e.g. Agile, Waterfall)
- Provides suitable oversight, project management and transparency
- Align on reporting schemes and metrics
“All of these parts combine synergistically,” says Dovga “to establish testing standards, optimize application quality and performance, improve alignment between business units and IT, and increase QA efficiency. The end goal of a TCoE is to operate as a service for the business.”
Building your TCoE
Moving to a TCoE could be an IT game-changer for many organizations. However, designing and building the model will take time, resources and effort. To maximize internal buy-in and value, we recommend firms follow these organizational best practices:
- Craft a compelling business case that is aligned with strategic goals and financial metrics
- Secure senior management and cross-functional alignment (IT and business)
- Aim for an end-to-end quality assurance mandate, built back from end user & customer needs
- Establish a centralized but flexible pool of resources that is readily deployable
- Encourage teamwork, collaboration and knowledge sharing within the TCoE as well as with outside partners
- Foster a culture of continuous improvement around practices, processes and tools
- Regularly engage stakeholders to ensure you are meeting their needs
Importantly, shifting to a TCoE is not an ‘all or nothing’ decision. In our consulting work, we have helped firms begin the TCoE journey. We begin by first diagnosing their situation and needs. For example, what are the key IT, business and customer needs? What is the current state of practices, processes and tools? And finally, what are the gaps and barriers to attaining better performance. Following this, we help them conduct a thorough financial and strategic analysis to ensure there is strong ROI for the initiative. Based on strong internal needs and a solid financial business case, organizations will move to designing and piloting a TCoE with established standards, optimized resources & processes and shared best practices. Once value is demonstrated and the model is refined, CIOs would then scale the TCoE into a mature, shared service structure that supports all business units. Finally, management should pay close attention to change management, talent management and cultural considerations when implementing this new testing unit.