Managed Testing: the Quiet, Fast-Growing Teenager?
Where our last column looked at one of the newer kids on the BPO block (social media outsourcing), this article turns its attention from BPO to Applications Outsourcing, to look at an activity that has quietly been one of the fastest growing areas for several years now: managed services around software testing.
NelsonHall’s first research study on the software testing outsourcing market was published in Q1 2008; since then there have been significant developments in market demand and vendor capability.
We should start with a quick definition: software testing is the activity undertaken to identify and reduce defects in software during the software development life cycle (SDLC).
In the provision of “traditional” testing services, this activity is bundled in with software development, and performed by non-full-time testing professionals.
With “specialist testing services”, testing is done by a separate unit from the team that has developed the software, a unit that is typically comprised of dedicated testers. The focus is on standardising the way testing is delivered, with the application of best practices, and on achieving economies of scale by creating testing centres. Specialist testing services units use a systematic approach based on sourcing (away from onsite to testing factories), and on automation and tools usage (such as HP, IBM Rational, Compuware and open source). Specialist testing service providers have also introduced new service offerings such as performance testing and security testing.
Third-party independent testing has been a fast growing business for some years now. To give one example of a vendor success story, Tata Consultancy Services’ testing services business achieved revenues of ~$890m in its FY 2013. Just four years ago, the unit’s revenues were around $320m – and this growth has been organic.
Clients want to move away from using testing as a late-stage ‘go/no go’ decision support to services that help move apps into production faster.
Adoption of Multi-Year Testing Contracts
Clients want to both improve the way they conduct testing and also decrease the costs of functional testing.
In recent years there has been a significant increase in demand for specialist testing, with some organisations, usually large enterprises, awarding multi-year contracts for these services. This multi-year approach has several benefits for clients.
- It means that clients can negotiate better prices with their testing services provider while enabling vendors to invest in tools and automation.
- It allows the parties to move from a HR discussion, for example around CVs, towards more focused topics, for example the vendor’s capability to ramp up and down within certain timeframes, or the number of test cases designed, created, automated and executed. In some cases, test SLAs are based on measures of software quality, with reduction of defects founds over a number of test cycles
- It fits well with applications found in the BFSI and telecoms sectors, with regular (three to five) releases/enhancements of applications being deployed per year. Multi-year contracts also allow some level of investment by clients in their testing organisation, be it through an improvement of their SDLC processes, or through more traditional investment in building test scripts for automating testing and procuring testing tool licenses.
Of course, the multi-year approach adds complexity. For example, the creation of testing shared service centres requires major change, and it typically involves the relocation of work offshore.
SLAs also create their own challenges: a productivity SLA such as a number of test cases may tend to inflate the number created and executed. And testing automation can drive an increasing number of test scripts, which need to be maintained over time. In short, testing contracts tend to drive and grow a testing ecosystem, which was not part of the initial objective. Specialist testing services do allow clients to improve the way they conduct testing. However, buying consulting services or testing applications more comprehensively increases the cost of testing.
Specialist testing services are visible costs (e.g. direct invoices), whereas the costs of conducting traditional testing services (the cost of non-quality) are not visible: the cost of defects in an application causing a production outage are, for the most part, non-attributed to budgets. As a result, clients look simultaneously at ways of lowering their visible testing costs, with functional testing the initial focus.
To give a sense of the level of growth in specialist testing services in the last decade, the number of career testers around the world is now close to 230,000; around 2002 it was negligible. Each year the industry is adding around 20,000 to 30,000 new testers worldwide. In short, the testing services ecosystem is now more or less the size of SAP’s own services ecosystem. The difference is that SAP’s growth has been over several decades, since its foundation in 1972. As well as reflecting growth in market adoption of specialist testing services, by organisations, the increasing numbers of specialist testers also indicates the fact that testing remains a highly labour-intensive activity.
The approach to quality assurance focuses on testing best practices and methodologies, such as test management and strategies, risk-based strategies and requirement analysis. The intent is two-fold:
1. Approaching testing activities in a professional manner
2. Enhancing software development practices to create defect-free software applications from the start. It is a commonly expressed opinion that “doing things right” from the start is far less expensive than identifying defects and correcting them afterwards.
Software testing vendors need to demonstrate their capabilities against a number of criteria. Two of these are:
- The breadth and depth of the offshore delivery capabilities (e.g. for activities such as test cases design, test scripting and execution) to complement onshore services
- The extent of their usage of tools, accelerators (such as reuse from test case repositories), and IP.
Many vendors have proprietary tools or frameworks. While IP can bring significant savings, the topic also has constraints. Several of those constraints are common to buying COTS e.g. developing assets and artefacts on a third-party product. Other constraints are specific to proprietary IP: proprietary tools are difficult to assess, and do not bring systematically a roadmap. In addition, it is always unclear how long will testing service vendors maintain IPs like repositories of test case and test scripts.
An interesting feature of software testing is that the services offered are in regular reinvention. Managed testing services portfolios have expanded from functional manual and automated testing to a large portfolio of specialised offerings. These include offerings that one would expect to see e.g. non-functional testing (performance and security testing) as well as ERP testing or website testing.
But testing has its own oddities. Let’s take a few examples:
- Gaming (or betting site) testing is becoming more popular. Game testing increasingly works on non-traditional models (for business) such as crowd testing, where testing is pay-per-bug based and delivery non-controlled by the client.
- Mobile app testing. Apps store vendors are looking at the right business model to test apps available in their store at a decent fee, considering the low (or free) pricing of mobile apps. Alternatively, client firms need to handle the vast diversity of devices, operating systems and networks in the different countries. Most vendors have set up a mobile app testing capability, based on a pay-per-use model. Again the economics do not always work and testing can sometimes be an expensive but necessary evil.
- Meanwhile cloud computing made its way into testing some years ago, driving a number of testing service activities, with different level of success. Testing applications hosted on cloud computing, whether public or private is not entirely different from testing an application ported from UNIX to Windows or LINUX.
Take up of other offerings such as bundled service and SaaS tool offerings has been disappointing: possibly because the pricing structure is too complex. Also, a number of portals were created by several IT services vendors for clients to order testing services, with the goal of limited human interaction and limiting client acquisition. Adoption of such portals has remained niche.
Offshore Scale an Important Factor
The industry has been changing. Scale is becoming more important, certainly for vendors to reassure clients of their financial strength and ability to accommodate personnel transfer.
Scale is also important in other areas, including HR tools and breadth of service offerings.
Demand for testing is growing fast and vendors are hiring testers literally by the hundreds. Because experienced testers are uncommon and therefore expensive in the market, most vendors focus on hiring new graduates and train them on their own methodologies and best practices. But that is not enough: testers also need to understand several business processes of their client industry and also applications functionality used in that industry. Training means a financial effort for vendors.
Scale is also an important element in IP and proprietary tools. Software testing remains highly labour-intensive. Efforts by vendors to further automate test execution, capture application knowledge from the brain of their personnel into the form of a tool, build repositories of test cases, or develop point solutions are a priority. And financing these tools is an investment.
We have just looked at a few examples of how testing service portfolios are expanding. It takes time and money to develop a new IT infrastructure testing methodology and execution service, for instance.
What Does The Vendor Landscape Look Like Today?
Given the importance of offshore scale, overwhelmingly in India, it is no surprise that NelsonHall counts six Indian-oriented service providers among the top ten testing services providers globally. Some of these have added local presence through relatively small acquisitions: in 2010, Cognizant acquired Galileo Performance, a small testing consultancy operating in France. Cognizant has also expanded its SAP testing capabilities with the acquisition of C1 in Germany. Similarly, TCS is gaining testing capabilities in the French banking sector with Alti.
Larger IT services vendors such as Accenture and IBM with large employee populations in India are growing fast enough to maintain their scale advantage.
Smaller testing specialists are mixing organic growth with M&A. One success story is Cigniti Technologies, which has grown from the tiny Chakkilam Infotech acquiring two US-centric players, Cigniti Inc. in Q3 2011, and Gallop Solutions in June 2013, the latter bringing in HP testing tools-related services.
Those IT services vendors that have not achieved scale in India are losing market share – unless they acquire. Two years ago, CSC acquired Indian software testing pure-play AppLabs. AppLabs, headquartered in Hyderabad, brought in 2,500 testers. CSC paid more than twice its annual revenue for AppLabs – and it was not the only suitor.
Meanwhile SQS has not made any acquisition for three years, during which time it focussed on building its managed services offering; it is now ready to make moves in the US or India.
Who is Driving Demand, and Why?
In his latest market analysis on software testing services, published in June 2013, NelsonHall’s Dominique Raviart identifies four distinct buyer segments with differing goals but all sharing a common need for lowering overall costs.
Details of each of these segments with their size and growth, plus our views of current and forecast take up in different verticals and geographies, are available to report buyers. It is no surprise that the adoption of testing has been strong in sectors that have large application estates (e.g. retail banking), or that are investing in new applications (e.g. telecoms service providers).
The NelsonHall report considers a number of likely developments in the software testing market per the next few years. The big challenge of the industry is to move to a non-linear model. A number of vendors now offer mobile testing as-a-service. SQS is expanding to offer SAP testing as-a-service and PLM as-a-service, a specialized automated service.
Overall, the opportunities for testing vendors continue to expand, and client demand for specialised offerings remains strong at this point. But in five years’ time, this will certainly be different, as clients re-tender contracts and move to second-generation outsourcing.
So, where social media outsourcing (see last column) can be seen as the brash new kid on the BPO block whose noise is far bigger than his size, managed testing services can be seen the quiet-spoken teenager that has been going through a growth spurt and is now commanding attention. But there is still more transformation to come before it is fully mature….
This article originally appeared in Outsource magazine Issue #33 Autumn 2013.