The main message from the 2019-20 World Quality Report (WQR) is that quality, and the processes that go into it, underpin the key business drivers of every major enterprise. Last year, the primary objective of QA and testing was to ensure end-user satisfaction. This year, that’s still considered a significant objective, but the top objective is contribution to business growth and positive business outcomes.
The WQR is a collaboration between Capgemini, Sogeti, and Micro Focus, and is the only global report analyzing software quality and testing trends. 1725 CIOs and other senior tech professionals from ten industries and 32 countries were interviewed for this year’s report.
Figure 1 Executive management objectives with QA and testing on a scale of 1-7
Many of the trends identified last year are still apparent, with some new ones joining the list.
The Agile and DevOps landscape is complex, with many moving parts. Team members must master an unprecedented range of skills, whether in software engineering, languages, test automation, performance engineering, security, artificial intelligence, data science, environment provisioning, and more. It’s becoming harder to define what ‘testing skills’ are, with such a broad range of abilities across IT disciplines in general demanded from team members as well.
Security continues to be a significant challenge for organizations, especially around ensuring compliance with security policies. So much so, in fact, that this year’s WQR includes a chapter dedicated to security and risk compliance. The chapter notes that more than half of respondents (53%) have found that as a result of automation, they have seen a reduction in their overall security risk. However, since many test processes have manual steps, this can hamper automation, especially for security testing.
Budget and costs
The proportion of budget for QA and testing is continuing the decline discussed in last year’s WQR. But given that the top objective of QA and testing is to contribute to business growth, that’s somewhat puzzling. There are a number of possible reasons for this. It could be the result of higher investments in previous years paying off, or that the tools and techniques adopted over the last few years are more cost-effective. But at the same time, it’s increasingly more difficult to track testing budgets in Agile and DevOps organizations.
A major challenge for Agile organizations is how to comprehensively test business processes end-to-end. One reason for this is that individual teams are focused on testing their own user stories, leaving a gap in testing bigger stories that cut across different teams and applications. To take control of end-to-end testing, organizations should connect their testing ecosystem together, and employ intelligent analytics to detect issues and adapt to change.
QA: An integral part of the picture
Agile and DevOps methodologies emphasize the importance of infusing quality assurance and testing throughout the entire process, and making it the responsibility of the entire team. The WQR observes that this is happening in practice, with QA being integrated into the overall development process.
The report lists the key findings that emerged from the research.
QA is enabling business growth and improving outcomes
End-user satisfaction was last year’s key QA objective. This year, the main motivation of QA is to contribute to business growth and outcomes. But at the same time, the WQR notes that QA budget continues to decline, and that could hamper business growth. That’s not necessarily the case though, because the report says that investments made in QA a few years back are now paying off, and the new tools and techniques that have been introduced in recent years are more cost-effective. The investments they’ve made in automation are also having a noticeable and positive influence on business outcomes. That said, testing budgets are becoming increasingly difficult to track in Agile and DevOps organizations, as the entire team is responsible for QA and testing, with QA no longer budgeted separately.
Automation rates are rising and delivering benefits
Organizations are continuing to grow automation, and are seeing the value. They are achieving better control of their test activities, more visibility, increased reuse of test assets, and improving their ability to detect, report, and resolve defects. But there are still challenges, the primary one being that applications change too frequently for automation to keep up.
AI-based testing, test data management on the rise
Artificial intelligence can make testing smarter, and teams are recognizing its potential. Organizations are introducing AI into their tools and processes, but there does seem to have been a decline since last year. That can be explained by the immaturity of AI solutions, together with a better understanding of the capabilities of AI and what it can offer. AI adoption is expected to increase, and it will be interesting to see how the trend continues next year.
Another challenge for testing, and especially test automation, is the difficulty of provisioning consistently reliable and accurate test environments. Organizations should look into test environment management and test data management to improve their automation rates. As with AI adoption, this is expected to increase moving forward
Security is now a key objective in QA and testing
This year’s WQR includes security in a separate section, indicating its importance in QA and testing. Organizations are more aware of security testing during development, and are raising awareness of security across the team and performing security testing earlier. However, a lot of that testing is still done manually, largely due to steps that can’t be automated. But more than half of those who have automated security testing report that it has led to better security and less risk.
The WQR offers more than just an overview of the state of the QA and testing art today. It also provides actionable recommendations for organizations looking to increase their Agile and DevOps adoption.
Fig 2. Challenges encountered in developing applications
Build a smart and connected testing ecosystem with intelligent analytics
To improve end-to-end testing of business processes, and to help align business goals with quality, organizations should connect their testing ecosystem together, with intelligent analytics to detect issues and adapt to change.
Expand Artificial Intelligence (AI) related skillsets within the team
Teams will need broader skills across the board, including AI. To take advantage of the benefits AI can bring, they will need new roles such as AI QA strategists, data scientists, and AI test experts.
Raise awareness and visibility of test environments
Lack of test environment and test data management continues to hinder automation efforts. Teams should increase their focus on this area to get more of the benefits that automation promises.
Adopt a center of excellence approach to test data management
Organizations should consider forming a center of excellence specifically for data, to help them pull in and maintain real-time test data from production systems.
Re-imagine test automation as a platform
Organizations should connect their diverse ecosystem as a platform, sharing tools and functions working intelligently from end-to-end, and aligned to business goals.
Raise your game on security
The WQR reported that teams who have increased security testing automation and introduced it early in development are seeing benefits in better security and reduced risk.
The WQR’s most important observation is that Agile and DevOps adoption is being hampered by lack of alignment between the goals of the business and execution. For organizations to improve, they must address the skills and security challenges that are standing in the way.
Download the report today to understand how your QA strategies compare to the report, and look at how you can adopt the key recommendations in your organization.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.