The 2021-22 World Quality Report (WQR) is a collaboration between Micro Focus, Capgemini and Sogeti, and is the only report to analyze software quality and testing trends on a global scale. In all, 1750 executives and professionals were interviewed for this report. From the C-suite to QA testing managers and quality engineers, this report breaks out ten industries and from 32 countries around the world.
“The World Quality Report is a one-of-a-kind global study, and this year’s survey highlights the effect of evolving pandemic-impacted requirements for applications across new deployment methods, as well as QA’s adoption of Agile and DevOps practices, and the continued growth of AI.” Said Rohit de Souza, GM of ADM, Micro Focus. “As our own customers look to improve their software quality, velocity, productivity, security, and their overall customer experience across a range of devices and environments, these findings show the need for organizations to advance their modernization initiatives through AI-powered continuous quality and test automation tools.”
A key message from the WQR is that during the COVID-19 pandemic, we saw the convergence of digital transformation and the adoption of Agile and DevOps practices in real-time. Also, QA is becoming a leader in adopting agile and DevOps practices, providing teams with the tools and processes to facilitate quality across the SDLC. In addition, the WQR highlights five specific themes around key findings and trends:
- COVID-19’s impact on QA organizations and software testing.
- Digital transformation’s real-time convergence with—and QA’s growing role in—DevOps and Agile adoption.
- How geographically distributed teams are adopting a business-oriented focus while deploying applications across environments.
- Ways in which artificial intelligence (AI) innovations augment Agile and DevOps and foster a growing culture of responsibility for quality across all teams.
- The use of AI-powered continuous testing and quality management tools to address customer experience priorities and rapidly changing pandemic-impacted requirements.
Key Findings & Trends
1. COVID-19’s impact on QA organizations and software testing
The COVID-19 pandemic had an immediate and real impact on almost every aspect of business, including QA operations. However, many QA organizations were able to adapt to the realities of a new hybrid work environment and transition to the new reality of working in distributed teams. This may have happened as there was already a growing trend to hybrid, distributed teams, which COVID simply accelerated. Also, rather than dedicating a specific chapter to COVID in this year’s WQR like the previous World Quality Report, 2020-21, we see it as having a varying degree of impact across most, some might say all, the QA key trends and findings.
The Customer Experience is King
COVID-19 has rightly placed the focus back on the customer and their experience. When asked to list the top two aspects of their IT strategy, among the top four responses, there was only a two-point spread. Also, replacing security in the top spot last year is this year’s top-rated aspect: enhancing the customer experience, chosen by 63% of respondents. Next is enhancing security (62%); followed by a tie between higher responsiveness to business demands (61%); and a higher quality of software solutions (61%).
From Custodian to Champion of Quality
This past year has also seen a reordering of testing and QA objectives. Whereas business outcomes and the custodians of quality were the clear leaders last year, this year there was a narrowing point spread among the leaders, with six of the seven options all falling within a two-point spread. There also was a three-way tie for first, with custodians of quality, quality at speed, and quality enablement across the team leading at 62%. There was another three-way tie for 2nd place, with business assurance, digital happiness, and automate at 61%. Based on these responses, it’s clear that QA teams are evolving from custodians of quality to champions of quality. QA teams are becoming dynamic leaders in organizations’ quality initiatives, supporting everyone on the team to achieve quality while contributing to business outcomes and growth.
2. Digital transformation’s real-time convergence with—and QA’s growing role in—DevOps and Agile adoption
The Push for Digital Transformation
This year saw digital transformation initiatives aligned with meeting pandemic requirements. For example, the digital transformation initiative of environments shifting to the cloud points in the same direction as the new pandemic requirement of facilitating hybrid and remote workforces. Prior to the onset of the pandemic, agile and DevOps were a growing trend. During the pandemic, we started to see that QA is now playing a key role in an organization's adoption of agile and DevOps, blurring the lines between development and test while creating a hybrid approach to quality.
Another interesting note is the drivers of digital transformation. At 47%, improved productivity and efficiency was the leader, followed by increased service/product quality at 46%. In third were speed, better agility, and flexibility, which makes sense. Customer experience was directly behind speed, following closely with cost reduction and the creation of innovation opportunities, with competitive differentiation being last. This makes sense, as competitive differentiation seems to be more of a side benefit of digital transformation, while digital transformation itself lends itself to efficiency, quality, speed, and an overall better customer experience.
QA’s Growing Role in DevOps and Agile Adoption—Guided by Business Priorities
This year we’re witnessing a major realignment of the needs of the business becoming more important than the needs of the technology stack. Compared to last year, the number of participants who gave weight to the technology stack dropped 16 points, which was replaced by the biggest riser: business priorities, which is now ranked first with an 11 point increase from last year. Also, significantly increasing from last year was culture/agility, gaining 21 points.
3. Geographically distributed teams are focused on business outcomes as they deploy applications across environments
Last year’s survey took place at the beginning of the global pandemic, which showed signs of the change required to meet business objectives with the new requirements of remote work and digital transformation. With this year’s survey, we saw that digital transformation continued, even in lockstep with the new pandemic-influenced work requirements. This accelerated companies’ plans to migrate workloads to the cloud, due in part to planned digital transformation initiatives, as well as the rapid shift to remote work, spurring the need for increased security.
As a result of the pandemic-influenced workplace, the highest-rated focus was remote access to test systems and test environments (using SaaS and Cloud). Supporting this remote access are secondary remote-based factors, including better collaboration tools for teams.
In order to support quality for modern applications, test environments themselves must also be modernized. This year we saw a growing satisfaction of organizations with their efforts to modernize their test environments with cloud and containers (top satisfaction rating), as well as improving booking and managing test environments (+16 points), providing visibility (+22 points), cost efficiency (+18 points).
4. Artificial intelligence (AI) augments Agile and DevOps, fostering a growing culture of responsibility for quality across all teams
AI continues to change how test automation is being built, and how testing, in general, is being executed. We saw a growing level of confidence in the level of AI-based testing within organizations, with almost half stating they already had the established repository of test execution data that AI and ML required, and almost half again stated they were willing to act on the intelligence that their AI and ML platforms would provide.
This year, we also asked participants to predict how likely they would be to leverage a range of approaches to speed up and optimize testing in agile and DevOps environments. Year over year, the greatest increases occurred in integrating test as automatic quality gates in the CI/CD pipeline (+5 points), with the implementation of smart and automated dashboards to enable continuous quality monitoring gaining the most (+9 points). What’s new this year was the addition of using AI over past cycles to optimize test cases, which tied for 2nd overall, falling just behind shift-left testing.
5. The use of AI-powered continuous testing and quality management tools to address customer experience priorities and rapidly changing pandemic-impacted requirements
This year we asked respondents about the perceived benefits of test automation. AI/ML was the fourth-highest perceived benefit—proving its potential, and real value. All of the year over year trends for the other benefits were trended down, as compared to last year. This shows the reality of the challenges of working in hybrid and distributed teams.
The top benefits were better control and transparency of activities, reduction of test cycle time, reduction of overall security risk, and better detection of defects, with better reduction of test costs and better test coverage rounding out the bottom. Although the perceived benefit of Security decreased year over year, the benefit value actually increased the most year over year.
Key Recommendations
QA Orchestration in Agile and DevOps
Focus on what matters most: the customer experience and business objectives. The goal is to meet the needs these two present with both efficiency and speed. Also, adopt an engineering mindset across your teams, and embrace multi-skilling and upskilling. A new trend that is quickly becoming the new normal is the SDET (software development engineer in test). Invest in insights, especially real-time insights that span the entire QA and test function. From short-term tactical initiatives to long-term planning and strategic direction, focus on developing smart dashboards with real-time KPIs.
Intelligent Test Automation
Standardize the use of test automation in QA by adopting an automation-first approach to software quality delivery. Expand automation across the end-to-end lifecycle, incorporating automation into all QA activities.
Artificial Intelligence and Machine Learning
Drive the use of AI—don’t be driven by it. AI and ML have the promise to deliver improvements on an exponential level, but use AI as a tool, not a replacement for the business decisions that you’re making. For example, use AI to shine a light on what to do and when to do it. It can also be helpful in not only identifying failures but why these failures are occurring. Also, focus AI on what matters most, pinpointing the most challenging quality areas of software delivery. Most importantly, if you haven’t incorporated AI into quality, now’s the best time to start.
Test Environment Management (TEM) and Test Data Management (TDM)
Cloud adoption is continuing is sure and steady growth but be careful to ensure that the future doesn’t obscure the needs of the present. The key to successful cloud adoption is to ensure integrity with legacy applications as well. Also, data analytics is now a key aspect of the test data management framework.
Security and the Intelligent Industry
Remote connectivity mandates that security and resilience are properly accounted for testing and the QA organization. Invest in innovation, in both your labs and your teams. Whether you begin with a POC to demonstrate feasibility, securing management’s support is crucial in implementing change.
For the complete listing of the trends, findings, and recommendations from this year’s survey, please download the full World Quality report at 2021-22 World Quality Report (WQR)
Key Links:
Support
Documentation
Learning Services
Partner Programs
