Defect retention and reduction
Defect retention and reduction of 80% to Production is a wonderful thing. Is there any sense or metrics on defect found in non-prod environments through the software stack? What's the breakdown in defects found from development, test and stage, for example? Has there been much of a shift in defects found earlier in the development lifecycle as well?
We conducted a recent survey that explored how organizations track the cost of defects, the breakdown of the cost of a defect, and whether Agile/DevOps approaches help to reduce the cost of a defect when compared to Waterfall software development.
The results of the study backed up two observations: defects (regardless of type) incur a greater cost the closer they are found to production, and the activities involved in a defect’s lifecycle become more costly as the defect moves from detection, through fixing, and out into production. While the advantages of Agile/DevOps is widely understood, we found that the nature and distribution of defects follows a similar pattern, regardless of the methodology. However, we noticed that there is a slight reduction in the abundance of high-impact defects in Agile/DevOps.
Since we focused mostly on cost impact, we didn’t include specifics on the number of defects into the study. The chart below illustrates effect of cost by development stage for an Agile/DevOps approach.
You can learn more about the results by downloading the study here.