Agile is a way to deliver software for many teams irrespective of company size or location around the globe. While many teams have mastered agile practises on individual team level, scaling the effort across the entire organization is a different story, regardless if it was born agile or is on a transformation path from the traditional waterfall method.
I recently took Implementing Scale Agile Framework (SAFe) class, as I wanted to understand better the Agile transformation journey that so many organizations are embarking on. The class was packed with folks from all sorts of backgrounds and industries- Dev Managers, PMOs, Business Executives, Release Engineers working at Oil, Finance, Hardware, Biotech and more- a shining example that the digital transformation is impacting everyone and every organization across the board. The class discussion were vibrant, with everyone sharing their organization’s challenges and bottlenecks. Without a surprise, testing is one of the biggest hurdles to overcome, and the usual hot topics kept on coming up: keeping up with the dev pace, predominantly manual testing, how much testing is enough to be “done,” and so on. So while teams have embraced the cultural change, harnessed DevOps practices, or launched Agile Release Trains, the reality is that the velocity of their releases is mainly dictated by their testing capabilities. Teams know that quality is everyone’s responsibilities, yet not all testing gets done, automation is not streamlined, and even though Scrum teams are self-organized and self-sufficient, not everyone can do all testing.
So, whether you are scaling Agile or simply trying to perfect your releases, let's have a quick look at best testing practices, who does what, and how to make sure quality software is indeed delivered. Preferably, continuously!
What to test and when to test?
The unit, load and vulnerability tests should be done continuous. Those are essential, though definitely not sufficient, quality controls and there should be no reason why they shouldn’t be mastered, as there are plenty of tools and frameworks to leverage (Hello, Selenium and JMeter).
Before continuing, a side note on load test as it’s often confused with performance tests: Load testing is a subset of performance testing and its goal is to determine the response time of a system given heavy demand. Performance testing is done to validate resource usage, scalability, reliability, and stability of a system for a given user load. It is a subset of performance engineering, which is focused on addressing performance issues in the design and architecture of a software product. So while one test will give you quick and easy guarantee that your soon to be hottest software on the market can withstand the crowds, the other one will guarantee that users can enjoy your product, while your system will hold and your servers won’t crash. Load test can be done quickly, hence you should do it along with the unit test.
The integration and the system tests can easily be done nightly as they tend to take a while, and though many teams execute them manually, there are ways to automated them as well. Yes, it will take time to automate even a handful of key scenarios. And yes, it will take even longer before you see the ROI, but there is no way to scale unless you automate as much as possible. So make friend with API testing tools and Scripting tools (TruClient and VUGen anyone?) to generate scripts and start automating.
It’s best to run the regression and performance tests weekly--they take a while to prepare and doing them weekly on several completed builds is ideal. Please don’t even think of skipping the full and thorough performance tests, because the performance of your software has the most immediate impact on business productivity and user experience. Lastly, run your compliance and security tests bi-monthly, because you know--there are regulations and after all, there is a dark, dark internet out there!
Is it possible to do all this testing in the typical short sprint cycles? Yes, if you double down on automation, and most importantly--if you put all those tests as part of your definition of “Done.” Then and only then all stakeholder will be aligned on what it means to be done with a story, a sprint, or a release.
Who does what?
In a typical Scrum teams, each member should be able to do everything that is required to release great software. So, theoretically anyone on the team can handle all the unit, vulnerability, integration system, performance, etc. testing.
Is it possible...probably! But I’d argue that if there is one person who can do it all, it can only be a woman. So to all those teams that have found your "Wonder Woman"--congratulations! ;-)
For all the rest, the reality most likely looks like this: The Developer will be the doing the unit test, the Quality Assurance team will be doing the regulatory, integration, system and regression testing, and specialist will be require for the load, performance and security tests. It will be only fair to say though that recently more and more developers have started to do the basic load test.
What do you need to validate quality across the lifecycle?
Tools. Many of them. Open source is fantastic and it will take you a long way. Is it enough? Not really. Root cause analysis and correlation, simulating services that are not there yet, spinning up an environment, executing several tests in parallel, or testing across endless operating systems and devices with a click of button is impossible with Open Source. Also, certain application - SAP, Oracle, Citrix, you know- the ones that are the backbone of your organization, cannot be tested with open source. This is where vendors come in. As you scale up and launch Solution or Portfolio Agile Release Trains, mixing open source and proprietary tool becomes inevitable. So choose your tools well and build your toolchain thoroughly--this is what will power your trains!
This post is brought to you by Micro Focus
Micro Focus has a rich portfolio of solutions that spans the whole development lifecycle. Micro Focus’s tools offer rich integrations with many Open Source and 3rd Party tools, so teams can leverage them to streamline and accelerate the DevOps process as much as possible. In the testing arena alone, the choices are vast and can help advance the testing capabilities of many team members. From environment provisioning, virtualizing services, factoring in network conditions, and spinning millions of load generators with several clicks, testing truly can be done by just about anyone, so Wonder Woman can be free to go save the world. To learn more about testing solutions or any other tool in the portfolio, click here. While browsing, feel free to take advantage of the free trials offered across the board!
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.