This post was written by Arthur Liberman from Performance Center R&D
One of the many new features in Performance Center 12.55 performance testing software is Automatic Trending.
Performance Center trend reports help paint a picture of an application’s performance consistency over multiple runs or how it behaves in different configurations. It’s an integral and crucial part in performance testing. It is also an important testing practice in the agile development process. Performance Trending allows users to compare multiple runs to a baseline, or a run to an earlier one. It is a perfect fit for nightly performance tests.
Automatic Trending makes the work process of performance engineers simpler and more efficient. It automates steps that engineers were required to perform manually and wait for each of these steps to finish before continuing with the next one. Automatic Trending saves valuable time and effort.
The simplest example for its use is running a performance test after hours (e.g. as a nightly process), and having the run analyzed, trended and ready to go over the data first thing in the morning.
Here are some guidelines for Automatic Trending:
- Automatic Trending can only be configured on a per-test basis, not per-run. Turning this feature on or off will only affect future runs. Runs which have already started or completed will retain the original configuration from when they were initiated.
- The run must be collated and analyzed in order to be added to a trend report. If the run is only collated, the system will automatically analyze it before adding to a trend report. As long as a run is not collated, it will not be automatically analyzed and trended.
Note: collating does not happen automatically. In case a run requires analysis, it will be analyzed first, and then it will be trended in the next cycle of the maintenance task.
- Runs that fail a step, such as collation or analysis will not be automatically trended, and will require going through the steps manually.
- Automatic Trending is a maintenance task and has a default interval of one hour.
Configuring Automatic Trending is easy:
Select or create and configure a new test as you normally would.
Click on the “Trending” ellipsis. You will now see the “Trend Settings” dialog.
- First we need to turn Automatic Trending on.
- Next we need to select the target report. If you want to create a new report, tick the “Create new report” check box, and the Report dropdown box will turn into a textbox where you can enter the name for the new report.
- Next we set the time range to trend. We can either let it trend the whole run or a certain part of it.
- The “Max runs in report” limits the number of runs that can exist in a given report, currently the maximum supported number of runs per report in automatic trending is 20.
- Next we can set the action automatic trending takes when the number of runs in the report reaches the set limit. The default setting will delete the oldest run and will insert a new run into the report. Rebasing occurs automatically. You can also opt for not adding any more runs to the report.
- Now we click OK, and then save the changes to the test.
That’s it! From now on, all runs of this test will be trended automatically.
API Support:
Automatic Trending can be configured using REST API.
It is a child element of the “Content” element in test creation XML, and this is the relevant section of XML code:
<Content>
...
<AutomaticTrending>
<ReportId>1</ReportId>
<MaxRunsInReport>19</MaxRunsInReport>
<TrendRangeType>CompleteRun</TrendRangeType>
<MaxRunsReachedOption>DoNotPublishAdditionalRuns</MaxRunsReachedOption>
</AutomaticTrending>
...
</Content>
It can be used as part of the usual REST API workflow, when you create or edit a test.
Automatic Trending can also be leveraged from the PC Jenkins plugin. It’s a great alternative to having to create your own Jenkins scripts that generate trend reports.
For further information about Performance Trending and Performance Center Jenkins plugin explore the following resources:
Support
Documentation
Learning Services
Partner Programs
