Measuring the user experience of AJAX websites with the timings the browser provides can be difficult. Often the perceived loading time of a web page differs considerably from the measured loading time. Therefore, Silk Performer introduced the time to interact. In this blog post, we will explain what exactly the time to interact is, why and when it is useful, and how to actually use the time to interact in Silk Performer.
When load testing modern websites, often the perceived loading time differs considerably from the measured loading time. A user can consider a web page as ready, although the processing in the background is not yet completed. This is illustrated in case A in the graphic below. Or the browser might have completed downloading resources, but the web page is not yet ready for the user at that point in time; essential page elements might be loaded asynchronously. This is shown in case B below. To get an indication of when a web page is ready for the user interaction, Silk Performer introduced the so-called time to interact, or TTI for short.
In Silk Performer terminology, the time to interact is defined as the time from a user interaction (such as navigating to a URL or a click on a link) until all relevant elements a user requires to interact with the page are ready; even if the page has not yet completely loaded.
Identifying the relevant elements of a page can not simply be automated, as it heavily depends on the use case and on the perspective which elements to consider relevant. For example: A web shop company can test their website from their own perspective and from the perspective of their customers. From the company-perspective, the elements that contain the special offers might be considered relevant. But from the customer-perspective, just the search field might be considered relevant.
Therefore, the performance engineer has to tag all TTI-relevant elements during recording. This is step one from the graphic below. The Recorder then generates a BrowserTtiIncludeElement() function for each of these tagged elements. During replay, Silk Performer measures how long it takes to load each TTI-relevant element and reports the maximum as the time to interact. This is step two from the graphic.
We will step through an example to show you how it is actually done. In Silk Performer, we create a new web browser-driven project and enter the URL of the application we want to test. For this example, we'll be recording our Silk Central dashboard. The Browser Application starts and we log in to Silk Central.
We want to find out how long it takes until the dashboard is ready for us to use. To be able to test this, we have to select some specific elements on the page that we want to measure. We go for one of the dashboard charts (1) and two metric elements (2). We hover the mouse over one of the elements (3), press Pause/Break on the keyboard to get the locator of the element (4), and then we click Include in TTI (5) on the bottom right. This way, we have tagged our first element as TTI-relevant. We perform the same steps for the other two elements and then we stop the recording.
Now we take a look at the script Silk Performer has created. We can find three BrowserTtiIncludeElement() functions, one function for each element that we've tagged. If you want to include additional elements, after you have recorded your script, you can easily do so during a trial run. Click Try Script on the Workflow bar, enable Visible client and Step by step execution and click Run. Step through the script up to the point where you want to include your additional elements. Then get the locator and click Include in TTI.
Complete the run and execute another trial run, this time without the visible client. If you do so, you will get a TrueLog that includes the newly tagged elements. When the trial run is completed, the Try Script Summary page displays.
Click Explore TrueLog file to open TrueLog Explorer. When you click the respective node, you can see the TTI measures in the Info tab (1). The tab includes the locator of each tagged element, the measured time values, and some colored bars. Green and yellow color tones refer to rather small TTI values, red color tones refer to big TTI values. When you click Toggle Heatmap (2), you can see the actual elements on the page, also with the color codings (3). The highest of your measured time values is the TTI (4).
On the Statistics tab (1), you can also find the TTI (2), as well as all the other measures Silk Performer collects for a browser-driven test (3). Also take a look at the diagram for the measures. In our example, this diagram shows that all of the conventional browser measures (4) are considerably lower than the TTI (5). So these measures can be absolutely misleading as indicators for user experience.
Once you understand what the time to interact is and how to set it up, it can be tremendously useful in collecting more meaningful metrics when load testing modern websites.
In the following video, you can watch how we performed the above described steps:
To learn more about all new features and enhancements Silk Performer 18.0 provides, take a look at the following blog post:
The Silk Performer Help is another comprehensive source of information.
If this article was useful for you, leave us a comment or like it. We appreciate your feedback.