(This post was written by Tsafrir Lantner from the LeanFT R&D Team)

When you are automating applications, there are several circumstances where verifying or manipulating application images is required. The same goes for recognizing image text, and text in other object types. For instance:

• You want to verify that your company’s logo or another image appears in a certain (or all of the) application page(s).
• Your application includes images which are generated dynamically based on the usage flow and inputs. You will want to verify that the generated images are as expected.
• You need to fetch the text which appears on a certain application image (or another control type) and verify its value. Because you cannot always fetch this text by querying a standard test object property for that control, you will need to use OCR capabilities to fetch the text.
• You want to automate clicking on a certain control but it is either impossible to identify this control in the standard way or, for technical reasons, you cannot click it using the test object’s API. To work around this, you think of looking for a location in which the control’s text appears, and then sending a “low-level” click operation request to be performed on that location.

Achieving all of the above with Micro Focus LeanFT continuous testing is easy, and is the subject of this blog post.

While these capabilities are demonstrated here on a web application and using the LeanFT’s .Net SDK, they are available for any application technology supported by LeanFT and also with the LeanFT’s Java SDK.

You can try LeanFT for yourself here.

Image verifications in LeanFT

LeanFT’s SDK provides utilities for verifying the existence of images, and for comparing expected and actual images.

Verifying the existence of images

The following LeanFT SDK’s method is provided for each test object and can be used to verify that an image exists in the visual scope of that test object:

The method expects to get an argument which represents the image to find, and an optional argument for image similarity in percentages. If the image is found, it returns the location (coordinates) of the image; if not, it returns a “null” value.

Let’s look at an example:

Suppose we want to verify that the logo exists in the web site: 

This can be achieved using the code below:

Note that “browser” is an object that was initialized earlier in the script and represents the browser we are working with.

Verifying a match between images

In addition to “VerifyImageExists”, LeanFT test objects’ API also includes image comparison methods. For example:

The usage of these methods is quite self-explanatory. The “maskArea” argument allows us to define areas in the compared image that we want to exclude from the comparison, or that we want to use as the relevant area against which to perform the comparison.

For instance, verifying the match of a test object’s visual with an expected image can be done as follows:

Tip 1: You can easily generate a test object image to compare the test object image against by spying this test object with the Object Identification Center and clicking the “Capture object image” button in the tool:

LeanFT captures an image of the test object and opens a dialog box that enables you to save the image to a file (to be referenced later on from your script).

Tip 2: LeanFT provides the image existence and comparison utilities as global utilities too (not in context of the test object API). For more details, see the “ImageUtils” class in the LeanFT .Net SDK reference.

Image Comparison Results in the Report

The image comparison results are automatically included in the LeanFT report. For instance, in the snapshot below you can see the result of a “VerifyImageMatch” call. The comparison in this case failed, and is reflected in the status of the step in the report. In addition, when selecting the step, the expected image, the actual test object image, and differences between the two, are presented:

LeanFT’s text recognition utilities

In addition to the image verifications and utilities, LeanFT provides text recognition utilities that enable getting text of test objects, as well as the location of the text in the visual scope of test objects. These capabilities use OCR mechanisms.

Each test object in the LeanFT SDK has the following methods to support this:

The first method returns an array containing rectangles of all of the locations where the text provided in the argument was found.

The second method returns the text which was recognized by the OCR mechanism, in the visual scope of the test object.

Let’s look at an example:

Suppose your application is a Chrome extension. When the page loads in your Chrome browser, you see Chrome’s developer mode extension warning dialog, with a “cancel” button. This dialog is using an internal Chrome UI mechaism, so LeanFT’s web controls might have difficulty finding this button. You can use LeanFT’s text recognition capabilities to identify the button and click on it:

Our test code will look as follows:

Tip1: To increase the accuracy of the text recognition methods, it is best to provide the smallest possible rectangle area.

Tip 2: Similar to the image exists and match utilities (described above), the text recognition utilities are also provided both from the test object API and as a global utility, using the ImageUtils class.

You can read more about LeanFT image and text utilities, in the LeanFT Help Center

About LeanFT

Lean Functional Testing (LeanFT), is a functional test automation solution, built specifically for continuous testing and continuous integration.

LeanFT enables advanced automation engineers and dev-testers in agile teams to leverage standard development IDEs for test automation. It integrates with Visual Studio and Eclipse, and enables creating automated functional tests using modern programming languages (C# and Java respectively).

In addition to its .NET and Java SDKs, LeanFT provides plugins for these IDEs, including powerful tools like the Object Identification Center and Application Models.

You can find more information about LeanFT in the LeanFT Help Center.

Thanks to Tsafrir Lantner from the LeanFT R&D Team for providing this article!

Try Lean Functional Testing for yourself!

Feel free to leave a comment in the box below.