Supporting HTML5 Canvas in UFT One

by in Application Delivery Management

Written by Chu Jun Dong (Aaron) from the UFT One R&D team.

This article provides an overview of how Micro Focus UFT One can support HTML5 Canvas elements.

HTML5 Canvas in Automated Testing

Canvas is an HTML5 element used to draw graphics on a web page via JavaScript. It can be used to draw shapes, images, and texts. On the web page, canvas is merely an area composed of some pixels without exposing any methods and properties that can be accessed by UFT One and used in an automated test script. Thus, it is challenging to use conventional ways to implement the automation.

For details of HTML5 Canvas elements, refer to w3schools online document.

Supporting HTML5 Canvas in UFT One

In UFT One, we can use two ways to identify objects in a Canvas, image recognition and OCR. In these ways, we can recognize objects inside the canvas, and extract their properties.

These two ways can be used to test different types of Canvas objects. Based on the two ways UFT One supports, we can think of Canvas objects as two types:

  1. Image-related
  2. Text-related

Image-related objects

For image related Canvas objects, like lines, circles, linear gradients, circular gradients, and images, we can use Insight. Insight mode is a way of identifying the object using image processing. With quick boundary detection algorithms (DFT & SVM), we can automatically identify the object that users want to interact with. And with context storage, which saves the surrounding area of the target object, UFT One can let the user further improve the result of object detection.

Text-related objects

For text objects within the Canvas, we can use OCR. UFT One supports multiple OCR engines for text recognition: ABBYY for text recognition on general images, and Tesseract for images that need customized processing.

On a side note, there is a type of Canvas object which is both image related and text related, which is stroke text. For this type of object, we can use both Insight Mode and OCR.

Working with HTML5 Canvas in UFT One

The following outlines how to add an Insight object, with additional information about recording and checkpoint options.

Add an Insight object

UFT One lets you add an Insight Test Object by saving a snapshot of the object, which can be used to locate and retrieve the properties of an object via image recognition.

Insight provides two ways to select an object on a web page: Manual and Automatic. In Manual mode, you can specify the target image using a draggable selector. In Automatic mode, the target image is selected intelligently by an image recognition algorithm.

  1. Open UFT One and create a new test. Click Resources on the tool bar on the top.


  1. Click Object Repository.


  1. Click Add Insight Object to Local in the object repository toolbar.


  1. In the popup window, choose either Automatic or Manual mode.


  1. Once the object is stored in the object repository, you can generate scripts that perform operations on the object or query its properties. For example, there is getVisibleText method to extract the words within the image.

Insight Recording

You can record and run dragging and dropping of draggable content. This is recorded as Drag and Drop operations.

During a recording session, change to Insight Recording mode using the drop-down in the Recording toolbar.



UFT One has various checkpoints to check the properties of objects in the Canvas. While recording, you can add Bitmap and Text Area checkpoints by clicking Add Checkpoint or Output Value in the Recording toolbar.


Bitmap checkpoint

A Bitmap checkpoint can help verify the image content pixel by pixel. Click in the list and select the area to check when running the step.


By default, the image recognition algorithm selects an area to check. You can further adjust the area by dragging the rectangle selector.


Text Area checkpoint

A Text Area checkpoint can check text within the canvas.


Real-world Example

Here is a chart of sales figures drawn on canvas and the objective is to identify and highlight the biggest figure.


Basically all the axis and spots are drawn dynamically by code. There is no information relating to the content in HTML.


In a conventional way of analyzing HTML, we can only identify it as the whole big web element. Thus, it is impossible to identify and perform operations on the first row, which is biggest.


In this case, UFT One’s Insight mode can further dive into the canvas. Using Insight, UFT One identifies the first row as an Insight object, which you can see when highlighting the object.


We can add a text checkpoint to examine the information in it.


Also, UFT One can record some mouse operations on the canvas and generate scripts. Another case is to identify the spots in a chart. In a recording session, UFT One can generate the script while clicking, dragging, and dropping spots on the canvas.

Wrap up

While this blog covers a subset of our functionality, more details about UFT One can be found on the Online Help Center.

Download UFT One today from the following locations:

To learn more about UFT One, visit us at: | Twitter | LinkedIn Group


Functional Testing