Testing multiple apps with a single script in Silk4J

Modern business transactions often span multiple devices and technologies. For example, a transaction can start on a mobile device, continue on the web, and end up in a native back-end system. As a tester, mapping this business transaction to consistent automation scripts can be tremendously challenging.

In this blog post we outline how you can solve this challenge with Silk4J and Selenium.

Workflow

To keep our example short, we will focus on a simple transaction spanning two devices. We will start on our native mobile sample application to calculate an insurance quote, and we will then switch over to the Firefox browser on the desktop and calculate the same quote. Then we will verify that the two quotes we have generated are equal.

Step 1: Mobile

We start by creating a new Silk4J project. In the New Silk4J Project wizard we select a mobile device, in this case an iOS simulator running on a remote Mac, and the appropriate app file.

Then we record the workflow on the mobile device. By selecting Keyword-Driven Test in the Start Recording dialog we are able to split the recorded workflow into different keywords right away. When creating keywords during the recording session, make sure to name the keywords appropriately to distinguish the mobile part of the test from the web part that you will create later. For example by suffixing the keywords with (Mobile). The recorded test in the Silk Recorder should look similar to the following.

Make sure to also name the Java class appropriately, in which the keyword implementation should be generated, for example by naming the class MobileKeywords.

After the code has been generated, rename the Start Application keyword to Start Application (Mobile), and adapt the keyword-driven test accordingly. After that, the keyword-driven test should look like the following.

And the corresponding implementation class should look like:

public class MobileKeywords {

	private Desktop desktop = new Desktop();

	@Keyword(value = "Start application (Mobile)", isBaseState = true)
	public void start_application() {
		MobileBaseState baseState = new MobileBaseState();
		baseState.execute(desktop);
	}

	@Keyword("Login (Mobile)")
	public void login_Mobile_() {
		desktop.find("Device.Email").setText("john.smith@gmail.com");
		desktop.find("Device.Password").setText("john");
		desktop.find("Device.Login").click();
	}

	@Keyword("Auto Quote (Mobile)")
	public void auto_Quote_Mobile_() {
		desktop.find("Device.menu").click();
		desktop.find("Device.Auto Quote").click();
		desktop.find("Device.Zip Code").setText("78731");
		desktop.find("Device.Male").click();
		desktop.find("Device.Good").click();
		desktop.find("Device.Car").click();
		desktop.find("Device.Year").setText("1999");
		desktop.find("Device.Select").click();
		desktop.find("Device.Toyota").click();
		desktop.find("Device.Select").click();
		desktop.find("Device.Hilux").click();
		desktop.find("Device").swipe(new Point(169, 531), new Point(144, 306));
		desktop.find("Device.Financed").click();
		desktop.find("Device.Submit").click();
		MobileObject yourInstantQuoteIs15600EveryTwelveMonths = desktop
				.find("Device.Your Instant Quote is 1560 0 every twelve months");
		Assert.assertEquals("Your Instant Quote is 1560.0 every twelve months.",
				yourInstantQuoteIs15600EveryTwelveMonths.getProperty("caption"));
		desktop.find("Device.Auto Quote Button").click();
	}

	@Keyword("Logout (Mobile)")
	public void logout_Mobile_() {
		desktop.find("Device.menu").click();
		desktop.find("Device.Logout").click();
	}
}

Step 2: Selenium

Next, we record the same workflow on the web application using Selenium. To combine both apps in the same Silk4J project, we will apply a little trick. We will move the silk4j.settings file from its original location to the project root directory, and we will rename the file to mobile.silk4j.settings.

Before:

After:

Then we create a new keyword-driven test and start recording again. Notice that Silk4J prompts you to configure the application again, but this time instead of selecting the mobile device, select the local desktop browser and the appropriate URL.

Once the recording session has started, switch the recording mode from Silk Test to WebDriver.

Record the workflow. To avoid name clashes with the mobile keywords, make sure to use different names for the new keywords. For example, name the new login keyword Login (Web) instead of Login (Mobile). When prompted to add a reference to the WebDriver client bindings, click Yes.

The recorded keyword-driven test should then look like the following.

This is the implementation of the test:

public class WebKeywords {

	private Desktop desktop = new Desktop();

	@Keyword("Login (Web)")
	public void login_Web_() {
		WebDriver driver = desktop.find("//BrowserApplication").getWebDriver();
		driver.findElement(By.id("login-form:email")).sendKeys("john.smith@gmail.com");
		driver.findElement(By.id("login-form:password")).sendKeys("john");
		driver.findElement(By.id("login-form:login")).click();
	}

	@Keyword("Auto Quote (Web)")
	public void auto_Quote_Web_() {
		WebDriver driver = desktop.find("//BrowserApplication").getWebDriver();
		new Select(driver.findElement(By.id("quick-link:jump-menu"))).selectByVisibleText("Auto Quote");
		driver.findElement(By.id("autoquote:zipcode")).clear();
		driver.findElement(By.id("autoquote:zipcode")).sendKeys("78731");
		driver.findElement(By.id("autoquote:e-mail")).clear();
		driver.findElement(By.id("autoquote:e-mail")).sendKeys("john.smith@gmail.com");
		driver.findElement(By.id("autoquote:vehicle:0")).click();
		driver.findElement(By.id("autoquote:next")).click();
		driver.findElement(By.id("autoquote:age")).clear();
		driver.findElement(By.id("autoquote:age")).sendKeys("38");
		driver.findElement(By.id("autoquote:gender:0")).click();
		driver.findElement(By.id("autoquote:type:1")).click();
		driver.findElement(By.id("autoquote:next")).click();
		driver.findElement(By.id("autoquote:year")).clear();
		driver.findElement(By.id("autoquote:year")).sendKeys("1999");
		driver.findElement(By.id("ext-gen4")).click();
		driver.findElement(By.xpath("//div[text()=\"Toyota\"]")).click();
		driver.findElement(By.id("ext-gen6")).click();
		driver.findElement(By.xpath("//div[text()=\"Hilux\"]")).click();
		driver.findElement(By.id("autoquote:finInfo:1")).click();
		driver.findElement(By.id("autoquote:next")).click();
		WebElement h1startswithtextYourInstantQuotei = driver
				.findElement(By.xpath("//h1[starts-with(text(),\"Your Instant Quote i\")]"));
		Assert.assertEquals("Your Instant Quote is USD 1.160,00 every twelve months.",
				h1startswithtextYourInstantQuotei.getText());
	}

	@Keyword("Logout (Web)")
	public void logout_Web_() {
		WebDriver driver = desktop.find("//BrowserApplication").getWebDriver();
		driver.findElement(By.xpath("//a[text()=\"Home\"]")).click();
		driver.findElement(By.id("logout-form:logout")).click();
	}
}

Note: Silk Test reused the Start Application (Mobile) keyword in the beginning of our test, which is not what we want. Create a new Start Application (Web) keyword in the WebKeywords class instead, with the following implementation.

@Keyword("Start application (Web)")
public void start_application_Web_() {
	BrowserBaseState baseState = new BrowserBaseState();
	baseState.execute(desktop);
}

Step 3: Handling the different application configurations

Next we have to make sure that the Start Application (Mobile) keyword actually starts the mobile app, and that the Start Application (Web) keyword actually starts the web application. To do that, we make use of the different settings files we have created.

Copy and paste the current silk4j.settings file, which now contains the configuration for the web application, to the root folder of the project, and rename it to web.silk4j.settings. Together with the mobile.silk4.settings file we have already added to the root folder, the project structure should look like the following.

Open the MobileKeywords Java class and adapt the code to refer to the mobile.silk4j.settings file:

@Keyword(value = "Start application (Mobile)", isBaseState = true)
public void start_application() {
	MobileBaseState baseState = new MobileBaseState(new File("mobile.silk4j.settings"));
	baseState.execute(desktop);
}

Do the same thing with the Start application (Web) keyword:

@Keyword("Start application (Web)")
public void start_application_Web_() {
	BrowserBaseState baseState = new BrowserBaseState(new File("web.silk4j.settings"));
	baseState.execute(desktop);
}

Step 4: Creating the combined test

Now we can create a combined keyword-driven test that makes use of both the web keywords as well as the mobile keywords. We will run the workflow against the mobile app first, and then against the desktop app.

With the changes in place, there is just a minor thing left to do before we can actually run the combined test. We need to disable the Select Browser dialog by unchecking Show 'Select Browser' dialog before record and playback in the Edit Application Configurations dialog.

Step 5: Passing data

As we have outlined before, we want to verify that the mobile app and the web app return the same quote. In order to verify that, we need to have both Auto Quote keywords return the value of the actual quote, and we need to introduce a keyword to compare both values. To do that, we will change both keyword methods to return a String, and return the text of the label that we used in the original verifications.

This is the new Auto Quote keyword method for the native mobile app:

@Keyword("Auto Quote (Mobile)")
public String auto_Quote_Mobile_() {
	desktop.find("Device.menu").click();
	desktop.find("Device.Auto Quote").click();
	desktop.find("Device.Zip Code").setText("78731");
	desktop.find("Device.Male").click();
	desktop.find("Device.Good").click();
	desktop.find("Device.Car").click();
	desktop.find("Device.Year").setText("1999");
	desktop.find("Device.Select").click();
	desktop.find("Device.Toyota").click();
	desktop.find("Device.Select").click();
	desktop.find("Device.Hilux").click();
	desktop.find("Device").swipe(new Point(169, 531), new Point(144, 306));
	desktop.find("Device.Financed").click();
	desktop.find("Device.Submit").click();
	
	MobileObject label = desktop.find("Device.Your Instant Quote is 1560 0 every twelve months");
	String labelText = label.getText();	
	
	desktop.find("Device.Auto Quote Button").click();
	
	return labelText;
}

And this is the new Auto Quote keyword method for the web application:

@Keyword("Auto Quote (Web)")
public String auto_Quote_Web_() {
	WebDriver driver = desktop.find("//BrowserApplication").getWebDriver();
	new Select(driver.findElement(By.id("quick-link:jump-menu"))).selectByVisibleText("Auto Quote");
	driver.findElement(By.id("autoquote:zipcode")).clear();
	driver.findElement(By.id("autoquote:zipcode")).sendKeys("78731");
	driver.findElement(By.id("autoquote:e-mail")).clear();
	driver.findElement(By.id("autoquote:e-mail")).sendKeys("john.smith@gmail.com");
	driver.findElement(By.id("autoquote:vehicle:0")).click();
	driver.findElement(By.id("autoquote:next")).click();
	driver.findElement(By.id("autoquote:age")).clear();
	driver.findElement(By.id("autoquote:age")).sendKeys("38");
	driver.findElement(By.id("autoquote:gender:0")).click();
	driver.findElement(By.id("autoquote:type:1")).click();
	driver.findElement(By.id("autoquote:next")).click();
	driver.findElement(By.id("autoquote:year")).clear();
	driver.findElement(By.id("autoquote:year")).sendKeys("1999");
	driver.findElement(By.id("ext-gen4")).click();
	driver.findElement(By.xpath("//div[text()=\"Toyota\"]")).click();
	driver.findElement(By.id("ext-gen6")).click();
	driver.findElement(By.xpath("//div[text()=\"Hilux\"]")).click();
	driver.findElement(By.id("autoquote:finInfo:1")).click();
	driver.findElement(By.id("autoquote:next")).click();
	
	WebElement label = driver.findElement(By.xpath("//h1[starts-with(text(),\"Your Instant Quote i\")]"));

	return label.getText();
}

Additionally, we will introduce a new keyword called Assert equals in one of our keyword classes with the following implementation:

@Keyword("Assert equals")
public void assertEquals(String value1, String value2) {
	Assert.assertEquals(value1, value2);
}

In the combined keyword-driven test, you can now make use of both the result values, store them in local variables, and pass them to the Assert equals keyword. The combined keyword-driven test should look as follows.

Run the keyword-driven test again. You should see a verification failure similar to the following being raised at the very last keyword:

org.junit.ComparisonFailure: expected:<...ur Instant Quote is [1560.]0 every twelve month...> but was:<...ur Instant Quote is [USD 1.160,0]0 every twelve month...>
	at org.junit.Assert.assertEquals(Assert.java:115)
	at org.junit.Assert.assertEquals(Assert.java:144)
	at WebKeywords.assertEquals(WebKeywords.java:70)

This verification failure is also reflected in the resulting TrueLog.

This means that our cross-technology test actually found a bug in one of our products, unless there is a business reason for giving customers on mobile devices a higher quote for the same level of car insurance.

Summary

By using keyword-driven testing in combination with different silk4j.settings files, we managed to run a cross-device test that started on a mobile app and then did some verifications on the web. Using the same approach you could do more sophisticated things as well. For example, you could create a workflow that spans multiple systems, devices, and tiers in order to validate a desired business flow.