Building reliable and effective functional tests for asynchronous/dynamic/single page web applications with Selenium and JUnit (1/2) – Basics

Selenium is a well established tool for browser automation and functional tests of web applications. The process to build reliable functional tests got harder since the advent of Web 2.0 applications that dynamically change the content of a page. Dima Kovalenko describes the main problem as follows.

Test automation was simpler in the good old days, before asynchronous page loading became mainstream. […] Now, an element might be missing for several seconds, and magically show up after an unspecifi ed delay. [1]

How can you build reliable and effective functional tests for this kind of applications?

Referencing elements

It is generally a good practice to reference WebElements by their id because the alternatives (including CSS selectors, XPath expressions, link texts,…) are more likely to break as development continues. Nowadays this is even more important because these selectors can be broken by dynamic DOM changes. Additionally you gain a performance boost in contrast of using the slower referencing options like CSS selectors or XPath.

Mind you, that there are situations where you simply can not use ids, e.g. when dealing with single entries of dynamic structures like lists or tables. Writing XPath expressions in these situation can be quite cumbersome, fortunately there are tools to help us developers. The Chrome developer tools come with the option to copy the XPath of a selected element. Just open the developer tools (F12), right-click an element in the “elements” view and select “Copy XPath”. Similar can be done with Firefox´ FirePath plugin.

Dealing with asynchronous/dynamic content changes

As stated before, the dynamic (dis-)appearance of elements on the page is the core problem to build reliable tests. The first idea that comes to mind is to use explicit sleeps/timeouts to wait until the requested data is loaded or a dialog is shown.

driver.findElement(By.id("btOpenDialog")).click();
try {
	Thread.sleep(5000);
}
catch (InterruptedException e) {
	e.printStackTrace();
}
driver.findElement(By.id("textFieldInDialog"))
               .sendKeys("DON`T DO THIS AT HOME!!!");

This approach is neither reliable nor efficient, because the time needed to load a web resource and/or change a pages content will always differ. You are now faced with two options. Unfortunately, they are equally bad. You could either use very long timeouts to ensure that the content will be loaded once you try to reference an element – which will lead to Selenium tests running all night – or you can choose to use rather short timeouts, resulting in tests that will probably fail in every 3rd try because the web server needed a little more time to respond this time.

The solution is to use Selenium´s FluentWait class. With help of this class we can create a mechanism to wait until a specified ExpectedCondition is met. In this case we want to check for the visibility of an element every 100ms up to 10 seconds in total.

public WebElement getElementWhenVisible(final By by) {
	final Wait<WebDriver> wait = new FluentWait<WebDriver>(driver)
			.withTimeout(10, TimeUnit.SECONDS)
			.pollingEvery(100, TimeUnit.MILLISECONDS)
			.ignoreAll(Arrays.asList(StaleElementReferenceException.class, NoSuchElementException.class));
	wait.until(ExpectedConditions.visibilityOfElementLocated(by));
	return driver.findElement(by);
}

Now the error-prone test code shown above could look like this and you won´t have to care about varying loading/rendering times anymore:

getElementWhenVisible(By.id("btOpenDialog")).click();
getElementWhenVisible(By.id("textFieldInDialog"))
               .sendKeys("DO THIS AT HOME!!!");

The PageObject pattern

Way too often you can see Selenium tests like this on sites like StackOverflow:

@Test
public void seleniumTestWithoutPageObject() {
	driver.findElement(By.name("j_username").click().clear().sendKeys("Name");
	driver.findElement(By.name("j_password").click().clear().sendKeys("Password");
	driver.findElement(By.xpath("//div[8]/div/div/form/div/div[3]/input[1]").
	click();
	...
}

Code like this has several problems:

  • Hard to write, read and unterstand
  • Very hard to maintain when the application is changed significantly if you use the same elements in multiple tests
  • Referencing of elements is mixed with test logic => no seperation of concerns!

In order to clearly separate those two aspects and create easy to write and read tests, use the Page Object pattern.
This pattern is based on four key principles:

  • Every page, dialog, etc. in the application is represented by its own “Page Object class”
  • Every element in the ui (buttons, labels,…) is modeled as a WebElement attribute in its associated Page Object
  • Every interaction with the page (clicking a button, typing text into a text field,…) is modeled by a method in a Page Object class
  • Every method in a Page Object returns either the Page Object itself, another Page Object to represent navigation in the application or a visible value from the view (e.g. the content of a text field.

A Page Object to encapsulate the element referencing and the page interaction for the code above could look like this:

public class LoginForm {
	
	@FindBy(name = "j_username")
	private WebElement tfName;
	
	@FindBy(name = "j_password")
	private WebElement tfPassword;
	
	@FindBy(xpath = "//div[8]/div/div/form/div/div[3]/input[1]")
	private WebElement btLogin;
	
	public LoginForm typeIntoNameTextField(final String text) {
		tfName.click();		
		tfName.clear();
		tfName.sendKeys(text);
		return this;
	}
	
	public LoginForm typeIntoPasswordTextField(final String text) {
		tfPassword.click();
		tfPassword.clear();
		tfPassword.sendKeys(text);
		return this;
	}
	
	public LoggedInPage login() {
		btLogin.click;
		return new LoggedInPage();
	}
}

Because every method returns either the object itself or a new page object, a Fluent API is created, resulting in very easy to read test cases. Furthermore you can easily reuse the Page Objects in various test cases.

@Test
public void seleniumTestWithPageObject() {
	loginForm
		.typeIntoNameTextfield("Name")
		.typeIntoPasswordTextfield("Password")
		.login();
		...
}

The biggest benefits of using the Page Object pattern are better readability, reusable code and better maintainability.

Important: Using the PageObject pattern can lead to StaleElementReferenceExceptions. This can happen, if you reuse WebElement references and the page was changed in the meantime, thus they are more likely to occur when you use the Page Object pattern (because WebElements are fields in a Page Object) instead of writing spaghetti code. I would recommend to use the @FindBy annotation until you face problems, because you will probably only encounter them in some test cases. However, if you face them, just use a constant for the way to reference the element and look it up anew in every method:

public class LoginForm {
	
	private final static String TF_USERNAME_NAME = "j_username";
	...
	
	public LoginForm typeIntoNameTextField(final String text) {
		WebElement tfName = getElementWhenVisible(By.name(TF_USERNAME_NAME));
		tfName.click();
		tfName.clear();
		tfName.sendKeys(text);
		return this;
	}
	...
}

Retry tests several times

Even if you follow these rules, due to small server/network problems and stuff like that, Selenium tests will probably never be as stable as plain unit tests. If this sort of problems occur in your project occasionally for only a very short time span, you can retry failed test several times automatically until they are considered a failure.

Retrying tests is very easy in TestNG, with JUnit you either have to write a custom TestRunner or a custom TestRule. I strongly recommend to use a TestRule because you can use several rules inside your test, but your test can only be run by one TestRunner.

public class RetryTestsRule implements TestRule {	
	private static final int TEST_RETRY_COUNT = 2;
	
	@Override
	public Statement apply(final Statement stmt, final Description description) {
		return process(stmt, description);
	}

	private Statement process(final Statement stmt, final Description description) {
		return new Statement() {
			@Override
			public void evaluate() throws Exception {
				Exception caughtExc = null;

				for (int i = 0; i < TEST_RETRY_COUNT; i++) {
					try {
						stmt.evaluate();
						return;
					}
					catch (final Exception e) {
						caughtExc = e;
					}
				}
				throw caughtExc;
			}
		};
	}
}

To use the defined rule in a test you just have to include the following line inside your test class:

@Rule
public RetryTestsRule retry = new RetryTestsRule();

Mind the order of your tests

In general every software test should be independent of the results of other tests. Therefore JUnits default behaviour to run test methods in a non-deterministic order can even be described with the famous words “it´s not a bug, it´s a feature”. If you use Maven to execute a number of test classes you will find the same non-deterministic ordering for test classes.

A big amount of Selenium tests can lead to tests that run several hours. So why should you execute tests that check the behaviour of the user account menu in your application when prior tests that check the login failed? Respectively why should you wait so long to see that half of your tests failed because of a basic problem?

Problems like this lead to the “fail-fast” approach for software tests, that basically says: “If a test fails, it should fail as fast as possible to give the developer feedback as fast as possible”. To implement this idea in your Selenium/JUnit tests you can specify the order of your test classes and methods. For test classes its easiest to group the test classes inside a TestSuite. The execution order of the single classes is the same as the order in which you listed them in the suite.

@RunWith(Suite.class)
@Suite.SuiteClasses({
	TestClass1.class,
	TestClass2.class,
	TestClass3.class
})
public class TestSuite {
}

Since JUnit 4.11 the annotation @FixMethodOrder was implemented in order to control the execution order of the methods inside your class. You can use MethodSorters.NAME_ASCENDING and numbered prefixes for your method names to run the tests in your desired order.

@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class TestClass1 {
	@Test
	public void t00_someTest() {
		...
	}
	@Test
	public void t01_anotherTest() {
		...
	}
}

Prior to JUnit 4.11 you can achieve the same behaviour with a custom TestRunner.

If you want to you can stop your tests once the first test failed with help of a RunListener or a TestWatcher.

Summary

  • Reference WebElements by their id if possible. If you need XPath expressions, use the Chrome developer tools or plugins for different browsers to ease the generation of correct XPath expressions
  • Use an abstract wait mechanism with ExpectedConditions instead of explicit wait intervalls
  • Spaghetti code is bad, always! Use the PageObject pattern
  • If you face StaleElementReferenceExceptions, look up associated WebElements individually before every page interaction
  • Retry Selenium tests several times until they are considered a failure to further increase their stability
  • Mind the “fail-fast” approach to get feedback more quickly
  • Follow general test best practices like independent test methods (do not rely on successfull precedent tests, use a clean environment when needed,…)

Follow-up

In the 2nd part we will show you how to build parallel cross-browser tests with Selenium Grid and JUnit.

[1] Kovalenko, Dima: Selenium Design Patterns and Best Practices. Packt Publishing, 2014. – ISBN 9781783982707

Short URL for this post: http://wp.me/p4nxik-2uo
This entry was posted in Java and Quality and tagged , , , . Bookmark the permalink.

One Response to Building reliable and effective functional tests for asynchronous/dynamic/single page web applications with Selenium and JUnit (1/2) – Basics

  1. Pingback: Selenium 3.0 released | techscouting through the java news

Leave a Reply