Performance testing with Selenium and JMeter

Mark Bakker

In this blog I will show a way to do performance testing with Selenium. The reason I use Selenium for performance testing is that some applications use proprietary protocols between the application layer in the browser and the server.

So just capturing the traffic between the server and replaying modified traffic is not that simple.

An example is testing GWT applications. In a previous blog I wrote why this is difficult.

To create a test script in Selenium the first thing I do is record a test with Selenium IDE
After recording a script I export the script to JUnit3 (Remote Control). This will generate a JUnit test script which can be run to test the application.

The next thing you need is a solution to run a lot of JUnit test cases at the same moment.

Here you see a visual representation of the whole test chain.


To archive this I created a lot of Virtual hosts, installed selenium server on the hosts and wrote the name of the server together with the selenium server port in a csv file.
After this I added the following code to a base class witch will be the base class for all the JUnit test classes:

protectedstaticinthostCounter = 0;

protectedstaticsynchronizedvoid initializeSeleniumHosts(){

† † //code to read file csv file with hosts

}

static{

† † initializeSeleniumHosts();

† † //other initializations

}

protectedstaticfinalThreadLocalseleniumRunner = newThreadLocal(){

† † protected Object initialValue(){

† † if(hostCounter<seleniumHosts.size()-1){

† † † † hostCounter++;

† † }elsehostCounter=0;

† † † † String[] hostFields = seleniumHosts.get(hostCounter).split(":");

† † † † returnnew DefaultSelenium(hostFields[0], Integer.parseInt(hostFields[1]), "*firefox", "http://targetHost+ĚbaseUrl"+"/");

† † }

};

publicvoid setUp(){

† † selenium = (DefaultSelenium)seleniumRunner.get();

† † selenium.start();

}

publicvoid tearDown(){

† † selenium.stop();

}

With this code each JUnit tests case will be run on the next host in the csv file.

During the first manual runs you can see where a script fails. But when you use > 10 vm's to do the testing you can't see where the script gives errors.

To solve this problem I introduced a custom fail method in the base class.
The fail method logs the line where if fails in a logfile. This helps fixing the script to be sure it does not fail on timeouts with fixed timeouts.
The fail method:

publicstaticvoid fail(String message){† † try{† † † † † † throw†new Exception();† †

}catch(Exception e){

† † † † StackTraceElement[] stacktrace = e.getStackTrace();

† † try{

† † † † logFile.write("FAIL: ;" + message+";"+stacktrace[1]+"\n");

† † † † logFile.flush();

† † }catch(Exception ee){

† † † † e.printStackTrace();

† † }

}

Assert.fail(message);

}

Here you see a code snippet on how to use it:

for (int second = 0;; second++) {† † if (second >= 15) fail("timeout");Thread.sleep(4000);† †† † try { if (selenium.isElementPresent("//input[@id='someId']"))

† † † † break;

† † } catch (Exception e) {}

}
selenium.click(
"id=someId");

This method of testing helped me a lot to do performance testing on GWT and applications and Java applets.

I hope this information will help you to do performance testing on frameworks which use custom protocols between the client and the server. Good luck.

Comments (3)

  1. R Gill - Reply

    November 23, 2013 at 2:48 pm

    Impressive!,

    however i can't imagine this setup to scale up for load testing where one needs to simulate hundreds or thousands of users unless you have the amount of vms (to run Selenium controlled browsers) available that you can control from the jmeter + junit setup, which in the end is really expensive setup. A rather cheaper option is to create custom samplers(Plugins) for those protocols only if not already available OOB jmeter, and try to simulate the browser behavior as closely as possible within jmeter, including any ajax and or other required protocols requests.

    you are absolutely right that its not easy but in my opinion and experience its easier then setting up a grid of selenium machines for performance testing.

    P.S. One jmeter machine with 2 CPU, 4 GB can easily simulate upto 1000 threads for a complicated script (~50 dynamic reqs, regex post samplers etc).

    • Mark Bakker - Reply

      November 25, 2013 at 11:21 am

      You are right, this setup will only work when you need a limited amount of users.
      The case where I use it is a setup with a limited amount of users (just 1000).

      Kind regards,

      Mark Bakker

  2. Dmitri T. - Reply

    November 26, 2013 at 4:51 pm

    There is an extension for JMeter which allows configuring and invoking Browser through WebDriver Sampler (you need to get it from JMeter Plugins site and drop contents to lib and lib/ext folders)

    http://blazemeter.com/blog/jmeter-webdriver-sampler

Add a Comment