Introduction to Xebium

Cirilo Wortel

Introduction

When testing web interfaces, it’s convenient to use an intuitive tool like Selenium IDE, it’s easy to use and can be used by non-technical people, but it is solely meant for record and playback of test-scripts. One of its limitations is that it misses sufficient options for documenting and managing tests. Furthermore it misses an interface with the backend of the system under test (SUT), to setup preconditions for a test or for instance to manipulate or read from a database.
Fitnesse is a great tool to do just that, it has the Wiki to manage tests and it by default has a setup and teardown mechanism, it’s easy to add non invasive testfixtures to interface directly with your SUT. The downside is that it is incapable of doing webtests.

We now have the glue that combines the two, it's called Xebium!

On several projects I’ve tried to find a way to combine the two, because both tools are free of use and both have their individual charm, but it seems combined they would master practically any situation. As a tester I miss the development skills to come up with a satisfying solution and I never convinced my team members to help me out in a satisfactory manner.

At our company we recently started app-incubator, a new form of knowledge exchange, in the form of mini projects, anyone can pitch a project idea and when enough colleagues sponsor the project, we can spend up to three days collaborating on it.  My pitch caught fire and with about five people we brainstormed on the solution. For me a prerequisite was that it should be possible to use the test scripts two directional, so record them with Selenium IDE, run and edit them in Fitnesse and playback from Selenium IDE again. During this first project session we came up with additional requirements, it should support data-driven testing and variable substitution. We wanted to keep it as simple as possible, no new DSL’s should be introduced and it should not require conversions between the two products. We explored some existing solutions such as Fitnium and Selenesse but came to the conclusion these (at the time) were fully based on their own DSL’s and did not support exchanging files between Selenium IDE and Fitnesse. So we decided to created something far more simple.

The Result

After several attempts we discovered that Webdriver (Selenium 2.0) would be the best engine for running our tests. It seems more future proof, turned out to be a lot more stable than Selenium Testrunner and its commands are more compatible with Selenium IDE than Selenium RC.
We introduced a Selenium IDE formatter which output can be directly copied into the Wiki. We created a Fitnesse fixture that is capable of running the pre-formatted scripts without additional interpretations of the Selenium commands.

The additional features we had in mind, the data-driven testing and variable substitution turned out to be standard Fitnesse features. What we did not realize when we started was that Fitnesse would introduce a mechanism which, in my opinion, added mind-blowing power to project: Scenario tables, which now enables us to create abstractions on top of our testscripts.

Conclusion

Xebium enables us to use both Selenium IDE (low learning curve) and FitNesse (ease of maintenance) to it's fullest and provides a complete solution when it comes to automated web testing.

Comments (7)

  1. Maxime - Reply

    May 26, 2011 at 3:08 pm

    Hi Cirilo,

    Nice post, thanks a lot ! Xebium seems to be a very powerfull solution for automated web testing. I installed it and made some tests but got some problems. Is there anything like a mailing list or something to get some support ? Or should I directly contact someone ?

    Thanks in advance and thanks to the xebia's team who developped this tool.

    Maxime

    • Cirilo Wortel - Reply

      May 26, 2011 at 4:48 pm

      Hi Maxime,

      Thanks for trying out Xebium! It is really valuable to us to gain more information on user experiences, I would be glad to help out with your problems. In order to do so we have created a google support group, please mail your questions to xebium@googlegroups.com

      Regards,

      Cirilo

  2. Albert - Reply

    May 8, 2012 at 10:39 pm

    I have 2 part question

    In Xebium how do I rename or remove Test Case from a Suite..

    If I have test case A that logs into google.com and test case B that clicks on the Inbox link. When I put test case A and B into a Suite and try to run them together
    Test Case B doesn't run because it doesn't have a part of the script that logs into google.

    Thank you...

    • Cirilo Wortel - Reply

      May 9, 2012 at 1:20 am

      Hi Albert,
      FitNesse runs the testcases in a suite sequentially based on the alphabetical order of the testcase name, so if the testcase are really named TestCaseA and TestCaseB is should work as you expect (if you do not open or close the browser in between). I would try to avoid though to make the testcases dependent upon each other as much as possible and would try to put both actions in one testcase. This results in more robust and standalone testcases.
      If the goal is to save time by not opening and closing the browser with each testcase, my preferred approach would be creating a SuiteSetUp page containing the "open browser" command and a SuiteTearDown page containing the "stop browser" command, and having all self containing testcases in your suite (using the same browser instance).

      * The SuiteSetUp and SuiteTearDown are FitNesse special page types that run only one for an individual test or an entire testsuite.
      Hope this is a satisfying answer to your question. Please also be aware of the Xebium discussion group which is meant for answering questions like yours.

      Regards,

      Cirilo

      • Cirilo Wortel - Reply

        May 9, 2012 at 1:23 am

        Renaming, Deleting or Moving testcases can be done through the Refactor functionality of FitNesse

  3. karan - Reply

    September 30, 2013 at 4:49 pm

    Albert,
    If you do not want to have the SuiteSetUp and teardown to have the log in for each test case and also you need in the first two test case you can include the first test case in second test case by using command include.(testpageA link). This will run the testcaseA again and open up a browser for you before executing the TestcaseB actual steps are carried out. Let me know if it sounds confusing.

    • Cirilo Wortel - Reply

      September 30, 2013 at 5:09 pm

      Hi Karan,
      You could do it that way, but there are more elegant solutions. Again here you are making tests depend on each other. If it is a function you want to reuse, like logging in it is more efficient to create a method or scenario for it, rather than including testcases.

      Stuff like opening the browser is typically something you want to do for an entire suite and the SuiteSetUp is intended for precisely that (it is executed once at the beginning of a suite).

      Stuff you like to repeat in each test, cleaning up data from the previous test for instance, can be done with the SetUp page (which is executed prior to each individual test in a suite).

Add a Comment