For Deployit, XebiaLabs' automated deployment product for Java EE applications, we are always building and modifying integrations with middleware systems such as IBM WebSphere, Oracle WebLogic and the JBoss application server. These integrations are small enough so that they can be rearranged to get many different deployment scenarios. A typical step, as we call these integrations, would be "Create WebSphere datasource" or "Restart WebLogic Server". So how do the test that code?
We've had some success using FitNesse and VMware to do integration tests on our deployment scenarios. But there were a few problems with this apporach:
- We could only test complete deployment scenarios in this way. If we wanted to test just a single step, we had to make a deployment scenario that used that step just to be able to test it.
- Because FitNesse does not provide any feedback while a test is running and the steps, let alone the deployment scenarios, can sometimes take a while to execute, there was little feedback on the progress.
- While it is possible to debug a FitNesse Fixture using Eclipse the process is not very convenient when debugging a technical component such as this step.
- To verify that a deployment scenario has executed succesfully we had to extend our FitNesse Fixture often. And while debugging code under test in FitNesse is complicated enough, debugging a Fixture is even harder!
Clearly we needed a different approach if we wanted to develop new steps easily.