Testing

Testing Feature Branches Remotely with Grunt

Misja Alma

At my current job we are working on multiple features simultaneously, using git feature branches. We have a Jenkins build server which we use for integration testing of the master branch, which runs about 20 jobs simultaneously for Protractor and Fitnesse tests. An individual job typically takes around 10 minutes to complete.

Our policy is to keep the master branch production ready at all times. Therefore we have a review process in place that should assure that feature branches are only pushed to master when they can't break the application.
This all works very well as long as the feature which you are working on requires only one or two integration test suites to test its functionality. But every once in a while you're working on something that could have effects all over the application, and you would like to run a larger number of integration test suites. And of course before you merge your feature branch to master.
Running all the integration suites on your local machine would take too way much time. And Jenkins is configured to run all its suites against the master branch. So what to do?
 Read more

Ready, Test, Go!

Viktor Clerc

The full potential of many an agile organization is hardly ever reached. Many teams find themselves redefining user stories although they have been committed to as part of the sprint. The ‘ready phase’, meant to get user stories clear and sufficiently detailed so they can be implemented, is missed. How will each user story result in high quality features that deliver business value? The ‘Definition of Ready’ is lacking one important entry: “Automated tests are available.” Ensuring to have testable and hence automated acceptance criteria before committing to user stories in a sprint, allows you to retain focus during the sprint. We define this as: Ready, Test, Go!

 Read more

Mutation Testing: How Good are your Unit Tests?

Misja Alma

You write unit tests for every piece of code you deliver. Your test coverage is close to 100%. So when the point comes when you have to make some small changes to the existing code, you feel safe and confident that your test suite will protect you against possible mistakes.
You make your changes, and all your tests still pass. You should be fairly confident now that you can commit your new code without breaking anything, right?

Well, maybe not. Maybe your unit tests were fooling you. Sure they covered every line of your code, but they could have performed the wrong assertions.
In this post I will introduce mutation testing. Mutation testing can help you find omissions in your unit tests.
 Read more

Then When Given

Kishen Simbhoedatpanday

People who practice ATDD all know how frustrating it can be to write automated examples. Especially when you get stuck overthinking the preconditions of examples.

This post describes an alternative approach to writing acceptance tests: write them backwards!

 Read more

Fast and Easy integration testing with Docker and Overcast

Paul van der Ende

Challenges with integration testing

Suppose that you are writing a MongoDB driver for java. To verify if all the implemented functionality works correctly, you ideally want to test it against a REAL MongoDB server. This brings a couple of challenges:

  • Mongo is not written in java, so we can not embed it easily in our java application
  • We need to install and configure MongoDB somewhere, and maintain the installation, or write scripts to set it up as part of our test run.
  • Every test we run against the mongo server, will change the state, and tests might influence each other. We want to isolate our tests as much as possible.
  • We want to test our driver against multiple versions of MongoDB.
  • We want to run the tests as fast as possible. If we want to run tests in parallel, we need multiple servers. How do we manage them?

Let's try to address these challenges. Read more

Integrating Geb with FitNesse using the Groovy ConfigSlurper

Kishen Simbhoedatpanday

We've been playing around with Geb for a while now and writing tests using WebDriver and Groovy has been a delight! Geb integrates well with JUnit, TestNG, Spock, and Cucumber. All there is left to do is integrating it with FitNesse ... or not :-).

 Read more

Hands-on Test Automation Tools session wrap up - Part1

Kishen Simbhoedatpanday

Last week we had our first Hands-on Test Automation sessions.
Developers and Testers were challenged to show and tell their experiences in Test Automation.
That resulted in lots of in depth discussions and hands-on Test Automation Tool shoot-outs.

In this blogpost we'll share the outcome of the different sessions, like the famous Cucumber vs. FitNesse debat.

Stay tuned for upcoming updates!

 Read more

Running unit tests on iOS devices

Jeroen Leenarts

When running a unit test target needing an entitlement (keychain access) it does not work out of the box on Xcode. You get some descriptive error in the console about a "missing entitlement". Everything works fine on the Simulator though.

Often this is a case of the executable bundle's code signature not begin valid anymore because a testing bundle was added/linked to the executable before deployment to the device. Easiest fix is to add a new "Run Script Build Phase" with the content:

codesign --verify --force --sign "$CODE_SIGN_IDENTITY" "$CODESIGNING_FOLDER_PATH"

codesign

Now try (cleaning and) running your unit tests again. Good chance it now works.

Are Testers still relevant?

Kishen Simbhoedatpanday

Last week I talked to one of my colleagues about a tester in his team. He told me that the tester was bored, because he had nothing to do. All the developers wrote and executed their tests themselves. Which makes sense, because the tester 2.0 tries to make the team test infected.

So what happens if every developer in the team has the Testivus? Are you still relevant on the Continuous Delivery train?
Come and join the discussion at the Open Kitchen Test Automation: Are you still relevant?

 Read more

How to verify Web Service State in a Protractor Test

Misja Alma

Sometimes it can be useful to verify the state of a web service in an end-to-end test. In my case, I was testing a web application that was using a third-party Javascript plugin that logged page views to a Rest service. I wanted to have some tests to verify that all our web pages did include the plugin, and that it was communicating with the Rest service properly when a new page was opened.
Because the webpages were written with AngularJS, Protractor was our framework of choice for our end-to-end test suite. But how to verify web service state in Protractor?
 Read more