7 rules for (Agile) Test Automation

Cirilo Wortel

In a previous project I had to compete against the established experts when trying to introduce a sensible test automation approach.

The reasoning why not to work with suitable tools was mainly based on the fear of change, the unwillingness to automate and the build up idea that when you do automate, the only way to overcome problems is by having expensive licenses.

There are several good reasons why you should automate, but most important is that the team is confident about the quality of the product being developed and gets reliable and fast feedback when making changes.

The specialists on a team should be capable of self-sufficiently building a product and dealing with problems, without having to be dependent on external experts. When working in an agile production environment service level agreements aren’t going to help you, when response times are two to three weeks. Whenever a problem occurs you need to be able to respond immediately, otherwise it will block your entire process flow.

Furthermore agile is all about transparency and collaboration. All skills should be bundled; collaboration is the key to a common level of knowledge and a shared understanding.

Most proprietary tools just don’t embrace the idea of collaboration, licenses are to expensive for tools to be made available to all team-members and often require tool specific programming skills.

When working out these thoughts I came up with the following rules for agile test automation:

  • 1. When working in an Agile environment Automation is of vital importance to keep up with the pace of the project, relying on only manual testing is no option.
  • 2. Automation must be possible within the multidisciplinary team, no external expertise should be required to create or maintain tests.
  • 3. A test tool should be flexible, making it easy to respond to change in terms of environmental, data and technical constrains.
  • 4. Creating automated test cases should be relatively simple and intuitive.
  • 5. Tests should be readable and understandable (for anybody). Documenting and formatting tests in unambiguous and ubiquitous language supports this.
  • 6. Tests should be easy accessible and interchangeable between team-members and preferably also for other stakeholders. Anyone in the team should be capable of adding, viewing and running tests.
  • 7. Test execution should be fast and reliable, since fast feedback is your main objective.

In the process of tool selection these rules can guide you in the right direction. Collaborative test automation tools like FitNesse, Cucumber and Twist support these concepts.

Keep in mind that the tool you decide to use is secondary to the process you follow. In the end critical thinking, discipline, commitment and collaboration are the most important drivers behind successful test automation.

Comments (6)

  1. George Dinwiddie - Reply

    July 29, 2013 at 5:12 pm

    Cirilo, rather than saying "manual testing is no option" I would suggest "manual testing is insufficient." The goal is not to eliminate manual testing, as it is to automate routine testing of known expected behavior, both for the fast feedback you mention but also to allow time for manual testing for unanticipated behavior.

    • Cirilo Wortel - Reply

      July 30, 2013 at 9:15 am

      Excellent point George, you're absolutely right. Automated tests can't replace valuable manual techniques like Exploratory Testing, but do create room for doing more of that!

  2. Kenneth Truyers - Reply

    August 1, 2013 at 12:04 am

    Very good post!
    I think indeed that collaboration and elimination of boundaries and bureaucracy is the way for a fast-moving industry (such as IT). Simplicity and transparency should be at the core of every process.

    One point of critique though:
    I fail to see what licensing costs and proprietary tools have to do with the whole story. While it's true that if you don't make tools available to everyone, it can have an impact on productivity and collaboration, but that's true for non-agile teams as well (that's even true in every branch, not just IT).
    Apart from that I'd like to hear the reasoning behind the statement that proprietary tools don't embrace the idea of collaboration. In my opinion, this has nothing to do with proprietary, closed source, open source, ... For example, if I look at Team Foundation Server, TeamCity and Jenkins, I can see equal value delivered and equal collaborative features (all with a different accent on the process). These three all have a different licensing model.

    • Cirilo Wortel - Reply

      December 9, 2013 at 10:03 pm

      Hi Kenneth,

      I completely missed your remarks, I am sorry I did not respond earlier.

      Maybe my argumentation is not that good, and I am not trying to say that proprietary tools are bad and open source is good.
      For some reason most automation tools known to me, that truly support collaboration, are open source tools, but there definitely are exceptions. I also believe that every environment has its own demands, so there is no one leading rule for success.
      What lead me to make that remark in is that I have been in situations where licensing prohibited team members from running or viewing tests, purely because of the extra costs per head.
      I do think there is a lot of value in making tools available to as much stakeholders as possible, which can become an obstacle when licenses are very expensive.

  3. Chad Wathington - Reply

    August 1, 2013 at 10:00 pm

    A great book that embodies a bunch of the concepts listed here is "Specification By Example" by Gojko Adzic.


    Test automation is hugely valuable (when done well) as part of a larger team approach to quality.

  4. Prashant - Reply

    August 5, 2013 at 4:52 pm

    The article is well written. I too agree with you that agile approach is the latest trend in software testing field. In agile testing the testers adapt change easily. You can see more about agile testing in this blog - http://blog.testing-whiz.com/2013/05/how-to-become-resourceful-agile-tester.html

Add a Comment