Blog

Mocking the ‘unmockable’: too much of a good thing?

06 Dec, 2009
Xebia Background Header Wave

Static calls, final classes, objects created in test code: there are few things some of the current mocking frameworks cannot handle. Using powerful approaches like bytecode instrumentation or custom class loaders, these libraries make code that was previously a ‘no go’ area amenable to unit testing. This, moreover, in an elegant and convenient manner that will feel familiar to developers used to ‘standard’ mocking frameworks.
The question is: does such power perhaps come with hidden dangers? Might it be possible that the ability to test more could actually result in less code quality?

Recently, I published some code samples from a comparison of test frameworks I undertook some while ago, and wrote a blog post about some of my impressions and thoughts.

Crossing the Final Frontier

In one section, I mentioned a couple of libraries that aim to cross TDD’s "Final Frontier", making it possible to test code such as the following, containing for instance static calls, final classes and instantiating collaborators in the code under test.
[java]
public final class ServiceA {
public void doBusinessOperationXyz(EntityX data)
throws InvalidItemStatus {
List<?> items = Database.find("select item from EntityY item where item.someProperty=?",
data.getSomeProperty());
BigDecimal total = new ServiceB().computeTotal(items);
data.setTotal(total);
Database.save(data);
}
}
public static final class ServiceB { …
[/java]
I remarked that I found the approaches of the two frameworks, JEasyTest and Jmockit, rather clunky. In the ensuing discussion I was happy to learn that my samples were rather out of date: Jmockit and PowerMock, a "mock extension" framework frequently mentioned in the comments, are now much more convenient, with a syntax very close to what one is used to from e.g. EasyMock or Mockito.1

Dangers of an undiscovered country?

Certainly, the existence of a Jmockit or a PowerMock is a tribute to the powerful techniques available via the JVM and Java language and, more so, the Java developer community. But equally I have a suspicion that, without carefully considered guidelines, the flexibiliy afforded by these "power mocking" frameworks could quickly lead to code that is as bad as it is (newly) testable.

Can ++testability = --quality?

Testability is frequently2 named as one of the ilities good software should aspire to, and I certainly agree with that3. Assuming for a minute that testability implies a high degree of unit testing4, two of the main benefits of a comprehensive test suite for me are:

  1. as a self-verifying functional description of the code
  2. ensuring code is unit testable helps avoid anti-patterns

From the perspective of the first point, power mocking frameworks are undoubtedly useful. The more code can be tested and thus described, the better – there will always be cases that standard mocks cannot address: calls to (for instance) Thread.currentThread(), legacy code5, invocations of final utility methods that may be expensive, such as cryptographic operations, etc.
It’s the second aspect, however, that set me thinking: what is the relationship between testable and good code, aside from any potential "intrinsic" value of testability? Referring to Miško Hevery’s frequently-cited list of testability nightmares, it is clear that there appears to be substantial overlap between patterns that make code hard to test and recognised anti-patterns that impact other aspects of software quality.
Whether it is static methods that may leak global state, or collaborators created in code that are potentially indicative of tightly-coupled systems, it (coincidentally?) looks as though code that used to be hard or impossible to attack with standard mocking6 was probably flawed in some more fundamental way.
Returning to our "canonical" example of hard-to-test code7:
[java]
public final class ServiceA {
public void doBusinessOperationXyz(EntityX data)
throws InvalidItemStatus {
List<?> items = Database.find("select item from EntityY item where item.someProperty=?",
data.getSomeProperty());
BigDecimal total = new ServiceB().computeTotal(items);
data.setTotal(total);
Database.save(data);
}
}
public static final class ServiceB { …
[/java]
In my opionion, creating a new collaborating service is not bad primarily because it’s untestable, but because it introduces coupling and undermines polymorphism, and similarly for the static invocations of the Database class.

When less can be more

Succinctly put, the inability to accomplish certain things using standard mocking frameworks may be something of a blessing in disguise. Their constraints mean that aiming for testability effectively forces the developer to avoid common anti-patterns that are detrimental to scalability, reusability, maintainability and many of the other important aspects of code quality.
Testability as a proxy for other (perhaps more?) important aspects that are often more difficult to quantify and explain, if you will.
Having said that, I should make it clear that I consider Jmockit, PowerMock and the others to be good things! I simply feel their use will need to be accompanied by carefully considered best practices if our software development process is to reap the real benefit: better code, rather than simply 100% unit test coverage.

Footnotes

  1. There are most likely further interesting “power mocking” frameworks out there that aren’t mentioned here. If you know of one please leave a comment!
  2. See here or here or here, to mention but a few.
  3. However, I don’t feel “testability” necessarily has to mean “convenient to write unit tests for”. My colleague Vincent Partington and I frequently get into discussions as to whether adding a package-level setter to a class to make the object more testable is justifiable, with me on the no side.
    Equally, whilst I find Google’s @VisibleForTesting annotation a valuable hint to the developer, I feel it shouldn’t really be necessary in the first place. But that’s a topic for another blog post…
  4. A debatable assumption, of course!
  5. A comment to the previous post outlined the interesting case of “reverse-documentating” the dependencies and effects of un- or partially known legacy code using power mocking.
  6. This is not to imply that I believe power mocking frameworks are recommending the use of statics etc., just because they make it possible to mock them.
  7. There is a bit of a weak argument since the example is clearly not particularly realistic.
Questions?

Get in touch with us to learn more about the subject and related solutions

Explore related posts