This is last, final, and 10th entry in the ten commandments of unit tests attributes that started here. And you should read all of them.
We usually talk about isolation in terms of mocking. Meaning, when we want to write unit tests, and the code has dependencies, we use mocking to fake those dependencies, and allow us to unit test the code in isolation.
That’s code isolation. But test isolation is different.
Isolated unit tests can run alone, in a suite, in any order, independent from other unit tests and give consistent results. We’ve already identified in footprint the different environment dependencies that can affect the result, and of course, the tested code has something to do with it.
Other unit tests can also create dependency, directly or not. In fact, sometimes we may be relying on the order of unit tests.
To give an example, I summon the main witness for the prosecution: The Singleton.
Here’s some basic code using a singleton:
public class Counter
{
private static Counter instance;
private int count = 0;
public static void Init()
{
instance = new Counter();
}
public static Counter GetInstance()
{
return instance;
}
public int GetValue()
{
return count++;
}
}
Pretty simple clean code: The static instance is initialized in a call to Init. We can write these unit tests:
[TestMethod]
public void CounterInitialized_WorksInIsolation()
{
Counter.Init();
var result = Counter.GetInstance().GetValue();
Assert.AreEqual(0, result);
}
[TestMethod]
public void CounterNotInitialized_ThrowsInIsolation()
{
var result = Counter.GetInstance().GetValue();
Assert.AreEqual(1, result);
}
Note that the second unit test passes when running after the first. But if you run it alone it crashes, because the instance is not initialized. Of course, that’s the kind of thing that gives singletons a bad name. And now you need to jump through hoops in order to check the second case.
By the way, we’re not just relying on the order of the unit tests – we’re relying on the way the test runner runs them. It could be in the order we’ve written them, but not necessarily.
While singletons mostly appear in the tested code, test dependency can occur because of the unit tests themselves. As long as you keep state in the test class, including mocking operations, there’s a chance that you’re depending on the order of the run.
Clean code no-nos
Do you know this trick?
public class MyTests: BaseTest {
///...
Why not put all common code in a base class, then derive the test class from it?
Well, apart of making readability suffer, and debugging excruciating, we now have all kinds of unit test setup and behavior that are located in another shared place. It may be that the unit test itself does not suffer interference from other unit tests, but we’re introducing this risk by putting shared code in the base class. Plus, you’ll need to no more about initialization order. And what if the base class is using a singleton? Antics ensue.
Test isolation issues show themselves very easily, because once they are out of order (ha-ha), you’ll get the red light. The problem is identifying the problem, because it may seem like an “irreproducible problem”.
In order to avoid isolation problems:
-
- Check the code. If you can identify patterns of usage like singelton, be aware of that and put it to use: either initialize the singleton before the whole run, or restart it before every test.
- Rearrange. If there are additional dependencies (like our counter increase), start thinking about rearranging the tests. Because the way the code is written, you’re starting to test more than just small operations.
- Don’t inherit. Test base classes create interdependence and hurt isolation.
- Mocking. Use mocking to control any shared dependency.
- Clean up. Make sure that the unit tests clean up after themselves. Or, instead before every run.
Isolation issues in unit tests are very annoying, because especially in unit tests, they can be easily avoided. Know the code, understand the dependencies, and never rely on another unit test to set up the state needed for the current one.
0 Comments