Test Attribute #8 – Truthiness

This is the 8th post, soon to be the 8th wonder of the world, in the Unit Tests Attribute series that started here. To learn more about testing, contact me.

I want to thank Steven Colbert for coining a word I can use in my title. Without him, all this would still be possible, had I not given up looking for a better word after a few minutes.

Tests (including unit tests) are about trust. We expect them to be reliable. Reliable unit tests tell us everything is ok when they pass, and that something is wrong when they fail.

The problem is that life is not black and white, and unit tests are not just green and red. Tests can give false positive (fail when they shouldn’t) or false negative (pass when they shouldn’t) results. We’ve encountered the false positive ones before – these are the fragile, dependent tests.

The unit tests that pass, instead of failing, are the problematic ones. They hide the real picture from us, and erode our trust, not just in those tests, but also in others. After all, when we find out a problematic tests, who can say the others we wrote are not problematic as well?

Truthiness (how much we feel the unit tests are reliable) comes into play.

Dependency Injection Example

Or rather, injecting an example of a dependency.

Let’s say we have a service (or 3rd party library) our tested code uses. It’s slow and communication is unreliable. All the things that give services a bad name. Our natural tendency is to mock the service in the unit test. By mocking the service, we can test our code in isolation.

So, in our case, our tested Hotel class uses a Service:

public class Hotel
{ 
   public string GetServiceName(Service service)
   {
      var result = service.GetName();
      return "Name: " + result;
   }
}

[TestMethod]
public void GetServiceName_RoomService_NameIsRoom()
{
   var fakeService = A.Fake<Service>();
   A.CallTo(() => fakeService.GetName()).Returns("Room");

   var hotel = new Hotel();
   Assert.AreEqual("Name: Room", hotel.GetServiceName(fakeService));
}

The Truthiness Is Out There

Mocking is an example of obstructing the real behavior by prescriptive tests, but it’s just an example. It can happen when we test a few cases, but don’t cover others.

Here’s one of my favorite examples. What’s the hidden test case here?

public int Increment()
{
   return counter++;
}

Much like differentiation, truthiness can not be examined by itself. The example works, but it hides a case we need another test for. We need to look at the collection of test cases, and see if we covered everything.

The solution doesn’t have to be a test of the same type. We can have a unit test for the service happy path, and an end-to-end test to cover the disconnection case. Of course, if you can think of other cases in the first place, why not unit test them?

So to level up your truthiness:

• • Ideate. Before writing the tests, and if you’re doing TDD – the code, write a list of test cases. On a notebook, a whiteboard, or my favorite: empty tests.
• Reflect. Often, when we write a few test, new test cases come to mind. Having a visual image of the code can help think of other cases.
• Beware the mock. We use mocks to prescribe dependency behavior in specific cases. Every mock you make can be a potential failure point, so think about other cases to mock.
• Review. Do it in pairs. Four eyes are better than two.

Aim for higher truthiness. Higher trust in your tests will help you sleep better.

Next time: Deterministic.
For training and coaching on topics like this, contact me.

Image source: https://www.flickr.com/photos/yodelanecdotal/3611536356/