The 4 Attributes of Testability : Operability

Code can be easy, or hard, to test. A unit, or API, or through the UI, automated or manual – in the end, we’d like to know how it works. But at what cost? Welcome to the world of testability.

Everything is testable. Like I mentioned my Dependency Injection post, it’s a matter of motivation, effort and skill.

We sometimes say code is “not testable”, but we mean something else – we’re not willing to put in the effort, or it takes too much time. It can even be a matter of “do we actually know how to do this?”.

Testability depends on many things, but we can divide it to:

1. Operability – How easily we can call the tested code
2. Observability – How easily we can observe the result or effect of the operation
3. Controllability – How easily we can control the conditions of the test
4. Reproducibility – How easily we can make sure the code executes repeatably

Let’s talk about the first one.

Operability is our ability to call the code we’d like to test. If we’re calling an API, for example, we invoke the code behind the API, all the way down. Depending on what “down” is – it could be all the way to the database, or simulator, or another service, or mocks. At the “end” of the code, a change has occurred in our systems, and we’d like to detect that (that’s observability, we’ll talk about it next time).

Can we even call it?

Let’s start with a small example. What if the code we want to check, lives in a private method? We immediately consider the private method “untestable”.

Yet ,we do have a couple of options.

• We can call a public method that calls our private method. This may require a bigger setup (we’ll talk about this soon).
• We can invoke the private method through reflection (depending on technology). This is considered a hi-tech-ey way, and sometimes considered a no-no, in some people’s view. I’m not above this..
• We can change the method to be public. And before I get the “somebody might call this”, remember that public/private is a language construct. In JavaScript we wouldn’t have this discussion in the first place. Check out my “Make it Public” Talk, then shout at me.
• Skip testing it altogether.

Our perception of if it’s possible to call the code (or change it), raises or lowers the testability level in our view.

Nothing but the code

Check out this amazing diagram:

There’s some code in a library, inside an API controller, inside a whole application. That’s our target code to test.

We can test the code in different ways. Directly, or through the API, or the UI.

APIs make it easy to call code (that’s why they’re there). But what if we want to call the internal code? Things get messier. We may need to call our code indirectly, through another API. When running more than our intended code, there can be side effects.

Side note: Why do we prefer unit tests for integrated tests? In unit tests, if something fails, the only code responsible is the tested code. While in integrated tests, the possibility of failure rises, because we can fail due to other dependencies, like the environment, or time, or place of the run. Plus a few thousand more reasons.

In testability terms, if we have code to test, we’d rather call it directly, or as close as we can get to it. Anything more – lowers the testability level in our eyes. It usually requires more effort, motivation and / or skill.

This makes even more sense when going bigger.

The further they are, the bigger the setup

When testing API code through the UI, we get further from our point of interest (if we’re interested in the API code, that is). The further we get from the point of interest, setup becomes bigger. We may need to run a server, instead of calling methods inside the test process. We may require to set up a database, which will need to reset for the next test.

We need to set up more things, like run separate services for the whole test to complete. maybe even in another technology (think selenium vs API calls, or SQL to setup the database, or scripts).

The bigger the setup, we lower the level testability – more effort, etc.

Operability is about how easy it is to call the code we want. That include the setup for the test, and how we call the code. Calling the same code can be done in different ways, and each one carries a different cost. We translate these costs into what we call testability.

Check out the my API testing and Clean Code workshops where I discuss testability: