Today, I want to talk about a special kind of tests: Verification tests. They are the time-savers you need.
What do we want from our tests? Not much. We want to know that everything’s okay (when they pass), and when they fail – that there’s something wrong (and hopefully where). But there’s a specific aspect we consider here: Functionality. Whatever we’re testing, we’re checking that based on our expectations, or requirements – the code functionality should work.
When the test finds something wrong, we really hope that it’s something we can fix. If not in the code, then in the underlying framework. If not there, in the database.
With this kind of failure, we know what to do – go in, find the leak, and plug it.
Imagine our surprise, if we go in and what we find is that the database is not connected. Or that it is, but there’s no data in it. Or that the wrong version of the framework is installed. On the wrong operating system. We may upgrade a package without updating a dependency. All these are errors we will find eventually and fix, but it will take some time to debug and identify the problem.
Wouldn’t that be embarrassing? But the real problem, as I said, is that we’re wasting time on the investigation. We shouldn’t find out about these problems when our functional tests run.
We want to find out before they run.
Verification Tests First
That’s what verification tests are for. They run before any suite of tests, and make sure that the functional tests are ready to be run. That when they fail, it’s not going to be a deployment problem, but a functional problem.
Verification tests can be done on any level, (although you probably won’t need them at the unit test level. they can check that a server is up, before running API tests. They can check that a browser of a certain version is installed, before running any UI test.
And they can check that the server can talk to the database, and it is setup with the right data and permissions. That all containers are up and running.
When all these pass, then we can run our tests. Then we’d know that when they fail, there’s a real problem.
An Example Would Be Handy
Here’s a minimal one. When you’re creating a Spring project in Java through Spring Boot, you’ll see a single empty test generated:
@SpringBootTest
class DemoApplicationTests {
@Test
void contextLoads() {
}
}Code language: Java (java)It looks like an empty test, waiting for you to fill it up. But the truth is, it can actually fail: As its name suggests, it checks that the Spring Context loads. Without it, any Spring-related code will fail. We’ll be surrounded by exceptions and wouldn’t know what hit us. We’ll figure this out by the end, but that will probably be a waste of minutes, sometimes more. Imagine a whole nightly run failing. With this test, you can enforce running the rest of the functional, API, whatever tests – only if it passed.
Verification tests can also be something else. For example, in my current development, I’m using an API test to tell me that the server and database are running. It’s a short test that gives me the confidence to run the rest of my longer suite.
Verification tests save us time. You can write them in whatever language you’re used to. Mostly, they are easy to write. They can save you a lot of time, although, I can’t guarantee that what the functional tests find won’t be embarrassing.
Speaking of embarrasing, check out the emberassingly great API Testing Workshop for developers. I also cover verification tests, of course.
0 Comments