I was asked in a recent webinar: What is the purpose of unit testing frameworks?
That’s a cool question, and while it seems an easy answer, the way we use a framework makes it more subtle.
In the beginning, a unit test framework was a tool to define, run and report tests. Each tool works differently, but includes the following functions:
- Identify tests (through annotation, name prefix, etc.)
- Run them
- Assert APIs for success or failure
- Report the results.
Some of them do more, but regardless of language or environment, these basics hold true.
Note that Karma, for example, is not a unit test framework. It is a test engine runner.
And as the tools for unit testing, they do their job well. Until we start using them for something else.
We can use these tools to run whole systems, that talk to other systems. This comes with a price, although that is not the tools fault – it’s how we use them.
To test APIs, components, or systems, we need databases, big setups, compare big JSONs. The same frameworks we use for unit tests will work as they do in unit mode, but we will work harder in integration mode when setting tests up and analyzing failures.
Are there better tools for bigger tests? Somewhat. We mostly rely on the generic tools out there, or sometimes in-house, customized versions for the organization’s specific purpose.
Are we doing it wrong? Not really. Unit test frameworks are general purpose tools, supposed to fit everyone’s needs. They have reached the limit of their generalization. The rest are specific needs, that can be answered by specialty recipes.
0 Comments