Why API Testing Is Complex – An Example

Years ago, I started working with a client, where the developers were doing all the API testing. They had great coverage, and it was easy to measure it, too. They had an Excel sheet with one column containing a list of all the APIs, and in the cell next to each API’s name, there was a cell with a single value, telling if it is covered by a test: Yes/No.

You probably get a feeling that this is not the best way of thinking about testing. I’ve already talked about the complexity of testing APIs, but for now, let’s narrow our topic to functionality. An API should mostly do one thing. We expect to see it directly from its definition.

A Failed Break In

Here’s a log-in API:

I know that’s not how you do a proper log-in, but bear with me .

Now, this is a simple, yet smart log-in API. When you fail a login once, you get a 200 OK status code, with a polite message in the response: “Try again!”.

If you tried again and failed, you get a less polite “Are you even trying, bro?” message (still with that 200 OK status). Logging in (and failing) persistently for the third time, gets you a big flashy Unauthorized 401 status code, and a stern “No Log-In For You” message. Also, every other API on the server gets locked.

But, why make it so simple? When you tried (and failed) logging in for that final third time, that whole server API lock-down is for a full one hour. After that, you can start over.

Remember I said we expect the same simple behavior from the definition?

We have a single API, behaving differently each time. Plus, the result of that API call has a direct effect on the other APIs too. We definitely can’t use that Excel sheet’s simple yes/no test coverage method for that.

API Testing Is Not That Simple

An API can be a simple thing to test. Like a “Get user details” API will get you the user’s data. The documentation says so. And in many data-driven application, it can be that simple. But even then, there will be other cases: user is not found, database is unavailable, or a user that you’re not allowed to see.

When we’re test planning, we need to cover all kinds of functionalities, and one test just doesn’t cover it.

And once you conquere your fear of complexity, check out my “API Automation Testing” for developers and “API Testing with Postman” for testers. Complexity is nothing to fear.