How to test the UI (like JavaFX )?

Testing UI is hard. Or is it?

It really depends on what we’re talking about when we’re saying “UI testing”.

This question came up recently in the “Unit Wars” webinar. And I think that it requires a deeper answer.

I remember writing Windows apps in C++, calling CreateWindow APIs, keeping score of window handles. I was down in the mine, kids.

Life’s much better today.

Using frameworks cuts down on coding time, coding boiler plate, and testing. We’re relying on the frameworks to do their job. And when we ask “well, does it work?”, we mean different things.

In ye olden days, it meant “is my code working”. But now, there’s the framework code, AND my code. How can I tell if my app works?

Let’s take an example – our UI code calls an API, gets a result, modifies it and renders those results in the UI.

Is the framework code working?

We assume yes. I’ve seen bugs and glitches. But statistically, the chances that you will find a bug in a framework are slim. In fact, chances are better that if you found a bug, it was already reported, and fixed.

So when we call the API, through framework calls, we assume the call to the API works. And when we pass something to the rendering engine, we assume the engine works correctly. Are we using them correctly? That’s another question.

Is my code working?

That’s where unit tests and TDD come in. If we test our logic, we’ll know our parts works. It doesn’t guarantee a good integration, but it’s a start.

What is “our code” in our scenario? “Our code” is calling the API, and returning the modified result. But like we said, API calls and graphics are handled by someone else.

So the next question is: What is the “interesting part” of our code in? That’s the API’s result modification. So we’ll concentrate on that piece of code in our unit tests.

Is my code easy to unit test?

Is it even possible to unit test? If that code is separated from the framework calls, it  definitely makes life easier. If it’s embedded or mixed, it may not be even runnable. This is what we mean by “testable code”. So in order to test it easily, the code needs to be written in a testable manner.

Right, so we don’t test frameworks, and we do test the logic. Is that enough? Probably not.

Is the integration working?

This is what the user sees, and the user doesn’t care if I have a 1000 unit tests passing. If the integration doesn’t work, the code doesn’t work.

So, we’d want to test that as well. And that takes time & effort & money. These automated tests are not easy to write or maintain, or debug. And they are brittle. And they can fail on what we don’t expect them to fail. But when they do pass, we’re much more confident that the code is working.

We need to prioritize what we check, since that wouldn’t be cheap too.
Let’s say our code’s result renders into a chart. How do we check that it displays correctly? At the pixel level? Do we allow a spectrum of space or color to consider the scenario a success? Do we interrogate the chart that it was built correctly? (if that’s even possible?)

When we talk about the integration tests, we need to consider our tools, the testability features of the framework, and the durability and effectiveness of our tests. And of course, taking skills of the team members into account.

Out of all the scenarios we can choose from, we need to pick the ones we want. We’d like to pick those that give us as much sleep at night, at the cheapest rate.

What was the question again?

The original question was how to tests a JavaFX app. That’s a lot of answer. It is not just technical. It’s about how you approach testing.

The more “testable” the application – logic separated, framework code segregated – it would be easier to unit test the app logic, and conversely rely less on automation of the UI. If all the code is jumbled together, it’s hard to unit test, and you’re left with integration tests, which is hard to do. And you’ll never get the coverage you want, and that means more bugs, less sleep.

Then there are cases that you’re rendering something special, that’s heavy work. This kind of testability needs to be baked into the system.

Testing UI code is not easy. It can be easier with careful planning, and following good principles. But it starts with thinking about what we expect from that testing.