Software testing

Let’s talk about software testing.
There has been a movement toward quality improvement of software since the last years.
One result of this movement is Test Driven Development. For those that don’t know it, in short you first write automated tests, before you initially write the code of your application.

Note that there are also other concepts, which let you write the software test before the code and other methodology, which tries to improve quality with software testing aspects.

There are several reasons why to write tests first, but the most important is that this will always make you write a test ;-)
It’s a common statement that the developer should not test its own code. This is because it’s said that, if you write the test for your own code, you remember how how it works and will align the software testing along with it.

Software testing the smart way

 

Relocation of the problem

Basically this was not a bad though, I’m sure.

When you write the software tests before the code, you will align your code that it always passes the test. In this case the risk is transferred from the code to the test, as there is no guarantee that your test is very meaningful.

Imagine you want an application multiplying two numbers. You write a test that passes 3 and 4 to be multiplied, which results in 12. The test is successful and you’re done, right? But what, if the result is always 12, independent on what numbers you pass?

Everybody will burn me on the stake now, “this is a very stupid example”. Yes it is a very stupid example. But if you read the meaning of Test Driven Development, it says “you do the less effort possible to pass the test”. And as you know the test – most likely it was you that wrote it – you know that 3 and 4 are passed and therefore the result must be 12. So the less effort possible is to return 12 and the test is successful.

Still stupid, I know. With my example I don’t want to say that software developers are silly. I want to show that this method does not necessarily lead to quality improvement, when the tests are focused wrong.

To summarize it, when code is written in front of the tests, the risk is to align the tests, so they always pass. When writing the tests first, you align the code, so that the tests are always successful.
The result remains almost the same.

Customer orientation

I mentioned quality improvement. What exactly means quality in respect of software testing? It means that the customer gets what he pays for (functional requirements) and the application supports some “soft quality goals”, such as usability (non-functional requirements). Ok non-functional requirements are sometimes hard to verify with software testing.

If you have scenarios for use cases, you should take these scenarios as a reference to state, whether or not you fulfill the requirements.
How you do that? Instead of testing isolated functions or classes, you should start implementing the scenarios with your automated tests. This also includes fail paths, which may result in an error state.
Sometimes it makes sense to implement more than one unit test per scenario (this normally happens, when the scenarios are not that detailed).
If you now ask “what the hell is that guy talking about”, you should really start inquire Google very quick!

Of course it can make sense to split tests to a deeper level.
Let’s say you implemented any kind of client, e.g. an FTP client. You built the whole client on your own (I mean you didn’t buy or download one), then it may be useful, to test the client standalone first. But this is just because you didn’t split your application into modules. The FTP client is something that could be a value to a customer by itself (refer to the onion principle).

Customer value

If you ask “what the hell is wrong with you, everybody says that tests should be as simple as possible, not to have errors in it?”, let me explain.

If you structure your application well and keep it simple, you normally have just small points of contact.
Let’s say as a customer, to create a new record in the database, it’s just required to fill a few fields and then press the save button – which is by the way a complete scenario.
Having a web service, which takes the data, the test would be very simple right? To validate the result, you may call the web service that returns the stored data. Otherwise you may also check the data stored within the database directly.

Remember that the customer does not pay for what you do inside the code. The customer pays for the behavior of the website, when he enters the data and presses the save button.

Let’s say you encrypt the data on client side and decrypt it on server side.

Now there is a bug and you again encrypt instead of decrypt it on server side. But when you do the same in reverse order, when returning the data, the customer will not see any difference.
In that case you pass the software tests, as the requirements are fulfilled.

Fulfilled Requirements

The customer pays to get the same data returned, when reading the record back. Nothing else.
He doesn’t want to mind about encoding, encryption, encreepment, embedded Linux on your grandmothers teeth brush or what so ever. This is your job! This is why you build this application instead of him.

Of course you may say that encryption is important for the customer. But it is not the desire. The desire is called security of data. How you ensure that, is your problem.
Maybe there is also a requirement that says “the data must not be read by rogue”. So you build a scenario for that and then build a test. Voilà, security guaranteed, requirement fulfilled.

What this is called in short is, focus on customer values. The customer never orders functions. He orders values, behaviors, solutions.

Software testing with the right focus

If this was way too fast to understand, let me summarize it.
I don’t want to convince you, not to use test driven development. In fact for me it doesn’t matter which tool you use for software testing. It’s not important, whether you build the tests before or after the code.
Important is that you focus on the customers value. What is the real desire of the customer and what does he pay for?

If you expect the application to be changed or extended in future and think you need to ensure some interface definition inside the code, just feel free to do it. I don’t say that it never makes sense to create tests other than just for scenarios.
But try to focus on the real desires, when checking your application for correctness.

Leave a Reply

Your email address will not be published. Required fields are marked *