What is Code Testing

Theoretical introduction of testing: what makes code reliable and professional

Contents of the Article

1. Introduction
2. Testing Approaches
3. Functional Tests
4. Structural Tests

Introduction

In professional software engineering, the creation of a program is done following some given specifications that indicate what the program should do. While the development is going on, it’s critical to check the correctness of the code it’s being written, to be sure the program will do exactly what the specifications say.

The most affordable technique is called “Formal Verification”: it’s done providing a formal proof of correctness as the result of the application of an abstract mathematical model of the program. Due to its complexity, it’s used only in high risk contexts.

On the contrary the most used technique is the so called “Testing”.

Testing and the linearity issue

Testing consists in executing a given program choosing some input datas and evaluating the correctness of the results.

In many engineering fields, when it comes to testing, it’s applied the “Linearity property”. This means that if an elevator is tested with 1000kg, and it is working properly, that means it will be properly working for all the lower weights and so there is no reason to test further more. In computer science errors cannot be considered linear, so we have to test every situation before stating that the system is certainly correct.

Hence, a test of a snippet of code is considered exhaustive only if it checks all the possible data inputs.

Unfortunately in real applications the possible combinations of inputs are so broad that the only thing that is feasible in a reasonable amount of time is to maximize the possibilty of detecting errors.

Program testing can be used to show the presence of bugs, but never to show their absence

— Dijkstra 1972

Testing Approaches

Random Tests

The data input is chosen randomly. This allows the tester to discover a program’s area or an input category that was ignoring and that could be cause of errors in the future.

Systematic Tests

The tester analyzes the input datas domain in order to find out which are the main critical issues that could most likely create problems.

There are two main categories of systematic tests you may take:

• Functional Tests: based on program specifications.
• Structural Tests: based on program implementation.

Functional Tests

Also called “Black Box Tests”, Functional Tests are:

• Simple to perform (Pro!)
• Do not require to know the code in question. They can therefore be performed by third parties or by the customer himself. (Pro!)
• It requires a specification that is as complete and exhaustive. (Cons!)

How to perform a functional test:

1. Identify the main software features
2. For each main feature identify the elementary features of parameters and referenced elements. For example a file (referenced element) can be empty or may not exist (elementary features).
3. For each elementary feature, classify the values ​​that can be assumed depending on whether they are normal, boundary, special or incorrect values. Particular attention must be paid to limit values. Below an example: if you pass a negative number as a parameter of processNumber() function, you will enter an infinite loop.

For this particular example, there are two solutions:

• Contract programming paradigm: the development team decides that the caller of processNumber will never pass as argument a negative number.
• Defensive programming paradigm: the method is enriched or with an instruction that compute the absolute value of number, otherwise an exception is thrown in case number is negative.

Structural Tests

Also called “White Box Tests”, the goal here is to verify each instruction of the code, therefore as a tester you have to create a set of tests which is broad enough to cover the whole program.

Generally speaking, structural tests are:

• Not always able to verify the entire code due to unreachable code. (Cons!)
• Generally more affordable than functional tests. (Pro!)
• Require complete knowledge of source code. (Cons!)

How to perform a structural test:

A structural test is performed checking the correctness of the code that is part of the analyzed system. Therefore, you don’t check only that a function returns correct answers, but you try to check the correctness of the code flows implementing it.

1. Structural Tests — Statement Coverage Approach

The hard task here is to build a set of tests that are able to test every instruction at least once. To achieve this, there are four different approaches you can follow, as described in the following paragraphs.

The amount of checked code is called covered code. With reference to the total amount of code, you can describe the coverage as the ratio of covered code over total code. In real world cases, it’s often hard to reach 100% of coverage: usually covering at least the 90% of code is considered fine.

Sometimes it can happen that after writing a complete set of tests, you spot an instruction that is never executed.

• Case #1: The instruction never executes because it’ss not covered by the whole tests set. In this case you should find a path that leads to the execution of such instruction, and build a test on top of that. Unfortunately this task is a non-deterministic problem, so it is not said that you can find the right path automatically: you may end up checking by hand going back through the instructions trying the various possible paths.
• Case #2: The instruction never executes because it’s in a “dead” part of the code, meaning there is no actual login flow that brings there. This situations are critical and must be inspected carefully.

2. Structural Tests — Edge Coverage Approach

Make sure that each logic branch of the code is covered.

For example if you have such a condition: “A and B”, you should test the behaviour of the code in case the condition gets true and in case it gets false. To do this actually, being an and condition, it would be enough testing when A is false (no matter B) and when A and B are true.

3. Structural Tests — Condition Coverage Approach

Make sure that each condition is covered. For example if you have this condition: “A and B” , you have to create test cases where at least one time you have A = true, A = false, B = true, B = false.

Important Notice: the && and || are short-circuit operators, meaning they don’t evaluate the right hand side if it isn’t necessary. This fact cause sometimes to consider a variable as “not-evaluated” that for the purposes of testing does not cover true or false. It is a “wasted” case for the not-evaluated variable, but sometimes it is unavoidable.

4. Structural Tests — Path Coverage Approach

The goal is to examinate all the possible path that can be followed during the code execution. In practice it is almost impossible to achieve. For example if you have a loop cycling 100 times that contains a call to a function that does not depend on the loop indexes and that can return in two different ways, you must test all the possible path that are 2 ^ 100 cases.