Home > Programming > Model based testing

Model based testing

What is the best way to test software?

Delivering quality code and keeping customers happy is the ultimate goal for any software development effort. Common problems people complain about software are that it does not do what they expect it to. Lot of times, particularly in R&D environments the specs are not clear and there are 2-3 iterations before the customers and developers agree that they have a working solution. While ambiguous specs do lead to some kind of re-work, in my personal experience developers do spend time fixing issues reported by the customer. Instead of adding new features, time is spent on fixing bugs in a previous release. If most of the bugs were to be found before the release, there is a good chance of project success as the customers get a stable ‘working’ build.

Software testing plays an important role in ensuring that the application released to customers does what it is supposed to do and without any errors. The definition for testing that I like most is “software testing is running the code in a controlled environment with the intent of finding errors”.  Automated testing has come a long way in ensuring that a software product is tested consistently through its various releases. Through automated testing it is possible to excercise so many scenarios that are impossible to be executed manually within a limtied time and with limited resources.

Helpful as it is, traditional automated testing suffers from the pesticide paradox. Just like how a pesticide can kill only a particular type of pests and how useless it becomes once those bugs have been killed, the throughput from automated testing drops drastically over time. Automated tests are still good for smoke tests and regression testing where you are interested to find if some core functionality is working or if old functionality remained unbroken.

However one of the bigger challenges for product release is to extensively test the software. All the functionalities have to be tested, all the code in the application has to be excercised with various inputs to ensure that the program behavior is verified in all its possible states.

This verification of program behavior in all its possible states is quite difficult with traditional testing techniques.

Here comes Model Based Testing.

In model based testing, a behavior model of the software is constructed that describes all the possible functionalities of the system and the manner in which the software system transitions from one state to another. A typical tool used to represent such behavior models are the finite state machines.

A typical model based testing framework consists of (or is developed as):

1. Create or define a model of the system under test. Examples of models are state transition table or the finite state machines.

2. A separate component then uses the model to generate test cases. From the state transition table or the FSM a directed graph can be built with the different states of the system as nodes and the transitions as the edges. Using graph traversal techniques it is possible to determine a sequence of input actions that leads the software to a particular state. The sequence of input actions form the test case and the final state is the excpected result.

3. As the software evolves, only the model needs to be updated with any new states and transitions. The test suite can then be updated automatically. Another advantage of this methodology is that the expected behavior (test oracle) is also built into the scheme.

An excellent and comprehensive reference to MBT is available at www.model-based-testing.org and www.geocities.com/harry_robinson_testing. This is maintained by Googler Harry Robinson.

In future posts, I will share examples of creating behavior models and also blog about creating a model based testing framework.

Advertisements
Categories: Programming Tags: ,
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: