Testing State Machines with SCTUnit

This example demonstrates how to write tests for statecharts using SCTUnit. It demonstrates how to write tests effectively, how to run them and use a coverage analysis to check if the model has been tested completely.

Example Application

As an example application we will use the light switch example with brightness adjustment from the Basic Tutorial.

The light switch model simply consists of two states, On and Off, and respectively two incoming events on_button and off_button. Each time the event on is received, the brightness is increased until a maximum brightness is reached. The maximum of the brightness will be calculated by the operation computeMaxBrightness(), which returns an integer. This operation will be mocked later on in the test. Additionally, the light will automatically turn off after 30 seconds. Thus, all testable modeling components are used:

• In Events
• Out Events
• Timed Events
• Variables
• Operations

Test Strategy

There are different ways of how to write tests. For this example series we are using the Given-When-Then (GWT) way to write the tests:

• "The given part describes the state of the world before you begin the behavior you're specifying in this scenario. You can think of it as the pre-conditions to the test."
• "The when section is that behavior that you're specifying."
• "Finally the then section describes the changes you expect due to the specified behavior."

See: https://martinfowler.com/bliki/GivenWhenThen.html

Let's transfer this to our example, where we want to test the behavior of the model. For example we want to test if the light will be turned off while it is on and the off button is pressed:

1. Given is a predefined state machine. In this case, the statechart shall be in theOn state.
2. When pressing the off button by raising the off_button event...
3. Then the state machine should be in the Off State. So we assert this.

Using this way of testing, we can test the model step by step in small, separated pieces. Thus, later extensions of the model can be adapted easily while ensuring that the functionalities, which may be mapped to requirements, are fullfilled.

SCTUnit Test

The SCTUnit test itself contains three different parts. The main part obviously are the tests, which are operations annotated with @Test. Additionally, there are other operations, which do two things:

• Set the state machine to a predefined state (Given).
• A pressUserButton operation, to raise events (When).
• Helper functions that do the assertions (Then).

The test can be executed with Right click -> Run As -> SCT Unit. Further informations on how to use the SCTUnit language can be found in our documentation.

Test implementation

This is the complete test implementation:

Coverage

Itemis CREATE supports another feature to improve the testing quality: Coverage. Each time a test is executed, a coverage analysis is started. The coverage score indicates how many of the state machine paths have been tested.

Let's take the LightSwitchTest and remove every test except for the very first one - testInitiallyTurnedOff. Executing the test will also run a coverage analysis, as mentioned before. The result of the coverage analysis is displayed in the coverage view: As only the initial state has been entered, the On state is not covered at all by the test. The Off state is only covered in parts, as no outgoing transition is taken. The coverage state is visualized in the model when you select an element from the coverage view:

Adding more and more test cases will increase the coverage up to 100%.