Masanori Maruyama

General Test Criteria

The scope of the test specification is unit test, integration test, and qualification test. These specifications should include the following criteria.

G.1: definition of test environment

G.2: definition of test procedure

G.3: definition of test case

• Note: The selection of test cases shall have sufficient coverage according to e.g. the software integration test strategy and the release plan.

G.4: definition of verification criteria

• Note: Verification criteria define the qualitative and quantitative criteria for verification of a requirement.

Unit Test Criteria

Create software unit test specification, and test the software units according to the unit test specification.

The unit test specification shall be suitable to provide evidence for compliance of the software units with the software detailed design and with the non-functional requirements.

U.1: Unit test specification should include the following criteria.

U.1.1: unit test cases

U.1.2: unit test data

U.1.3: static verification

U.1.4: coverage goals

U.1.5: coding standards (e.g. MISRA rules)

U.2: Verify that software unit conforms to the software detailed design based on the following points.

U.2.1: input data format 

U.2.2: output data format 

U.2.3: prototype

U.2.4: flow chart

U.2.5: sequence chart

U.2.6: entity relationship diagram

U.2.7: pseudo code

etc.

Integration Test Criteria

Create software integration test specification, and test the integrated software according to the integration test specification.

The software integration test specification shall be suitable to provide evidence for compliance of the integrated software items with the software architectural design.

Note: Compliance to the architectural design means that the specified integration tests are suitable to prove that the interfaces between the software units and between the software items fulfill the specification given by the software architectural design.

I.1: Verify that software conforms to the the software architectural design based on the following points.

I.1.1: inter-task/inter-process communication

I.1.2: relationship and dependency between software elements

I.1.3: interrupts with their priorities

I.1.4: tasks with cycle time and priority

I.1.5: relationship between software elements

I.1.6: where the data (such as parameters) are stored and which measures (e.g. checksums, redundancy) are taken to prevent data corruption

I.1.7: which data is persistent and under which conditions

I.2: Verify that the dynamic behavior of the software conforms to the software architectural design based on the following points.

I.2.1: start-up

I.2.2: shutdown

I.2.3: software update

I.2.4: error handling

I.2.5: recovery

etc.

I.3: Verify that software conforms to the resource consumption characteristics based on the following points.

I.3.1: ROM Usage

I.3.2: RAM Usage

I.3.3: EEPROM Usage

I.3.4: CPU Usage

I.3.5: GPU Usage

etc.

Qualification

Create specification for software qualification test, and test the release software package according to the qualification test specification.

The test specification shall be suitable to provide evidence for compliance of the integrated software with the software requirements.

Q.1: Verify that software conforms to the the requirement specification based on the following points.

Q.1.1: relationship between the system elements and the software

Q.1.2: how variants for different model series or configurations are derived

Q.1.3: any database design requirements

Q.1.4: any required error handling and recovery attributes

Q.1.5: any reliability requirements

Q.1.6: any security requirements

Q.1.7: any diagnosis requirements

End