Software Test Matrix

What is Software Test Matrix?

It is used to convey progress of our project status report.A test matrix is simply a spreadsheet that suggests test and captures test results by laying them out in the form of a table. In other words it is a measurement of the work we have done in the project to improve our process and check where we are lagging.

What is use of Software Test Matrix?

A test matrix can be used for a number of purposes:

• to record a consistent set of tests;
• to document a desired level of test coverage based on an intersection of two criteria and, when filled out, to be used as evidence that the desired amount of testing has been done.
• as a quick visual indication of how much testing has been done on those criteria.
• to help to identify and refine the specific environments in which a defect occurs.
• When it is completely filled out, as a checklist to assure that some exhaustive task has been completed.

Test Process and Test Product Metrics

QA recognize two subsets of Software Testing Metrics:

 Test Process Metrics

These measures provide information about preparation for testing, test execution and test progress. They don’t provide information about the test state of the product and are primarily of use in measuring  progress of the Test Phase.

 Test Product Metrics

These measures provide information about the test state of the product and are generated by test  execution and code fixes or deferment. Using these metrics we can gauge the products test state and  indicative level of quality, useful for product release decisions. 

What are Test factors to be considered?

Test Matrix is also called as Test responsibility matrix. It consists of Test Factors like--

·         correctness

·         compliance

·         coupling

·         continuity of process

·         authorization

·         access control

·         audit check

·         performance

·         portability

·         Ease of use

·         Ease of operation

·         Maintainability

·         Reliability

·         File integrity

·         secure levels

To create a test matrix we have to:

·         Put the objects that you’re testing on the rows.

·         Show the tests on the columns.

·         Check off the tests that you actually completed in the cells.

Terminology used in Software Test Metrix:

·         Test Procedure coverage: No. of test procedures Vs total number of test procedures execution

·         Error Density: Total number of errors Vs no. of test procedures execution.

·         Test Case coverage: Total number of test cases Vs no. of test cases execution and no. of test cases passed.

·         Defects:  Severity and Priority, time to find defect, time to fix defect , turnaround time to defect.

What is Traceability Matrix?

·         The Requirements Traceability Matrix (RTM) captures the complete user and system requirements for the system, or a portion of the system. The RTM captures all requirements and their traceability in a single document, and is a mandatory deliverable at the conclusion of the lifecycle.

·         The RTM is used to record the relationship of the requirements to the design, development, testing and release of the software as the requirements are allocated to a specific release of the software. Changes to the requirements are also recorded and tracked in the RTM. The RTM is maintained throughout the lifecycle of the release, and is reviewed and baselined at the end of the release.
It is very useful document to track Time, Change Management and Risk Management in the Software Development.

The RTM Template shows the Mapping between the actual Requirement and User Requirement/System Requirement.
Any changes that happens after the system has been built we can trace the impact of the change on the Application through RTM Matrix. This is also the mapping between actual Requirement and Design Specification. This helps us in tracing the changes that may happen with respect to the Design Document during the Development process of the application. Here we will give specific Document unique ID, which is associated with that particular requirement to easily trace that particular document.

Instructions for Requirements Traceability Matrix Template.

Requirements traceability matrix (RTM) provides traceability between the [workspace/workgroup] approved requirements, design specifications, and test scripts.
The table below displays the RTM for the requirements that were approved for inclusion in [Application Name/Version]. The following information is provided for each requirement:
1. Requirement ID
2. Risks
3. Requirement Type (User or System)
4. Requirement Description
5. Trace to User Requirement/Trace From System Requirement
6. Trace to Design Specification
7. UT * Unit Test Cases
8. IT * Integration Test Cases
9. ST * System Test Cases
10. UAT * User Acceptance Test Cases
11. Trace to Test Script

 Disadvantages of not using Traceability Matrix

What happens if the Traceability factor is not considered while developing the software?
a) The system that is built may not have the necessary functionality to meet the customers and users needs and expectations
b) If there are modifications in the design specifications, there is no means of tracking the changes
c) If there is no mapping of test cases to the requirements, it may result in missing a major defect in the system
d) The completed system may have Extra functionality that may have not been specified in the design specification , resulting in wastage of manpower, time and effort.
e) If the code component that constitutes the customers high priority requirements is not known, then the areas that need to be worked first may not be known thereby decreasing the chances of shipping a useful product on schedule
f) A seemingly simple request might involve changes to several parts of the system and if proper Traceability process is not followed, the evaluation of the work that may be needed to satisfy the request may not be correctly evaluated