Value Cognizant Test Case Prioritization for Regression Testing Using Integration Based Test Data

Abstract

Test case prioritization (TCP) techniques are beneficial in regression testing tasks as they allow us to find test cases of high priority based on test data and metrics of previous executions. Existing TCP approaches can be categorized as value-neutral or value-neutral approaches. The value-neutral approaches are those where the underlying assumption is that all test cases, requirements, and faults have the same severity. Value-based approaches are approaches where a particular value of test cases, requirements, and faults is calculated and used in the TCP process. However, the majority of available methods and approaches are value-neutral where we propose a requirements’ business value quantification framework. This framework is proposed in such a manner that the impact of individual developers’ experience or the involved engineers’ professional experience is minimal. We also consider test case features and aspects such that we do test case prioritization considering the business value of a requirement as well as the test case execution data concerning faults found and their severity. We validate our framework using a previously established framework evaluation method (KANO), TOPSIS, and a case study. We use previously established evaluation criteria while validating through case studies, such as Average Percentage of Faults Detected (APFD), and Normalized Average Percentage of Faults Detected (NAPFD) for performance evaluation. We, finally, use white-box test data and provide a mechanism through extracting elements from Message-Method paths through the system under test (SUT) so that we may allow testers and developers to consider whitebox test cases for TCP as well. We make use of previously published criteria of Average Percentage of Element Coverage (APEC) for coverage analysis which is a novel contribution to our investigation. We compare our results with industrial benchmark results and we outperform them.