Applications Of Value-Centric Regression Testing For Software Applications

Abstract

In the current era, businesses are Information Technology (IT) reliant, but most companies are deteriorating to maximize the value of their IT initiatives to their businesses. IT professionals do not know the value of distinct software features to the business. Likewise, they do not know the business value of diverse software quality attributes to the business. Therefore, they prioritize their project tasks based on their perceptions without considering formally measured business value. Ignoring value in software processes, practices, and artifacts is a value-neutral approach. In regression testing, software testers cannot re-execute all the test cases to find out the ripple effects of the changes due to time and budget constraints. No company can afford exhaustive regression testing in rapidly growing applications. Therefore, software testing professionals need a way through which they can prioritize their test cases for regression testing to uncover maximum bugs and side effects by utilizing minimum time and cost. Test Case Prioritization (TCP) is one of the processes to address this challenge. TCP is a smart way for regression testing to handle testing resource constraints. The main advantage of TCP is to save time through the prioritization of critical tests earlier. Current TCP techniques can be categorized as Value-Neutral (VN) and Value-Based (VB) approaches. In a VB approach, the cost of test cases and severity of faults are considered while, in a VN approach these are not considered. The VN approach is dominant over VB approach, and it assumes that all test cases have identical costs and that all software faults have same severity. But this notion seldom holds in practice. Therefore, VN TCP techniques are likely to deliver unreliable results. To fill this gap, focus should be shifted from VN to VB test prioritization. Presently, limited research work is done in a VB approach. To address this issue, a Systematic Literature Review (SLR) of VB TCP techniques is performed, and its results are presented in this thesis. Its purpose is to combine the overall knowledge related to VB TCP techniques and to highlight some open research issues in this domain. The literature review yields that value-orientation is vital in the TCP process to achieve its targeted goals and this is a potential area for further research. Many TCP techniques are available, and their performance is usually measured through a metric Average Percentage of Fault Detection (APFD). This metric is value-neutral because it only works well when all test cases have identical costs, and all faults have equal severity. Using APFD for performance evaluation of test case orders where test case cost or fault severity varies is prone to produce false results. Therefore, using the right metric for performance evaluation of TCP techniques is very important to get reliable and correct results. To the best of the author’s knowledge, there is no formal technique available to quantify business value based on which test cases can be prioritized. To overcome this problem, a business value quantification model has been proposed in this work to estimate fault severities and test case cost. The proposed model supports the business value measurement of software requirements. We use the term software features as functional requirements and software quality attributes as nonfunctional requirements. The business value calculation of software features and quality attributes is based on three factors client priority, feature complexity, and feature usage. To compute the value of client priority, the proposed model utilizes five business success factors including profitability, productivity, operational efficiency, customer satisfaction, and time to market. Software fault severity and test case cost are estimated through the business value of requirements because different test cases and faults are directly associated with some requirements. Business value has been incorporated into the TCP process through the proposed model. The model is validated through two working examples. Based on the proposed model, two value-based TCP techniques have been introduced in this thesis using Genetic Algorithms (GA). These techniques are Value-Cognizant Fault Detection-Based TCP (VCFDB-TCP) and Value-Cognizant Requirements Coverage-Based TCP (VCRCB-TCP). Two novel value-based performance evaluation metrics are also introduced for value-based TCP including the APFDv and Average Percentage of Requirements Coverage per value (APRCv). Two case studies are performed to validate proposed techniques and performance evaluation metrics quantitatively. A statistical analysis of the results is performed by a statistical test. The statistical results reveal that the proposed approaches provide significantly better results than traditional value-neutral TCP techniques.