scholarly journals Can fault-exposure-potential estimates improve the fault detection abilities of test suites?

2002 ◽  
Vol 12 (4) ◽  
pp. 197-218 ◽  
Author(s):  
Wei Chen ◽  
Roland H. Untch ◽  
Gregg Rothermel ◽  
Sebastian Elbaum ◽  
Jeffery von Ronne
Keyword(s):  
Fault Detection ◽  
Exposure Potential ◽  
Test Suites

scholarly journals A Tester-Assisted Methodology for Test Redundancy Detection

2010 ◽  
Vol 2010 ◽  
pp. 1-13 ◽  
Author(s):  
Negar Koochakzadeh ◽  
Vahid Garousi
Keyword(s):  
Fault Detection ◽  
False Positive ◽  
Test Suite ◽  
Test Case ◽  
Maintenance Costs ◽  
Detection Ability ◽  
Analysis Technique ◽  
Inspection Process ◽  
Redundancy Detection ◽  
Test Suites

Test redundancy detection reduces test maintenance costs and also ensures the integrity of test suites. One of the most widely used approaches for this purpose is based on coverage information. In a recent work, we have shown that although this information can be useful in detecting redundant tests, it may suffer from large number of false-positive errors, that is, a test case being identified as redundant while it is really not. In this paper, we propose a semiautomated methodology to derive a reduced test suite from a given test suite, while keeping the fault detection effectiveness unchanged. To evaluate the methodology, we apply the mutation analysis technique to measure the fault detection effectiveness of the reduced test suite of a real Java project. The results confirm that the proposed manual interactive inspection process leads to a reduced test suite with the same fault detection ability as the original test suite.

A Revisit of Metrics for Test Case Prioritization Problems

2020 ◽  
Vol 30 (08) ◽  
pp. 1139-1167
Author(s):  
Ziyuan Wang ◽  
Chunrong Fang ◽  
Lin Chen ◽  
Zhiyi Zhang
Keyword(s):  
Fault Detection ◽  
Detection Efficiency ◽  
Formal Analysis ◽  
Weighted Average ◽  
Test Case ◽  
Test Cases ◽  
Test Case Prioritization ◽  
Relative Cost ◽  
Parallel Testing ◽  
Test Suites

For the test case prioritization problems, the average percent of faults detected (APFD) and its variant versions are widely used as metrics to evaluate prioritized test suite’s efficiency of fault detection. By a revisit of metrics for test case prioritization, we observe that APFD is only available for the scenarios where all test suites under evaluation contain the same number of test cases. Such a limitation is often overlooked, and lead to incorrect results when comparing fault detection efficiency of test suites with different sizes. Moreover, APFD cannot precisely illustrate the process of fault detection in the real world. Besides the APFD, most of its variants, including the NAPFD and the APFD[Formula: see text], have similar problems. This paper points out these limitations in detail by analyzing the physical explanation of APFD series metrics formally. In order to eliminate these limitations, we propose a series of improved metrics, including the relative average percent of faults detected (RAPFD) and the relative cost-cognizant weighted average percent of faults detected (RAPFD[Formula: see text]), to evaluate the efficiency of the test suite. Furthermore, for the scenario of parallel testing, a series of metrics including the relative average percent of faults detected in parallel testing ([Formula: see text]-RAPFD) and the relative cost-cognizant weighted average percent of faults detected in parallel testing ([Formula: see text]-RAPFD[Formula: see text]) are proposed too. All the proposed metrics refer to both the speed of fault detection and the constraint of the testing resource. A formal analysis and some examples show that all the proposed metrics provide much more precise illustrations of the fault detection process.

Sign in / Sign up

Export Citation Format

Share Document