scholarly journals An Effective Strategy to Build Up a Balanced Test Suite for Spectrum-Based Fault Localization

2016 ◽  
Vol 2016 ◽  
pp. 1-13
Author(s):  
Ning Li ◽  
Rui Wang ◽  
Yu-li Tian ◽  
Wei Zheng
Keyword(s):  
Fault Localization ◽  
Regression Testing ◽  
Test Suite ◽  
Test Case ◽  
Test Cases ◽  
Software Faults ◽  
Localization Accuracy ◽  
Software Debugging ◽  
Faults Diagnosis ◽  
Automated Software

During past decades, many automated software faults diagnosis techniques including Spectrum-Based Fault Localization (SBFL) have been proposed to improve the efficiency of software debugging activity. In the field of SBFL, suspiciousness calculation is closely related to the number of failed and passed test cases. Studies have shown that the ratio of the number of failed and passed test case has more significant impact on the accuracy of SBFL than the total number of test cases, and a balanced test suite is more beneficial to improving the accuracy of SBFL. Based on theoretical analysis, we proposed an PNF (Passed test cases, Not execute Faulty statement) strategy to reduce test suite and build up a more balanced one for SBFL, which can be used in regression testing. We evaluated the strategy making experiments using the Siemens program and Space program. Experiments indicated that our PNF strategy can be used to construct a new test suite effectively. Compared with the original test suite, the new one has smaller size (average 90% test case was reduced in experiments) and more balanced ratio of failed test cases to passed test cases, while it has the same statement coverage and fault localization accuracy.

Test Pair Selection for Test Case Prioritization in Regression Testing for WS-BPEL Programs

2013 ◽  
Vol 10 (1) ◽  
pp. 73-102 ◽  
Author(s):  
Lijun Mei ◽  
Yan Cai ◽  
Changjiang Jia ◽  
Bo Jiang ◽  
W.K. Chan
Keyword(s):  
Web Services ◽  
Structural Similarity ◽  
Regression Testing ◽  
Test Pair ◽  
Test Suite ◽  
Test Case ◽  
Test Cases ◽  
Test Case Prioritization ◽  
Empirical Results ◽  
Selection For

Many web services not only communicate through XML-based messages, but also may dynamically modify their behaviors by applying different interpretations on XML messages through updating the associated XML Schemas or XML-based interface specifications. Such artifacts are usually complex, allowing XML-based messages conforming to these specifications structurally complex. Testing should cost-effectively cover all scenarios. Test case prioritization is a dimension of regression testing that assures a program from unintended modifications by reordering the test cases within a test suite. However, many existing test case prioritization techniques for regression testing treat test cases of different complexity generically. In this paper, the authors exploit the insights on the structural similarity of XML-based artifacts between test cases in both static and dynamic dimensions, and propose a family of test case prioritization techniques that selects pairs of test case without replacement in turn. To the best of their knowledge, it is the first test case prioritization proposal that selects test case pairs for prioritization. The authors validate their techniques by a suite of benchmarks. The empirical results show that when incorporating all dimensions, some members of our technique family can be more effective than conventional coverage-based techniques.

scholarly journals K-Step Crossover Method based on Genetic Algorithm for Test Suite Prioritization in Regression Testing

2021 ◽  
Vol 27 (2) ◽  
pp. 170-189
Author(s):  
P. K. Gupta
Keyword(s):  
Genetic Algorithm ◽  
Regression Testing ◽  
Test Suite ◽  
Test Case ◽  
Test Cases ◽  
Test Case Prioritization ◽  
Average Percentage ◽  
Code Coverage ◽  
Test Case Selection ◽  
Case Selection

Software is an integration of numerous programming modules  (e.g., functions, procedures, legacy system, reusable components, etc.) tested and combined to build the entire module. However, some undesired faults may occur due to a change in modules while performing validation and verification. Retesting of entire software is a costly affair in terms of money and time. Therefore, to avoid retesting of entire software, regression testing is performed. In regression testing, an earlier created test suite is used to retest the software system's modified module. Regression Testing works in three manners; minimizing test cases, selecting test cases, and prioritizing test cases. In this paper, a two-phase algorithm has been proposed that considers test case selection and test case prioritization technique for performing regression testing on several modules ranging from a smaller line of codes to huge line codes of procedural language. A textual based differencing algorithm has been implemented for test case selection. Program statements modified between two modules are used for textual differencing and utilized to identify test cases that affect modified program statements. In the next step, test case prioritization is implemented by applying the Genetic Algorithm for code/condition coverage. Genetic operators: Crossover and Mutation have been applied over the initial population (i.e. test cases), taking code/condition coverage as fitness criterion to provide a prioritized test suite. Prioritization algorithm can be applied over both original and reduced test suite depending upon the test suite's size or the need for accuracy. In the obtained results, the efficiency of the prioritization algorithms has been analyzed by the Average Percentage of Code Coverage (APCC) and Average Percentage of Code Coverage with cost (APCCc). A comparison of the proposed approach is also done with the previously proposed methods and it is observed that APCC & APCCc values achieve higher percentage values faster in the case of the prioritized test suite in contrast to the non-prioritized test suite.

COST-COGNIZANT COMBINATORIAL TEST CASE PRIORITIZATION

2011 ◽  
Vol 21 (06) ◽  
pp. 829-854 ◽  
Author(s):  
ZIYUAN WANG ◽  
LIN CHEN ◽  
BAOWEN XU ◽  
YAN HUANG
Keyword(s):  
Physical Meaning ◽  
Resource Constraints ◽  
Regression Testing ◽  
Experimental Results ◽  
Test Suite ◽  
Test Case ◽  
Test Cases ◽  
Test Case Prioritization ◽  
Combinatorial Testing

Combinatorial testing has been widely used in practice. People usually assume all test cases in combinatorial test suite will run completely. However, in many scenarios where combinatorial testing is needed, for example the regression testing, the entire combinatorial test suite is not run completely as a result of test resource constraints. To improve the efficiency of testing, combinatorial test case prioritization technique is required. For the scenario of regression testing, this paper proposes a new cost-cognizant combinatorial test case prioritization technique, which takes both combination weights and test costs into account. Here we propose a series of metrics with physical meaning, which assess the combinatorial coverage efficiency of test suite, to guide the prioritization of combinatorial test cases. And two heuristic test case prioritization algorithms, which are based on total and additional techniques respectively, are utilized in our technique. Simulation experimental results illustrate some properties and advantages of proposed technique.

Hybrid Regression Testing Based on Path Pruning

2015 ◽  
Vol 5 (1) ◽  
pp. 35-55 ◽  
Author(s):  
Varun Gupta ◽  
Durg Singh Chauhan ◽  
Kamlesh Dutta
Keyword(s):  
Fault Detection ◽  
Regression Testing ◽  
Test Suite ◽  
Test Case ◽  
Test Cases ◽  
Testing Time ◽  
Ripple Effect ◽  
Web Based ◽  
Testing Approach ◽  
Globally Distributed

Regression testing has been studied by various researchers for developing and testing the quality of software. Regression testing aims at re-execution of evolved software code to ensure that no new errors had been introduced during the process of modification. Since re-execution of all test cases is not feasible, selecting manageable number of test cases to execute modified code with good fault detection rate is a problem. In past few years, various hybrid based regression testing approaches have been proposed and successfully employed for software testing, aiming at reduction in the number of test cases and higher fault detection capabilities. These techniques are based on sequence of selections, prioritizations and minimization of test suite. However, these techniques suffer from major drawbacks like improper consideration of control dependencies, neglection of unaffected fragments of code for testing purpose. Further, these techniques have been employed on hypothetical or simple programs with test suites of smaller size. Present paper proposes hybrid regression testing, a combination of test case selections, test case prioritizations and test suite minimization. The technique works at statement level and is based on finding the paths containing statements that affects or gets affected by the addition/deletion or modification (both control and data dependency) of variables in statements. The modification in the code may cause ripple effect thereby resulting into faulty execution of the code. The hybrid regression testing approach is aimed at detecting such faults with lesser number of test cases. Reduction in number of test cases is possible because of the decreased number of paths to be tested. A web based framework to automate and parallelize this testing technique to maximum extend, making it well suited for globally distributed environments is also proposed in the present paper. Framework when implemented as a tool can handle large pool of test cases and will make use of parallel MIMD architectures like multicore systems. Technique is applied on prototype live system and results are compared with recently proposed hybrid regression testing approach against parameters of interest. Obtained optimized results are indicators of effectiveness of approach in terms of reduction in effort, cost as well as testing time in general and increment delivery time in particular.

Test Suite Minimization in Regression Testing Using Hybrid Approach of ACO and GA

2021 ◽  
pp. 133-150
Author(s):  
Abhishek Pandey ◽  
Soumya Banerjee
Keyword(s):  
Hybrid Approach ◽  
Regression Testing ◽  
Minimum Time ◽  
Test Suite ◽  
Test Case ◽  
Test Cases ◽  
Test Case Prioritization ◽  
Ant Colony Optimization Algorithm ◽  
Test Case Selection

This article describes about the application of search-based techniques in regression testing and compares the performance of various search-based techniques for software testing. Test cases tend to increase exponentially as the software is modified. It is essential to remove redundant test cases from the existing test suite. Regression testing is very costly and must be performed in restricted ways to ensure the validity of the existing software. There exist different methods to improve the quality of test cases in terms of the number of faults covered, opposed to the number of statements covered in a minimum time. Different methods exist for this purpose, such as minimization, test case selection, and test case prioritization. In this article, search-based methods are applied to improve the quality of the test suite in order to select a minimum set of test cases which covers all the statements in a minimum time. The whole approach is named search based regression testing. In this paper, the performance of different metaheuristics for test suite minimization problem is also compared with a hybrid approach of ant colony optimization algorithm and genetic algorithm.

Test Suite Minimization in Regression Testing Using Hybrid Approach of ACO and GA

2018 ◽  
Vol 9 (3) ◽  
pp. 88-104
Author(s):  
Abhishek Pandey ◽  
Soumya Banerjee
Keyword(s):  
Hybrid Approach ◽  
Regression Testing ◽  
Minimum Time ◽  
Test Suite ◽  
Test Case ◽  
Test Cases ◽  
Ant Colony Optimization Algorithm ◽  
Test Case Selection ◽  
Case Selection

This article describes about the application of search-based techniques in regression testing and compares the performance of various search-based techniques for software testing. Test cases tend to increase exponentially as the software is modified. It is essential to remove redundant test cases from the existing test suite. Regression testing is very costly and must be performed in restricted ways to ensure the validity of the existing software. There exist different methods to improve the quality of test cases in terms of the number of faults covered, opposed to the number of statements covered in a minimum time. Different methods exist for this purpose, such as minimization, test case selection, and test case prioritization. In this article, search-based methods are applied to improve the quality of the test suite in order to select a minimum set of test cases which covers all the statements in a minimum time. The whole approach is named search based regression testing. In this paper, the performance of different metaheuristics for test suite minimization problem is also compared with a hybrid approach of ant colony optimization algorithm and genetic algorithm.

scholarly journals Using Global Constraints to Automate Regression Testing

AI Magazine ◽  
2017 ◽  
Vol 38 (1) ◽  
pp. 73-87
Author(s):  
Arnaud Gotlieb ◽  
Dusica Marijan
Keyword(s):  
Product Line ◽  
Regression Testing ◽  
Test Suite ◽  
Global Constraints ◽  
Test Case ◽  
Delivery Mode ◽  
Test Cases ◽  
High Quality ◽  
Test Suite Reduction ◽  
Test Requirements

Nowadays, any communicating or autonomous systems rely on high-quality software-based components. To ensure a sufficient level of quality, these components must be thoroughly verified before being released and being deployed in operational settings. Regression testing is a crucial verification process that executes any new release of a software-based component against previous versions of the component, with existing test cases. However, the selection of test cases in regression testing is challenging as the time available for testing is limited and some selection criteria must be respected. This problem, coined as Test Suite Reduction (TSR), is usually addressed by validation engineers through manual analysis or by using approximation techniques. Even if the underlying optimization problem is untractable in theory, solving it in practice is crucial when there are pressing needs to release high-quality components while at the same time reducing the time-to-market of new software releases. In this paper, we address the TSR problem with sound Artificial intelligence techniques such as Constraint Programming (CP) and global constraints. By associating each test case a cost-value aggregating distinct criteria, such as execution time, priority or importance due to the error-proneness of each test case, we propose several constraint optimization models to find a subset of test cases covering all the test requirements and optimizing the overall cost of selected test cases. Our models are based on a combination of NVALUE, GLOBALCARDINALITY, and SCALAR_PRODUCT, three well-known global constraints that can faithfully encode the coverage relation between test cases and test requirements. Our contribution includes the reuse of existing preprocessing rules to simplify the problem before solving it and the design of structure-aware heuristics, which take into account the notion of costs, associated with test cases. The work presented in this paper has been motivated by an industrial application in the communication domain. Our overall goal is to develop a constraint-based approach of test suite reduction that can be deployed to test a complete product line of conferencing systems in continuous delivery mode. By implementing this approach in a software prototype tool and experimentally evaluated it on both randomly generated instances and industrial instances, we hope to foster a quick adoption of the technology.

scholarly journals Model-Based Test Suite Generation Using Mutation Analysis for Fault Localization

2019 ◽  
Vol 9 (17) ◽  
pp. 3492 ◽  
Author(s):  
Choi ◽  
Lim
Keyword(s):  
Mutation Analysis ◽  
Fault Localization ◽  
Test Suite ◽  
Test Case Generation ◽  
Test Case ◽  
Test Cases ◽  
Generation Process ◽  
State Chart ◽  
Generation Cost ◽  
Test Suite Generation

Fault localization techniques reduce the effort required when debugging software, as revealed by previous test cases. However, many test cases are required to reduce the number of candidate fault locations. To overcome this disadvantage, various methods were proposed to reduce fault-localization costs by prioritizing test cases. However, because a sufficient number of test cases is required for prioritization, the test-case generation cost remains high. This paper proposes a test-case generation method using a state chart to reduce the number of test suites required for fault localization, minimizing the test-case generation and execution times. The test-suite generation process features two phases: fault-detection test-case generation and fault localization in the test cases. Each phase uses mutation analysis to evaluate test cases; the results are employed to improve the test cases according to the objectives of each phase, using genetic algorithms. We provide useful guidelines for application of a search-based mutational method to a state chart; we show that the proposed method improves fault-localization performance in the test-suite generation phase.

A Passed Test Case Cluster Method to Improve Fault Localization

2020 ◽  
pp. 2150053
Author(s):  
Weibo Wang ◽  
Yonghao Wu ◽  
Yong Liu
Keyword(s):  
Negative Impact ◽  
Empirical Studies ◽  
Fault Localization ◽  
Recall Rate ◽  
Cluster Method ◽  
Test Case ◽  
Test Cases ◽  
Localization Accuracy ◽  
Final Output ◽  
Negative Impacts

Coverage-Based Fault Localization (CBFL) techniques are based on the conjecture that the program elements executed by more failed test cases and less passed test cases have more chance to be faulty. Coincidental Correct (CC) test case is one of the negative impacts on CBFL, for the reason that the CC test cases execute the faulty element but not propagate the faulty status to the final output. To alleviate the negative impact of CC test cases, in this paper, we propose a cluster-based technique to identify CC test cases from the passed test suite. To evaluate the effectiveness of our method, we conduct empirical studies on 102 versions from six programs. The experimental results show that, when using our method, it can accurately recognize CC test cases, where the precision and recall rate are both higher than 85%. A further study shows that, after removing identified CC test cases, the fault localization accuracy of SBFL can be improved apparently.

scholarly journals Swarm Intelligence Techniques and Genetic Algorithms for Test Case Prioritization

2020 ◽  
Vol 9 (4) ◽  
pp. 465-469
Keyword(s):  
Genetic Algorithms ◽  
Swarm Intelligence ◽  
Regression Testing ◽  
Test Suite ◽  
Test Case ◽  
Test Cases ◽  
Random Testing ◽  
Development Life Cycle ◽  
Software Development Life Cycle ◽  
Time Required

Regression testing is a technique which is carried out to ascertain that the changes that were done in the source code have not negatively damped its performance. Hence, it is a crucial and an expensive step of the software development life cycle. It re-establishes confidence in correctness of the software after changes were made to it. A test suite is used to test the software, but often it becomes time consuming to re-execute each test case every time regression testing is done. Therefore, it becomes essential to decrease the number of the test cases by prioritizing them based on some criterion. This ensures maximum detection of faults in least amount of time. In this paper, author has compared swarm intelligence techniques with genetic algorithms for such a test suite prioritization. In particular, by taking a sample GCD program Ant Colony Optimization (ACO) has been compared with Genetic Algorithms (GA) for the purpose of test suite minimization. Unit of comparison has been execution time required for prioritization of test cases. Further, experimental results have been compared with time taken by both with random testing.

Sign in / Sign up

Export Citation Format

Share Document