Parallel Test Prioritization

Although regression testing is important to guarantee the software quality in software evolution, it suffers from the widely known cost problem. To address this problem, existing researchers made dedicated efforts on test prioritization, which optimizes the execution order of tests to detect faults earlier; while practitioners in industry leveraged more computing resources to save the time cost of regression testing. By combining these two orthogonal solutions, in this article, we define the problem of parallel test prioritization, which is to conduct test prioritization in the scenario of parallel test execution to reduce the cost of regression testing. Different from traditional sequential test prioritization, parallel test prioritization aims at generating a set of test sequences, each of which is allocated in an individual computing resource and executed in parallel. In particular, we propose eight parallel test prioritization techniques by adapting the existing four sequential test prioritization techniques, by including and excluding testing time in prioritization. To investigate the performance of the eight parallel test prioritization techniques, we conducted an extensive study on 54 open-source projects and a case study on 16 commercial projects from Baidu , a famous search service provider with 600M monthly active users. According to the two studies, parallel test prioritization does improve the efficiency of regression testing, and cost-aware additional parallel test prioritization technique significantly outperforms the other techniques, indicating that this technique is a good choice for practical parallel testing. Besides, we also investigated the influence of two external factors, the number of computing resources and time allowed for parallel testing, and find that more computing resources indeed improve the performance of parallel test prioritization. In addition, we investigated the influence of two more factors, test granularity and coverage criterion, and find that parallel test prioritization can still accelerate regression testing in parallel scenario. Moreover, we investigated the benefit of parallel test prioritization on the regression testing process of continuous integration, considering both the cumulative acceleration performance and the overhead of prioritization techniques, and the results demonstrate the superiority of parallel test prioritization.

SpecTest: Specification-Based Compiler Testing

Compilers are error-prone due to their high complexity. They are relevant for not only general purpose programming languages, but also for many domain specific languages. Bugs in compilers can potentially render all programs at risk. It is thus crucial that compilers are systematically tested, if not verified. Recently, a number of efforts have been made to formalise and standardise programming language semantics, which can be applied to verify the correctness of the respective compilers. In this work, we present a novel specification-based testing method named SpecTest to better utilise these semantics for testing. By applying an executable semantics as test oracle, SpecTest can discover deep semantic errors in compilers. Compared to existing approaches, SpecTest is built upon a novel test coverage criterion called semantic coverage which brings together mutation testing and fuzzing to specifically target less tested language features. We apply SpecTest to systematically test two compilers, i.e., the Java compiler and the Solidity compiler. SpecTest improves the semantic coverage of both compilers considerably and reveals multiple previously unknown bugs.

Smart Test Case Quantifier Based on Mc/Dc Coverage Criterion with Uml Sequence Diagram

Software testing is an inevitable phase in the Software Development Life Cycle (SDLC). It plays a vital role in ensuring the quality of the product. Testing can be performed on requirement, design and code. Traditional test case generation approaches majorly focuses on designing and implementation models, as a huge percentage of errors in the software are caused due to the lack of understanding in the early phases. So, it’s advisory to follow testing in the initial phase in SDLC as most of the errors/faults can be eliminated and can be prevented without disseminating to the next phase. So, testing must not be secluded to a single phase alone in SDLC. In software testing, test case generation is the fundamental and challenging part. The major purpose to design test case is to create set of tests that are valuable in validation. Due to the increase in software size, it’s not quite advisable to have the choice as manual testing to test the code as its error prone, complex and time consuming. Automation of testing would help the tester to test effectively and timely. Applications of new techniques improve the test process and cut down the tester’s effort. The proposed work presents a sandwich of black box and white box testing. Design-based testing, a black box approach is implemented, that aids in fixing the errors at initial phase. Unified Modeling Language (UML) Use case diagram, Activity diagram and Sequence Diagram are considered for designing. The code-based generation of test cases, is a white box testing approach, test coverage criteria Modified Condition/Decision Condition (MC/DC) has been used to ensure the maximum coverage of the code during testing phase for generating the test cases. The objective of this paper is to automate the generation of minimal number of test cases required to test a system with maximum coverage by removing the redundant test cases using MC/DC criterion. This present an idea to the beginners of the testing about minimal test cases they necessitate for testing their system.

Automatic Test Data Generation Using the Activity Diagram and Search-Based Technique

In software testing, generating test data is quite expensive and time-consuming. The manual generation of an appropriately large set of test data to satisfy a specified coverage criterion carries a high cost and requires significant human effort. Currently, test automation has come at the cost of low quality. In this paper, we are motivated to propose a model-based approach utilizing the activity diagram of the system under test as a test base, focusing on its data flow aspect. The technique is incorporated with a search-based optimization heuristic to fully automate the test data generation process and deliver test cases with more improved quality. Our experimental investigation used three open-source software systems to assess and compare the proposed technique with two alternative approaches. The experimental results indicate the improved fault-detection performance of the proposed technique, which was 11.1% better than DFAAD and 38.4% better than EvoSuite, although the techniques did not differ significantly in terms of statement and branch coverage. The proposed technique was able to detect more computation-related faults and tends to have better fault detection capability as the system complexity increases.

Simulation-Based Hardware Verification with a Graph-Based Specification

Simulation-based verification continues to be the primary technique for hardware verification due to its scalability and ease of use; however, it lacks exhaustiveness. Although formal verification techniques can exhaustively prove functional correctness, they are limited in terms of the scale of their design due to the state-explosion problem. Alternatively, semiformal approaches can involve a compromise between scalability, exhaustiveness, and resource costs. Therefore, we propose an event-driven flow graph-based specification, which can describe the cycle-accurate functional behaviors without the exploration of whole state space. To efficiently generate input sequences according to the proposed specification, we introduce a functional automatic test pattern generation (ATPG) approach, which involves the proposed intelligent redundancy-reduction strategy to solve problems of random test vectors. We also proposed functional coverage criterion based on the formal specification to support a more reliable measure of verification. We implement a verification platform based on the proposed semiformal approach and compare the proposed semiformal approach with the constrained randomized test (CRT) approach. The experiment results show that the proposed semiformal verification method ensures a more exhaustive and effective exploration of the functional correctness of designs under verification (DUVs).