Purpose
Testing is one of the indispensable activities in software development and is being adopted as an independent course by software engineering (SE) departments at universities worldwide. The purpose of this paper is to carry out an investigation of the performance of learners about testing, given the tendencies in the industry and motivation caused by the unavailability of similar studies in software testing field.
Design/methodology/approach
This study is based on the data collected over three years (between 2012 and 2014) from students taking the software testing course. The course is included in the second year of undergraduate curriculum for the bachelor of engineering (SE).
Findings
It has been observed that, from the performance perspective, automated testing outperforms structural and functional testing techniques, and that a strong correlation exists among these three approaches. Moreover, a strong programming background does help toward further success in structural and automated testing, but has no effect on functional testing. The results of different teaching styles within the course are also presented together with an analysis exploring the relationship between students’ gender and success in the software testing course, revealing that there is no difference in terms of performance between male and female students in the course. Moreover, it is advisable to introduce teaching concepts one at a time because students find it difficult to grasp the ideas otherwise.
Research limitations/implications
These findings are based on the analysis conducted using three years of data collected while teaching a course in testing. Obviously, there are some limitations to this study. For example, student’s strength in programming is calculated using the score of C programming courses taken in previous year/semester. Such scores may not reflect their current level of programming knowledge. Furthermore, attempt was made to ensure that the exercises given for different testing techniques have similar difficulty level to guarantee that the difference in success between these testing techniques is due to the inherent complexity of the technique itself and not because of different exercises. Still, there is small probability that a certain degree of change in success may be due to the difference in the difficulty levels of the exercises. As such, it is obviously premature to consider the present results as final since there is a lack of similar type of studies, with which the authors can compare the results. Therefore, more work needs to be done in different settings to draw sound conclusions in this respect.
Originality/value
Although there are few studies (see e.g. Chan et al., 2005; Garousi and Zhi, 2013; Ng et al., 2004) exploring the preference of testers over distinct software testing techniques in the industry, there appears to be no paper comparing the preferences and performances of learners in terms of different testing techniques.
A New Approach for Evaluating Software Testing Automation Tools