One Size Doesn’t Fit All: Using Factor Analysis to Gather Validity Evidence When Using Surveys in Your Research

Across all sciences, the quality of measurements is important. Survey measurements are only appropriate for use when researchers have validity evidence within their particular context. Yet, this step is frequently skipped or is not reported in educational research. This article briefly reviews the aspects of validity that researchers should consider when using surveys. It then focuses on factor analysis, a statistical method that can be used to collect an important type of validity evidence. Factor analysis helps researchers explore or confirm the relationships between survey items and identify the total number of dimensions represented on the survey. The essential steps to conduct and interpret a factor analysis are described. This use of factor analysis is illustrated throughout by a validation of Diekman and colleagues’ goal endorsement instrument for use with first-year undergraduate science, technology, engineering, and mathematics students. We provide example data, annotated code, and output for analyses in R, an open-source programming language and software environment for statistical computing. For education researchers using surveys, understanding the theoretical and statistical underpinnings of survey validity is fundamental for implementing rigorous education research.

The Role of Test Anticipation in the Link Between Performance-Approach Goals and Academic Achievement

Performance-approach goals (i.e., the desire to outperform others) have been found to be positive predictors of test performance, but research has also revealed that they predict surface learning strategies. The present research investigates whether the high academic performance of students who strongly adopt performance-approach goals stems from test anticipation and preparation, which most educational settings render possible since examinations are often scheduled in advance. We set up a longitudinal design for an experiment conducted in high-school classrooms within the context of two science, technology, engineering, and mathematics (STEM) disciplines, namely, physics and chemistry. First, we measured performance-approach goals. Then we asked students to take a test that had either been announced a week in advance (enabling strategic preparation) or not. The expected interaction between performance-approach goal endorsement and test anticipation was moderated by the students’ initial level: The interaction appeared only among low achievers for whom the pursuit of performance-approach goals predicted greater performance – but only when the test had been scheduled. Conversely, high achievers appeared to have adopted a regular and steady process of course content learning whatever their normative goal endorsement. This suggests that normative strivings differentially influence the study strategies of low and high achievers.

Do Achievement Goals Mediate Stereotype Threat?: An Investigation on Females’ Soccer Performance

This research investigated stereotype threat effects on women’s performance in sports and examined the mediation of this effect by achievement goals. The influence of two stereotypes—relative to the poor athletic ability and the poor technical soccer ability of women—were studied. Fifty-one female soccer players were randomly assigned to one of three conditions, introducing the task as diagnostic of athletic ability, technical soccer ability, or sports psychology. Next, they filled out a questionnaire measuring achievement goals and performed a soccer dribbling task. Results showed that compared with the control condition, females’ performance significantly decreased in the athletic ability condition and tended to decrease in the technical soccer ability condition. Moreover, participants endorsed a performance-avoidance (relative to performance-approach) goal when the stereotypes were activated. However, this goal endorsement was not related to performance. The implications of these results for understanding the role of stereotypes in gender inequalities in sports are discussed.