Prediction Consistency: A Test of the Equivalence Assumption across Different Indicators of the Same Construct

Prominent theoretical constructs such as the Big Five personality factors often inspire the development and use of different inventories. This practice rests on the vital assumption that different indicators equivalently assess the same construct—otherwise, it would often be inappropriate to draw conclusions on the construct level. In comparison to the evidence typically relied on to support this equivalence assumption, we argue that a direct test of prediction consistency will provide further insights: prediction consistency is a necessary condition for the equivalence assumption that indicators from different inventories predict an external criterion to the same extent. Here, we outline guidelines how to design studies to establish prediction consistency and illustrate this approach in an experiment testing the prediction consistency of the Agreeableness indicators from three prominent Big Five inventories. Specifically, we considered prediction consistency with respect to honesty (vs. cheating) as the behavioral criterion for which a specific a priori hypothesis can be derived on theoretical grounds. Results contradicted predictions consistency and thus the equivalence assumption by showing qualitatively different relations to behavioral honesty, thereby also emphasizing that the interchangeability of inventories should generally be subjected to a strict test, rather than assumed. Copyright © 2016 European Association of Personality Psychology