Issues in Survey Development: Assumption Checking, Behavior Coding, Cognitive Interviewing, and Design Testing

Survey research can benefit from pretesting and greatly enhance the effectiveness of data collection and analytic processes. For this purpose, a comprehensive process involving four aspects of testing is proposed, namely, assumption checking, behavior coding, cognitive interviewing, and design testing. In this article, I briefly explain each aspect with relevant illustrations.

Coding Body Language in Sports: The Nonverbal Behavior Coding System for Soccer Penalties

Nonverbal behavior (NVB) plays an important role in sports. However, it has been difficult to measure, as no coding schemes exist to objectively measure NVB in sports. Therefore, the authors adapted the Body Action and Posture Coding System to the context of soccer penalties, validated it, and initially used this system (Nonverbal Behavior Coding System for Soccer Penalties [NBCSP]) to explore NVB in penalties. Study 1 demonstrated that the NBCSP had good to excellent intercoder reliability regarding the occurrence and temporal precision of NVBs. It also showed that the coding system could differentiate certain postures and behaviors as a function of emotional valence (i.e., positive vs. negative emotional states). Study 2 identified differences in NVB for successful and missed shots in a sample of penalties (time spent looking toward the goal, toward the ground, right arm movement, and how upright the body posture was). The authors discuss the utility of the coding system for different sport contexts.

Exploring the Validity of Behavior Coding

Behavior coding is used to detect issues in questionnaires by flagging interactions between survey respondents and interviewers when their interactions deviate from standardized interviewing. It has become a popular method for evaluating survey questionnaire quality and has been used mostly in Western countries, with little research being done in Asia. However, because of its dependence on overt behaviors observed during the coding, it was hypothesized that it might not be as effective in a country where different communicative norms are present. We conducted an experiment in Korea using two sets of survey questionnaires with embedded problematic questions in the experimental group to investigate the validity of behavior coding as reference to the control group. Using 200 audio recordings from a telephone survey, we investigated whether behavior-coded data were helpful in identifying embedded flaws. The results indicated behavior coding in Korean surveys may not be as effective as in Western countries.