The Logic of the Survey Experiment Reexamined

2007 ◽  
Vol 15 (1) ◽  
pp. 1-20 ◽  
Author(s):  
Brian J. Gaines ◽  
James H. Kuklinski ◽  
Paul J. Quirk
Keyword(s):  
Political Science ◽  
Survey Data ◽  
Political Behavior ◽  
Experimental Approach ◽  
Experimental Designs ◽  
Survey Experiment ◽  
Alternative Strategies ◽  
Survey Experiments

Scholars of political behavior increasingly embed experimental designs in opinion surveys by randomly assigning respondents alternative versions of questionnaire items. Such experiments have major advantages: they are simple to implement and they dodge some of the difficulties of making inferences from conventional survey data. But survey experiments are no panacea. We identify problems of inference associated with typical uses of survey experiments in political science and highlight a range of difficulties, some of which have straightforward solutions within the survey-experimental approach and some of which can be dealt with only by exercising greater caution in interpreting findings and bringing to bear alternative strategies of research.

Improving Data Quality in Face-to-Face Survey Research

2019 ◽  
Vol 53 (1) ◽  
pp. 46-50
Author(s):  
Carolyn Logan ◽  
Pablo Parás ◽  
Michael Robbins ◽  
Elizabeth J. Zechmeister
Keyword(s):  
Best Practices ◽  
Data Quality ◽  
Political Science ◽  
Survey Research ◽  
Survey Data ◽  
Political Behavior ◽  
High Quality ◽  
Face To Face ◽  
The World

ABSTRACTData quality in survey research remains a paramount concern for those studying mass political behavior. Because surveys are conducted in increasingly diverse contexts around the world, ensuring that best practices are followed becomes ever more important to the field of political science. Bringing together insights from surveys conducted in more than 80 countries worldwide, this article highlights common challenges faced in survey research and outlines steps that researchers can take to improve the quality of survey data. Importantly, the article demonstrates that with the investment of the necessary time and resources, it is possible to carry out high-quality survey research even in challenging environments in which survey research is not well established.

Measuring the Rural Continuum in Political Science

2021 ◽  
pp. 1-20
Author(s):  
Zoe Nemerever ◽  
Melissa Rogers
Keyword(s):  
Construct Validity ◽  
Political Science ◽  
Survey Data ◽  
Political Behavior ◽  
Rural Politics ◽  
Rural Geography ◽  
Rural Identity ◽  
Urban Rural ◽  
Legislative Districts ◽  
State Legislative

Abstract Recent accounts of American politics focus heavily on urban–rural gaps in political behavior. Rural politics research is growing but may be stymied by difficulties defining and measuring which Americans qualify as “rural.” We discuss theoretical and empirical challenges to studying rurality. Much existing research has been inattentive to conceptualization and measurement of rural geography. We focus on improving estimation of different notions of rurality and provide a new dataset on urban–rural measurement of U.S. state legislative districts. We scrutinize construct validity and measurement in two studies of rural politics. First, we replicate Flavin and Franko (2020, Political Behavior, 845–864) to demonstrate empirical results may be sensitive to measurement of rural residents. Second, we use Mummolo and Nall’s (2017, The Journal of Politics, 45–59) survey data to show rural self-identification is not well-captured with objective, place-based classifications, suggesting a rethinking of theoretical and empirical accounts of rural identity. We conclude with strategies for operationalizing rurality using readily available tools.

Demand Effects in Survey Experiments: An Empirical Assessment

2018 ◽  
Vol 113 (2) ◽  
pp. 517-529 ◽  
Author(s):  
JONATHAN MUMMOLO ◽  
ERIK PETERSON
Keyword(s):  
Social Science ◽  
Political Science ◽  
Financial Incentives ◽  
Online Survey ◽  
Experimental Designs ◽  
Limited Ability ◽  
Research Participants ◽  
Survey Experiments ◽  
Demand Effects ◽  
Positive Results

Survey experiments are ubiquitous in social science. A frequent critique is that positive results in these studies stem from experimenter demand effects (EDEs)—bias that occurs when participants infer the purpose of an experiment and respond so as to help confirm a researcher’s hypothesis. We argue that online survey experiments have several features that make them robust to EDEs, and test for their presence in studies that involve over 12,000 participants and replicate five experimental designs touching on all empirical political science subfields. We randomly assign participants information about experimenter intent and show that providing this information does not alter the treatment effects in these experiments. Even financial incentives to respond in line with researcher expectations fail to consistently induce demand effects. Research participants exhibit a limited ability to adjust their behavior to align with researcher expectations, a finding with important implications for the design and interpretation of survey experiments.

What Can We Learn From Factorial Surveys About Human Behavior?

Methodology ◽  
2019 ◽  
Vol 15 (1) ◽  
pp. 19-30 ◽  
Author(s):  
Knut Petzold ◽  
Tobias Wolbring
Keyword(s):  
Social Sciences ◽  
Behavioral Intentions ◽  
Actual Behavior ◽  
Full Time ◽  
Factorial Survey ◽  
Survey Experiment ◽  
Intended Behavior ◽  
The Social ◽  
Survey Experiments ◽  
High Level

Abstract. Factorial survey experiments are increasingly used in the social sciences to investigate behavioral intentions. The measurement of self-reported behavioral intentions with factorial survey experiments frequently assumes that the determinants of intended behavior affect actual behavior in a similar way. We critically investigate this fundamental assumption using the misdirected email technique. Student participants of a survey were randomly assigned to a field experiment or a survey experiment. The email informs the recipient about the reception of a scholarship with varying stakes (full-time vs. book) and recipient’s names (German vs. Arabic). In the survey experiment, respondents saw an image of the same email. This validation design ensured a high level of correspondence between units, settings, and treatments across both studies. Results reveal that while the frequencies of self-reported intentions and actual behavior deviate, treatments show similar relative effects. Hence, although further research on this topic is needed, this study suggests that determinants of behavior might be inferred from behavioral intentions measured with survey experiments.

Are voters too afraid to tackle corruption? Survey and experimental evidence from Mexico

2021 ◽  
pp. 1-19
Author(s):  
Omar García-Ponce ◽  
Thomas Zeitzoff ◽  
Leonard Wantchekon
Keyword(s):  
Political Behavior ◽  
Causal Effects ◽  
Willingness To Accept ◽  
Survey Experiment ◽  
Drug War ◽  
Mexican Drug War ◽  
The Face ◽  
Victims Of Crime ◽  
Nationally Representative

Abstract Are individuals in violent contexts reluctant to tackle corruption for fear of future violence? Or does violence mobilize them to fight corruption? We investigate these questions looking at the effects of fear and violence stemming from the Mexican Drug War on attitudes toward corruption. We conducted two surveys before the 2012 Mexican presidential election. First, as part of a nationally representative survey, we find a positive correlation between fear of violence and willingness to accept corruption in exchange for lower levels of violence. To disentangle causal effects, we conducted a follow-up survey experiment in Greater Mexico City where we manipulated fear over the Drug War. We find that individuals within this context are not easily scared. Those who received a common fear-inducing manipulation do not report higher levels of fear and are less willing to tolerate corruption. Conversely, we find strong evidence that individuals who have been victims of crime are more likely to report both higher levels of fear and willingness to accept corruption if it lowers violence. Our findings suggest that voters are more strategic and resilient in the face of violence than many extant theories of political behavior suggest.

An Experimental Approach to the Introductory Political Science Course

1982 ◽  
Vol 35 ◽  
pp. 9-11
Author(s):  
William McClure ◽  
Michael Stohl
Keyword(s):  
Higher Education ◽  
Political Science ◽  
Experimental Approach ◽  
Expert Knowledge ◽  
Educational Process ◽  
Introductory Course ◽  
Teacher’S Role ◽  
Teacher's Role ◽  
Receiver Model ◽  
To Receive

The conventional introductory course rests upon the pedagogical assumption that the teacher's function is to transmit information (or knowledge) and that the student's function is to receive it. According to this transmitter-receiver model of the educational process, teaching begins with a “knower” who “transmits” what he knows to a “learner.” In higher education, certain euphemisms are employed to soften and furnish a color of legitimacy to this model: the teacher is a “scholar,” and “authority,” in his field; he possesses an “expert knowledge” which the student has come to school to “learn“; the student is the “learner.” The teacher's role, accordingly, is the active one of transmitting information and the student's role, accordingly, is the passive one of receiving and recording (or memorizing) this information.

Increasing Precision without Altering Treatment Effects: Repeated Measures Designs in Survey Experiments

2021 ◽  
pp. 1-18
Author(s):  
SCOTT CLIFFORD ◽  
GEOFFREY SHEAGLEY ◽  
SPENCER PISTON
Keyword(s):  
Treatment Effect ◽  
Repeated Measures ◽  
Treatment Effects ◽  
Repeated Measurements ◽  
Experimental Designs ◽  
Treatment Effect Size ◽  
Standard Design ◽  
Survey Experiments ◽  
Repeated Measures Designs ◽  
Alternative Designs

The use of survey experiments has surged in political science. The most common design is the between-subjects design in which the outcome is only measured posttreatment. This design relies heavily on recruiting a large number of subjects to precisely estimate treatment effects. Alternative designs that involve repeated measurements of the dependent variable promise greater precision, but they are rarely used out of fears that these designs will yield different results than a standard design (e.g., due to consistency pressures). Across six studies, we assess this conventional wisdom by testing experimental designs against each other. Contrary to common fears, repeated measures designs tend to yield the same results as more common designs while substantially increasing precision. These designs also offer new insights into treatment effect size and heterogeneity. We conclude by encouraging researchers to adopt repeated measures designs and providing guidelines for when and how to use them.

scholarly journals Employment Lapses and Subsequent Hiring Disadvantages: An Experimental Approach Examining Types of Discrimination and Mechanisms

2021 ◽  
Vol 7 ◽  
pp. 237802312110198
Author(s):  
Katherine Weisshaar
Keyword(s):  
Labor Market ◽  
Job Performance ◽  
Experimental Approach ◽  
Cognitive Biases ◽  
Job Applicants ◽  
Survey Experiment ◽  
Hiring Decisions ◽  
Positive Information ◽  
Employment Gaps ◽  
Types Of Information

Employment interruption is a common experience in today’s labor market, most frequently due to unemployment from job loss and temporary lapses to care for family or children. Although existing research shows that employment lapses cause disadvantages at the hiring interface compared to individuals with no employment disruptions, competing theories predict different mechanisms explaining these hiring penalties. In this study, the author uses an original conjoint survey experiment to causally assess perceptions of fictitious job applicants, focusing on a comparison of unemployed applicants and nonemployed caregiver applicants, who left work to care for family, to currently employed applicants. The author examines whether disadvantages for job applicants with employment gaps are receptive to positive information (and therefore represent a form of “informational bias”) or are resistant to information (reflecting “cognitive bias”) and further assesses which types of information affect or do not affect levels of bias in fictitious hiring decisions. Results show that positive information on past job performance and social skills essentially eliminates disadvantages faced by unemployed job applicants, but nonemployed caregiver applicants remain disadvantaged even with multiple types of positive information. These findings suggest that unemployed applicants face informational biases but that nonemployed caregiver applicants face cognitive biases that are rigid even with rich forms of positive or counter-stereotypical information. This study has implications for understanding the career consequences of employment disruption, which is especially relevant to consider in light of labor market disruptions during the recent pandemic.

Exploration of the technocratic mentality among European civil servants

2021 ◽  
pp. 002085232199642
Author(s):  
Ringa Raudla ◽  
James W. Douglas ◽  
Zachary Mohr
Keyword(s):  
Organizational Performance ◽  
Political Science ◽  
Public Administration ◽  
Survey Data ◽  
Civil Servants ◽  
European Countries ◽  
Positive Interactions ◽  
Technical Knowledge ◽  
High Level ◽  
Proper Balance

Civil servants vary in the degree to which they hold technocratic attitudes. We explore whether bureaucrats’ exposure to politics and politicians is associated with the depoliticization dimension of the technocratic mentality. We use survey data of high-level executives in 19 European countries to explore factors that are associated with executives’ perceptions that removing issues and activities from the realms of politics leads to more farsighted policies. We find that respondents’ level of exposure to politics and politicians is indeed negatively associated with technocratic mentality. Bureaucrats have studied political science or public administration, work closer to politicians (in terms of type of organization), interact with them more frequently, and have more positive perceptions of these interactions tend to have lower levels of technocratic attitudes. Points for practitioners Beliefs affect behaviors and behaviors affect outcomes. Technocratic attitudes may limit the ability of civil servants to work effectively with politicians. We show that educational degrees that promote democratic values and exposure to politicians (particularly positive interactions) are associated with lower levels of technocratic attitudes. Given that a proper balance between political and technical knowledge can enhance organizational performance (Krause et al., 2006), these findings should be taken into account when staffing and structuring public organizations.

scholarly journals Echo chamber and trench warfare dynamics in online debates

2017 ◽  
Vol 32 (3) ◽  
pp. 257-273 ◽  
Author(s):  
Rune Karlsen ◽  
Kari Steen-Johnsen ◽  
Dag Wollebæk ◽  
Bernard Enjolras
Keyword(s):  
Online Survey ◽  
Experimental Approach ◽  
Online Discussion ◽  
Collective Learning ◽  
Survey Experiments ◽  
Echo Chambers ◽  
Echo Chamber ◽  
Trench Warfare

In this article, we take issue with the claim by Sunstein and others that online discussion takes place in echo chambers, and suggest that the dynamics of online debates could be more aptly described by the logic of ‘trench warfare’, in which opinions are reinforced through contradiction as well as confirmation. We use a unique online survey and an experimental approach to investigate and test echo chamber and trench warfare dynamics in online debates. The results show that people do indeed claim to discuss with those who hold opposite views from themselves. Furthermore, our survey experiments suggest that both confirming and contradicting arguments have similar effects on attitude reinforcement. Together, this indicates that both echo chamber and trench warfare dynamics – a situation where attitudes are reinforced through both confirmation and disconfirmation biases – characterize online debates. However, we also find that two-sided neutral arguments have weaker effects on reinforcement than one-sided confirming and contradicting arguments, suggesting that online debates could contribute to collective learning and qualification of arguments.

Sign in / Sign up

Export Citation Format

Share Document