Rethinking Policy Evaluation - Do Simple Neural Nets Bear Comparison with Synthetic Control Method?

In 2014, the most prominent anti-corruption investigation in Latin America called Lava Jato, exposed a Brazilian corruption scheme with reverberations in 61 countries, resulting in legal judgments for nearly 5 billion USD in reimbursements thus far. This article applies the synthetic control method on data from 135 countries (2002–2018) to test the hypothesis that Lava Jato impacts the Worldwide Governance Indicators in Brazil. The findings reveal that Lava Jato negatively affects control of corruption, the rule of law, and regulatory quality. There are signs of possible improvement in at least the corruption and the rule of law measures. This paper brings value to the criminological body of literature, notably lacking in the Global South.

Download Full-text

When is a Match Sufficient? A Score-based Balance Metric for the Synthetic Control Method

Journal of Causal Inference ◽

10.1515/jci-2020-0013 ◽

2020 ◽

Vol 8 (1) ◽

pp. 209-228

Author(s):

Layla Parast ◽

Priscillia Hunt ◽

Beth Ann Griffin ◽

David Powell

Keyword(s):

Los Angeles ◽

Control Method ◽

Control Unit ◽

Treated Group ◽

Control Group ◽

Synthetic Control ◽

Donor Pool ◽

Synthetic Control Method ◽

The Mean ◽

The Impact

AbstractIn some applications, researchers using the synthetic control method (SCM) to evaluate the effect of a policy may struggle to determine whether they have identified a “good match” between the control group and treated group. In this paper, we demonstrate the utility of the mean and maximum Absolute Standardized Mean Difference (ASMD) as a test of balance between a synthetic control unit and treated unit, and provide guidance on what constitutes a poor fit when using a synthetic control. We explore and compare other potential metrics using a simulation study. We provide an application of our proposed balance metric to the 2013 Los Angeles (LA) Firearm Study [9]. Using Uniform Crime Report data, we apply the SCM to obtain a counterfactual for the LA firearm-related crime rate based on a weighted combination of control units in a donor pool of cities. We use this counterfactual to estimate the effect of the LA Firearm Study intervention and explore the impact of changing the donor pool and pre-intervention duration period on resulting matches and estimated effects. We demonstrate how decision-making about the quality of a synthetic control can be improved by using ASMD. The mean and max ASMD clearly differentiate between poor matches and good matches. Researchers need better guidance on what is a meaningful imbalance between synthetic control and treated groups. In addition to the use of gap plots, the proposed balance metric can provide an objective way of determining fit.

Download Full-text

Campaign Finance Regulations and Public Policy

American Political Science Review ◽

10.1017/s0003055421000149 ◽

2021 ◽

pp. 1-8

Author(s):

MARTIN GILENS ◽

SHAWN PATTERSON ◽

PAVIELLE HAINES

Keyword(s):

Government Policy ◽

Policy Making ◽

Campaign Finance ◽

Control Method ◽

Synthetic Control ◽

Synthetic Control Method ◽

Citizens United ◽

Independent Expenditures ◽

American Elections ◽

The U.S

Abstract Despite a century of efforts to constrain money in American elections, there is little consensus on whether campaign finance regulations make any appreciable difference. Here we take advantage of a change in the campaign finance regulations of half of the U.S. states mandated by the Supreme Court’s Citizens United decision. This exogenously imposed change in the regulation of independent expenditures provides an advance over the identification strategies used in most previous studies. Using a generalized synthetic control method, we find that after Citizens United, states that had previously banned independent corporate expenditures (and thus were “treated” by the decision) adopted more “corporate-friendly” policies on issues with broad effects on corporations’ welfare; we find no evidence of shifts on policies with little or no effect on corporate welfare. We conclude that even relatively narrow changes in campaign finance regulations can have a substantively meaningful influence on government policy making.

Download Full-text