Experimenter Effects in Experimental Program Evaluation

2019 ◽  
Author(s):  
Alec Brandon
Keyword(s):  
Program Evaluation ◽  
Experimental Program ◽  
Experimenter Effects

scholarly journals Experimentation at Scale

2017 ◽  
Vol 31 (4) ◽  
pp. 103-124 ◽  
Author(s):  
Karthik Muralidharan ◽  
Paul Niehaus
Keyword(s):  
Developing Countries ◽  
Program Evaluation ◽  
Randomized Controlled Trials ◽  
Experimental Program ◽  
Controlled Trials ◽  
Randomized Experiments ◽  
Sampling Frame ◽  
Randomized Controlled

This paper makes the case for greater use of randomized experiments “at scale.” We review various critiques of experimental program evaluation in developing countries, and discuss how experimenting at scale along three specific dimensions—the size of the sampling frame, the number of units treated, and the size of the unit of randomization— can help alleviate the concerns raised. We find that program-evaluation randomized controlled trials published over the last 15 years have typically been “small” in these senses, but also identify a number of examples—including from our own work— demonstrating that experimentation at much larger scales is both feasible and valuable.

Testing Conceptual Frameworks of Nonexperimental Program Evaluation Designs Using Structural Equation Modeling

2011 ◽  
Vol 32 (4) ◽  
pp. 480-493 ◽  
Author(s):  
Omolola A. Adedokun ◽  
Amy L. Childress ◽  
Wilella D. Burgess
Keyword(s):  
Program Evaluation ◽  
Structural Equation Modeling ◽  
Program Implementation ◽  
Structural Equation ◽  
Large Scale ◽  
Conceptual Models ◽  
Equation Modeling ◽  
Experimental Program ◽  
Quasi Experimental ◽  
Analyze Data

A theory-driven approach to evaluation (TDE) emphasizes the development and empirical testing of conceptual models to understand the processes and mechanisms through which programs achieve their intended goals. However, most reported applications of TDE are limited to large-scale experimental/quasi-experimental program evaluation designs. Very few (limited) examples of the relevance of TDE to nonexperimental program evaluation designs exist in literature. Using the method of structural equation modeling to analyze data from the Interns for Indiana (IfI) program, this study demonstrates how evaluation practitioners can test logical and sequential relationships among tiers of outcomes of nonexperimental programs, especially programs with limited datasets. The study also describes how the empirical feedback can be used to understand program dynamics and improve program implementation and evaluation.

scholarly journals Vankeusaikaisten kuntoutusohjelmien vaikutusarvioinnit ja arvioitavuus

Kriminologia ◽  
2021 ◽  
Vol 1 (1) ◽  
pp. 39-59
Author(s):  
Sasu Tyni ◽  
Mikko Aaltonen
Keyword(s):  
Program Evaluation ◽  
Statistical Power ◽  
Evaluation Research ◽  
Cost Effective ◽  
Effect Sizes ◽  
Experimental Program ◽  
Prison Programs ◽  
True Effect ◽  
Rehabilitation Programs ◽  
Quasi Experimental

Suomalaisissa vankiloissa on tehty systemaattista ohjelmatoimintaa noin 20 vuoden ajan. Ohjelmatoiminnan tärkeimpänä tavoitteena on edistää rikoksetonta elämäntapaa, ja siten vähentää vapautumisen jälkeistä uusintarikollisuutta. Uusintarikollisuusvaikutusten todentaminen edellyttää vaikutustutkimuksiin perustuvaa tutkimusnäyttöä, jota on vähitellen alkanut kertymään myös Suomesta. Tässä artikkelissa käydään läpi yksittäisten ohjelmien uusintarikollisuusvaikutuksia koskevien kotimaisten arviointitutkimusten keskeiset tulokset, sekä pohditaan ohjelmatoiminnan vaikutusten arvioinnin mahdollisuuksia tilastollisten tutkimusmenetelmien, tutkimusaineistojen ja ohjelmatoiminnan volyymin näkökulmista. Ohjelmatoiminnan vaikutuksista uusintarikollisuuteen Suomessa ei ole toistaiseksi saatu selvää näyttöä. Vaikka ohjelmien vaikutusten tutkimiseen tarvittavat aineistot ovatkin parantuneet 2000-luvulla selvästi, uskottavaa arviointia vaikeuttavat edelleen hyvien tutkimusasetelmien puute sekä yksittäisten ohjelmien pienet osallistujamäärät. Vaikutusarviointien menetelmällisessä laadussa on parantamisen varaa. Koska yksittäisen ohjelman vaikutus uusintarikollisuuteen on saatavilla olevan tutkimustiedon valossa luultavasti aika maltillinen, kovin pienillä tutkimusaineistoilla vaikutuksia ei kannata jatkossa yrittää tilastollisesti tutkia. Pienet vaikutukset eivät kuitenkaan tarkoita sitä, että ohjelmat eivät voisi olla kannattavia ja kustannustehokkaita. Merkittävä osa nykyisestä ohjelmatoiminnasta on osallistujamääriltään niin pienimuotoista, ettei niiden toimivuudesta tai toimimattomuudesta ole helppo saada luotettavaa tietoa. Tämä tilanne haastaa tutkimustietoon perustuvan ohjelmatoiminnan kehittämistä.   Sasu Tyni and Mikko Aaltonen: Evaluation research on rehabilitation programs in prison. Rehabilitation programs have been used systematically in Finnish prisons now for two decades. The main aim of these programs is to promote desistance and reduce recidivism after release. Several Finnish evaluations of prison programs have been published during the recent years. In this article, we start by reviewing the key results of these studies and consider the possibilities and limitations of quasi-experimental program evaluation in the light of available register data and the scale of program uptake. So far none of the Finnish evaluations have shown evidence of programs reducing recidivism. Even though the datasets needed for evaluation have improved clearly in the 21st century, credible evaluation of programs’ causal effects is still hampered by lack of strong research designs and low participant rates in most programs. The methodological quality of evaluations should be improved. Given that the available evidence suggests that true effect sizes of programs are likely to be relatively small, large samples are needed to attain enough statistical power to detect effects or their absence. At the same time, even if true effect sizes are small, the programs can still be cost-effective. The low numbers of participants in most prison programs present a challenge to evidence-based program development. Keywords: prison rehabilitation – recidivism – program evaluation – statistical power – causal inference

The 1-MeV Electron Microscope Installation at the University of Colorado

1973 ◽  
Vol 31 ◽  
pp. 8-9 ◽  
Author(s):  
Mircea Fotino
Keyword(s):  
Electron Microscopy ◽  
Developmental Biology ◽  
Transmission Electron Microscopy ◽  
Electron Microscope ◽  
National Institutes Of Health ◽  
Experimental Program ◽  
University Of Colorado ◽  
Transmission Electron ◽  
The University ◽  
Research Resources

A new 1-MeV transmission electron microscope (Model JEM-1000) was installed at the Department of Molecular, Cellular and Developmental Biology of the University of Colorado in Boulder during the summer and fall of 1972 under the sponsorship of the Division of Research Resources of the National Institutes of Health. The installation was completed in October, 1972. It is installed primarily for the study of biological materials without many of the limitations hitherto unavoidable in standard transmission electron microscopy. Only the technical characteristics of the installation are briefly reviewed here. A more detailed discussion of the experimental program under way is being published elsewhere.

Program Evaluation

2020 ◽  
Author(s):  
Allen Rubin
Keyword(s):  
Program Evaluation

Theoretical and Methodological Problems of a 10-Year Follow-Up Program Evaluation Study

2002 ◽  
Vol 18 (3) ◽  
pp. 229-241 ◽  
Author(s):  
Kurt A. Heller ◽  
Ralph Reimann
Keyword(s):  
Program Evaluation ◽  
Evaluation Studies ◽  
Evaluation Study ◽  
Control Groups ◽  
Cross Sectional ◽  
Long Term Study ◽  
Partial Inclusion ◽  
Methodological Problems

Summary In this paper, conceptual and methodological problems of school program evaluation are discussed. The data were collected in conjunction with a 10 year cross-sectional/longitudinal investigation with partial inclusion of control groups. The experiences and conclusions resulting from this long-term study are revealing not only from the vantage point of the scientific evaluation of new scholastic models, but are also valuable for program evaluation studies in general, particularly in the field of gifted education.

Mental health Program Evaluation in the 1980s

1982 ◽  
Vol 27 (11) ◽  
pp. 878-879
Author(s):  
Lawrence H. Cohen
Keyword(s):  
Mental Health ◽  
Program Evaluation ◽  
Health Program ◽  
Mental Health Program
Sign in / Sign up

Export Citation Format

Share Document