Identifying and respecting differences among participants in evaluation studies

Abstract. Introduction: While alcohol treatment predominantly focuses on abstinence, drug treatment objectives include a variety of outcomes related to consumption and quality of life. Consequently harm reduction programs tackling psychoactive substances are well documented and accepted by practitioners, whereas harm reduction programs tackling alcohol are under-researched and met with resistance. Method: The paper is mainly based on key-person interviews with eight program providers conducted in Switzerland in 2009 and up-dated in 2015, and the analysis of reports and mission statements to establish an inventory and description of drinking under control programs (DUCPs). A recent twin program in Amsterdam and Essen was included to exemplify conditions impeding their implementation. Firstly, a typology based on the type of alcohol management, the provided support and admission criteria is developed, complemented by a detailed description of their functioning in practice. Secondly, the case studies are analyzed in terms of factors promoting and impeding the implementation of DUCPs and efforts of legitimize them and assess their success. Results: Residential and non-residential DUCPs show high diversity and pursue individualized approaches as the detailed case descriptions exemplify. Different modalities of proactively providing and including alcohol consumption are conceptualized in a wider framework of program objectives, including among others, quality of life and harm reduction. Typically DUCPs represent an effort to achieve public or institutional order. Their implementation and success are contingent upon their location, media response, type of alcohol management and the response of other substance-oriented stake holders in the treatment system. The legitimization of DUCPs is hampered by the lack of evaluation studies. DUCPs rely mostly – also because of limited resources – on rudimentary self-evaluations and attribute little importance to data collection exercises. Conclusions: Challenges for participants are underestimated and standard evaluation methodologies tend to be incompatible with the rationale and operational objectives of DUCPs. Program-sensitive multimethod approaches enabled by sufficient financing for monitoring and accompanying research is needed to improve the practice-oriented implementation of DUCPs. Barriers for these programs include assumptions that ‘alcohol-assisted’ help abandons hope for recovery and community response to DUCPs as locally unwanted institutions (‘not in my backyard’) fuelled by stigmatization.

Download Full-text

Theoretical and Methodological Problems of a 10-Year Follow-Up Program Evaluation Study

European Journal of Psychological Assessment ◽

10.1027//1015-5759.18.3.229 ◽

2002 ◽

Vol 18 (3) ◽

pp. 229-241 ◽

Cited By ~ 6

Author(s):

Kurt A. Heller ◽

Ralph Reimann

Keyword(s):

Program Evaluation ◽

Evaluation Studies ◽

Evaluation Study ◽

Control Groups ◽

Cross Sectional ◽

Long Term Study ◽

Partial Inclusion ◽

Methodological Problems

Summary In this paper, conceptual and methodological problems of school program evaluation are discussed. The data were collected in conjunction with a 10 year cross-sectional/longitudinal investigation with partial inclusion of control groups. The experiences and conclusions resulting from this long-term study are revealing not only from the vantage point of the scientific evaluation of new scholastic models, but are also valuable for program evaluation studies in general, particularly in the field of gifted education.

Download Full-text

Reversion Toward the Mean Independently of Regression Toward the Mean

Methodology ◽

10.1027/1614-2241.5.1.3 ◽

2009 ◽

Vol 5 (1) ◽

pp. 3-6 ◽

Cited By ~ 2

Author(s):

Merton S. Krause

Keyword(s):

Random Error ◽

Evaluation Studies ◽

Internal Validity ◽

Intervention Evaluation ◽

Population Mean ◽

Experimental Intervention ◽

Sample Data ◽

The Mean ◽

Regression Toward The Mean

There is another important artifactual contributor to the apparent improvement of persons subjected to an experimental intervention which may be mistaken for regression toward the mean. This is the phenomenon of random error and extreme selection, which does not at all involve the population regression of posttest on pretest scores but involves a quite different and independent reversion of subjects’ scores toward the population mean. These two independent threats to the internal validity of intervention evaluation studies, however, can be detected and differentiated on the sample data of such studies.

Download Full-text

ACT program evaluation studies

PsycEXTRA Dataset ◽

10.1037/e507572016-001 ◽

2016 ◽

Keyword(s):

Program Evaluation ◽

Evaluation Studies

Download Full-text

Conceptualizing and Measuring "Healthy Marriage" For Empirical Research and Evaluation Studies: A Review of the Literature and Annotated Bibliography (Task Three)

PsycEXTRA Dataset ◽

10.1037/e552172011-001 ◽

2013 ◽

Author(s):

Jacinta Bronte-Tinkew ◽

Lina Guzman ◽

Suzanne Jekielek ◽

Kristin A. Moore ◽

Suzanne Ryan ◽

...

Keyword(s):

Empirical Research ◽

Evaluation Studies ◽

Annotated Bibliography ◽

Review Of The Literature

Download Full-text

The Apple Classrooms of Tomorrow: The UCLA Evaluation Studies

PsycEXTRA Dataset ◽

10.1037/e670272011-001 ◽

1993 ◽

Cited By ~ 1

Author(s):

Eva L. Baker ◽

Maryl Gearhart ◽

Joan L. Herman

Keyword(s):

Evaluation Studies

Download Full-text

Overview and initial results of the National Center for Health Statistics’ Research and Development Survey1

Statistical Journal of the IAOS ◽

10.3233/sji-200678 ◽

2020 ◽

Vol 36 (4) ◽

pp. 1199-1211

Author(s):

Jennifer Parker ◽

Kristen Miller ◽

Yulei He ◽

Paul Scanlon ◽

Bill Cai ◽

...

Keyword(s):

Survey Data ◽

Evaluation Studies ◽

Research Area ◽

Production Cycle ◽

Response Patterns ◽

Health Statistics ◽

Mode Effects ◽

Research Areas ◽

Statistical Agencies ◽

Initial Results

The National Center for Health Statistics is assessing the usefulness of recruited web panels in multiple research areas. One research area examines the use of close-ended probe questions and split-panel experiments for evaluating question-response patterns. Another research area is the development of statistical methodology to leverage the strength of national survey data to evaluate, and possibly improve, health estimates from recruited panels. Recruited web panels, with their lower cost and faster production cycle, in combination with established population health surveys, may be useful for some purposes for statistical agencies. Our initial results indicate that web survey data from a recruited panel can be used for question evaluation studies without affecting other survey content. However, the success of these data to provide estimates that align with those from large national surveys will depend on many factors, including further understanding of design features of the recruited panel (e.g. coverage and mode effects), the statistical methods and covariates used to obtain the original and adjusted weights, and the health outcomes of interest.

Download Full-text