scholarly journals Identification of Complex Health Interventions Suitable for Evaluation: Development and Validation of the 8-Step Scoping Framework (Preprint)

2018 ◽  
Author(s):  
Rosemary Davidson ◽  
Gurch Randhawa ◽  
Stephanie Cash
Keyword(s):  
Local Government ◽  
Evaluation Research ◽  
Complex Interventions ◽  
Evidence Base ◽  
Outcome Evaluation ◽  
Early Years ◽  
Extensive Literature ◽  
Evaluability Assessment ◽  
Government Organization ◽  
Process And Outcome

BACKGROUND There is extensive literature on the methodology of evaluation research and the development and evaluation of complex interventions but little guidance on the formative stages before evaluation and how to work with partner organizations that wish to have their provision evaluated. It is important to be able to identify suitable projects for evaluation from a range of provision and describe the steps required, often with academic institutions working in partnership with external organizations, in order to set up an evaluation. However, research evaluating programs or interventions rarely discusses these stages. OBJECTIVE This study aimed to extend work on evaluability assessment and pre-evaluation planning by proposing an 8-Step Scoping Framework to enable the appraisal of multiple programs in order to identify interventions suitable for evaluation. We aimed to add to the literature on evaluability assessment and more recent evaluation guidance by describing the processes involved in working with partner organizations. METHODS This paper documents the steps required to identify multiple complex interventions suitable for process and outcome evaluation. The steps were developed using an iterative approach by working alongside staff in a local government organization, to build an evidence base to demonstrate which interventions improve children’s outcomes. The process of identifying suitable programs for evaluation, thereby establishing the pre-evaluation steps, was tested using all Flying Start provision. RESULTS The 8-Step Scoping Framework was described using the example of the local government organization Flying Start to illustrate how each step contributes to finding projects suitable for process and outcome evaluation: (1) formulating overarching key questions that encompass all programs offered by an organization, (2) gaining an in-depth understanding of the work and provision of an organization and engaging staff, (3) completing a data template per project/program offered, (4) assessing the robustness/validity of data across all programs, (5) deciding on projects suitable for evaluation and those requiring additional data, (6) negotiating with chosen project leads, both within and outside the organization, (7) developing individual project evaluation protocols, and (8) applying for ethical approval from the university and partner organization. CONCLUSIONS This paper describes the processes involved in identifying suitable projects for evaluation. It adds to the existing literature on the assessment of specific programs suitable for evaluation and guidance for conducting evaluations by establishing the formative steps required to identify suitable programs from a range of provision. This scoping framework particularly relates to academic partners and organizations tasked with delivering evidence-based services designed to meet local needs. The steps identified have been described in the context of early years provision but can be applied to a range of community-based evaluations, or more generally, to cases where an academic partner is working with external stakeholders to identify projects suitable for academic evaluation.

The Sign 4 Big Feelings intervention to improve early years outcomes in preschool children: Outcome evaluation (Preprint)

2020 ◽  
Author(s):  
Rosemary Davidson ◽  
Gurch Randhawa
Keyword(s):  
Economic Status ◽  
Well Being ◽  
Outcome Evaluation ◽  
Preschool Age ◽  
Early Years ◽  
Children’S Wellbeing ◽  
Preschool Age Children ◽  
Measuring Change ◽  
Quasi Experimental ◽  
Clinically Significant

BACKGROUND Any delays in language development may affect learning, profoundly influencing personal, social, and professional trajectories. The effectiveness of the Sign 4 Big Feelings (S4BF) intervention was investigated by measuring change in early years outcomes after a three month period. OBJECTIVE To determine whether Early Years Outcomes (EYOs) significantly improve (beyond typical expected development), if children’s wellbeing improves after the S4BF intervention period, and if there are differences between boys and girls in any progress made. METHODS An evaluation of S4BF was conducted with 111 preschool age children in early years settings in Luton, United Kingdom. Listening, speaking, understanding, and managing feelings and behaviour, in addition to Leuven well-being scales were used in a quasi-experimental study design to measure outcomes pre- and postintervention. RESULTS Statistically and clinically significant differences were found for each of the seven pre- and post measures taken: words understood and spoken, well-being scores, and the four EYO domains. Gender differences were negligible in all analyses undertaken. CONCLUSIONS Children of all abilities may benefit considerably from S4BF, but a language-based intervention of this nature may be transformational for children who are behind developmentally, with EAL needs or of lower socio-economic status. CLINICALTRIAL ISRCTN42025531; https://doi.org/10.1186/ISRCTN42025531

Evaluating a Functional Family Therapy Intervention: Challenges and Lessons Learned

2021 ◽  
pp. 104973152098560 ◽  
Author(s):  
Katarzyna Celinska
Keyword(s):  
Family Therapy ◽  
Service Providers ◽  
Evaluation Research ◽  
Evaluation Process ◽  
Outcome Evaluation ◽  
Lessons Learned ◽  
Functional Family Therapy ◽  
Functional Family ◽  
Family Based

Purpose: This case study is the introspective account of the evaluation process of Functional Family Therapy (FFT) as implemented in Middlesex County in New Jersey between 2005 and 2011. The study presents challenges and issues in evaluation falling into three main categories. Methods: The case study is based on the recollections and documented experiences of the author who was responsible for all major aspects of the evaluation including designing the study, collecting the data, and handling daily evaluation activities. Results: The author differentiated among three main categories of challenges. In respect to research design, the relative merits of experimental versus nonexperimental designs and quantitative versus qualitative research methods are discussed. The second set of issues involves developing and exercising the social competence skills necessary to form working partnerships with service providers. The third set encompasses logistical barriers encountered during daily evaluation activities. Conclusions: The challenges and lessons learned from conducting the outcome evaluation of FFT are situated within scholarly debates on evaluation research, with the goal of providing further insights into the on-the-ground implementation and process of program evaluations. The experiences, recollections and processes illustrate challenges and solutions applicable to evaluations of other family-based violence prevention interventions.

scholarly journals Developing a whole-school mental health and wellbeing intervention through pragmatic formative process evaluation: a case-study of innovative local practice within The School Health Research network

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Nina Gobat ◽  
Hannah Littlecott ◽  
Andy Williams ◽  
Kirsten McEwan ◽  
Helen Stanton ◽  
...  
Keyword(s):  
Mental Health ◽  
Process Evaluation ◽  
Evidence Base ◽  
Outcome Evaluation ◽  
Research Network ◽  
Programme Theory ◽  
Student Mental Health ◽  
Local Practice ◽  
Health And Wellbeing ◽  
Whole School

Abstract Background The evidence-base for whole school approaches aimed at improving student mental health and wellbeing remains limited. This may be due to a focus on developing and evaluating de-novo, research-led interventions, while neglecting the potential of local, contextually-relevant innovation that has demonstrated acceptability and feasibility. This study reports a novel approach to modelling and refining the programme theory of a whole-school restorative approach, alongside plans to scale up through a national educational infrastructure in order to support robust scientific evaluation. Methods A pragmatic formative process evaluation was conducted of a routinized whole-school restorative approach aimed at improving student mental health and wellbeing in Wales. Results The study reports the six phases of the pragmatic formative process evaluation. These are: 1) identification of innovative local practice; 2) scoping review of evidence-base to identify potential programme theory; outcomes; and contextual characteristics that influence implementation; 3) establishment of a Transdisciplinary Action Research (TDAR) group; 4) co-production and confirmation of an initial programme theory with stakeholders; 5) planning to optimise intervention delivery in local contexts; and 6) planning for feasibility and outcome evaluation. The phases of this model may be iterative and not necessarily sequential. Conclusions Formative, pragmatic process evaluations can support researchers, policy-makers and practitioners in developing robust scientific evidence-bases for acceptable and feasible local innovations that do not already have a clear evidence base. The case of a whole-school restorative approach provides a case example of how such an evaluation may be undertaken.

scholarly journals Understanding the Impact of a Community-led HIV Prevention Program in South Africa: Context, Conceptual Framework and Methodology

1999 ◽  
Vol 5 (4) ◽  
pp. 9 ◽  
Author(s):  
Catherine Campbell ◽  
Brian Williams
Keyword(s):  
Hiv Prevention ◽  
Prevention Program ◽  
Hiv Transmission ◽  
Evaluation Research ◽  
Evaluation Study ◽  
Social Contexts ◽  
Outcome Evaluation ◽  
Evaluation Methodology ◽  
Sexually Transmitted ◽  
The Impact

In this paper we provide an account of our multi-dimensional evaluation of a community led HIV-prevention program in the southern African mining community of Carletonville. The Mothusimpilo Project has three pillars: peer education and condom distribution, syndromic management of sexually transmitted diseases (STDs), and stakeholder mobilisation. Substantial efforts are being made to evaluate the impact of the intervention and in this paper we outline the theoretical rationale, research design and some preliminary results of the evaluation. The first section provides the setting for the evaluation work, viz an intervention which seeks to contextualise traditional biomedical and behavioural approaches to HIV-transmission within their broader community and social contexts. In the second section we outline the theoretical assumptions underlying the evaluation (which has both 'outcome evaluation' and 'process evaluation' components). In particular, we discuss the way in which the concepts of identity, empowerment and social capital are used to understand the processes involved in health-enhancing behaviour change. In the third section we describe our multi-disciplinary evaluation methodology and present some preliminary findings from our on-going evaluation study. One important goal of our evaluation research is to demonstrate the extent to which community level factors serve to assist or hinder the project in achieving its goals. In this way we hope to contribute to understandings of the role of community participation in influencing the outcomes of community-based health promotional projects.

scholarly journals REORGANIZATION OF REGIONAL ENVIRONMENTAL INSTITUTION

Yuridika ◽  
2017 ◽  
Vol 32 (1) ◽  
pp. 1
Author(s):  
Bagus Oktafian Abrianto ◽  
Suparto Wijoyo
Keyword(s):  
Local Government ◽  
Central Government ◽  
Management Policy ◽  
Environment Management ◽  
Government Organization ◽  
Local Government Authority ◽  
Integrated Environment ◽  
Government Authority

Nowadays, The Institutional of environmental in regional experinced many changes, its cause of stipulated of Law No. 23 of 2014 on Local Government and Government Act No. 18 of 2016 on Local Government Organization. Rule about central government authority and local government authority on environment aspect that ruled by Act No. 32 of 2009 on Protection and Management of Environment must be adjusted with Act No. 23 of 2014 on Local Government and Government Act No. 18 of 2016 on Local Government Organization. This research analyze about institutional harmonization on statute approaches. It is expected that this research give a knowledge of affairs which central government authority, province government authority, and district/regency government authority clearly especially on environment aspect. The minister of environment and forestry in order to implementing coordinative function, should be obliged to synchronize working program of environment institution between local and central government for making integrated environment management (policy) approaches.   

scholarly journals BUDAYA KERJA GALIE: (Studi Kasus Budaya Kerja Kalangan Pegawai Negeri Sipil Etnik Minangkabau di Kabupaten Pasaman Barat)

Humanus ◽  
2012 ◽  
Vol 12 (2) ◽  
pp. 103
Author(s):  
Aldri Frinaldi
Keyword(s):  
Data Analysis ◽  
Local Government ◽  
Civil Servant ◽  
Study Data ◽  
Qualitative Approach ◽  
Civil Servants ◽  
Work Culture ◽  
Government Organization ◽  
The Difference ◽  
To Come

The work culture of civil servants in this decentralization era tends to come fromthe working culture of each ethnic. The difference of culture applied causes thedifference of civil servants’ way, behavior, and action when performing their work. Thisresearch article aims to discuss one of the working cultures of the civil servants fromMinangkabau ethnic group named galie. This research was conducted by usingdescriptive-qualitative approach, and the data was collected trough observation,interview, and documentation study. Data is validated by triangulation and sourcetriangulation methods.Tthe data analysis is based on Miles and Huberman (2000). Theresult of the research shows that the galie work culture has usually been inherited in thefamily and society for a period time; hence the galie work culture is also implementedwhile working in the local government organization of Pasaman Barat regency. Theresearch concluded that the galie work culture does not cause work culture that canharm others, but whenever the work culture exists, the civil servants would be annoyed.Civil servants who have this work culture tend to avoid risks and prefer simpler workthan their colleagues.Key words: work ethnict culture, galie, civil servant, Minangkabau ethict

scholarly journals Welsh Government Flying Start Programme Evaluation Using Linked Data

2018 ◽  
Vol 3 (2) ◽  
Author(s):  
Sarah Lowe ◽  
Laura McGinn ◽  
Marcos Quintela ◽  
Luke Player ◽  
Karen Tingay
Keyword(s):  
Evidence Base ◽  
Development Project ◽  
Early Years ◽  
Local Authorities ◽  
Parenting Support ◽  
Individual Level ◽  
Level Data ◽  
Part Time ◽  
The Individual ◽  
Families With Children

BackgroundFlying Start (FS) is the Welsh Government’s (WG) flagship Early Years programme for families with children aged less than 4 years of age. Running since 2006, the four entitlements are: Free part-time childcare for 2-3 year olds Enhanced Health Visiting Parenting support Speech, language, and communication support ObjectivesCurrently, while we know which areas in Wales are receiving FS support, individual-level data on which child received what entitlements is not available. Area-level outcomes can be used as proxy indicators but the individual impact of receiving FS support cannot be examined.The project aims to evaluate FS by linking the FS cohort to a range of outcomes including health, education and social care. MethodsA Dataflow Development Project (DDP) has been launched to install SAIL (Secure Anonymised Information Linkage) appliances into 6 pilot Local Authorities in Wales which will test acquiring and linking the individual level FS data from pilot Local Authorities with other datasets in SAIL. FindingsThe project will report some emerging findings from the analysis of pilot data. ImplicationsThere is a growing interest in using linked administrative data to evaluate government initiatives, and mounting enthusiasm in Local Government. If successful, this model is likely to be adopted by related WG programmes; improving the evidence base, facilitating effective evaluation, and adding to the data available for re-use in Wales.

scholarly journals Current practices in clinical neurofeedback with functional MRI—Analysis of a survey using the TIDieR checklist

2018 ◽  
Vol 50 ◽  
pp. 28-33 ◽  
Author(s):  
Elizabeth Randell ◽  
Rachel McNamara ◽  
Leena Subramanian ◽  
Kerenza Hood ◽  
David Linden
Keyword(s):  
Functional Mri ◽  
Online Survey ◽  
Scientific Evidence ◽  
Complex Interventions ◽  
Evidence Base ◽  
Web Based ◽  
Study Protocols ◽  
Survey Responses ◽  
Intervention Description ◽  
Standard Terms

AbstractBackgroundA core principle of creating a scientific evidence base is that results can be replicated in independent experiments and in health intervention research. The TIDieR (Template for Intervention Description and Replication) checklist has been developed to aid in summarising key items needed when reporting clinical trials and other well designed evaluations of complex interventions in order that findings can be replicated or built on reliably. Neurofeedback (NF) using functional MRI (fMRI) is a multicomponent intervention that should be considered a complex intervention. The TIDieR checklist (with minor modification to increase applicability in this context) was distributed to NF researchers as a survey of current practice in the design and conduct of clinical studies. The aim was to document practice and convergence between research groups, highlighting areas for discussion and providing a basis for recommendations for harmonisation and standardisation.MethodsThe TIDieR checklist was interpreted and expanded (21 questions) to make it applicable to neurofeedback research studies. Using the web-based Bristol Online Survey (BOS) tool, the revised checklist was disseminated to researchers in the BRAINTRAIN European research collaborative network (supported by the European Commission) and others in the fMRI-neurofeedback community.ResultsThere were 16 responses to the survey. Responses were reported under eight main headings which covered the six domains of the TIDieR checklist: What, Why, When, How, Where and Who.ConclusionsThis piece of work provides encouraging insight into the ability to be able to map neuroimaging interventions to a structured framework for reporting purposes. Regardless of the considerable variability of design components, all studies could be described in standard terms of diagnostic groups, dose/duration, targeted areas/signals, and psychological strategies and learning models. Recommendations are made which include providing detailed rationale of intervention design in study protocols.

Sign in / Sign up

Export Citation Format

Share Document