Ecological validity of performance-based measures of executive functions: Is face validity necessary for prediction of daily functioning?

2019 ◽  
Vol 31 (11) ◽  
pp. 1307-1318 ◽  
Author(s):  
Rosemary E. Ziemnik ◽  
Yana Suchy
Keyword(s):  
Executive Functions ◽  
Ecological Validity ◽  
Face Validity ◽  
Daily Functioning

Technologies for Functionally Relevant Neuropsychological Assessment

2017 ◽  
Author(s):  
Maria T. Schultheis ◽  
Matthew Doiron
Keyword(s):  
Functional Performance ◽  
Ecological Validity ◽  
Brain Lesion ◽  
Face Validity ◽  
Education Law ◽  
Everyday Functioning ◽  
Different Populations ◽  
Brain Behavior ◽  
Neuropsychological Methods ◽  
Everyday Function

Over the course of its history, the field of neuropsychology has shifted its focus to meet the demands of the medical landscape. Before the advent of neuroimaging, neuropsychologists were relied on to determine brain lesion location and to diagnose brain-behavior pathologies. As time progressed, neuroimaging was able to provide faster and more consistent lesion identification and neuropsychology began to adapt its skills and services for other related fields, such as education, law, and rehabilitation. As a result, some neuropsychological methods were adapted to assess broader cognitive functions in a variety of populations and the general public; however, these assessments have been heavily rooted in the field’s diagnostically focused past, which creates limitations in the ecological validity of this approach. Ecological validity can be generally defined as a measure’s ability to predict functional performance or mimic activities of everyday living (i.e., performance at work, driving). For example, batteries of neuropsychological tests and questionnaires have been used to infer level of function and general performance at work or school. These batteries were developed due their statistical associations with different populations, concordance with neurological theories and constructs, and general face validity. However, very few assessments resembled any activity a person would perform in daily life. For many measures, ecological validity was defined by correlating performance with everyday functioning (verdicality; Franzen & Wilhelm, 1996). In contrast, another approach to ecological validity involved designing measures to resemble or mimic an everyday function (verisimilitude; Franzen & Wilhelm, 1996). The major difference between the two approaches determines the primary goal of designing the measure at the onset. It must be decided if the measure will prioritize construct validity at the onset and subsequently infer a link to everyday function, or vice versa. Many researchers interested in predicting functional outcome have relied on verisimilitude, as it more closely resembles “real-world” performance; however, it often comes at a cost of interpretability within the context of current neuropsychological frame­works and models.

scholarly journals M51. EFFICACY OF “PRAGMACOM TRAINING” IN SCHIZOPHRENIA: A RCT ON A NOVEL PRAGMATIC INTERVENTION

2020 ◽  
Vol 46 (Supplement_1) ◽  
pp. S153-S153
Author(s):  
Giulia Agostoni ◽  
Elisabetta Tonini ◽  
Mariachiara Buonocore ◽  
Margherita Bechi ◽  
Maria Paola Silvestri ◽  
...  
Keyword(s):  
Quality Of Life ◽  
Executive Functions ◽  
Controlled Trial ◽  
Effect Sizes ◽  
Daily Functioning ◽  
Control Group ◽  
Abstract Thinking ◽  
Metaphor Comprehension ◽  
Quality Of Life Scale

Abstract Background A pragmatic disruption is observed in several clinical conditions and especially in schizophrenia. It is estimated that over 75% of patients affected by schizophrenia present a pervasive and wide impairment of pragmatic abilities, encompassing both comprehension and production abilities, thus confirming the hypothesis of a wide Pragmatic Language Disorder in schizophrenia. More specifically, the comprehension of figurative languages, such as metaphors, idioms, and irony, is the most compromised domain in schizophrenia. Poor pragmatics has a relevant impact on daily functioning, by contributing to social isolation and lower quality of life. Only few pragmatic treatments have been developed and tested in schizophrenia, focusing only on specific pragmatic features and without using a Randomized Controlled Trial (RCT) design. This study aimed at investigating the efficacy of PragmaCom Training (PT), a novel 12-weeks intervention specifically developed to enhance pragmatics in schizophrenia. Methods 30 patients with schizophrenia, according to DSM 5, were randomly assigned to PT to an active control group (ACG). All patients were assessed for global pragmatics with the Assessment of Pragmatic Abilities and Cognitive Substrates test, APACS), metaphor comprehension (Physical and Mental Metaphors task, PMM), executive functions (Brief Assessment of Cognition in Schizophrenia, Tower of London score, BACS), abstract thinking (Positive and Negative Syndrome Scale for Schizophrenia, N5 score, PANSS), and daily functioning (Quality of Life Scale, QLS). To quantify the magnitude of changes after the PT, effect sizes were estimated using Cohen’s d (Cohen, 1988) for APACS, PMM, PANSS N5 Score, and QLS only in PT Group. The effect of PT in enhancing pragmatic abilities, the abstract thinking and daily functioning was tested between groups by means of several ANCOVA, entering post-training measures as dependent variables, measures at the baseline and executive functions as covariates, and treatment (PT vs ACG) as grouping variables. Results Patients treated with PT showed small to medium-large effect-sizes in global pragmatics (0.25), metaphors comprehension (0.72), functioning (0.23), and abstract thinking (-0.18). ANCOVAs revealed a significant effect of PT in enhancing global pragmatic abilities (F=5.4, p=.03), metaphor comprehension (F=8.94, p=.007) and abstract thinking (F=8.1, p=.01). No significant effect was found for functioning. Discussion This is the first study using a RCT design to test the efficacy of a pragmatic training. PT is a novel training specifically developed to target the pragmatic impairments that characterize schizophrenia. This study confirms the efficacy of PT in improving multiple domains, encompassing global pragmatics, the comprehension of figurative language, and abstract thinking in schizophrenia. Impact on functioning is also expected. However, in line with literature, it is likely to occur at later time points, since it requires the chance to apply the enhanced abilities in ecological contexts.

Time-Based Prospective Memory Is Associated with Functional Performance in Persons with MS

2019 ◽  
Vol 25 (10) ◽  
pp. 1035-1043
Author(s):  
Erica Weber ◽  
Nancy D. Chiaravalloti ◽  
John DeLuca ◽  
Yael Goverover
Keyword(s):  
Executive Functions ◽  
Prospective Memory ◽  
Functional Performance ◽  
Ecological Validity ◽  
Community Dwelling ◽  
Everyday Functioning ◽  
Significant Relationships ◽  
Increased Risk ◽  
Memory For Intentions ◽  
Event Based

AbstractObjective:Persons with multiple sclerosis (MS) often report prospective memory (PM) failures that directly impact their everyday life. However, it is not known whether PM deficits confer an increased risk of poorer everyday functioning. The aims of this study were to: (1) compare time- (Time-PM) and event-based PM (Event-PM) performance between persons with MS and healthy controls (HCs), (2) examine the neuropsychological correlates of PM in MS, and (3) examine the relationship between PM and everyday functioning in MS.Method:A between-subjects design was used to examine 30 adults with MS and 30 community-dwelling HC. Participants were administered the Memory for Intentions Screening Test (MIST) to assess PM skills, the Actual Reality™ (AR) to assess everyday functioning, and a battery of cognitive tests.Results:The MS group performed significantly worse on Time-PM compared to HC but not on Event-PM tasks. While both Time-PM and Event-PM subscales were correlated with retrospective learning and memory, the MIST Time-PM subscale was correlated with executive functions. Significant correlations were observed between AR and the MIST Time-PM, but not Event-PM, subscales.Conclusions:The results highlight the role of executive functions on Time-PM. Furthermore, significant relationships with AR extend the ecological validity of the MIST to MS populations.

scholarly journals Ecological Validity and Neuroanatomical Correlates of the NIH EXAMINER Executive Composite Score

2013 ◽  
Vol 20 (1) ◽  
pp. 20-28 ◽  
Author(s):  
Katherine L. Possin ◽  
Amanda K. LaMarre ◽  
Kristie A. Wood ◽  
Dan M. Mungas ◽  
Joel H. Kramer
Keyword(s):  
Executive Function ◽  
Executive Functions ◽  
Real World ◽  
Cognitive Abilities ◽  
Ecological Validity ◽  
Brain Regions ◽  
Dorsolateral Prefrontal ◽  
Scale Scores ◽  
Frontal Systems ◽  
Behavior Scale

AbstractExecutive functions refer to a constellation of higher-level cognitive abilities that enable goal-oriented behavior. The NIH EXAMINER battery was designed to assess executive functions comprehensively and efficiently. Performance can be summarized by a single score, the “Executive Composite,” which combines measures of inhibition, set-shifting, fluency, and working memory. We evaluated the ecological validity of the Executive Composite in a sample of 225 mixed neurological patients and controls using the Frontal Systems Behavior Scale (FrSBe), an informant-based measure of real-world executive behavior. In addition, we investigated the neuroanatomical correlates of the Executive Composite using voxel-based morphometry in a sample of 37 participants diagnosed with dementia, mild cognitive impairment, or as neurologically healthy. The Executive Composite accounted for 28% of the variance in Frontal Systems Behavior Scale scores beyond age. Even after including two widely used executive function tests (Trails B and Stroop) as covariates, the Executive Composite remained a significant predictor of real-world behavior. Anatomically, poorer scores on the Executive Composite were associated with smaller right and left dorsolateral prefrontal volumes, brain regions critical for good executive control. Taken together, these results suggest that the Executive Composite measures important aspects of executive function not captured by standard measures and reflects the integrity of frontal systems. (JINS, 2013,19, 1–9)

scholarly journals Orbitofrontal cortex-related executive functions in children and adolescents: their assessment and its ecological validity

2010 ◽  
Author(s):  
Michele Poletti
Keyword(s):  
Executive Functions ◽  
Children And Adolescents ◽  
Orbitofrontal Cortex ◽  
Ecological Validity ◽  
Delay Of Gratification ◽  
Delay Aversion ◽  
Child Neuropsychology

This paper discusses the ecological validity of recently proposed tasks for the assessment of executive functions related to the Orbitofrontal Cortex, in children and adolescents. This topic is discussed considering two conceptually distinct approaches to ecological validity, verisimilitude and veridicality. The issue of ecological validity has been increasingly recognized in child neuropsychology, but an attention to this topic seems to lack in studies that assessed the development of Orbitofrontal cortex-related executive functions in children and adolescents with gambling tasks or delay of gratification tasks. An exception is represented by ADHD population, where specially delay aversion tasks seem to have a good veridicality-oriented ecological validity. The weakness of literature about the ecological validity of these executive measures must encourage to keep in mind ecological validity during the development of new tasks.

Handwriting Measures and Executive Functions Among Adolescents Referred to OT

2021 ◽  
Vol 75 (Supplement_2) ◽  
pp. 7512505220p1-7512505220p1
Author(s):  
Liat Hen-Herbst
Keyword(s):  
Executive Function ◽  
Executive Functions ◽  
Underlying Mechanism ◽  
Daily Functioning ◽  
Daily Function ◽  
Primary Author

Abstract Date Presented Accepted for AOTA INSPIRE 2021 but unable to be presented due to online event limitations. Handwriting difficulties (HD) are a top reason for referral to OT. Executive function (EF) was described as an underlying mechanism of daily functioning and HD. This study found differences between handwriting of adolescents with HD and their peers with other daily function difficulties but no relation between EFs and legibility. The Handwriting Legibility Scale is suggested for evaluation of legibility in adolescents. Primary Author and Speaker: Liat Hen-Herbst

scholarly journals Super-unsupervised text classification for labeling online political hate

2021 ◽  
Author(s):  
Stig Hebbelstrup Rye Rasmussen ◽  
Alexander Bor ◽  
Mathias Osmundsen ◽  
Michael Bang Petersen
Keyword(s):  
Unsupervised Learning ◽  
Discriminant Validity ◽  
Ecological Validity ◽  
Face Validity ◽  
Social Scientists ◽  
Psychological Constructs ◽  
Convergent And Discriminant Validity ◽  
Novel Approach ◽  
Learning Techniques ◽  
Data Source

We live in a world of text but the sheer magnitude of social media data coupled with a need to measure complex psychological constructs have made this important source of data difficult to use for many social scientists. Either researchers engage in costly hand-coding of thousands of texts using supervised techniques or in unsupervised techniques where the measurement of predefined constructs are difficult. We propose a novel approach which we call super-unsupervised learning using the psychologically complex construct online political hate. This approach draws on the best features from both supervised and unsupervised learning techniques: Measurements of complex psychological constructs without a single labelled data source. We first outline the approach and then provide tests of (i) face validity, (ii) convergent and discriminant validity, (iii) criterion validity, (iv) external validity and (v) ecological validity.

Face validity of the youth Multiple Errands Test (yMET) in the community: A focus group and pilot study

2018 ◽  
Vol 82 (4) ◽  
pp. 248-258 ◽  
Author(s):  
Vanessa L Hanberg ◽  
Diane E MacKenzie ◽  
Joan Versnel
Keyword(s):  
Pilot Study ◽  
Focus Group ◽  
Executive Functions ◽  
Test Performance ◽  
Early Adulthood ◽  
Group Analysis ◽  
Community Setting ◽  
Face Validity ◽  
Shopping Mall ◽  
Multiple Errands Test

Introduction During late adolescence and early adulthood, maturation of cognitive functions including executive functions are occurring. The multiple errands test is an assessment of real-world executive functions and, to date, non-virtual reality multiple errands test research has focused primarily on adults with acquired brain injury in hospital settings. There is poor evidence across multiple errands test studies for content and face validity and limited studies in the community. This study aimed to explore multiple errands test face validity for typically developing youth (age 16–24 years) and describe their community setting performance on a youth multiple errands test. Methods A youth focus group ( N = 5) was conducted to explore perceptions of the multiple errands test. From their input, the youth multiple errands test was developed and pilot tested ( N = 9) in a shopping mall. Results Two themes emerged from focus group analysis and limited changes, relevant to youth, were made to develop the youth multiple errands test. The focus group and pilot study found the youth multiple errands test was acceptable and cognitively challenging for youth, with older youth performing better than younger youth. Overall youth multiple errands test performance suggests similarities to healthy adults in previous studies. Conclusion Findings must be interpreted with caution since the sample was small, but preliminary results indicate that future studies with the youth multiple errands test are feasible and warranted.

Sign in / Sign up

Export Citation Format

Share Document