Five shades of gray: Conceptual and methodological issues around multivariate models of performance validity

2021 ◽  
pp. 1-35
Author(s):  
Laszlo A. Erdodi
Keyword(s):  
Signal Detection ◽  
Psychiatric Symptoms ◽  
Theoretical Models ◽  
Performance Validity ◽  
Multivariate Models ◽  
Free Standing ◽  
Validity Assessment ◽  
Performance Validity Assessment ◽  
Neurocognitive Profile ◽  
Validity Tests

OBJECTIVE: This study was designed to empirically investigate the signal detection profile of various multivariate models of performance validity tests (MV-PVTs) and explore several contested assumptions underlying validity assessment in general and MV-PVTs specifically. METHOD: Archival data were collected from 167 patients (52.4%male; M Age = 39.7) clinicially evaluated subsequent to a TBI. Performance validity was psychometrically defined using two free-standing PVTs and five composite measures, each based on five embedded PVTs. RESULTS: MV-PVTs had superior classification accuracy compared to univariate cutoffs. The similarity between predictor and criterion PVTs influenced signal detection profiles. False positive rates (FPR) in MV-PVTs can be effectively controlled using more stringent multivariate cutoffs. In addition to Pass and Fail, Borderline is a legitimate third outcome of performance validity assessment. Failing memory-based PVTs was associated with elevated self-reported psychiatric symptoms. CONCLUSIONS: Concerns about elevated FPR in MV-PVTs are unsubstantiated. In fact, MV-PVTs are psychometrically superior to individual components. Instrumentation artifacts are endemic to PVTs, and represent both a threat and an opportunity during the interpretation of a given neurocognitive profile. There is no such thing as too much information in performance validity assessment. Psychometric issues should be evaluated based on empirical, not theoretical models. As the number/severity of embedded PVT failures accumulates, assessors must consider the possibility of non-credible presentation and its clinical implications to neurorehabilitation.

scholarly journals Symptom and Performance Validity Assessment in European Countries: an Update

2021 ◽  
Author(s):  
Thomas Merten ◽  
Brechje Dandachi-FitzGerald ◽  
Vicki Hall ◽  
Thomas Bodner ◽  
Luciano Giromini ◽  
...  
Keyword(s):  
European Countries ◽  
Performance Validity ◽  
Validity Assessment ◽  
Performance Validity Assessment ◽  
And Performance

Geographic Variation and Instrumentation Artifacts: in Search of Confounds in Performance Validity Assessment in Adults with Mild TBI

2019 ◽  
Vol 12 (2) ◽  
pp. 127-145 ◽  
Author(s):  
Jonathan D. Lichtenstein ◽  
Matthew K. Greenacre ◽  
Laura Cutler ◽  
Kaitlyn Abeare ◽  
Shannon D. Baker ◽  
...  
Keyword(s):  
Geographic Variation ◽  
Performance Validity ◽  
Mild Tbi ◽  
Validity Assessment ◽  
Performance Validity Assessment

scholarly journals Multidimensional Malingering Criteria for Neuropsychological Assessment: A 20-Year Update of the Malingered Neuropsychological Dysfunction Criteria

2020 ◽  
Vol 35 (6) ◽  
pp. 735-764 ◽  
Author(s):  
Elisabeth M S Sherman ◽  
Daniel J Slick ◽  
Grant L Iverson
Keyword(s):  
Neuropsychological Assessment ◽  
Psychiatric Symptoms ◽  
Detection Methods ◽  
Performance Validity ◽  
Neurocognitive Dysfunction ◽  
Neuropsychological Dysfunction ◽  
Malingered Neurocognitive Dysfunction ◽  
Validity Tests ◽  
External Incentives

Abstract Objectives Empirically informed neuropsychological opinion is critical for determining whether cognitive deficits and symptoms are legitimate, particularly in settings where there are significant external incentives for successful malingering. The Slick, Sherman, and Iversion (1999) criteria for malingered neurocognitive dysfunction (MND) are considered a major milestone in the field’s operationalization of neurocognitive malingering and have strongly influenced the development of malingering detection methods, including serving as the criterion of malingering in the validation of several performance validity tests (PVTs) and symptom validity tests (SVTs) (Slick, D.J., Sherman, E.M.S., & Iverson, G. L. (1999). Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research. The Clinical Neuropsychologist, 13(4), 545–561). However, the MND criteria are long overdue for revision to address advances in malingering research and to address limitations identified by experts in the field. Method The MND criteria were critically reviewed, updated with reference to research on malingering, and expanded to address other forms of malingering pertinent to neuropsychological evaluation such as exaggeration of self-reported somatic and psychiatric symptoms. Results The new proposed criteria simplify diagnostic categories, expand and clarify external incentives, more clearly define the role of compelling inconsistencies, address issues concerning PVTs and SVTs (i.e., number administered, false positives, and redundancy), better define the role of SVTs and of marked discrepancies indicative of malingering, and most importantly, clearly define exclusionary criteria based on the last two decades of research on malingering in neuropsychology. Lastly, the new criteria provide specifiers to better describe clinical presentations for use in neuropsychological assessment. Conclusions The proposed multidimensional malingering criteria that define cognitive, somatic, and psychiatric malingering for use in neuropsychological assessment are presented.

scholarly journals Stronger Together: The Wechsler Adult Intelligence Scale—Fourth Edition as a Multivariate Performance Validity Test in Patients with Traumatic Brain Injury

2019 ◽  
Vol 35 (2) ◽  
pp. 188-204 ◽  
Author(s):  
Laszlo A Erdodi ◽  
Christopher A Abeare
Keyword(s):  
Classification Accuracy ◽  
Digit Span ◽  
Performance Validity ◽  
Adult Intelligence Scale ◽  
Intelligence Scale ◽  
Wechsler Adult Intelligence Scale ◽  
Adult Intelligence ◽  
Validity Assessment ◽  
Performance Validity Assessment ◽  
Wechsler Adult Intelligence

Abstract Objective This study was designed to evaluate the classification accuracy of a multivariate model of performance validity assessment using embedded validity indicators (EVIs) within the Wechsler Adult Intelligence Scale—Fourth Edition (WAIS-IV). Method Archival data were collected from 100 adults with traumatic brain injury (TBI) consecutively referred for neuropsychological assessment in a clinical setting. The classification accuracy of previously published individual EVIs nested within the WAIS-IV and a composite measure based on six independent EVIs were evaluated against psychometrically defined non-credible performance. Results Univariate validity cutoffs based on age-corrected scaled scores on Coding, Symbol Search, Digit Span, Letter-Number-Sequencing, Vocabulary minus Digit Span, and Coding minus Symbol Search were strong predictors of psychometrically defined non-credible responding. Failing ≥3 of these six EVIs at the liberal cutoff improved specificity (.91–.95) over univariate cutoffs (.78–.93). Conversely, failing ≥2 EVIs at the more conservative cutoff increased and stabilized sensitivity (.43–.67) compared to univariate cutoffs (.11–.63) while maintaining consistently high specificity (.93–.95). Conclusions In addition to being a widely used test of cognitive functioning, the WAIS-IV can also function as a measure of performance validity. Consistent with previous research, combining information from multiple EVIs enhanced the classification accuracy of individual cutoffs and provided more stable parameter estimates. If the current findings are replicated in larger, diagnostically and demographically heterogeneous samples, the WAIS-IV has the potential to become a powerful multivariate model of performance validity assessment. Brief Summary Using a combination of multiple performance validity indicators embedded within the subtests of theWechsler Adult Intelligence Scale, the credibility of the response set can be establishedwith a high level of confidence. Multivariatemodels improve classification accuracy over individual tests. Relying on existing test data is a cost-effective approach to performance validity assessment.

Sign in / Sign up

Export Citation Format

Share Document