Are Items More Than Indicators? An examination of psychometric homogeneity, item-specific effects, and consequences for structural equation models

Concerns of measurement error often motivate researchers to aggregate item information, using simple heuristics (e.g., sum scores) or latent variable methods, to mitigate unwanted effects such as parameter bias and attenuation. These approaches are often invoked without acknowledging that many scales in practice likely fail to possess the necessary properties for these models to be sufficient (i.e., positive conditional association and vanishing conditional dependence). We argue that measures which are not psychometrically homogeneous likely contain item specific effects particularly when examined in conjunction with external variables. We demonstrate this using a clinical empirical example assessing risk factors for suicidal ideation and show that measures constructed in alignment with principles of psychometric homogeneity are most appropriately modeled at the scale (or subscale) level while other measures should be considered at the item level. As a result, latent variable applications to such instruments are susceptible to interpretational confounding. The effects of interpretational confounding on R2, root mean square error, and model parameters are evaluated in a small simulation study. We conclude that item specific effects are not uncommon in practice and impact both explanatory and predictive research. Our findings suggest that classical approaches to addressing measurement error are insufficient to fully capture the breadth of instruments implemented in practice. Careful consideration of both the scale construction process and roles of scale items in the broader psychological theory are necessary prior to the application of traditional measurement methods.

Latent Cognitive Class at Enrollment Predicts Future Cognitive Trajectories of Decline in a Community Sample of Older Adults

Background: Methods that can identify subgroups with different trajectories of cognitive decline are crucial for isolating the biologic mechanisms which underlie these groupings. Objective: This study grouped older adults based on their baseline cognitive profiles using a latent variable approach and tested the hypothesis that these groups would differ in their subsequent trajectories of cognitive change. Methods: In this study we applied time-varying effects models (TVEMs) to examine the longitudinal trajectories of cognitive decline across different subgroups of older adults in the Rush Memory and Aging Project. Results: A total of 1,662 individuals (mean age = 79.6 years, SD = 7.4, 75.4%female) participated in the study; these were categorized into five previously identified classes of older adults differing in their baseline cognitive profiles: Superior Cognition (n = 328, 19.7%), Average Cognition (n = 767, 46.1%), Mixed-Domains Impairment (n = 71, 4.3%), Memory-Specific Impairment (n = 274, 16.5%), and Frontal Impairment (n = 222, 13.4%). Differences in the trajectories of cognition for these five classes persisted during 8 years of follow-up. Compared with the Average Cognition class, The Mixed-Domains and Memory-Specific Impairment classes showed steeper rates of decline, while other classes showed moderate declines. Conclusion: Baseline cognitive classes of older adults derived through the use of latent variable methods were associated with distinct longitudinal trajectories of cognitive decline that did not converge during an average of 8 years of follow-up.

Developing and Validating Clinical Questionnaires

Coinciding with the development and revision of conceptual models of psychopathology, there has been a proliferation in the number of self-report clinical questionnaires and studies evaluating their psychometric properties. Unfortunately, many clinical measures are constructed and evaluated using suboptimal methods. This review provides current guidelines for the conceptualization, development, and psychometric validation of clinical questionnaires using latent variable methods. A two-stage exploratory-confirmatory framework is provided. The exploratory stage includes item selection and revision, initial structural evaluation, and preliminary tests of concurrent validity (e.g., convergent and discriminant). The confirmatory stage involves replicating factor structure using a more restrictive model, identifying areas of model strain, conducting additional tests of concurrent and predictive validity, and evaluating measurement invariance. Recommendations are provided for ( a) item generation, ( b) how to use different types of exploratory and confirmatory factor models to determine structure, and ( c) evaluating reliability and validity using a latent variable measurement model approach. Expected final online publication date for the Annual Review of Clinical Psychology, Volume 17 is May 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Dual Sensory Impairment and Cognitive Performance, MCI, and Dementia in Older Adults: The ARIC Neurocognitive Study

Dual sensory impairment (DSI) affects 11.3% of adults aged ≥80 years. Hearing and vision impairments are each associated with cognitive decline and dementia, but DSI’s impact is unknown. All-cause dementia and mild cognitive impairment (MCI) were adjudicated using longitudinal cognitive information. Ten neurocognitive tests were summarized using latent variable methods. Hearing was measured using pure tone better-ear thresholds (0.5-4 kHz) and vision with better-eye presenting distance visual acuity and/or contrast sensitivity. In 881 adults (79±4 years, 44% black, 64% female), DSI (vs. no hearing or vision impairment) was cross-sectionally associated with -0.17 standard deviations (SD) [95% confidence interval (CI): -0.32, -0.02] lower global cognitive score and an 87% increased odds (95% CI: 1.01, 3.45) of combined MCI/dementia, after full adjustment for demographic and clinical factors. Future longitudinal research should elucidate the mechanism underlying this association to determine if treatment can delay cognitive decline and MCI/dementia in older adults.

An observational study on the rate of reporting of adverse event on healthcare staff in a mental health setting: An application of Poisson expectation maximisation analysis on nurse staffing data

Evidence highlights the intrinsic link between nurse staffing and expertise, and outcomes for service users of healthcare, and that workforce retention is linked to the clinical and organisational experiences of employees. However, this understanding is less well established in mental health. This study comprises a retrospective observational study carried out on routinely collected data from a large mental healthcare provider. Two databases comprising nurse staffing levels and adverse events were modelled using latent variable methods to account for the presence of multiple underlying behaviours. The analysis reveals a strong dependence of the rate of adverse events on the location and perceived clinical demand of the wards, and a reduction in adverse events where registered nurses exceed ‘clinically required levels’. In the first study of its kind, these findings present significant implications for nursing workforce policy and present an opportunity to not only improve safety but potentially impact nurse retention.

Hybrid Measurement Models for Technology-Enhanced Assessments Through mIRT-bayes

Technology-enhanced assessments (TEAs) are rapidly emerging in educational measurement. In contexts such as simulation and gaming, a common challenge is handling complex streams of information, for which new statistical innovations are needed that can provide high quality proficiency estimates for the psychometrics of complex TEAs. Often in educational assessments with formal measurement models, latent variable models such as item response theory (IRT) are used to generate proficiency estimates from evidence elicited. Such robust techniques have become a foundation of educational assessment, when models fit. Another less common approach to compile evidence is through Bayesian networks, which represent a set of random variables and their conditional dependencies via a directed acyclic graph. Network approaches can be much more flexibly designed for complex assessment tasks and are often preferred by task developers, for technology-enhanced settings. However, the Bayesian network-based statistical models often are difficult to validate and to gauge the stability and accuracy, since the models make assumptions regarding conditional dependencies that are difficult to test. Here a new measurement model family, mIRT-bayes, is proposed to gain advantages of both latent variable models and network techniques combined through hybridization. Specifically, the technique described here embeds small Bayesian networks within an overarching multidimensional IRT model (mIRT), preserving the flexibility for task design while retaining the robust statistical properties of latent variable methods. Applied to simulation-based data from Harvard's Virtual Performance Assessments (VPA), the results of the new model show acceptable fit for the overarching mIRT model, along with reduction of the standard error of measurement through the embedded Bayesian networks, compared to use of mIRT alone. Overall for respondents, a finer grain-size of inference is made possible without additiona testing time or scoring resources, showing potentially promise for this family of new hybrid models.

Music self-concept and self-esteem formation in adolescence: A comparison between individual and normative models of importance within a latent framework

We examined the possible effects of six dimensions of music self-concept on determination of self-esteem, through the application of models based on individual and normative-group importance. Previous studies have supported the individual model of importance in narrowly defined self-domains such as spiritual self-concept that might be unimportant for most people, but very important for some people. However, results from more recent studies of spiritual, academic, and physical self-concepts involving latent variable methodologies support the normative-group model. Here, we extended the use of latent variable methods to music self-concept using a sample of 512 junior high students (11–16 years old). Our results for music-reading skills supported the individual importance model rather than the normative-group importance model. Additional results revealed that singing, instrument playing, and the importance of instrument playing had direct rather than interactive linkages with self-esteem. Collectively, these results highlight differential effects of performance (singing, instrument playing) and knowledge (reading) on self-esteem, and imply that strategies to enhance self-esteem may vary within different domains of music instruction and participation. At a more general level, the findings together with those from previous studies indicate that interconnections between specific and global aspects of self-concept vary across domains and are more complex than previously thought.