Impact of Item Orientation on the Structural Validity of Self-Report Measures

2014 ◽  
Vol 33 (3) ◽  
pp. 278-290 ◽  
Author(s):  
Tony Mowbray ◽  
Christopher Boyle ◽  
Kate Jacobs
Keyword(s):  
Self Report ◽  
Structural Validity ◽  
Item Orientation

scholarly journals A self-report measure of perfectionism: A confirmatory factor analysis of the Swedish version of the Clinical Perfectionism Questionnaire

2021 ◽  
Vol 3 (4) ◽  
Author(s):  
Allison Parks ◽  
Jakob Clason van de Leur ◽  
Marcus Strååt ◽  
Fredrik Elfving ◽  
Gerhard Andersson ◽  
...  
Keyword(s):  
Factor Analysis ◽  
Confirmatory Factor Analysis ◽  
Interpersonal Relationships ◽  
Self Report ◽  
Negative Consequences ◽  
Structural Validity ◽  
Swedish Version ◽  
Confirmatory Factor ◽  
Best Fit ◽  
Report Measure

Background Perfectionism is often defined as the strive for achievement and high standards, but can also lead to negative consequences. In addition to affecting performance and interpersonal relationships, perfectionism can result in mental distress. A number of different self-report measures have been put forward to assess perfectionism. Specifically intended for clinical practice and research, the Clinical Perfectionism Questionnaire (CPQ) was developed and is presently available in English and Persian. To promote its use in additional contexts, the current study has translated and investigated the psychometric properties of the Swedish version of the CPQ. Method A Confirmatory Factor Analysis was performed to examine the best fit with data, using a priori-models and a sample of treatment-seeking participants screened for eligibility to receive Internet-based cognitive behavior therapy (n = 223). Results The results indicated a lack of fit with data. A two-factor structure without the two reversed items (2 and 8) exhibited the best fit, perfectionistic strivings and perfectionistic concerns, but still had poor structural validity. Correlations with self-report measures of perfectionism, depression, anxiety, dysfunctional beliefs, self-criticism, quality of life, and self-compassion were all in the expected directions. Eight-week test-retest correlation was Pearson r = .62, 95% Confidence Interval [.45, .74], using data from 72 participants in the wait-list control, and the internal consistency for the CPQ, once removing the reversely scored items, was Cronbach’s α = .72. Conclusion The CPQ can be used as a self-report measure in Swedish, but further research on its structural validity is needed.

scholarly journals Hidden Invalidity Among 15 Commonly Used Measures in Social and Personality Psychology

2020 ◽  
Vol 3 (2) ◽  
pp. 166-184 ◽  
Author(s):  
Ian Hussey ◽  
Sean Hughes
Keyword(s):  
Measurement Invariance ◽  
Internal Consistency ◽  
Degrees Of Freedom ◽  
Self Report ◽  
Structural Validity ◽  
Data Set ◽  
Personality Psychology ◽  
Reliability Factor ◽  
And Gender ◽  
Gender Groups

It has recently been demonstrated that metrics of structural validity are severely underreported in social and personality psychology. We comprehensively assessed structural validity in a uniquely large and varied data set ( N = 144,496 experimental sessions) to investigate the psychometric properties of some of the most widely used self-report measures ( k = 15 questionnaires, 26 scales) in social and personality psychology. When the scales were assessed using the modal practice of considering only internal consistency, 88% of them appeared to possess good validity. Yet when validity was assessed comprehensively (via internal consistency, immediate and delayed test-retest reliability, factor structure, and measurement invariance for age and gender groups), only 4% demonstrated good validity. Furthermore, the less commonly a test was reported in the literature, the more likely the scales were to fail that test (e.g., scales failed measurement invariance much more often than internal consistency). This suggests that the pattern of underreporting in the field may represent widespread hidden invalidity of the measures used and may therefore pose a threat to many research findings. We highlight the degrees of freedom afforded to researchers in the assessment and reporting of structural validity and introduce the concept of validity hacking ( v-hacking), similar to the better-known concept of p-hacking. We argue that the practice of v-hacking should be acknowledged and addressed.

Structural Validity of a Self-Report Concussion-Related Symptom Scale

2006 ◽  
Vol 38 (1) ◽  
pp. 27-32 ◽  
Author(s):  
SCOTT G. PILAND ◽  
ROBERT W. MOTL ◽  
KEVIN M. GUSKIEWICZ ◽  
MICHAEL MCCREA ◽  
MICHAEL S. FERRARA
Keyword(s):  
Self Report ◽  
Symptom Scale ◽  
Structural Validity ◽  
Related Symptom

scholarly journals Structural Validity of the Rheumatology Attitudes Index in Systemic Sclerosis: Analysis from the UCLA Scleroderma Quality of Life Study

2017 ◽  
Vol 44 (6) ◽  
pp. 795-798
Author(s):  
Shadi Gholizadeh ◽  
Sarah D. Mills ◽  
Rina S. Fox ◽  
Erin L. Merz ◽  
Scott C. Roesch ◽  
...  
Keyword(s):  
Quality Of Life ◽  
Factor Analysis ◽  
Systemic Sclerosis ◽  
Los Angeles ◽  
Self Report ◽  
University Of California ◽  
Structural Validity ◽  
Life Study ◽  
The University

Objective.To evaluate the structural validity of the Rheumatology Attitudes Index (RAI), a widely used measure of rheumatic disease–related helplessness in patients with systemic sclerosis (SSc).Methods.Patients with physician-confirmed SSc from the University of California, Los Angeles (UCLA) Scleroderma Quality of Life Study (n = 208) received clinical examinations and completed self-report questionnaires. The structural validity of the RAI was examined through confirmatory and exploratory factor analysis (CFA/EFA).Results.A tenable factor structure was not identified through CFA or EFA.Conclusion.The present structural analysis did not support the use of the RAI with SSc patients.

Structural Validity of the HBSC Bullying Measure: Self-Report Rating Scales of Youth Victimization and Perpetration Behavior

2017 ◽  
Vol 36 (6) ◽  
pp. 628-643 ◽  
Author(s):  
Anthony J. Roberson ◽  
Tyler L. Renshaw
Keyword(s):  
Rating Scales ◽  
Measurement Model ◽  
Self Report ◽  
Assessment Instruments ◽  
Structural Validity ◽  
School Aged Children ◽  
Diverse Student Populations ◽  
Diverse Student ◽  
Survey Results ◽  
Perpetrators And Victims

Bullying involvement among youth has consistently been linked to potentially serious consequences for both perpetrators and victims. To help clarify the nature and scope of youth bullying involvement, empirically validated assessment instruments measuring victimization and perpetration behaviors are needed for use in research and practice. The present study investigated the latent factor structure of the 22 victimization and perpetration items within the 2009-2010 Health Behavior in School-Aged Children (HBSC) self-report survey. Structural validity analyses were conducted using a representative sample of U.S. youth in Grades 5 to 10 ( N = 11,449) obtained from the national administration of the HBSC self-report survey. Results suggested a two-factor latent structure comprised of bullying victimization and perpetration was the most theoretically and psychometrically sound measurement model for these data. In addition, multigroup measurement and structural invariance analyses showed that this model functioned equitably across student race/ethnicity, sex, and grade level, supporting the measure’s use with diverse student populations.

scholarly journals Commentary_on_Hussey&Hughes(2020)

2020 ◽  
Author(s):  
Eunike Wetzel ◽  
Brent Roberts
Keyword(s):  
Measurement Invariance ◽  
Factor Structure ◽  
Self Report ◽  
Measurement Properties ◽  
Structural Validity ◽  
Consistency Test ◽  
Personality Psychology ◽  
Reliability Factor ◽  
Psychological Scales ◽  
Test Retest Reliability

Hussey and Hughes (2020) analyzed four aspects (internal consistency, test-retest reliability, factor structure, and measurement invariance) relevant to the structural validity of psychological scales in 15 self-report questionnaires and concluded that social and personality psychology has a “hidden invalidity” problem. We argue that their argument that the field ignores structural validity (hence “hidden”) is incorrect because many published papers specifically investigate the measurement properties of instruments applied in social and personality psychology. Furthermore, we show that the models they used to test structural validity do not match the construct space for many of the measures. Lastly, we argue that their conclusion that measures are invalid based on a pass/fail decision for measurement invariance is overly simplistic. Rather, partial measurement invariance and the effect size of the noninvariance should be considered. Moving forward, we think it would be important for all researchers to more actively engage with prior measurement research, know the limits of existing measures, and invest in a deeper examination of the psychometric properties of their own measures in each of their studies.

Evaluating the construct validity and internal consistency of the Sickness Questionnaire in a Swedish sample of adults with longstanding pain

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Jenny Åström ◽  
Linda Holmström ◽  
Bianka Karshikoff ◽  
Anna Andreasson ◽  
Mike K. Kemani
Keyword(s):  
Construct Validity ◽  
Internal Consistency ◽  
Behavioral Responses ◽  
Sickness Behavior ◽  
Model Fit ◽  
Self Report ◽  
Low Grade ◽  
Structural Validity ◽  
Adequate Model ◽  
Swedish Sample

Abstract Objectives Low-grade inflammation is a possible contributing factor in the development and persistence of chronic primary pain syndromes. Related to inflammatory activity is sickness behavior, a set of behavioral responses including increased pain sensitivity, fatigue, malaise, fever, loss of appetite, as well as depressive behavior and anhedonia. To capture these behavioral responses and their relation to longstanding pain, psychometrically sound self-report questionnaires are needed. The Sickness Questionnaire (SicknessQ) was developed to assess self-reported sickness behavior based on studies on acute immune activation while maintaining relevance for persistent conditions. The aim of the current study was to evaluate aspects of the validity and reliability of the SicknessQ in a Swedish sample of persons with longstanding pain. Methods Aspects of construct validity were evaluated by means of performing a confirmatory factor analysis (CFA) (testing structural validity) and by relevant hypothesis testing i.e., that ratings of sickness behavior in combination with other related factors (e.g., depression and anxiety) would be significantly related to ratings of avoidance. Reliability was evaluated by means of analyzing the internal consistency of items. Results Following the CFA, a non-significant Chi-Square test (χ2 [32, N=190] = 42.95, p=0.094) indicated perfect model fit. Also, the relative fit indices supported adequate model fit (CFI = 0.978; TLI = 0.969; RMSEA = 0.0430). Sickness behavior (p<0.0001), depression (p<0.05) and pain duration (p<0.05) significantly contributed to the regression model, explaining 45% of the total variance in avoidance. Internal consistency was adequate, as indicated by a Cronbach’s α value of 0.82 for the entire questionnaire. Conclusions Results indicate that the SicknessQ has adequate structural validity as well as adequate internal consistency, and is significantly associated with avoidance. The SicknessQ appears to have utility as a self-report questionnaire to assess symptoms of sickness behavior for adults with longstanding pain.

scholarly journals Using Classical Test Theory to Determine the Psychometric Properties of the Deglutition Handicap Index

Dysphagia ◽  
2021 ◽  
Author(s):  
Renée Speyer ◽  
Reinie Cordier ◽  
Clara Bouix ◽  
Yohan Gallois ◽  
Virginie Woisard
Keyword(s):  
Hypothesis Testing ◽  
Psychometric Properties ◽  
Internal Consistency ◽  
Diagnostic Performance ◽  
Test Theory ◽  
Self Report ◽  
Factor Analyses ◽  
Health Measurement ◽  
Structural Validity ◽  
Report Measure

AbstractThe Deglutition Handicap Index (DHI) is a self-report measure for patients at risk of oropharyngeal dysphagia on deglutition-related aspects of functional health status (FHS) and health-related quality of life (HR-QoL). The DHI consists of 30 items which are subsumed within the Symptom, Functional and Emotional subscales. The purpose of this study was to evaluate the psychometric properties of the DHI using Classic Test Theory according to the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) criteria. A total of 453 patients with dysphagia with different aetiologies were recruited concurrently at two academic hospitals. Dysphagia was confirmed by fiberoptic endoscopic and/or videofluoroscopic evaluation of swallowing. In addition, a healthy control group of 132 participants were recruited. Structural validity was determined using exploratory and confirmatory factor analyses and internal consistency by calculating Cronbach’s alpha coefficients. Hypothesis testing was evaluated using Mann–Whitney U-tests, linear regression analysis and correlations analysis. Diagnostic performance and receiver operating characteristic curves analysis were calculated. Factor analyses indicated that the DHI is a unidimensional measure. The DHI has good internal consistency with some indication of item redundancy, weak to moderate structural validity and strong hypothesis testing for construct validity. The DHI shows high diagnostic performance as part of criterion validity. These findings support that the DHI is an appropriate choice as a patient self-report measure to evaluate FHS and HR-QoL in dysphagia. Ongoing validation to assess the measure for possible item redundancy and to examine the dimensionality of the DHI using item response theory is recommended.

scholarly journals Psychometric validation of the Arthritis Helplessness Index in systemic lupus erythematosus

Lupus ◽  
2018 ◽  
Vol 27 (12) ◽  
pp. 1980-1984 ◽  
Author(s):  
S Gholizadeh ◽  
D R Azizoddin ◽  
S D Mills ◽  
G Zamora-Racaza ◽  
H M K Potemra ◽  
...  
Keyword(s):  
Systemic Lupus Erythematosus ◽  
Internal Consistency ◽  
Lupus Erythematosus ◽  
Convergent Validity ◽  
Internal Consistency Reliability ◽  
Self Report ◽  
Psychometric Validation ◽  
Systemic Lupus ◽  
Structural Validity ◽  
Fit Index

Objective Helplessness is a relevant construct in systemic lupus erythematosus (SLE), an unpredictable chronic illness with no known cure characterized by relapsing and remitting features. However, no measure of helplessness has been validated in this population. The present study examined the structural validity, reliability, and convergent validity of the Arthritis Helplessness Index, a measure initially developed for rheumatoid arthritis populations, in a sample of patients with SLE. Methods Patients with SLE ( N = 136) receiving medical care at a private hospital completed the Arthritis Helplessness Index and other self-report measures. The structural validity of the Arthritis Helplessness Index was examined using confirmatory factor analysis. Internal consistency reliability was evaluated with Cronbach’s coefficient alpha. Pearson product–moment correlations were used to examine convergent validity with measures of depression, anxiety and mastery. Results The five-item Arthritis Helplessness Index–Helplessness measure demonstrated a tenable factor structure (comparative fit index 0.98, root mean square error of approximation 0.06, standardized root mean residual 0.04). Internal consistency reliability was fair (α = 0.69). Convergent validity was evidenced by significant correlations with measures of depression, anxiety and mastery. Conclusion The five-item Arthritis Helplessness Index–Helplessness scale can confidently be used as a measure of helplessness in SLE.

Sign in / Sign up

Export Citation Format

Share Document