Applications of generalizability theory and their relations to classical test theory and structural equation modeling.

2018 ◽  
Vol 23 (1) ◽  
pp. 1-26 ◽  
Author(s):  
Walter P. Vispoel ◽  
Carrie A. Morris ◽  
Murat Kilinc
Keyword(s):  
Structural Equation Modeling ◽  
Structural Equation ◽  
Generalizability Theory ◽  
Classical Test Theory ◽  
Test Theory ◽  
Equation Modeling ◽  
Classical Test

Testing the Assumption of Uncorrelated Errors for Short Scales by Means of Structural Equation Modeling

2014 ◽  
Vol 35 (4) ◽  
pp. 201-211 ◽  
Author(s):  
André Beauducel ◽  
Anja Leue
Keyword(s):  
Structural Equation Modeling ◽  
Negative Correlation ◽  
Structural Equation ◽  
Error Score ◽  
Classical Test Theory ◽  
Test Theory ◽  
Equation Modeling ◽  
Eysenck Personality Questionnaire ◽  
Personality Questionnaire ◽  
True Score

It is shown that a minimal assumption should be added to the assumptions of Classical Test Theory (CTT) in order to have positive inter-item correlations, which are regarded as a basis for the aggregation of items. Moreover, it is shown that the assumption of zero correlations between the error score estimates is substantially violated in the population of individuals when the number of items is small. Instead, a negative correlation between error score estimates occurs. The reason for the negative correlation is that the error score estimates for different items of a scale are based on insufficient true score estimates when the number of items is small. A test of the assumption of uncorrelated error score estimates by means of structural equation modeling (SEM) is proposed that takes this effect into account. The SEM-based procedure is demonstrated by means of empirical examples based on the Edinburgh Handedness Inventory and the Eysenck Personality Questionnaire-Revised.

scholarly journals Improving generalizability coefficient estimate accuracy: A way to incorporate auxiliary information

2018 ◽  
Vol 11 (2) ◽  
pp. 205979911879139 ◽  
Author(s):  
Zhehan Jiang ◽  
Kevin Walker ◽  
Dexin Shi ◽  
Jian Cao
Keyword(s):  
Structural Equation Modeling ◽  
Structural Equation ◽  
Generalizability Theory ◽  
Theory Model ◽  
Bias Reduction ◽  
Equation Modeling ◽  
Coefficient Estimate ◽  
Auxiliary Variables ◽  
Modeling Approach ◽  
Generalizability Coefficient

Initially proposed by Marcoulides and further expanded by Raykov and Marcoulides, a structural equation modeling approach can be used in generalizability theory estimation. This article examines the utility of incorporating auxiliary variables into the structural equation modeling approach when missing data is present. In particular, the authors assert that by adapting a saturated correlates model strategy to structural equation modeling generalizability theory models, one can reduce any biased effects caused by missingness. Traditional approaches such as an analysis of variance do not possess such a feature. This article provides detailed instructions for adding auxiliary variables into a structural equation modeling generalizability theory model, demonstrates the corresponding benefits of bias reduction in generalizability coefficient estimate via simulations, and discusses issues relevant to the proposed approach.

scholarly journals Validation of the Simplified Chinese Version of FACT-Hep for Patients with Hepatocellular Carcinoma Based on Combinations of Classical Test Theory and Generalizability Theory

2020 ◽  
Author(s):  
Pingguang lei ◽  
zheng yang ◽  
wei li ◽  
jingqing ou ◽  
yingli cun ◽  
...  
Keyword(s):  
Hepatocellular Carcinoma ◽  
Generalizability Theory ◽  
Classical Test Theory ◽  
Well Being ◽  
Chinese Version ◽  
Test Theory ◽  
Validity And Reliability ◽  
Classical Test ◽  
Simplified Chinese ◽  
Reliability Coefficients

Abstract Background Quality of life (QOL) is now concerned worldwide in cancer clinical fields and the specific instrument FACT-Hep (Functional Assessment of Cancer Therapy- Hepatobiliary questionnaire) is widely used in English-spoken countries. However, the specific instruments for hepatocellular carcinoma patients in China were seldom and no formal validation on the Simplified Chinese Version of the FACT-Hep was carried out. This study was aimed to validate the Chinese FACT-Hep based on Combinations of Classical Test Theory and Generalizability Theory. Methods The Chinese Version of FACT-Hep and the QLICP-LI were used to measure QOL three times before and after treatments from a sample of 114 in-patients of hepatocellular carcinoma. The scale were evaluated by indicators such as validity and reliability coefficients Cronbach α, Pearson r, intra-class correlation (ICC), and standardized response mean. The Generalizability Theory (G theory) was also applied to addresses the dependability of measurements and estimation of multiple sources of variance. Results The Internal consistency Cronbach’s α coefficients were greater than 0.70 for all domains, and test-retest reliability coefficients for all domains and the overall were greater than 0.80 (exception of emotional Well-being 0.74) with the range from 0.81 to 0.96. G-coefficients and Ф-coefficients confirmed the reliability of the scale further with exact variance components. The domains of PWB, FWB and the overall scale had significant changes after treatments with SRM ranging from 0.40 to 0.69. Conclusions The Chinese version of FACT-Hep has good validity, reliability, and responsiveness, and can be used to measure QOL for patients with hepatocellular carcinoma in China.

Competence- and Integrity-Based Trust in Interorganizational Relationships: Which Matters More?

2015 ◽  
Vol 44 (3) ◽  
pp. 919-945 ◽  
Author(s):  
Brian L. Connelly ◽  
T. Russell Crook ◽  
James G. Combs ◽  
David J. Ketchen ◽  
Herman Aguinis
Keyword(s):  
Structural Equation Modeling ◽  
Transaction Costs ◽  
Structural Equation ◽  
Interorganizational Relationships ◽  
Technical Skills ◽  
Negative Information ◽  
Test Theory ◽  
Equation Modeling ◽  
Asymmetric Effects

Trust is an important factor for managing transaction costs within interorganizational relationships (IORs). Research on trust indicates that separate dimensions of trust arise from a partner’s competence (i.e., technical skills, experience, and reliability) and integrity (i.e., motives, honesty, and character), and that these dimensions have potentially unique effects. Because scholars rarely apply this distinction within IOR research, past studies may have masked important relationships involving competence- and integrity-based trust. In response, we build and test theory that explains how competence- and integrity-based trust have asymmetric effects on different kinds of transaction costs. In particular, we build on theory that describes how parties process positive and negative information about others’ behavior to predict that integrity-based trust in IORs is more potent for reducing transaction costs than is competence-based trust. We also theorize that building strong IORs requires more up-front investment with competence-based but not with integrity-based trust. By applying meta-analytic structural equation modeling to data on 37,366 IORs drawn from 150 samples, we find that integrity-based trust is about 10 times more effective at reducing these costs. A key implication is that managers seeking to improve the efficiency of their IORs may do well by performing competently, but they can do even better by building perceptions of integrity.

Does the Length of a Questionnaire Matter?

2014 ◽  
Vol 35 (4) ◽  
pp. 250-261 ◽  
Author(s):  
Matthias Ziegler ◽  
Arthur Poropat ◽  
Julija Mell
Keyword(s):  
Generalizability Theory ◽  
Classical Test Theory ◽  
Test Construction ◽  
Test Theory ◽  
Research And Practice ◽  
Advantages And Disadvantages ◽  
Classical Test ◽  
Reliability Estimates ◽  
Clear Description ◽  
Scale Length

Short personality questionnaires are increasingly used in research and practice, with some scales including as few as two to five items per personality domain. Despite the frequency of their use, these short scales are often criticized on the basis of their reduced internal consistencies and their purported failure to assess the breadth of broad constructs, such as the Big 5 factors of personality. One reason for this might be the use of principles routed in Classical Test Theory during test construction. In this study, Generalizability Theory is used to compare psychometric properties of different scales based on the NEO-PI-R and BFI, two widely-used personality questionnaire families. Applying both Classical Test Theory (CTT) and Generalizability Theory (GT) allowed to identify the inner workings of test shortening. CTT-based analyses indicated that longer is generally better for reliability, while GT allowed differentiation between reliability for relative and absolute decisions, while revealing how different variance sources affect test score reliability estimates. These variance sources differed with scale length, and only GT allowed clear description of these internal consequences, allowing more effective identification of advantages and disadvantages of shorter and longer scales. Most importantly, the findings highlight the potential error proneness of focusing solely on reliability and scale length in test construction. Practical as well as theoretical consequences are discussed.

Contemporary process to test the theory of a research model through covariance-based structural equation modeling in business research

2015 ◽  
Vol 27 (4) ◽  
pp. 447-458 ◽  
Author(s):  
Göran Svensson
Keyword(s):  
Structural Equation Modeling ◽  
Iterative Process ◽  
Structural Equation ◽  
Design Methodology ◽  
Test Theory ◽  
Equation Modeling ◽  
Research Model ◽  
Business Research ◽  
Content Type ◽  
Practical Implications

Purpose – The purpose of this paper is to describe potential flaws and pitfalls in the contemporary process of testing the theory of a research model in business research through the use of covariance-based structural equation modeling (CB-SEM). Design/methodology/approach – This paper offers a foundation for discussion, debate and questioning regarding the contemporary process of testing the theory of a research model in business research through CB-SEM. Findings – The contemporary process to test theory of a research model through CB-SEM in business research lacks to a large extent a stepwise and iterative process of an accumulation of knowledge to build sound and rigorous business theory that is both reliable and valid over time as well as across contexts. Research limitations/implications – This paper provides an awakening toward further debate and discussion on the relevance and suitability of the contemporary process to test the theory of a research model through CB-SEM in business research – is it science, quasi-science or just nonsense? Practical implications – The primary implication of this paper is that its content will challenge most readers ' preconceptions of the topic and stimulate debate. Subsequently, it is the author’s hope that the content is thought-provoking and counterintuitive. Some scholars might reject the content, while others may find it valuable. Originality/value – The paper intends to provide counterintuitive thoughts regarding the contemporary process of testing the theory of a research model in business research through the use of CB-SEM. CB-SEM offers potentially valuable merits in business research settings, if applied and performed properly.

Sign in / Sign up

Export Citation Format

Share Document