Basic Empathy Scale: A Systematic Review and Reliability Generalization Meta-Analysis

The Basic Empathy Scale (BES) has been internationally used to measure empathy. A systematic review including 74 articles that implement the instrument since its development in 2006 was carried out. Moreover, an evidence validity analysis and a reliability generalization meta-analysis were performed to examine if the scale presented the appropriate values to justify its application. Results from the systematic review showed that the use of the BES is increasing, although the research areas in which it is being implemented are currently being broadened. The validity analyses indicated that both the type of factor analysis and reliability are reported in validation studies much more than the consequences of testing are. Regarding the meta-analysis results, the mean of Cronbach’s α for cognitive empathy was 0.81 (95% CI: 0.77–0.85), with high levels of heterogeneity (I2 = 98.81%). Regarding affective empathy, the mean of Cronbach’s α was 0.81 (95% CI: 0.76–0.84), with high levels of heterogeneity. It was concluded that BES is appropriate to be used in general population groups, although not recommended for clinical diagnosis; and there is a moderate to high heterogeneity in the mean of Cronbach’s α. The practical implications of the results in mean estimation and heterogeneity are discussed.

Reliability, and Convergent and Discriminant Validity of Gaming Disorder Scales: A Meta-Analysis

Background: An association between gaming disorder (GD) and the symptoms of common mental disorders is unraveled yet. In this preregistered study, we quantitatively synthesized reliability, convergent and discriminant validity of GD scales to examine association between GD and other constructs.Methods: Five representative GD instruments (GAS-7, AICA, IGDT-10, Lemmens IGD-9, and IGDS9-SF) were chosen based on recommendations by the previous systematic review study to conduct correlation meta-analyses and reliability generalization. A systematic literature search was conducted through Pubmed, Proquest, Embase, and Google Scholar to identify studies that reported information on either reliability or correlation with related variables. 2,124 studies were full-text assessed as of October 2020, and 184 were quantitatively synthesized. Conventional Hedges two-level meta-analytic method was utilized.Results: The result of reliability generalization reported a mean coefficient alpha of 0.86 (95% CI = 0.85–0.87) and a mean test-retest estimate of 0.86 (95% CI = 0.81–0.89). Estimated effect sizes of correlation between GD and the variables were as follows: 0.33 with depression (k = 45; number of effect sizes), 0.29 with anxiety (k = 37), 0.30 with aggression (k = 19), –0.22 with quality of life (k = 18), 0.29 with loneliness (k = 18), 0.56 with internet addiction (k = 20), and 0.40 with game playtime (k = 53), respectively. The result of moderator analyses, funnel and forest plots, and publication bias analyses were also presented.Discussion and Conclusion: All five GD instruments have good internal consistency and test-retest reliability. Relatively few studies reported the test-retest reliability. The result of correlation meta-analysis revealed that GD scores were only moderately associated with game playtime. Common psychological problems such as depression and anxiety were found to have a slightly smaller association with GD than the gaming behavior. GD scores were strongly correlated with internet addiction. Further studies should adopt a rigorous methodological procedure to unravel the bidirectional relationship between GD and other psychopathologies.Limitations: The current study did not include gray literature. The representativeness of the five tools included in the current study could be questioned. High heterogeneity is another limitation of the study.Systematic Review Registration: [https://www.crd.york.ac.uk/PROSPERO/], identifier [CRD42020219781].

Reliability Generalization Meta-analyses in Mathematics Education Research: A Research Synthesis

The purpose of this systematic review was to characterize the implementation of reliability generalization meta-analytic (RGM) practices within mathematics education-related empirical research. RGM studies are used to investigate and generalize the reliability of a measure across various studies. An exhaustive literature search was conducted to locate studies related to mathematics education, including RGM studies of psychological tests. The literature search included articles as well as grey literature (e.g., conference proceedings, dissertations, theses). Of the 27 RGM studies examined, five were on scales that related to mathematics education research, five were on scales related to motivation and/or learning, four related to self-esteem, self-concept, and/or self-efficacy, six related to perceptions, well-being, and/or anxiety, and seven related to personality or behavior. Of the mathematics education-related RGM studies, 85.5% (N=9,184) of the articles examined across studies had no mention of reliability or fell into the convention of citing previously reported reliabilities. Increasing awareness of RGM studies could lead to an increase in RGM studies conducted on mathematics education research scales, leading to increased understanding of mathematics education scales. This paper contributes to the literature on the practical and empirical importance of RGM for mathematics and STEM education praxis.

Reliability Generalization Study of the Person-Centered Care Assessment Tool

The so-called Person-Centered Care (PCC) model identifies three fundamental principles: changing the focus of attention from the disease to the person, individualizing care, and promoting empowerment. The Person-Centered Care Assessment Tool (P-CAT) has gained wide acceptance as a measure of PCC in recent years due to its brevity and simplicity, as well as its ease of application and interpretation. The objective of this study is to carry out a reliability generalization meta-analysis to estimate the internal consistency of the P-CAT and analyze possible factors that may affect it, such as the year of publication, the care context, the application method, and certain sociodemographic properties of the study sample. The mean value of α for the 25 samples of the 23 studies in the meta-analysis was 0.81 (95% CI: 0.79–0.84), with high heterogeneity (squared-I = 85.83%). The only variable that had a statistically significant relationship with the reliability coefficient was the mean age of the sample. The results show that the P-CAT gives acceptably consistent scores when its use is oriented toward the description and investigation of groups, although it may be affected by variables such as the age of participants.

A Test-Retest Reliability Generalization Meta-Analysis of Judgments Via the Policy-Capturing Technique

Policy capturing is a widely used technique, but the temporal stability of policy-capturing judgments has long been a cause for concern. This article emphasizes the importance of reporting reliability, and in particular test-retest reliability, estimates in policy-capturing studies. We found that only 164 of 955 policy-capturing studies (i.e., 17.17%) reported a test-retest reliability estimate. We then conducted a reliability generalization meta-analysis on policy-capturing studies that did report test-retest reliability estimates—and we obtained an average reliability estimate of .78. We additionally examined 16 potential methodological and substantive antecedents to test-retest reliability (equivalent to moderators in validity generalization studies). We found that test-retest reliability was robust to variation in 14 of the 16 factors examined but that reliability was higher in paper-and-pencil studies than in web-based studies and was higher for behavioral intention judgments than for other (e.g., attitudinal and perceptual) judgments. We provide an agenda for future research. Finally, we provide several best-practice recommendations for researchers (and journal reviewers) with regard to (a) reporting test-retest reliability, (b) designing policy-capturing studies for appropriate reportage, and (c) properly interpreting test-retest reliability in policy-capturing studies.