Supplemental Material for Heterogeneity in Direct Replications in Psychology and Its Association With Effect Size

2020 ◽  
Keyword(s):  
Effect Size

Evaluation von Gesundheitsförderungsprogrammen - Methodische Stolpersteine und pragmatische Empfehlungen

2000 ◽  
Vol 8 (1) ◽  
pp. 18-24 ◽  
Author(s):  
Gert Kaluza ◽  
Hans-Henning Schulze
Keyword(s):  
Effect Size ◽  
Power Analysis ◽  
Prävention Und Gesundheitsförderung ◽  
Size Display ◽  
Methodische Probleme

Zusammenfassung. Die Evaluation von Interventionen zur Prävention und Gesundheitsförderung stellt ein zentrales Aufgabenfeld der gesundheitspsychologischen Forschung dar. Häufige methodische Probleme entsprechender Evaluationsstudien betreffen 1. Ausgangswert-Unterschiede bei nicht randomisierten Studiendesigns, 2. Abhängigkeit von Beobachtungen bei Gruppeninterventionsstudien, 3. Kapitalisierung von Irrtumswahrscheinlichkeiten aufgrund einer Vielzahl von abhängigen Variablen und 4. Beurteilung der praktischen Relevanz statistisch signifikanter Interventionseffekte. Zu deren pragmatischer Lösung werden u.a. 1. die Anwendung kovarianzanalytischer Auswertungsstrategien, 2. die Berechnung von Intraclass-Korrelationen und ggf. eine Datenauswertung auf der Ebene der Gruppenmittelwerte, 3. eine Reduktion der Anzahl der abhängigen Variablen mittels Hauptkomponentenanalyse sowie eine Alpha-Adjustierung unter Berücksichtigung der Teststärke (“compromise power analysis”) und 4. die Umrechnung gängiger Effektstärken in prozentuale Erfolgsraten (“binomial effect size display”) empfohlen.

The Level of Reaction Time Determines the ERP Correlates of Auditory Negative Priming

2006 ◽  
Vol 20 (3) ◽  
pp. 186-194 ◽  
Author(s):  
Susanne Mayr ◽  
Michael Niedeggen ◽  
Axel Buchner ◽  
Guido Orgs
Keyword(s):  
Reaction Time ◽  
Negative Priming ◽  
Effect Size ◽  
Priming Effect ◽  
Reaction Times ◽  
Negative Priming Effect ◽  
Episodic Retrieval ◽  
Time Level ◽  
Fast Reactions ◽  
Priming Condition

Responding to a stimulus that had to be ignored previously is usually slowed-down (negative priming effect). This study investigates the reaction time and ERP effects of the negative priming phenomenon in the auditory domain. Thirty participants had to categorize sounds as musical instruments or animal voices. Reaction times were slowed-down in the negative priming condition relative to two control conditions. This effect was stronger for slow reactions (above intraindividual median) than for fast reactions (below intraindividual median). ERP analysis revealed a parietally located negativity of the negative priming condition compared to the control conditions between 550-730 ms poststimulus. This replicates the findings of Mayr, Niedeggen, Buchner, and Pietrowsky (2003) . The ERP correlate was more pronounced for slow trials (above intraindividual median) than for fast trials (below intraindividual median). The dependency of the negative priming effect size on the reaction time level found in the reaction time analysis as well as in the ERP analysis is consistent with both the inhibition as well as the episodic retrieval account of negative priming. A methodological artifact explanation of this effect-size dependency is discussed and discarded.

Clinically Meaningful Change

Methodology ◽  
2019 ◽  
Vol 15 (3) ◽  
pp. 97-105
Author(s):  
Rodrigo Ferrer ◽  
Antonio Pardo
Keyword(s):  
Effect Size ◽  
False Negative ◽  
False Negative Rate ◽  
Point Of View ◽  
Skewed Distribution ◽  
Effect Sizes ◽  
False Negatives ◽  
Large Size ◽  
Before And After ◽  
Post Test

Abstract. In a recent paper, Ferrer and Pardo (2014) tested several distribution-based methods designed to assess when test scores obtained before and after an intervention reflect a statistically reliable change. However, we still do not know how these methods perform from the point of view of false negatives. For this purpose, we have simulated change scenarios (different effect sizes in a pre-post-test design) with distributions of different shapes and with different sample sizes. For each simulated scenario, we generated 1,000 samples. In each sample, we recorded the false-negative rate of the five distribution-based methods with the best performance from the point of view of the false positives. Our results have revealed unacceptable rates of false negatives even with effects of very large size, starting from 31.8% in an optimistic scenario (effect size of 2.0 and a normal distribution) to 99.9% in the worst scenario (effect size of 0.2 and a highly skewed distribution). Therefore, our results suggest that the widely used distribution-based methods must be applied with caution in a clinical context, because they need huge effect sizes to detect a true change. However, we made some considerations regarding the effect size and the cut-off points commonly used which allow us to be more precise in our estimates.

How to Detect Publication Bias in Psychological Research

2019 ◽  
Vol 227 (4) ◽  
pp. 261-279 ◽  
Author(s):  
Frank Renkewitz ◽  
Melanie Keiner
Keyword(s):  
Publication Bias ◽  
Effect Size ◽  
Statistical Power ◽  
Type I Error ◽  
Psychological Research ◽  
Type I ◽  
True Effect Size ◽  
Questionable Research Practices ◽  
True Effect ◽  
Meta Analyses

Abstract. Publication biases and questionable research practices are assumed to be two of the main causes of low replication rates. Both of these problems lead to severely inflated effect size estimates in meta-analyses. Methodologists have proposed a number of statistical tools to detect such bias in meta-analytic results. We present an evaluation of the performance of six of these tools. To assess the Type I error rate and the statistical power of these methods, we simulated a large variety of literatures that differed with regard to true effect size, heterogeneity, number of available primary studies, and sample sizes of these primary studies; furthermore, simulated studies were subjected to different degrees of publication bias. Our results show that across all simulated conditions, no method consistently outperformed the others. Additionally, all methods performed poorly when true effect sizes were heterogeneous or primary studies had a small chance of being published, irrespective of their results. This suggests that in many actual meta-analyses in psychology, bias will remain undiscovered no matter which detection method is used.

The Relationship Between Immorality and Cleansing

2018 ◽  
Vol 49 (5) ◽  
pp. 303-309 ◽  
Author(s):  
Jedidiah Siev ◽  
Shelby E. Zuckerman ◽  
Joseph J. Siev
Keyword(s):  
Effect Size ◽  
Meta Analysis ◽  
Weighted Mean ◽  
The Relationship

Abstract. In a widely publicized set of studies, participants who were primed to consider unethical events preferred cleansing products more than did those primed with ethical events ( Zhong & Liljenquist, 2006 ). This tendency to respond to moral threat with physical cleansing is known as the Macbeth Effect. Several subsequent efforts, however, did not replicate this relationship. The present manuscript reports the results of a meta-analysis of 15 studies testing this relationship. The weighted mean effect size was small across all studies (g = 0.17, 95% CI [0.04, 0.31]), and nonsignificant across studies conducted in independent laboratories (g = 0.07, 95% CI [−0.04, 0.19]). We conclude that there is little evidence for an overall Macbeth Effect; however, there may be a Macbeth Effect under certain conditions.

Caffeine supplementation for 4-day, followed by acute ingestion, did not impact triathlete output power after submaximal intensity exercise

2019 ◽  
Vol 18 (3) ◽  
pp. 118
Author(s):  
Anderson Pontes Morales ◽  
Felipe Sampaio-Jorge ◽  
Thiago Barth ◽  
Alessandra Alegre De Matos ◽  
Luiz Felipe Da Cruz Rangel ◽  
...  
Keyword(s):  
Blood Lactate ◽  
Output Power ◽  
Body Mass ◽  
Effect Size ◽  
Power Output ◽  
Anaerobic Power ◽  
Lactate Concentration ◽  
Wingate Test ◽  
Large Effect Size ◽  
Acute Ingestion

Introduction: The aim of this study was to test the hypothesis that caffeine supplementation (6 mg·kg-1 body mass) for 4-days, followed by acute intake, would impact five male triathletes output power after performed submaximal intensity exercise. Methods: This was a randomized, double-blind, placebo-controlled crossover study, placebo (4-day) - placebo (acute) PP, placebo (4-days) -caffeine (acute) PC, and caffeine (4-day) - caffeine (acute) CC. Participants abstained from dietary caffeine sources for 4 days and ingested capsules containing either placebo or caffeine (6 mg.kg-1 body mass day in one absorption). The acute trials the capsules containing placebo or caffeine (6 mg.kg-1 body mass day in one absorption) were ingested 60min before completing exercise in a treadmill for 40min (80% VO2max) and to perform the Wingate test. Results: Blood lactate was determined before, 60min after ingestion, and immediately after the exercise on the treadmill, the Wingate test, and after the recovery (10-min). CC and PC trials did not change the cardiopulmonary variables (P>0.05) and the anaerobic power variables (peak/mean power output and fatigue index) (P>0.05). The PC trial compared with PP promoted improvements in the curve power output in 2 sec by 31.19% (large effect-size d = 1.08; P<0.05) and 3 sec by 20% (large effect-size d = 1.19; P<0.05). A 10min recovery was not sufficient to reduce blood lactate concentration in the PC trial compared with PP (PC, 13.73±2.66 vs. PP, 10.26±1.60 mmol.L-1; P<0.05, respectively) (P<0.05). Conclusion: In conclusion, these results indicate that caffeine supplementation (6 mg·kg-1 body mass) for 4 days, followed by acute ingestion, did not impact the triathletes output power after performed submaximal intensity exercise. Nutritional interventions may help researchers and athletes to adapt strategies for manipulating caffeine use.Key-words: caffeine metabolism, Wingate test, blood lactate, performance.

Sign in / Sign up

Export Citation Format

Share Document