scholarly journals Deciding what to replicate: A decision model for replication study selection under resource and knowledge constraints.

2021 ◽  
Author(s):  
Peder Mortvedt Isager ◽  
Robbie C. M. van Aert ◽  
Štěpán Bahník ◽  
Mark J. Brandt ◽  
K. Andrew DeSoto ◽  
...  
Keyword(s):  
Decision Model ◽  
Replication Study ◽  
Study Selection

scholarly journals Replication value as a function of citation impact and sample size

2021 ◽  
Author(s):  
Peder Mortvedt Isager ◽  
Anna Elisabeth van 't Veer ◽  
Daniel Lakens
Keyword(s):  
Sample Size ◽  
Expected Utility ◽  
Citation Count ◽  
Citation Impact ◽  
Selection Procedure ◽  
Replication Study ◽  
Selection Strategy ◽  
Original Research ◽  
Study Selection ◽  
Utility Gain

Researchers seeking to replicate original research often need to decide which of several relevant candidates to select for replication. Several strategies for study selection have been proposed, utilizing a variety of observed indicators as criteria for selection. However, few strategies clearly specify the goal of study selection and how that goal is related to the indicators that are utilized. We have previously formalized a decision model of replication study selection in which the goal of study selection is to maximize the expected utility gain of the replication e?ort. We further define the concept of replication value as a proxy for expected utility gain (Isager et al., 2020). In this article, we propose a quantitative operationalization of replication value. Wefirst discuss how value and uncertainty - the two concepts used to determine replication value – could be estimated via information about citation count and sample size. Second, we propose an equation for combining these indicators into an overall estimate of replication value, which we denote RVCn. Third, we suggest how RVCn could be implemented as part of a broader study selection procedure. Finally, we provide preliminary data suggesting that studies that were in fact selected for replication tend to have relatively high RVCn estimates. The goal of this article is to explain how RVCn is intended to work and, in doing so, demonstrate the many assumptions that should be explicit in any replication study selection strategy.

scholarly journals Deciding what to replicate: A formal definition of “replication value” and a decision model for replication study selection.

2020 ◽  
Author(s):  
Peder Mortvedt Isager ◽  
Robbie Cornelis Maria van Aert ◽  
Štěpán Bahník ◽  
Mark John Brandt ◽  
Kurt Andrew DeSoto ◽  
...  
Keyword(s):  
Expected Utility ◽  
Decision Model ◽  
General Rule ◽  
Difficult Problem ◽  
Current Evidence ◽  
Replication Studies ◽  
Study Selection ◽  
Utility Gain ◽  
Replication Research ◽  
Definition Of

Robust scientific knowledge is contingent upon replication of original findings. However, researchers who conduct replication studies face a difficult problem; there are many more studies in need of replication than there are funds available for replicating. To select studies for replication efficiently, we need to understand which studies are the most in need of replication. In other words, we need to understand which replication efforts have the highest expected utility. In this article we propose a general rule for study selection in replication research based on the replication value of the claims considered for replication. The replication value of a claim is defined as the maximum expected utility we could gain by replicating the claim, and is a function of (1) the value of being certain about the claim, and (2) uncertainty about the claim based on current evidence. We formalize this definition in terms of a causal decision model, utilizing concepts from decision theory and causal graph modeling. We discuss the validity of using replication value as a measure of expected utility gain, and we suggest approaches for deriving quantitative estimates of replication value.

Visual Text Mining: Ensuring the Presence of Relevant Studies in Systematic Literature Reviews

2015 ◽  
Vol 25 (05) ◽  
pp. 909-928 ◽  
Author(s):  
Katia Romero Felizardo ◽  
Ellen Francine Barbosa ◽  
Rafael Messias Martins ◽  
Pedro Henrique Dias Valle ◽  
José Carlos Maldonado
Keyword(s):  
Text Mining ◽  
Replication Study ◽  
Primary Study ◽  
Original Experiment ◽  
Study Selection ◽  
Literature Reviews ◽  
Selection Step ◽  
Visual Text ◽  
And Performance ◽  
Level Of Experience

One of the activities associated with the Systematic Literature Review (SLR) process is the selection review of primary studies. When the researcher faces large volumes of primary studies to be analyzed, the process used to select studies can be arduous. In a previous experiment, we conducted a pilot test to compare the performance and accuracy of PhD students in conducting the selection review activity manually and using Visual Text Mining (VTM) techniques. The goal of this paper is to describe a replication study involving PhD and Master students. The replication study uses the same experimental design and materials of the original experiment. This study also aims to investigate whether the researcher's level of experience with conducting SLRs and research in general impacts the outcome of the primary study selection step of the SLR process. The replication results have confirmed the outcomes of the original experiment, i.e., VTM is promising and can improve the performance of the selection review of primary studies. We also observed that both accuracy and performance increase in function of the researcher's experience level in conducting SLRs. The use of VTM can indeed be beneficial during the selection review activity.

Children's Dissociative Experiences Scale and Posttraumatic Symptom Inventory: A Replication Study

2007 ◽  
Author(s):  
Bradley C. Stolbach ◽  
Renee Z. Dominguez ◽  
Vikki Rompala ◽  
Courtney Fleisher ◽  
Tanja Gazibara ◽  
...  
Keyword(s):  
Replication Study ◽  
Dissociative Experiences Scale ◽  
Symptom Inventory ◽  
Dissociative Experiences

A Fixed Observation Boundary Decision Model for Detection of Change

1974 ◽  
Author(s):  
Graham J. Burkheimer ◽  
William E. Stein ◽  
Amnon Rapoport
Keyword(s):  
Decision Model

Virtual Reality and In Vivo Exposure for Fear of Flying: A Phase II Replication Study

2006 ◽  
Author(s):  
Ken Graap ◽  
Barbara O. Rothbaum ◽  
Page Anderson ◽  
Elana Zimand ◽  
Larry Hodges ◽  
...  
Keyword(s):  
Virtual Reality ◽  
Phase Ii ◽  
Replication Study ◽  
Fear Of Flying
Sign in / Sign up

Export Citation Format

Share Document