Peer review expert selection method research based on knowledge set theory

2011 ◽  
Author(s):  
Haifeng Li ◽  
Ning Chen
Keyword(s):  
Peer Review ◽  
Set Theory ◽  
Selection Method ◽  
Expert Selection ◽  
Knowledge Set

scholarly journals Metascience on Peer Review: Testing the Effects of a Study’s Originality and Statistical Significance in a Field Experiment

2019 ◽  
Author(s):  
Malte Elson ◽  
Markus Huff ◽  
Sonja Utz
Keyword(s):  
Field Experiment ◽  
Peer Review ◽  
Gold Standard ◽  
Primary Outcome ◽  
Review Process ◽  
Peer Review Process ◽  
Statistical Significance ◽  
Selection Method ◽  
Psychological Theories ◽  
Peer Reviewers

Peer review has become the gold standard in scientific publishing as a selection method and a refinement scheme for research reports. However, despite its pervasiveness and conferred importance, relatively little empirical research has been conducted to document its effectiveness. Further, there is evidence that factors other than a submission’s merits can substantially influence peer reviewers’ evaluations. We report the results of a metascientific field experiment on the effect of the originality of a study and the statistical significance of its primary outcome on reviewers’ evaluations. The general aim of this experiment, which was carried out in the peer-review process for a conference, was to demonstrate the feasibility and value of metascientific experiments on the peer-review process and thereby encourage research that will lead to understanding its mechanisms and determinants, effectively contextualizing it in psychological theories of various biases, and developing practical procedures to increase its utility.

scholarly journals Metascience on Peer Review: Testing the Effects of a Study’s Originality and Statistical Significance in a Field Experiment

2019 ◽  
Author(s):  
Malte Elson ◽  
Markus Huff ◽  
Sonja Utz
Keyword(s):  
Field Experiment ◽  
Peer Review ◽  
Gold Standard ◽  
Primary Outcome ◽  
Review Process ◽  
Peer Review Process ◽  
Statistical Significance ◽  
Selection Method ◽  
Psychological Theories ◽  
Peer Reviewers

Peer review has become the gold standard in scientific publishing as a selection method and a refinement scheme for research reports. However, despite its pervasiveness and conferred importance, relatively little empirical research has been conducted to document its effectiveness. Further, there is evidence that factors other than a submission’s merits can substantially influence peer reviewers’ evaluations. We report the results of a metascientific field experiment on the effect of the originality of a study and the statistical significance of its primary outcome on reviewers’ evaluations. The general aim of this experiment, which was carried out in the peer-review process for a conference, was to demonstrate the feasibility and value of metascientific experiments on the peer-review process and thereby encourage research that will lead to understanding its mechanisms and determinants, effectively contextualizing it in psychological theories of various biases, and developing practical procedures to increase its utility.

scholarly journals Metascience on Peer Review: Testing the Effects of a Study’s Originality and Statistical Significance in a Field Experiment

2020 ◽  
Vol 3 (1) ◽  
pp. 53-65
Author(s):  
Malte Elson ◽  
Markus Huff ◽  
Sonja Utz
Keyword(s):  
Field Experiment ◽  
Peer Review ◽  
Gold Standard ◽  
Primary Outcome ◽  
Review Process ◽  
Peer Review Process ◽  
Statistical Significance ◽  
Selection Method ◽  
Psychological Theories ◽  
Peer Reviewers

Peer review has become the gold standard in scientific publishing as a selection method and a refinement scheme for research reports. However, despite its pervasiveness and conferred importance, relatively little empirical research has been conducted to document its effectiveness. Further, there is evidence that factors other than a submission’s merits can substantially influence peer reviewers’ evaluations. We report the results of a metascientific field experiment on the effect of the originality of a study and the statistical significance of its primary outcome on reviewers’ evaluations. The general aim of this experiment, which was carried out in the peer-review process for a conference, was to demonstrate the feasibility and value of metascientific experiments on the peer-review process and thereby encourage research that will lead to understanding its mechanisms and determinants, effectively contextualizing it in psychological theories of various biases, and developing practical procedures to increase its utility.

Sign in / Sign up

Export Citation Format

Share Document