scholarly journals The cases for and against double-blind reviews

PeerJ ◽  
2019 ◽  
Vol 7 ◽  
pp. e6702 ◽  
Author(s):  
Amelia R. Cox ◽  
Robert Montgomerie
Keyword(s):  
Gender Bias ◽  
Behavioral Ecology ◽  
Peer Review Process ◽  
Impact Factors ◽  
Double Blind ◽  
Blind Review ◽  
Female Authors ◽  
Female Authorship ◽  
Blind Reviews ◽  
Single Blind

To date, the majority of authors on scientific publications have been men. While much of this gender bias can be explained by historic sexism and discrimination, there is concern that women may still be disadvantaged by the peer review process if reviewers’ biases lead them to reject publications with female authors more often. One potential solution to this perceived gender bias in the reviewing process is for journals to adopt double-blind reviews whereby neither the authors nor the reviewers are aware of each other’s identity and gender. To test the efficacy of double-blind reviews in one behavioral ecology journal (Behavioral Ecology, BE), we assigned gender to every authorship of every paper published for 2010–2018 in that journal compared to four other journals with single-blind reviews but similar subject matter and impact factors. While female authorships comprised only 35% of the total in all journals, the double-blind journal (BE) did not have more female authorships than its single-blind counterparts. Interestingly, the incidence of female authorship is higher at behavioral ecology journals (BE and Behavioral Ecology and Sociobiology) than in the ornithology journals (Auk, Condor, Ibis) for papers on all topics as well as those on birds. These analyses suggest that double-blind review does not currently increase the incidence of female authorship in the journals studied here. We conclude, at least for these journals, that double-blind review no longer benefits female authors and we discuss the pros and cons of the double-blind reviewing process based on our findings.

The Case For and Against Double-blind Reviews

2018 ◽  
Author(s):  
Amelia R Cox ◽  
Robert Montgomerie
Keyword(s):  
Peer Review ◽  
Gender Bias ◽  
Behavioral Ecology ◽  
Peer Review Process ◽  
Double Blind ◽  
Blind Review ◽  
Female Authors ◽  
Female Authorship ◽  
Blind Reviews ◽  
Single Blind

To date, the majority of authors on scientific publications have been men. While much of this gender bias can be explained by historic sexism and discrimination, there is concern that women may still be disadvantaged by the peer review process if reviewers' unconscious biases lead them to reject publications with female authors more often. One potential solution to this perceived gender bias in the reviewing process is for journals to adopt double-blind reviews whereby neither the authors nor the reviewers are aware of each other's identities and genders. To test the efficacy of double-blind reviews, we assigned gender to every authorship of every paper published in 5 different journals with different peer review processes (double-blind vs. single blind) and subject matter (birds vs. behavioral ecology) from 2010-2018 (n = 4865 papers). While female authorships comprised only 35% of the total, the double-blind journal Behavioral Ecology did not have more female authorships than its single-blind counterparts. Interestingly, the incidence of female authorship is higher at behavioral ecology journals (Behavioral Ecology and Behavioral Ecology and Sociobiology) than in the ornithology journals (Auk, Condor, Ibis), for papers on all topics as well as those on birds. These analyses suggest that double-blind review does not currently increase the incidence of female authorship in the journals studied here. We conclude, at least for these journals, that double-blind review does not benefit female authors and may, in the long run, be detrimental.

scholarly journals Double blind reviews in software engineering venues: Practicability, promises and perils

2016 ◽  
Author(s):  
Moritz Beller ◽  
Alberto Bacchelli
Keyword(s):  
Software Engineering ◽  
Scientific Method ◽  
Review Process ◽  
Peer Review Process ◽  
Double Blind ◽  
Blind Review ◽  
Blind Reviews ◽  
Hidden Biases ◽  
Benefits And Costs ◽  
Single Blind

The peer review process is central to the scientific method, the advancement and spread of research as well as crucial for individual careers. However, the single blind review process currently used in most Software Engineering (SE) venues is susceptible towards apparent and hidden biases, since reviewers know the identity of authors. In this paper, we perform a study on the benefits and costs that are associated with introducing double-blind reviews in SE venues. We surveyed the SE community’s opinion, interviewed experts on double-blind reviewing, and estimated the likelihood of reviewers being able to guess the authors. Our results indicate that double-blind reviewing could be introduced in large SE conferences at lower-than-generally believed costs and that the majority of the SE community is in favor of introducing it.

Does double-blind peer review impact gender authorship trends? An evaluation of two leading neurosurgical journals from 2010 to 2019

2020 ◽  
pp. 1-9
Author(s):  
Uma V. Mahajan ◽  
Harsh Wadhwa ◽  
Parastou Fatemi ◽  
Samantha Xu ◽  
Judy Shan ◽  
...  
Keyword(s):  
Peer Review ◽  
Board Certification ◽  
Current Status ◽  
Double Blind ◽  
Blind Review ◽  
Female Authorship ◽  
Before And After ◽  
Single Blind ◽  
Single Versus ◽  
Status Of Women

OBJECTIVEPublications are key for advancement within academia. Although women are underrepresented in academic neurosurgery, the rates of women entering residency, achieving board certification, and publishing papers are increasing. The goal of this study was to assess the current status of women in academic neurosurgery publications. Specifically, this study sought to 1) survey female authorship rates in the Journal of Neurosurgery (JNS [not including JNS: Spine or JNS: Pediatrics]) and Neurosurgery from 2010 to 2019; 2) analyze whether double-blind peer review (started in Neurosurgery in 2011) altered female authorship rates relative to single-blind review (JNS); and 3) evaluate how female authorship rates compared with the number of women entering neurosurgery residency and obtaining neurosurgery board certification.METHODSGenders of the first and last authors for JNS and Neurosurgery articles from 2010 to 2019 were obtained. Data were also gathered on the number and percentage of women entering neurosurgery residency and women obtaining American Board of Neurological Surgeons (ABNS) certification between 2010 and 2019.RESULTSWomen accounted for 13.4% (n = 570) of first authors and 6.8% (n = 240) of last authors in JNS and Neurosurgery publications. No difference in rates of women publishing existed between the two journals (first authors: 13.0% JNS vs 13.9% Neurosurgery, p = 0.29; last authors: 7.3% JNS vs 6.0% Neurosurgery, p = 0.25). No difference existed between women first or last authors in Neurosurgery before and after initiation of double-blind review (p = 0.066). Significant concordance existed between the gender of first and last authors: in publications with a woman last author, the odds of the first author being a woman was increased by twofold (OR 2.14 [95% CI 1.43–3.13], p = 0.0001). Women represented a lower proportion of authors of invited papers (8.6% of first authors and 3.1% of last authors were women) compared with noninvited papers (14.1% of first authors and 7.4% of last authors were women) (first authors: OR 0.576 [95% CI 0.410–0.794], p = 0.0004; last authors: OR 0.407 [95% CI 0.198–0.751], p = 0.001). The proportion of women US last authors (7.4%) mirrors the percentage of board-certified women neurosurgeons (5.4% in 2010 and 6.8% in 2019), while the percentage of women US first authors (14.3%) is less than that for women entering neurosurgical residency (11.2% in 2009 and 23.6% in 2018).CONCLUSIONSThis is the first report of female authorship in the neurosurgical literature. The authors found that single- versus double-blind peer review did not impact female authorship rates at two top neurosurgical journals.

scholarly journals Peer review blinding practices of highly ranked dental journals: analysis and discussion

BDJ ◽  
2021 ◽  
Author(s):  
Clovis Mariano Faggion Jr
Keyword(s):  
Logistic Regression ◽  
Peer Review ◽  
Odds Ratio ◽  
Review Process ◽  
Peer Review Process ◽  
Linear Regression Analysis ◽  
Double Blind ◽  
Lower Odds ◽  
Blind Review ◽  
Single Blind

AbstractObjectives To evaluate the type of peer review blinding used in highly ranked dental journals and to discuss the influence of the blinding approaches on the peer review process.Methods All 91 dental journals classified by impact factor (IF) had their websites scrutinised for the type of peer review blinding used for submissions. If the information was not reported, the journals were contacted to obtain the information. Linear and logistic regression were applied to evaluate the association between type of peer review blinding and IF.Results The selected journals reported the following peer review blinding approaches: single-blind (N = 36, 39.6%), double-blind (N = 46, 50.5%), transparent (N = 2, 2.2%) and open (N = 1, 1.1%). Information from six (6.6%) journals was not available. A linear regression analysis demonstrated that journals with lower IFs were associated with double-blind review (p = 0.001). A logistic regression suggested lower odds of association between single-blind peer review and journals with IFs below a threshold of 2 (odds ratio 0.157, confidence interval 0.059 to 0.417, p <0.001).Conclusions The majority of highly ranked dental journals had single- and double-blind peer review; journals with higher IFs presented single-blind peer review and those with lower IFs reported double-blind peer review.

scholarly journals Double-blind review in software engineering venues

2016 ◽  
Author(s):  
Moritz Beller ◽  
Alberto Bacchelli
Keyword(s):  
Side Effects ◽  
Software Engineering ◽  
Scientific Method ◽  
Review Process ◽  
Peer Review Process ◽  
Double Blind ◽  
Blind Review ◽  
Hidden Biases ◽  
Benefits And Costs ◽  
Single Blind

The peer review process is central to the scientific method, the advancement and spread of research, as well as crucial for individual careers. However, the single-blind review mode currently used in most Software Engineering (SE) venues is susceptible to apparent and hidden biases, since reviewers know the identity of authors. We perform a study on the benefits and costs that are associated with introducing double- blind review in SE venues. We surveyed the SE community’s opinion and interviewed experts on double-blind reviewing. Our results indicate that the costs, mostly logistic challenges and side effects, outnumber its benefits and mostly regard difficulty for authors in blinding papers, for reviewers in understanding the increment with respect to previous work from the same authors, and for organizers to manage a complex transition. While the surveyed community largely consents on the costs of DBR, only less than one-third disagree with a switch to DBR for SE journals, all SE conferences, and, in particular, ICSE; the analysis of a survey with authors of submitted papers at ICSE 2016 run by the program chairs of that edition corroborates our result.

scholarly journals Double-blind review in software engineering venues

2016 ◽  
Author(s):  
Moritz Beller ◽  
Alberto Bacchelli
Keyword(s):  
Side Effects ◽  
Software Engineering ◽  
Scientific Method ◽  
Review Process ◽  
Peer Review Process ◽  
Double Blind ◽  
Blind Review ◽  
Hidden Biases ◽  
Benefits And Costs ◽  
Single Blind

The peer review process is central to the scientific method, the advancement and spread of research, as well as crucial for individual careers. However, the single-blind review mode currently used in most Software Engineering (SE) venues is susceptible to apparent and hidden biases, since reviewers know the identity of authors. We perform a study on the benefits and costs that are associated with introducing double- blind review in SE venues. We surveyed the SE community’s opinion and interviewed experts on double-blind reviewing. Our results indicate that the costs, mostly logistic challenges and side effects, outnumber its benefits and mostly regard difficulty for authors in blinding papers, for reviewers in understanding the increment with respect to previous work from the same authors, and for organizers to manage a complex transition. While the surveyed community largely consents on the costs of DBR, only less than one-third disagree with a switch to DBR for SE journals, all SE conferences, and, in particular, ICSE; the analysis of a survey with authors of submitted papers at ICSE 2016 run by the program chairs of that edition corroborates our result.

scholarly journals Single- or double-blind review? A field study of system preference, reliability, bias, and validity

2021 ◽  
Author(s):  
Timothy Joseph Pleskac ◽  
Ellie Kyung ◽  
Gretchen B. Chapman ◽  
Oleg Urminsky
Keyword(s):  
Field Study ◽  
Scientific Work ◽  
Annual Conference ◽  
High Stakes ◽  
Double Blind ◽  
Anna Karenina ◽  
Blind Review ◽  
Subsequent Publication ◽  
Blind Reviews ◽  
Single Blind

Many scientists experience a practice-preference gap about peer review. Single-blind review---where authors' identities are revealed to reviewers---is often used for evaluation. Yet, double-blind review---where authors' identities are concealed---is seen as more fair. To understand this gap, we compared both systems in a high-stakes field study: submissions to the Society for Judgment and Decision Making’s annual conference, the leading international conference on this topic. Each submission received both review types. Reviewers were randomly assigned to the review system and submissions. Selected conference talks were evaluated for quality, popularity, and subsequent publication status. We assessed the two systems on reliability, bias, and validity. On reliability, while both systems had moderate reliability, agreement was higher on what constituted a poor submission than a strong one (Anna Karenina Principle). On bias, double-blind reviews showed a slight bias against submission by women (Matilda Effect), while single-blind reviews showed a preference for submissions with senior co-authors (Matthew Effect). On validity, neither system predicted talk quality or popularity, but both predicted publication status. Author characteristics did not consistently predict outcomes. Thus, we suggest the costs of single-blind review do not outweigh its benefits. Yet, double-blind review is also not a perfect solution. We propose an equitable approach for selecting scientific work may be an informed lottery: use double-blind review to identify submissions of merit, then randomly choose from this set.

scholarly journals Double-blind reviewing and gender biases at EvoLang conferences: An update

2019 ◽  
Author(s):  
Christine Cuskley ◽  
Seán G Roberts ◽  
Stephen Politzer-Ahles ◽  
Tessa Verhoef
Keyword(s):  
Gender Bias ◽  
Evaluation Process ◽  
Evolution Of Language ◽  
Double Blind ◽  
Blind Review ◽  
Female Authors ◽  
Gender Biases ◽  
Significant Difference ◽  
Comprehensive Picture ◽  
And Gender

Abstract A previous study of reviewing at the Evolution of Language conferences found effects that suggested that gender bias against female authors was alleviated under double-blind review at EvoLang 11. We update this analysis in two specific ways. First, we add data from the most recent EvoLang 12 conference, providing a comprehensive picture of the conference over five iterations. Like EvoLang 11, EvoLang 12 used double-blind review, but EvoLang 12 showed no significant difference in review scores between genders. We discuss potential explanations for why there was a strong effect in EvoLang 11, which is largely absent in EvoLang 12. These include testing whether readability differs between genders, though we find no evidence to support this. Although gender differences seem to have declined for EvoLang 12, we suggest that double-blind review provides a more equitable evaluation process.

scholarly journals Open, single-blind, double-blind: which peer review process do you prefer?

2014 ◽  
Vol 15 (1) ◽  
Author(s):  
Elizabeth C Moylan ◽  
Simon Harold ◽  
Ciaran O’Neill ◽  
Maria K Kowalczuk
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Double Blind ◽  
Single Blind

Does double-blind review favor female authors?

2008 ◽  
Vol 6 (7) ◽  
pp. 354-354 ◽  
Author(s):  
Katrin Hammerschmidt ◽  
Klaus Reinhardt ◽  
Jens Rolff
Keyword(s):  
Double Blind ◽  
Blind Review ◽  
Female Authors
Sign in / Sign up

Export Citation Format

Share Document