scholarly journals To fund or not to fund?

eLife ◽  
2017 ◽  
Vol 6 ◽  
Author(s):  
Sarah Shailes
Keyword(s):  
Peer Review ◽  
Funding Agencies ◽  
Grant Proposals

Funding agencies use many different criteria and peer review strategies to assess grant proposals.

scholarly journals Measuring bias, burden and conservatism in research funding processes

F1000Research ◽  
2019 ◽  
Vol 8 ◽  
pp. 851 ◽  
Author(s):  
Susan Guthrie ◽  
Daniela Rodriguez Rincon ◽  
Gordon McInroy ◽  
Becky Ioppolo ◽  
Salil Gunashekar
Keyword(s):  
Peer Review ◽  
Research Funding ◽  
Assessment Process ◽  
Grant Funding ◽  
Funding Allocation ◽  
Complex Process ◽  
Funding Agencies ◽  
Grant Proposals ◽  
Funding Scheme ◽  
Measuring Bias

Background: Grant funding allocation is a complex process that in most cases relies on peer review. A recent study identified a number of challenges associated with the use of peer review in the evaluation of grant proposals. Three important issues identified were bias, burden, and conservatism, and the work concluded that further experimentation and measurement is needed to assess the performance of funding processes. Methods: We have conducted a review of international practice in the evaluation and improvement of grant funding processes in relation to bias, burden and conservatism, based on a rapid evidence assessment and interviews with research funding agencies. Results: The evidence gathered suggests that efforts so far to measure these characteristics systematically by funders have been limited. However, there are some examples of measures and approaches which could be developed and more widely applied. Conclusions: The majority of the literature focuses primarily on the application and assessment process, whereas burden, bias and conservatism can emerge as challenges at many wider stages in the development and implementation of a grant funding scheme. In response to this we set out a wider conceptualisation of the ways in which this could emerge across the funding process.

scholarly journals The Participation and Motivations of Grant Peer Reviewers: A Comprehensive Survey

2018 ◽  
Author(s):  
Stephen A Gallo ◽  
Lisa A Thompson ◽  
Karen B Schmaling ◽  
Scott R Glisson
Keyword(s):  
Peer Review ◽  
Selection Process ◽  
Funding Agency ◽  
Grant Applications ◽  
Time Period ◽  
Funding Agencies ◽  
Integral Role ◽  
Comprehensive Survey ◽  
Peer Reviewers ◽  
Grant Proposals

AbstractScientific peer reviewers play an integral role in the grant selection process, yet very little has been reported on the levels of participation or the motivations of scientists to take part in peer review. AIBS developed a comprehensive peer review survey that examined the motivations and levels of participation of grant reviewers. The survey was disseminated to 13,091 scientists in AIBS’s proprietary database. Of the 874 respondents, 76% indicated they had reviewed grant applications in the last 3 years; however, the number of reviews was unevenly distributed across this sample. Higher review loads were associated with respondents who had submitted more grant proposals over this time period, some of whom were likely to be study section members for large funding agencies. The most prevalent reason to participate in a review was to give back to the scientific community (especially among frequent grant submitters) and the most common reason to decline an invitation to review was lack of time. Interestingly, few suggested that expectation from the funding agency was a motivation to review. Most felt that review participation positively influenced their careers through improving grantsmanship and exposure to new scientific ideas. Of those who reviewed, respondents reported dedicating 2-5% of their total annual work time to grant review and, based on their self-reported maximum review loads, it is estimated they are participating at 56%-89% of their capacity, which may have important implications regarding the sustainability of the system. Overall, it is clear that participation in peer review is uneven and in some cases near capacity, and more needs to be done to create new motivations and incentives to increase the future pool of reviewers.

scholarly journals Grant reviewer perceptions of the quality and effectiveness of panel discussion

2019 ◽  
Author(s):  
Stephen A. Gallo ◽  
Karen B. Schmaling ◽  
Lisa A. Thompson ◽  
Scott R. Glisson
Keyword(s):  
Peer Review ◽  
Panel Discussion ◽  
Limited Participation ◽  
Funding Agencies ◽  
Potential Sources ◽  
Grant Proposals ◽  
Proposal Review ◽  
Method Analysis ◽  
Research Grant

AbstractBackgroundFunding agencies have long used panel discussion in the peer review of research grant proposals as a way to utilize a set of expertise and perspectives in making funding decisions. Little research has examined the quality of panel discussions and how effectively they are facilitated.MethodsHere we present a mixed-method analysis of data from a survey of reviewers focused on their perceptions of the quality and facilitation of panel discussion from their last peer review experience.ResultsReviewers indicated that panel discussions were viewed favorably in terms of participation, clarifying differing opinions, informing unassigned reviewers, and chair facilitation. However, some reviewers mentioned issues with panel discussions, including an uneven focus, limited participation from unassigned reviewers, and short discussion times. Most reviewers felt the discussions affected the review outcome, helped in choosing the best science, and were generally fair and balanced. However, those who felt the discussion did not affect the outcome were also more likely to evaluate panel communication negatively, and several reviewers mentioned potential sources of bias related to the discussion. While respondents strongly acknowledged the importance of the chair in ensuring appropriate facilitation of the discussion to influence scoring and to limit the influence of potential sources of bias from the discussion on scoring, nearly a third of respondents did not find the chair of their most recent panel to have performed these roles effectively.ConclusionsIt is likely that improving chair training in the management of discussion as well as creating review procedures that are informed by the science of leadership and team communication would improve review processes and proposal review reliability.

scholarly journals New approaches to peer review in the age of online, open-access publishing

2019 ◽  
Author(s):  
Melanie J Hopkins
Keyword(s):  
Open Access ◽  
Peer Review ◽  
Electronic Publishing ◽  
Peer Review Process ◽  
Financial Burden ◽  
Open Access Publishing ◽  
Non Profit ◽  
Funding Agencies ◽  
Scientific Results ◽  
Peer Community

“Electronic publishing” can mean a variety of things, but for the dissemination of scientific results, there are two major categories: 1) materials that have not gone through peer-review, such as community-database entries, presentations from conferences, and manuscripts posted on preprint servers; and 2) materials that have gone through peer-review and are subsequently posted online. In the latter case, the process of peer-review is usually managed by a body of editors associated with a journal. If a manuscript is published by such a journal, the reader can be assured that it went through the peer-review process successfully. In the last decade or so, journals have started to abandon printed issues of peer-reviewed articles and are now publishing exclusively online; there have also been a proliferation of new online-only journals. Concurrently, there has been a shift towards open-access publishing, which, while making scientific studies more broadly available, has also transferred the financial burden from the reader or subscriber to the authors and funding agencies. Lastly, there has been a shift in how manuscripts on preprint servers are viewed, and it is increasingly common in many scientific fields for authors to post a finalized manuscript to a preprint server prior to submission to a journal. This talk will describe the “Peer Community In” (PCI) Project, which is a non-profit organization that was established in response to these major shifts in scientific publishing. The PCI Project is comprised of communities of researchers working in different fields (including paleontology), who peer review and recommend research articles publicly available on preprint servers. The goal is to promote rigorous scientific study by providing an alternative to traditional avenues for peer-reviewed publishing.

Peer Review or Lottery? A Critical Analysis of Two Different Forms of Decision-making Mechanisms for Allocation of Research Grants

2019 ◽  
Vol 44 (6) ◽  
pp. 994-1019 ◽  
Author(s):  
Lambros Roumbanis
Keyword(s):  
Decision Making ◽  
Peer Review ◽  
Review Process ◽  
Selection Process ◽  
Peer Review Process ◽  
Dynamic Selection ◽  
Funding Agencies ◽  
Epistemic Diversity ◽  
Alternative Decision ◽  
Made In

At present, peer review is the most common method used by funding agencies to make decisions about resource allocation. But how reliable, efficient, and fair is it in practice? The ex ante evaluation of scientific novelty is a fundamentally uncertain endeavor; bias and chance are embedded in the final outcome. In the current study, I will examine some of the most central problems of peer review and highlight the possible benefits of using a lottery as an alternative decision-making mechanism. Lotteries are driven by chance, not reason. The argument made in the study is that the epistemic landscape could benefit in several respects by using a lottery, thus avoiding all types of bias, disagreement, and other limitations associated with the peer review process. Funding agencies could form a pool of funding applicants who have minimal qualification levels and then select randomly within that pool. The benefits of a lottery would not only be that it saves time and resources, but also that it contributes to a more dynamic selection process and increases the epistemic diversity, fairness, and impartiality within academia.

scholarly journals New approaches to peer review in the age of online, open-access publishing

2019 ◽  
Author(s):  
Melanie J Hopkins
Keyword(s):  
Open Access ◽  
Peer Review ◽  
Electronic Publishing ◽  
Peer Review Process ◽  
Financial Burden ◽  
Open Access Publishing ◽  
Non Profit ◽  
Funding Agencies ◽  
Scientific Results ◽  
Peer Community

“Electronic publishing” can mean a variety of things, but for the dissemination of scientific results, there are two major categories: 1) materials that have not gone through peer-review, such as community-database entries, presentations from conferences, and manuscripts posted on preprint servers; and 2) materials that have gone through peer-review and are subsequently posted online. In the latter case, the process of peer-review is usually managed by a body of editors associated with a journal. If a manuscript is published by such a journal, the reader can be assured that it went through the peer-review process successfully. In the last decade or so, journals have started to abandon printed issues of peer-reviewed articles and are now publishing exclusively online; there have also been a proliferation of new online-only journals. Concurrently, there has been a shift towards open-access publishing, which, while making scientific studies more broadly available, has also transferred the financial burden from the reader or subscriber to the authors and funding agencies. Lastly, there has been a shift in how manuscripts on preprint servers are viewed, and it is increasingly common in many scientific fields for authors to post a finalized manuscript to a preprint server prior to submission to a journal. This talk will describe the “Peer Community In” (PCI) Project, which is a non-profit organization that was established in response to these major shifts in scientific publishing. The PCI Project is comprised of communities of researchers working in different fields (including paleontology), who peer review and recommend research articles publicly available on preprint servers. The goal is to promote rigorous scientific study by providing an alternative to traditional avenues for peer-reviewed publishing.

scholarly journals Ethics issues identified by applicants and ethics experts in Horizon 2020 grant proposals

F1000Research ◽  
2021 ◽  
Vol 10 ◽  
pp. 471
Author(s):  
Ivan Buljan ◽  
David G Pina ◽  
Ana Marušić
Keyword(s):  
Health And Safety ◽  
Personal Data ◽  
Horizon 2020 ◽  
Ethics Review ◽  
Research And Innovation ◽  
Research Organizations ◽  
Funding Agencies ◽  
Grant Proposals ◽  
Further Development ◽  
Ethical Clearance

Background: We assessed the ethics review of proposals selected for funding under the Marie Skłodowska-Curie Actions (MSCA) and the European Research Council (ERC) in Horizon 2020, EU’s framework programme for research and innovation, 2014-2020. Methods: We analysed anonymized datasets for 3,054 MSCA individual fellowships (IF), 417 MSCA Innovative Training Networks (ITN), and 1,465 ERC main-listed proposals with ethics conditional clearance, over four years (2016 to 2019). The datasets included the information on ethics issues identified by applicants in their proposal and ethics issues and requirements identified by ethics experts during the ethics review. Results: 42% of proposals received ethical clearance. For proposals with conditional ethics clearance (n=3546), most of the identified ethics issues by both applicants and ethics experts were in the ethics categories related to humans; protection of personal data; environment, health and safety; and non-EU countries. Ethics experts identified twice as many ethics issues compared to applicants across funding schemes, years, and from high- and low-research performing countries. ERC grants had the highest number of ethics requirements per proposal (median (Md)=8, interquartile range (IQR=4-14), compared to ITN (Md=6, IQR=3-13) and IF grants (Md=3, IQR=2-6). The majority of requirements had to be fulfilled after grant agreement: 99.4% for IF, 99.5% for ITN, and 26.0% for ERC. For 9% of the proposals, the requirements included the appointment of an independent ethics advisor and 1% of the proposals had to appoint an ethics advisory board. Conclusions: Many applicants for highly competitive H2020 funding schemes lack awareness of ethics issues raised by their proposed research. There is a need for better training of researchers at all career stages about ethics issues in research, more support to researchers from research organizations to follow the funding agencies requirements, as well as further development and harmonization of the ethics appraisal process during grant assessment.

scholarly journals On the pitfalls of peer review

F1000Research ◽  
2015 ◽  
Vol 4 ◽  
pp. 1244 ◽  
Author(s):  
Wilfred van Gunsteren
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Scientific Literature ◽  
Scientific Research ◽  
Research Institutions ◽  
Proper Functioning ◽  
Funding Agencies

The review process of academic, scientific research and its basic tenets is considered, thereby distinguishing between (i) reviewing of manuscripts to be published in the scientific literature, (ii) reviewing of research proposals to be financed by funding agencies, (iii) reviewing of educational or research institutions with respect to their proper functioning, and (iv) reviewing of scientists with the aim of appointing or tenuring faculty.

Sign in / Sign up

Export Citation Format

Share Document