scholarly journals Grant reviewer perceptions of the quality and effectiveness of panel discussion

2019 ◽  
Author(s):  
Stephen A. Gallo ◽  
Karen B. Schmaling ◽  
Lisa A. Thompson ◽  
Scott R. Glisson
Keyword(s):  
Peer Review ◽  
Panel Discussion ◽  
Limited Participation ◽  
Funding Agencies ◽  
Potential Sources ◽  
Grant Proposals ◽  
Proposal Review ◽  
Method Analysis ◽  
Research Grant

AbstractBackgroundFunding agencies have long used panel discussion in the peer review of research grant proposals as a way to utilize a set of expertise and perspectives in making funding decisions. Little research has examined the quality of panel discussions and how effectively they are facilitated.MethodsHere we present a mixed-method analysis of data from a survey of reviewers focused on their perceptions of the quality and facilitation of panel discussion from their last peer review experience.ResultsReviewers indicated that panel discussions were viewed favorably in terms of participation, clarifying differing opinions, informing unassigned reviewers, and chair facilitation. However, some reviewers mentioned issues with panel discussions, including an uneven focus, limited participation from unassigned reviewers, and short discussion times. Most reviewers felt the discussions affected the review outcome, helped in choosing the best science, and were generally fair and balanced. However, those who felt the discussion did not affect the outcome were also more likely to evaluate panel communication negatively, and several reviewers mentioned potential sources of bias related to the discussion. While respondents strongly acknowledged the importance of the chair in ensuring appropriate facilitation of the discussion to influence scoring and to limit the influence of potential sources of bias from the discussion on scoring, nearly a third of respondents did not find the chair of their most recent panel to have performed these roles effectively.ConclusionsIt is likely that improving chair training in the management of discussion as well as creating review procedures that are informed by the science of leadership and team communication would improve review processes and proposal review reliability.

scholarly journals Grant Review Feedback: Appropriateness and Usefulness

2021 ◽  
Vol 27 (2) ◽  
Author(s):  
Stephen A. Gallo ◽  
Karen B. Schmaling ◽  
Lisa A. Thompson ◽  
Scott R. Glisson
Keyword(s):  
Peer Review ◽  
Funding Agency ◽  
Constructive Feedback ◽  
Competitive Funding ◽  
Grant Proposals ◽  
Funding Success ◽  
Negative Perceptions ◽  
Grant Review ◽  
Research Grant

AbstractThe primary goal of the peer review of research grant proposals is to evaluate their quality for the funding agency. An important secondary goal is to provide constructive feedback to applicants for their resubmissions. However, little is known about whether review feedback achieves this goal. In this paper, we present a multi-methods analysis of responses from grant applicants regarding their perceptions of the effectiveness and appropriateness of peer review feedback they received from grant submissions. Overall, 56–60% of applicants determined the feedback to be appropriate (fair, well-written, and well-informed), although their judgments were more favorable if their recent application was funded. Importantly, independent of funding success, women found the feedback better written than men, and more white applicants found the feedback to be fair than non-white applicants. Also, perceptions of a variety of biases were specifically reported in respondents’ feedback. Less than 40% of applicants found the feedback to be very useful in informing their research and improving grantsmanship and future submissions. Further, negative perceptions of the appropriateness of review feedback were positively correlated with more negative perceptions of feedback usefulness. Importantly, respondents suggested that highly competitive funding pay-lines and poor inter-panel reliability limited the usefulness of review feedback. Overall, these results suggest that more effort is needed to ensure that appropriate and useful feedback is provided to all applicants, bolstering the equity of the review process and likely improving the quality of resubmitted proposals.

scholarly journals Grant Review Feedback: Appropriateness and Usefulness

2020 ◽  
Author(s):  
Stephen Gallo ◽  
Karen Schmaling ◽  
Lisa Thompson ◽  
Scott Glisson
Keyword(s):  
Peer Review ◽  
Funding Agency ◽  
Constructive Feedback ◽  
Competitive Funding ◽  
Grant Proposals ◽  
Funding Success ◽  
Negative Perceptions ◽  
Grant Review ◽  
Research Grant

AbstractThe primary goal of the peer review of research grant proposals is to evaluate their quality for the funding agency. An important secondary goal is to provide constructive feedback to applicants for their resubmissions. However, little is known about whether review feedback achieves this goal. In this paper, we present a mixed methods analysis of responses from grant applicants regarding their perceptions of the effectiveness and appropriateness of peer review feedback they received from grant submissions. Overall, 56%-60% of applicants determined the feedback to be appropriate (fair, well-written, and well-informed), although their judgments were more favorable if their recent application was funded. Importantly, independent of funding success, women found the feedback better written than men, and more white applicants found the feedback to be fair than non-white applicants. Also, perceptions of a variety of biases were specifically reported in respondents’ feedback. Less than 40% of applicants found the feedback to be very useful in informing their research and improving grantsmanship and future submissions. Further, negative perceptions of the appropriateness of review feedback were positively correlated with more negative perceptions of feedback usefulness. Importantly, respondents suggested that highly competitive funding pay-lines and poor inter-panel reliability limited the usefulness of review feedback. Overall, these results suggest that more effort is needed to ensure that appropriate and useful feedback is provided to all applicants, bolstering the equity of the review process and likely improving the quality of resubmitted proposals.

Panel discussion does not improve reliability of peer review for medical research grant proposals

2012 ◽  
Vol 65 (1) ◽  
pp. 47-52 ◽  
Author(s):  
Mikael Fogelholm ◽  
Saara Leppinen ◽  
Anssi Auvinen ◽  
Jani Raitanen ◽  
Anu Nuutinen ◽  
...  
Keyword(s):  
Medical Research ◽  
Peer Review ◽  
Panel Discussion ◽  
Grant Proposals ◽  
Improve Reliability ◽  
Research Grant

scholarly journals Measuring bias, burden and conservatism in research funding processes

F1000Research ◽  
2019 ◽  
Vol 8 ◽  
pp. 851 ◽  
Author(s):  
Susan Guthrie ◽  
Daniela Rodriguez Rincon ◽  
Gordon McInroy ◽  
Becky Ioppolo ◽  
Salil Gunashekar
Keyword(s):  
Peer Review ◽  
Research Funding ◽  
Assessment Process ◽  
Grant Funding ◽  
Funding Allocation ◽  
Complex Process ◽  
Funding Agencies ◽  
Grant Proposals ◽  
Funding Scheme ◽  
Measuring Bias

Background: Grant funding allocation is a complex process that in most cases relies on peer review. A recent study identified a number of challenges associated with the use of peer review in the evaluation of grant proposals. Three important issues identified were bias, burden, and conservatism, and the work concluded that further experimentation and measurement is needed to assess the performance of funding processes. Methods: We have conducted a review of international practice in the evaluation and improvement of grant funding processes in relation to bias, burden and conservatism, based on a rapid evidence assessment and interviews with research funding agencies. Results: The evidence gathered suggests that efforts so far to measure these characteristics systematically by funders have been limited. However, there are some examples of measures and approaches which could be developed and more widely applied. Conclusions: The majority of the literature focuses primarily on the application and assessment process, whereas burden, bias and conservatism can emerge as challenges at many wider stages in the development and implementation of a grant funding scheme. In response to this we set out a wider conceptualisation of the ways in which this could emerge across the funding process.

scholarly journals Navigating the Road to Success: Guidelines for Preparing Competitive Grant Proposals

2007 ◽  
Vol 2 (1) ◽  
pp. 23 ◽  
Author(s):  
Lynn L. Langille ◽  
Theresa Mackenzie
Keyword(s):  
Best Practices ◽  
Research Funding ◽  
Research Process ◽  
Grant Writing ◽  
Original Research ◽  
The Road ◽  
Grant Applications ◽  
Funding Agencies ◽  
Grant Proposals ◽  
Research Grant

Purpose - Difficulty in securing research funding has been cited as one barrier to the involvement of more librarians and information professionals in conducting original research. This article seeks to support the work of librarians who wish to secure research funding by describing some key approaches to the creation of successful grant applications. Approach - The authors draw on more than 15 years experience in supporting the development of successful research grant proposals. Twelve grant-writing best practices or ‘key approaches’ are described, and a planning timeline is suggested. Conclusions - Use of these best practices can assist researchers in creating successful research grant proposals that will also help streamline the research process once it is underway. It is important to recognize the competitive nature of research grant competitions, to obtain feedback from an internal review panel, and to use feedback from funding agencies to strengthen future grant applications.

scholarly journals The Keys to Preparing Successful Research Grant Proposals

2001 ◽  
Vol 33 (3) ◽  
pp. 605-612
Author(s):  
Mary A. Marchant
Keyword(s):  
Peer Review ◽  
Review Process ◽  
Peer Review Process ◽  
Grant Writing ◽  
Application Process ◽  
Department Of Agriculture ◽  
Grant Programs ◽  
Research Initiative ◽  
Grant Proposals ◽  
Research Grant

AbstractThis article seeks to demystify the competitive grant recommendation process of scientific peer review panels. The National Research Initiative Competitive Grants Program (NRICGP) administered by the U.S. Department of Agriculture-Cooperative State Research, Extension, and Education Service (USDA-CSREES) serves as the focus of this article. This article provides a brief background on the NRICGP and discusses the application process, the scientific peer review process, guidelines for grant writing, and ways to interpret reviewer comments if a proposal is not funded. The essentials of good grant writing discussed in this article are transferable to other USDA competitive grant programs.

scholarly journals The Participation and Motivations of Grant Peer Reviewers: A Comprehensive Survey

2018 ◽  
Author(s):  
Stephen A Gallo ◽  
Lisa A Thompson ◽  
Karen B Schmaling ◽  
Scott R Glisson
Keyword(s):  
Peer Review ◽  
Selection Process ◽  
Funding Agency ◽  
Grant Applications ◽  
Time Period ◽  
Funding Agencies ◽  
Integral Role ◽  
Comprehensive Survey ◽  
Peer Reviewers ◽  
Grant Proposals

AbstractScientific peer reviewers play an integral role in the grant selection process, yet very little has been reported on the levels of participation or the motivations of scientists to take part in peer review. AIBS developed a comprehensive peer review survey that examined the motivations and levels of participation of grant reviewers. The survey was disseminated to 13,091 scientists in AIBS’s proprietary database. Of the 874 respondents, 76% indicated they had reviewed grant applications in the last 3 years; however, the number of reviews was unevenly distributed across this sample. Higher review loads were associated with respondents who had submitted more grant proposals over this time period, some of whom were likely to be study section members for large funding agencies. The most prevalent reason to participate in a review was to give back to the scientific community (especially among frequent grant submitters) and the most common reason to decline an invitation to review was lack of time. Interestingly, few suggested that expectation from the funding agency was a motivation to review. Most felt that review participation positively influenced their careers through improving grantsmanship and exposure to new scientific ideas. Of those who reviewed, respondents reported dedicating 2-5% of their total annual work time to grant review and, based on their self-reported maximum review loads, it is estimated they are participating at 56%-89% of their capacity, which may have important implications regarding the sustainability of the system. Overall, it is clear that participation in peer review is uneven and in some cases near capacity, and more needs to be done to create new motivations and incentives to increase the future pool of reviewers.

scholarly journals To fund or not to fund?

eLife ◽  
2017 ◽  
Vol 6 ◽  
Author(s):  
Sarah Shailes
Keyword(s):  
Peer Review ◽  
Funding Agencies ◽  
Grant Proposals

Funding agencies use many different criteria and peer review strategies to assess grant proposals.

The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation

1991 ◽  
Vol 14 (1) ◽  
pp. 119-135 ◽  
Author(s):  
Domenic V. Cicchetti
Keyword(s):  
Peer Review ◽  
Nuclear Physics ◽  
Cultural Anthropology ◽  
Medical Specialty ◽  
Physical Sciences ◽  
Grant Proposals ◽  
Low Levels ◽  
Editorial Decisions ◽  
Scientific Value

AbstractThe reliability of peer review of scientific documents and the evaluative criteria scientists use to judge the work of their peers are critically reexamined with special attention to the consistently low levels of reliability that have been reported. Referees of grant proposals agree much more about what is unworthy of support than about what does have scientific value. In the case of manuscript submissions this seems to depend on whether a discipline (or subfield) is general and diffuse (e.g., cross-disciplinary physics, general fields of medicine, cultural anthropology, social psychology) or specific and focused (e.g., nuclear physics, medical specialty areas, physical anthropology, and behavioral neuroscience). In the former there is also much more agreement on rejection than acceptance, but in the latter both the wide differential in manuscript rejection rates and the high correlation between referee recommendations and editorial decisions suggests that reviewers and editors agree more on acceptance than on rejection. Several suggestions are made for improving the reliability and quality of peer review. Further research is needed, especially in the physical sciences.

Automation of selected brazes' dynamic properties in high-temperatures' measurement process

2013 ◽  
Vol 18 (2-3) ◽  
pp. 33-41
Author(s):  
Dominik Sankowski ◽  
Marcin Bakala ◽  
Rafał Wojciechowski
Keyword(s):  
Surface Tension ◽  
High Temperatures ◽  
Dynamic Properties ◽  
Automatic Measurement ◽  
Measurement Methodology ◽  
Joining Process ◽  
Wetting Force ◽  
Research Grant

Abstract The good quality of several manufactured components frequently depends on solidliquid interactions existing during processing. Nowadays, the research in material engineering focuses also on modern, automatic measurement methods of joining process properties, i.a. wetting force and surface tension, which allows for quantitative determination of above mentioned parameters. In the paper, the brazes’ dynamic properties in high-temperatures’ measurement methodology and the stand for automatic determination of braze’s properties, constructed and implmented within the research grant nr KBN N N519 441 839 - An integrated platform for automatic measurement of wettability and surface tension of solders at high temperatures, are widely described

Sign in / Sign up

Export Citation Format

Share Document