Auditing with Data and Analytics: External Reviewer Perceptions of Audit Quality and Effort

2020 ◽  
Author(s):  
Scott A. Emett ◽  
Steven E. Kaplan ◽  
Elaine Mauldin ◽  
Jeffrey Scott Pickerd
Keyword(s):  
Audit Quality ◽  
External Reviewer

scholarly journals Gender Bias Impacts Top-Merited Candidates

2021 ◽  
Vol 6 ◽  
Author(s):  
Emma Rachel Andersson ◽  
Carolina E. Hagberg ◽  
Sara Hägg
Keyword(s):  
Peer Review ◽  
Gender Bias ◽  
Assistant Professor ◽  
Gender Segregation ◽  
External Reviewer ◽  
Men And Women ◽  
Fair Competition ◽  
Track Record ◽  
Mesh Terms ◽  
Senior Researcher

Expectations of fair competition underlie the assumption that academia is a meritocracy. However, bias may reinforce gender inequality in peer review processes, unfairly eliminating outstanding individuals. Here, we ask whether applicant gender biases peer review in a country top ranked for gender equality. We analyzed peer review assessments for recruitment grants at a Swedish medical university, Karolinska Institutet (KI), during four consecutive years (2014–2017) for Assistant Professor (n = 207) and Senior Researcher (n = 153). We derived a composite bibliometric score to quantify applicant productivity and compared this score with subjective external (non-KI) peer reviewer scores of applicants' merits to test their association for men and women, separately. To determine whether there was gender segregation in research fields, we analyzed publication list MeSH terms, for men and women, and analyzed their overlap. There was no gendered MeSH topic segregation, yet men and women with equal merits are scored unequally by reviewers. Men receive external reviewer scores resulting in stronger associations (steeper slopes) between computed productivity and subjective external reviewer scores, meaning that peer reviewers “reward” men's productivity with proportional merit scores. However, women applying for assistant professor or senior researcher receive only 32 or 92% of the score men receive, respectively, for each additional composite bibliometric score point. As productivity increases, the differences in merit scores between men and women increases. Accumulating gender bias is thus quantifiable and impacts the highest tier of competition, the pool from which successful candidates are ultimately chosen. Track record can be computed, and granting organizations could therefore implement a computed track record as quality control to assess whether bias affects reviewer assessments.

scholarly journals 2286 A CTSA External Reviewer Exchange Consortium: Description and lessons learned

2018 ◽  
Vol 2 (S1) ◽  
pp. 2-2
Author(s):  
Margaret Schneider ◽  
Tanya Mathew ◽  
Madeline Gibson ◽  
Christine Zeller ◽  
Hardeep Ranu ◽  
...  
Keyword(s):  
Review Process ◽  
Tracking System ◽  
Lessons Learned ◽  
Conference Calls ◽  
Self Assessment ◽  
Pilot Studies ◽  
External Reviewer ◽  
The Future ◽  
The Status ◽  
Original Objective

OBJECTIVES/SPECIFIC AIMS: To share the experience gained and lessons learned from a cross CTSA collaborative effort to improve the review process for Pilot Studies awards by exchanging external reviewers. METHODS/STUDY POPULATION: The CEREC process is managed by a web-based tracking system that enables all participating members to view at any time the status of reviewer invitations. This online tracking system is supplemented by monthly conference calls during which new calls for proposals are announced and best practices are identified. Each CTSA hub customized the CEREC model based on their individual pilot program needs and review process. Some hubs have supplemented their internal reviews by only posting proposals on CEREC that lack reviewers with significant expertise within their institutions. Other hubs have requested 1–3 external reviewers for each of their proposals or a selection of most promising proposals. In anticipation of potential scoring discrepancies, several hubs added a self-assessment of reviewer expertise and confidence at the end of each review. If a proposal is on the cusp of fundability, then the reviewers’ self-assessment may be taken into account. In addition to the tracking data collected by the online system, a survey of CEREC reviewers was conducted using Qualtrics. RESULTS/ANTICIPATED RESULTS: Across the 144 proposals submitted for reviews, CEREC members issued a total of 396 email invitations to potential reviewers. The number of invitations required to yield a reviewer ranged from 1 to 17. A total of 224 invitations were accepted, for a response rate of 56%. An external reviewer was unable to be located for 5 proposals (3%). Ultimately, 196 completed reviews were submitted, for a completion rate of 87%. The most common reasons for non-completion after acceptance of an invitation included reviewer illness and discovery of a conflict of interest. CEREC members found the process extremely useful for locating qualified reviewers who were not in conflict with the proposal being reviewed and for identifying reviewers for proposals related to highly specialized topics. The survey of CEREC reviewers found that they generally found the process easy to navigate and intellectually rewarding. Most would be willing to review additional CEREC proposals in the future. External reviewer comments and scores were generally in agreement with internal reviewer comments and scores. Thus, hubs could factor in external reviewer scores equally to internal reviewer scores, without feeling compelled to calibrate external reviewer scores. Overall, through CEREC external reviewers, mainly due to the stronger matching of scientific expertise and reduction of potential bias, the quality of reviews appear to be higher and more pertinent. DISCUSSION/SIGNIFICANCE OF IMPACT: Some aspects of the process emerged that will be addressed in the future to make the system more efficient. One issue that arose was the burden on the system during multiple simultaneous calls for proposals. Future plans call for harmonizing review cycles to avoid these overlaps. Efficiency also will be improved by optimizing the timing of reviewer invitations to minimize the probability of obtaining more reviews than requested. In addition to the original objective of CEREC, the collaboration has led to additional exchange of information regarding methods and processes related to running the Pilot Funding programs. For example, one site developed a method using REDCap to manage their reviewer database; an innovation that is being shared with the other CEREC partners. Another site has a well-developed process for integrating community reviewers into their review process and is sharing their training materials with the remaining CEREC partners.

scholarly journals Auditing with Data and Analytics: External Reviewer Perceptions of Audit Quality and Effort

2020 ◽  
Author(s):  
Elaine Mauldin ◽  
Scott Emett ◽  
Steve E. Kaplan ◽  
Jeffrey Scott Pickerd
Keyword(s):  
Audit Quality ◽  
External Reviewer

scholarly journals Atrial fibrillation in unexplained syncope: observations from the Reveal LINQ registry

2021 ◽  
Vol 42 (Supplement_1) ◽  
Author(s):  
M Del Greco ◽  
A Natale ◽  
K Kusano ◽  
A Verma ◽  
S Beinart ◽  
...  
Keyword(s):  
Atrial Fibrillation ◽  
Private Company ◽  
Detection Rates ◽  
External Reviewer ◽  
Funding Sources ◽  
Kaplan Meier ◽  
History Of ◽  
Unexplained Syncope ◽  
Implantable Loop Recorders

Abstract Background Implantable loop recorders (ILRs) have come to play an important role in the workup of patients with recurrent syncope of uncertain origin. In addition to detecting bradyarrhythmias related to syncope, which is the main diagnostic focus in these patients, ILRs are also capable of uncovering subclinical atrial fibrillation (AF). Purpose We sought to determine the percentage of patients monitored with an ILR for unexplained syncope who have AF detected and to describe clinical actions taken in these patients. Methods Patients enrolled in the Reveal LINQ Registry who received an ILR for unexplained syncope and had at least one follow-up form were included. The device automatically detects AF episodes lasting ≥2 minutes. Patients were considered to have AF based on an AF diagnosis made by the treating physician during follow-up or if device-detected AF was adjudicated as true AF by an external reviewer. AF detection rates were calculated using Kaplan-Meier methods. Results In total, 498 patients (aged 61.8±20.0 years, 49.6% female, CHA2DS2VASc score 2.2±1.7) were included and followed for 22±12 months. A history of AF was present in 97 (20%) patients, while 401 patients had no history. By 18 months, the incidence of AF was 70.9% (95% CI, 60.8%, 80.3%) in patients with a history of AF and 21.4% (95% CI, 17.4%, 26.1%) in patients without (Figure). AF detection in those with (30.4%) and without (30.1%) syncope during follow-up was similar. By the end of follow-up, and among patients with newly detected AF, 29/86 (33.7%) were on oral anticoagulation, 7 (8.1%) underwent AF ablation, 6 (7.0%) underwent other type of ablation, and 2 (2.3%) received cardioversion. Other actions among the whole cohort included implant of an IPG, ICD, or CRT in 98/498 (19.7%). Conclusion Among patients monitored with ILRs to determine the cause of recurrent syncope episodes, approximately 1 in 5 patients had new AF detected. In addition to improving the management of patients with syncope, ILR data served to support AF-related clinical decisions. FUNDunding Acknowledgement Type of funding sources: Private company. Main funding source(s): Medtronic Inc Incidence of AF according to baseline AF

scholarly journals Association Between Mechanical, Physiological, and Technical Parameters With Canoe Slalom Performance: A Systematic Review

2021 ◽  
Vol 12 ◽  
Author(s):  
Leonardo Henrique Dalcheco Messias ◽  
Ivan Gustavo Masselli Dos Reis ◽  
Viktor Bielik ◽  
Ana Luíza Paula Garbuio ◽  
Claudio Alexandre Gobatto ◽  
...  
Keyword(s):  
Total Sample ◽  
Physiological Data ◽  
Published Data ◽  
Relevant Parameter ◽  
Male Athletes ◽  
External Reviewer ◽  
Race Time ◽  
Technical Parameters ◽  
Force Development ◽  
Meta Analyses

This study aimed to systematically review studies that evaluated and compared mechanical, physiological, and technical parameters with the performance of slalom athletes. PubMed, SPORTDiscuss, and Scopus databases were searched until September 10, 2021, with no restriction of published data. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses guided the study's screening and quality assessment performed by an external reviewer using a 16-checklist item. A search of the databases identified 125 studies, but only eight were eligible, including a total sample of 117 male athletes. Four reports only associated mechanical or technical parameters with the performance of the paddler. Concerning the remaining studies, only one correlated physiological data, and the others associated more than one parameter with race time. Most of the eligible reports presented significant associations between mechanical/physiological components and slalom performance. Eligible studies support that high-force development during a slalom race is a relevant parameter for performance. Aerobic metabolism is highly required during slalom tasks and is inversely associated with race time, although it may not increase the chances of winning medals. Few reports have associated canoe slalom performance with technical components, and further research should focus on this matter.

Sign in / Sign up

Export Citation Format

Share Document