scholarly journals Novel Bibliometric Scores for Evaluating Research Quality and Output: A Correlation Study with Established Indexes

2016 ◽  
Vol 31 (4) ◽  
pp. 451-455 ◽  
Author(s):  
Valeria Scotti ◽  
Annalisa De Silvestri ◽  
Luigia Scudeller ◽  
Paola Abele ◽  
Funda Topuz ◽  
...  

Introduction Novel bibliometric indexes (commonly known as altmetrics) are gaining interest within the scientific community and might represent an important alternative measure of research quality and output. Aims We evaluate how these new metrics correlate with established bibliometric indexes such as the impact factor (IF), currently used as a measure of scientific production as well as a criterion for scientific research funding, and how they might be helpful in assessing the impact of research. Methods We calculated altmetrics scores for all the articles published at our institution during a single year and examined the correlation between altmetrics scores and IFs as a measure of research quality and impact in all departments. Results For all articles from the various departments published in a single year, the altmetrics score and the sum of all IFs showed a strong and significant correlation (Spearman's rho 0.88). The correlation was significant also when the major components of altmetrics, including Facebook, Twitter and Mendeley, were analyzed. The implementation of altmetrics has been found to be easy and effective at both the researcher and librarian levels. Conclusions The novel bibliographic index altmetrics is consistent and reliable and can complement or be considered a valid alternative to standard bibliometric indexes to benchmark output and quality of research for academic and funding purposes.

2015 ◽  
Vol 116 (11/12) ◽  
pp. 661-676 ◽  
Author(s):  
Nedra Ibrahim ◽  
Anja Habacha Chaibi ◽  
Mohamed Ben Ahmed

Purpose – This paper aims to propose a new qualitative indicator for the evaluation of the productions of researchers in any discipline. Design/methodology/approach – Based on the study of existing quantitative indicators, the authors’ approach consisted of the hybridization of two indicators. This hybridization is based on the individual H_index (Hi_index) and H_index contemporary (Hc_index) weighted by qualitative factors. The initial sources of the data are online bibliographic databases, such as Google Scholar and Publish or Perish. Findings – A new scientometric indicator was used to compare the scientific production quality of researchers and their classification (as part of a research community) as the classification of national and international research institutions. The authors have applied a new indicator to compare and classify the members of their laboratory, RIADI, according to their quality of scientific production. Practical implications – The indicator is an improvement of the H_index. It is a measure that can have an impact on society (influencing research attitudes, affecting quality of research). By this contribution, the authors measure more than one aspect by involving all the external factors that can affect the quality of research. Originality/value – This paper fulfils a gap in the literature concerning the absence of a qualitative indicator among the set of existing quantitative measures. Additionally, this paper addresses the limitations of the existing qualitative practices, such as peer review and citation analysis. In the new qualitative indicator, the authors involve all of these qualitative aspects: the influence of the age of the paper, the number of co-authors, the order of the co-authors, the impact factor of journals and the conference rankings.


2018 ◽  
Vol 37 (1) ◽  
Author(s):  
Olli Poropudas

Performance-based research funding systems are often assumed to create an increasing pressure to publish among academics. Presumption is that national level incentives put pressure on researchers to increase quantity of publishing at the expense of quality (publishing pressure hypothesis). Janne Pölönen and Otto Auranen argued in their article in Information Studies 2 (36) that the hypothesis does not apply to Finland. The purpose of this article is to continue the discussion on the hypothesis. I repeat the empirical study by Pölönen and Auranen, and I will end up with somewhat opposite results. Secondly, I discuss the factors affecting the productivity and quality of research. Changes in productivity and quality of research are not just related to the introduction of publications as a criterion for research funding, but the range of explanatory factors is much wider. I will highlight the make-up of researchers, the amount of funding and competitive funding, and look at their explanatory power in respect to the changes of the productivity and the impact of Finnish scientific research.


2020 ◽  
Vol 8 (2) ◽  
pp. 225-264
Author(s):  
Parviz Tarikhi

Sanctions on the scientific community of Iran go back to the days following the victory of the 1979 revolution when the severe pressures on the country commenced. Although it is still difficult to estimate quantitatively the impacts of sanctions on the Iranian scientific community, the qualitative estimations and in situ observations bolster the idea that sanctions, particularly those in recent decades, have led to an obvious decrease in the quality of Iranian scientific production. It has led to further cosmopolitanism of the Iranian scientific community in diaspora, and has strengthened extremism and predomination of hardliner attitudes domestically. This paper demonstrates the impact of foreign sanctions on the scientific community of Iran in addition to the already deplorable pressure initiated by the post-revolutionary ruling system and its administration.


F1000Research ◽  
2019 ◽  
Vol 8 ◽  
pp. 1353 ◽  
Author(s):  
Teresa Gomez-Diaz ◽  
Tomas Recio

Background:Evaluation of the quality of research software is a challenging and relevant issue, still not sufficiently addressed by the scientific community.Methods:Our contribution begins by defining, precisely but widely enough, the notions of research software and of its authors followed by a study of the evaluation issues, as the basis for the proposition of a sound assessment protocol: the CDUR procedure.Results:CDUR comprises four steps introduced as follows:Citation, to deal with correct RS identification,Dissemination, to measure good dissemination practices,Use, devoted to the evaluation of usability aspects, andResearch, to assess the impact of the scientific work.Conclusions:Some conclusions and recommendations are finally included. The evaluation of research is the keystone to boost the evolution of the Open Science policies and practices.  It is as well our belief that research software evaluation is a fundamental step to induce better research software practices and, thus, a step towards more efficient science.


F1000Research ◽  
2019 ◽  
Vol 8 ◽  
pp. 1353 ◽  
Author(s):  
Teresa Gomez-Diaz ◽  
Tomas Recio

Background: Evaluation of the quality of research software is a challenging and relevant issue, still not sufficiently addressed by the scientific community. Methods: Our contribution begins by defining, precisely but widely enough, the notions of research software and of its authors followed by a study of the evaluation issues, as the basis for the proposition of a sound assessment protocol: the CDUR procedure. Results: CDUR comprises four steps introduced as follows: Citation, to deal with correct RS identification, Dissemination, to measure good dissemination practices, Use, devoted to the evaluation of usability aspects, and Research, to assess the impact of the scientific work. Conclusions: Some conclusions and recommendations are finally included. The evaluation of research is the keystone to boost the evolution of the Open Science policies and practices.  It is as well our belief that research software evaluation is a fundamental step to induce better research software practices and, thus, a step towards more efficient science.


2020 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Nethal K. Jajo ◽  
Shelton Peiris

PurposeThe purpose of this paper is to explore the impact of the Excellence Research in Australia (ERA) process in boosting research quality at Australian universities, this paper presents an analysis of a policy initiative, ERA, and compares the results of its measures as calculated in 2018 with those observed in previous implementation, namely, 2015 and 2012.Design/methodology/approachTwo approaches are implemented in this study; Excellence Index (EI) scores for both cited and peer-reviewed 4 digits FoR codes and citation per paper (CPP) approach for the cited 4 digits FoR codes.FindingsThe authors show that the higher education providers' (HEPs') performance in the cited FoRs in ERA in 2018 was improved by 27% compared to that in 2015, and that HEPs' performance in the cited FoR codes in ERA 2015 was improved by 80% compared to that in 2012. A reason for this visibility of research improvement may be due to the universities are simply getting better at reporting outcomes using ERA-driven criteria. Moreover, even though EI scores steadily increased in ERA rounds, there is no significant statistical evidence available of improvement in research quality between two consecutive ERA rounds.Originality/valueThese findings underpin the importance of more future research and deep analysis using the other complementary variables, like Relative Citation Impact (RCI), citation centiles and distribution of papers based on the centiles and RCI classes and more transparency and data availability from the Australian Research Council (ARC) site. Given the introduction of the Engagement and Impact Assessment by the ARC to accompany the ERA exercise in 2018, the authors expect that the results of these findings will be useful as well as prompting further debate and scholarship to the relevance and value of the ERA process.


2020 ◽  
Vol 93 (6) ◽  
pp. 343-350
Author(s):  
Molly O. Regelmann ◽  
Rushika Conroy ◽  
Evgenia Gourgari ◽  
Anshu Gupta ◽  
Ines Guttmann-Bauman ◽  
...  

<b><i>Background:</i></b> Pediatric endocrine practices had to rapidly transition to telemedicine care at the onset of the novel coronavirus disease 2019 (COVID-19) pandemic. For many, it was an abrupt introduction to providing virtual healthcare, with concerns related to quality of patient care, patient privacy, productivity, and compensation, as workflows had to change. <b><i>Summary:</i></b> The review summarizes the common adaptations for telemedicine during the pandemic with respect to the practice of pediatric endocrinology and discusses the benefits and potential barriers to telemedicine. <b><i>Key Messages:</i></b> With adjustments to practice, telemedicine has allowed providers to deliver care to their patients during the COVID-19 pandemic. The broader implementation of telemedicine in pediatric endocrinology practice has the potential for expanding patient access. Research assessing the impact of telemedicine on patient care outcomes in those with pediatric endocrinology conditions will be necessary to justify its continued use beyond the COVID-19 pandemic.


1997 ◽  
Vol 11 (5) ◽  
pp. 304-313
Author(s):  
W. O. George ◽  
A. N. Hill

In this paper, the origins and characteristics of the 102 current UK universities are briefly traced and the outcomes of recent assessments of research quality are summarized for all universities and for the 69 subject units within which assessment was made. The quality of research in a subject unit, group of subject units or complete institution is measured by a weighted average score based on a peer rating of submitted subject units from each university and the numerical values obtained are described within the limitations of the methodology developed. The authors consider the scores in terms of the characteristics of each university and the broad subject areas, science, engineering, social sciences and humanities. They then discuss the industrial link with research in terms of recent government policy inputs, university research outcomes and the impact of market forces on universities from diminishing patterns of some income streams.


Blood ◽  
2019 ◽  
Vol 134 (Supplement_1) ◽  
pp. 2174-2174
Author(s):  
Michelle Neier ◽  
Michele P. Lambert ◽  
Rachael F. Grace ◽  
Kerry Hege ◽  
Stephanie Chiu ◽  
...  

Background: Immune thrombocytopenia (ITP) is an immune mediated bleeding disorder characterized by isolated thrombocytopenia. ITP can have a variety of presentations from asymptomatic to life threatening bleeding. Although childhood ITP is most often a self-resolving illness which can be closely observed without intervention, it can be associated with significant impact on quality of life (QoL). Prospective studies of QoL in ITP patients show that there is not always a correlation with treatment or disease severity. The pathway from initial presentation to final diagnosis varies and may include encounters with emergency room, primary care or specialty providers. There have been no published studies to date showing the impact of factors prior to the diagnosis of ITP on treatment decision making and QoL. Objective: To identify the role of physician-patient and physician-caregiver interactions on the QoL and emotional well-being of patients and their families. Ascertaining the impact of pre-diagnosis factors may provide an opportunity to improve access and quality of care provided. Methods: The ITP Consortium of North America (ICON) "Pathways" study was a multicenter observational prospective cohort study focused on the pathways to diagnosis of ITP. The study was supported by a Foundation for Morristown Medical Center Research Fund Grant. Subjects were included if they had presumed primary ITP and were age >12 months to <18 years. Subjects were excluded if they had secondary ITP, including Evans syndrome. Treatment was determined by the physician. Subjects were consented and presented with questionnaires to be completed at the conclusion of the initial hematology visit. The hematologist also completed survey data at that time. Survey data forms included demographic form, physician form, Peds QL Family Impact Questionnaire, Kids ITP tools (KIT) Parent Impact Report and parent proxy report, and child (patient) KIT self-report. There was a parent questionnaire which included a question about worry with a scale from 0 to 10. Study data were collected and managed using REDCap electronic data capture tools hosted at Atlantic Health System. Correlation between variables were calculated using Pearson coefficient or Spearman's rho depending on the distribution of the data variables. Results: Sixty subjects and caregivers were enrolled at 6 ICON centers; 52 were eligible for inclusion. The majority (40%) had Grade 1 bleeding. Most patients (82%) were seen in outpatient hematology clinic by the hematologist and had been referred by the emergency room (73%). The median time to consultation with a hematologist from onset of symptoms was 7 days (1-199) and the median time to diagnosis by hematologist from initial contact with a health care provider was 5 days (0-154). Most subjects had seen 2 health care providers prior to the hematologist. KIT proxy report cumulative scores were a mean of 76.03 (SD 14.72). There was no significant difference between the time to diagnosis or the time from initial encounter with health care provider to hematologist and initial level of worry (p=0.70 and 0.90, respectively). There was also no significant difference between the time to diagnosis or the time from initial encounter with health care provider to hematologist and KIT proxy scores (p=0.96 and 0.50, respectively). However, there was a significant decline in level of worry (scale 0-10) prior to the hematologist visit (median 8, range 1-10) to after the visit (median 4, range 1-10). The association between number of medical providers encountered prior to diagnosis and KIT proxy scores was not significant (p=0.45) (Table). Conclusions: In this study at 6 teaching institutions, we were unable to detect a significant difference in proxy-reported KIT scores relative to the number of health care providers seen or time from diagnosis until the first encounter with the hematologist. We were, however, able to detect a significant change in the level of caregiver worry pre- and post- visit with the pediatric hematologist, supporting a benefit of specialist care to the caregivers of children with ITP. This study was limited by its small sample size and retrospective design. ITP is considered a benign disease but is associated with a significant amount of worry and impact on QoL for patients and caregivers which warrants further investigation. Disclosures Lambert: CSL Behring: Consultancy; Amgen: Consultancy, Other; Bayer: Other: Ad boards; Novartis: Other: Ad boards, Research Funding; Shionogi: Consultancy; Kedrion: Consultancy; Sysmex: Consultancy; AstraZeneca: Research Funding; PDSA: Research Funding. Grace:Agios Pharmaceuticals, Inc: Consultancy, Membership on an entity's Board of Directors or advisory committees, Research Funding; Novartis: Research Funding.


Sign in / Sign up

Export Citation Format

Share Document