scholarly journals Journals in Economic Sciences: Paying Lip Service to Reproducible Research?

10.29173/iq6 ◽  
2017 ◽  
Vol 41 (1-4) ◽  
pp. 16
Author(s):  
Sven Vlaeminck ◽  
Felix Podkrajac
Keyword(s):  
Research Literature ◽  
Data Availability ◽  
Economic Research ◽  
Reproducible Research ◽  
Research Papers ◽  
Program Code ◽  
Replication Studies ◽  
Data Archives ◽  
Data Policy ◽  
Share Of Articles

The findings of numerous replication studies in economics have raised serious concerns regarding the credibility and reliability of published applied economic research. Literature suggests that economic research often is not replicable because (i) only a small proportion of journals in the field have implemented functional policies on the disclosure of employed datasets and program code, (ii) authors frequently do not comply with these data policies and (iii) editorial offices do not ensure that these policies are enforced. In this paper, we focus on the aspect last mentioned. We empirically evaluate 599 articles published in 37 journals with a data availability policy. We present the share of articles that fall under a data policy, because replication data is needed to verify the published results. Afterwards, we check the journal data archives and supplemental information section of each article for the availability of replication files. For a reduced sub-sample of 245 data-based articles, we check in depth whether the replication files we found are compliant with the requirements of the journal’s respective data policy. Thereby, we are able to determine how much journals in economic sciences enforce their data policies. Our findings suggest a mixed picture: While some journals achieve high compliance rates, a significant share of journals only sporadically provides replication files for data-based research papers. Overall, 47.5% of all articles analysed honour the data policy of the respective journal. Our findings also provide evidence that voluntary data policies are not effective in fostering replicable research.

scholarly journals Do economics journals foster replicable research?

2017 ◽  
Author(s):  
Sven Vlaeminck ◽  
Ralf Toepfer
Keyword(s):  
Compliance Rate ◽  
Data Availability ◽  
Economic Research ◽  
Reproducible Research ◽  
Necessary Condition ◽  
Scientific Integrity ◽  
Empirical Economics ◽  
Replication Studies ◽  
Data Archives ◽  
Data Policy

Watch the VIDEO here.Replications are pivotal for the credibility of empirical economics. It is widely recognized in economics that replication studies are a necessary condition for scientific integrity. Alarmingly, several studies indicate that a significant share of empirical economics research cannot be replicated. At the same time, the awareness among researchers, that empirically-based research is often based on shaky grounds, has increased in the last years. It becomes more and more evident that there is a need for more replications in economics to regain trust and credibility in empirical economics research.Though established scholarly journals have adopted replication policies in recent years, replication activities only slightly increased. Against this background our talk investigates if and how journals in economics foster replicable research. For this purpose, we will address two aspects:Journals’ data policies and their effective enforcement in economics: The first part of our talk presents the findings of a new study, in which we evaluated almost 600 articles published in 37 well-regarded journals with a data availability policy. First, we highlight the share of articles that fall under the data policy. Subsequently, the talk contrasts for how many of these data-based articles replication files were available in journals’ data archives and/or the supplemental information section of the article. Moreover, the exact requirements of journals’ data policies have been contrasted to the replication files available on journals’ web pages (respectively in their data archives).We developed a ‘compliance rate’ for each journal in our study. The higher the compliance rate the more do journals enforce their data policy. In the first part of our talk, we also discuss the question whether voluntary data policies are effective in fostering replicable research. For this purpose, we compare the compliance rate of journals with a voluntary data policy to their mandatory counterparts.Journals as publication outlets for replication studies: Though researchers agree that replication studies are needed to regain trust and credibility in empirical economic research, replication activities only slightly increased in recent years.  One reason for that finding can be that in the current system replicating other people’s result does not progresses researcher’s career. Another reason is the paucity of publication outlets for such replication studies. In this part of our talk we also discuss whether established journals should implement replication sections or whether a journal which is entirely dedicated to replication, would be a better way to foster the publication of replication studies. As a showcase we will briefly introduce the newly founded “International Journal for Re-Views in Empirical Economics” (IREE).To conclude, we sketch the current and potential future developments in economics when it comes to reproducible research. 

scholarly journals Text Analysis of Disciplinary Research Papers

2020 ◽  
Author(s):  
Emily Carey ◽  
James Gopsill ◽  
Linda Newnes
Keyword(s):  
Content Analysis ◽  
Social Science ◽  
Text Analysis ◽  
Unique Feature ◽  
Research Literature ◽  
Research Papers ◽  
Venn Diagrams

Research literature terminology illustrates that publications claim to pertain to “disciplinary” approaches and researcher’s align themselves to specific, multi-, inter- or trans-disciplinarities. Ambiguity exists in definition and application of disciplinarity, hence there is need to establish a coherent application of disciplinarity. We present results of content analysis of research literature claiming to be inter-, multi-, or transdisciplinary to assist in ascertaining commonalities or differences for those disciplinarities. We analyse the abstracts and keywords of 8834 papers, using n-grams and bi-grams, dating from 1970 until 2018, extracting a list of 76,552 terms for comparison. The top 15 most frequent terms characterise each disciplinarity and Venn diagrams of the top 15 features illustrate differences and overlap. A total of six terms appear common to all approaches in the abstracts, with four shared by multi- and inter-, two between inter- and trans-, and none common to multi- and trans-. The term “social science(s)” appears to be a unique feature in the trans- abstracts and our findings identify common text terms such as the “research” feature, common to all disciplinarities. This supports characterising the nature of transdisciplinarity and its unique differences from other approaches such as inclusion of social science(s).

scholarly journals Practices and Perspectives of Mental Health in the Balkan Countries: A Narrative Review

2019 ◽  
Author(s):  
Maya E. Lee
Keyword(s):  
Mental Health ◽  
Mental Health Treatment ◽  
Mental Healthcare ◽  
Research Literature ◽  
Economic Research ◽  
Health Issues ◽  
The Balkans ◽  
Historical Perspectives ◽  
Yugoslav Republic ◽  
Balkan Region

Mental health and wellness are integral parts to person’s overall health and happiness. Globally, there has been an increased initiative to treat and support people living with mental health issues and disease; the Balkan region of southeastern Europe is no exception. A literary review researching the background of mental health treatment and how it intersects with the unique history and current administrative environment within the nations of the former Yugoslav Republic was conducted. Existing literature about mental health prevalence and practices within the region was analyzed and contextualized with historical perspectives. Significant gaps in research literature were identified, including lack of research into everyday mental disorders in the region that are not to do with the recent civil war, a need for standardized data collection about where mental health infrastructure exists within the region and how effective it is in treating patients, and finally economic research to determine how and by which governing body national healthcare systems should be funded. Filling these gaps in knowledge would greatly reduce barriers to mental healthcare and overall wellness within the Balkans.

The Studies on Veteran Entrepreneurship in the Past Decade in China

2021 ◽  
pp. 100-118
Author(s):  
Hua Geng ◽  
Benzhao Yang ◽  
Qingan Huang
Keyword(s):  
Research Literature ◽  
Research Trend ◽  
China National Knowledge Infrastructure ◽  
Research Papers ◽  
The Public ◽  
The Past ◽  
Knowledge Infrastructure ◽  
Current State ◽  
The Status ◽  
Literature Mapping

To have a thorough understanding of the current state of veteran entrepreneurship in China, a literature mapping was made on the basis of 235 related articles and research literature published in the past decade (from 2010 to 2019), which are open to the public and available on the China National Knowledge Infrastructure (CNKI). Via combing through the above findings, the authors tried to unlock a phase with prominent rise of corresponding research papers and its possible reason, reveal the leading research power and their correlation, and also explore a potential research trend in the future. On the basis of the overall context and generalization of the status quo and limitations of this field during the past 10 years, reference and guidance can be provided for future study and policy making.

Factors influencing the data sharing behavior of researchers in sociology and political science

2018 ◽  
Vol 74 (5) ◽  
pp. 1053-1073 ◽  
Author(s):  
Wolfgang Zenk-Möltgen ◽  
Esra Akdeniz ◽  
Alexia Katsanidou ◽  
Verena Naßhoven ◽  
Ebru Balaban
Keyword(s):  
Political Science ◽  
Data Sharing ◽  
Behavioral Control ◽  
Statistical Tests ◽  
Data Availability ◽  
Impact Factors ◽  
Data Set ◽  
Content Type ◽  
Data Policy ◽  
Sharing Behavior

Purpose Open data and data sharing should improve transparency of research. The purpose of this paper is to investigate how different institutional and individual factors affect the data sharing behavior of authors of research articles in sociology and political science. Design/methodology/approach Desktop research analyzed attributes of sociology and political science journals (n=262) from their websites. A second data set of articles (n=1,011; published 2012-2014) was derived from ten of the main journals (five from each discipline) and stated data sharing was examined. A survey of the authors used the Theory of Planned Behavior to examine motivations, behavioral control, and perceived norms for sharing data. Statistical tests (Spearman’s ρ, χ2) examined correlations and associations. Findings Although many journals have a data policy for their authors (78 percent in sociology, 44 percent in political science), only around half of the empirical articles stated that the data were available, and for only 37 percent of the articles could the data be accessed. Journals with higher impact factors, those with a stated data policy, and younger journals were more likely to offer data availability. Of the authors surveyed, 446 responded (44 percent). Statistical analysis indicated that authors’ attitudes, reported past behavior, social norms, and perceived behavioral control affected their intentions to share data. Research limitations/implications Less than 50 percent of the authors contacted provided responses to the survey. Results indicate that data sharing would improve if journals had explicit data sharing policies but authors also need support from other institutions (their universities, funding councils, and professional associations) to improve data management skills and infrastructures. Originality/value This paper builds on previous similar research in sociology and political science and explains some of the barriers to data sharing in social sciences by combining journal policies, published articles, and authors’ responses to a survey.

A Review of Asphalt Binders Characterized by X-ray Diffraction

2019 ◽  
Vol 9 (1) ◽  
pp. 28-40 ◽  
Author(s):  
John Shirokoff ◽  
Leonard Lye
Keyword(s):  
Asphalt Binder ◽  
Research Literature ◽  
Oil Fields ◽  
Asphalt Binders ◽  
Data Sets ◽  
X Ray Diffraction ◽  
Research Papers ◽  
X Ray ◽  
Box Plots ◽  
Petroleum Oil

Asphalt binder research papers characterized by X-ray diffraction are reviewed. The Xray results reveal the aromaticity and crystallite size parameters of asphaltene in asphalt binders as reported in the research literature. The research data is discussed in terms of asphalt molecular structure on a length scale, methodology, and petroleum oil fields. Box plots and summary statistics created for each parameter are compared and discussed as they relate to eleven data sets from various asphaltene-crude-oil-geological-regions.

scholarly journals Conflict-Related Population Health Research in Syria, 2011-2019: An Assessment of Information Gaps, Research Needs and Challenges, for The Lancet - AUB Commission on Syria

2020 ◽  
Author(s):  
Marian Abouzeid ◽  
Manal Elzalabany ◽  
Iman Nuwayhid ◽  
Samer Jabbour
Keyword(s):  
Scoping Review ◽  
The United States ◽  
Research Literature ◽  
Humanitarian Assistance ◽  
Syrian Refugees ◽  
Limited Information ◽  
Health Issues ◽  
Research Papers ◽  
Wide Range ◽  
Operational Activities

Abstract Backgorund. While there has been a growth of research on health and the Syrian conflict, most such research covers Syrian refugees. There is limited information about the scope and focus of research on health and population inside Syria. While there are several systemtic and scoping literarure reviews of health of Syrian refugees, there has not been a sccoping review of research literature on health issues inside Syria. Methods. As part of a broader scoping review covering January 2011 to December 2019, we examined English-language conflict-related research papers that studied health issues inside Syria and focused on Syrians or those permanently resident in Syria. We classified research articles based on the major thematic areas studied. We abstracted bibliometric information, study characteristics, funding statements and reported key limitations and challenges of conducting research. To gain additional insights or data, we examined separately publications reporting field and operational activities as well as personal reflections and narrative accounts of first-hand experiences inside Syria. Results. Of 2,088 papers identified in the scoping review, 708 (34%) exclusively focus on health issues of Syrians inside Syria, of which 350 (49%) are conflict-related with. Of conflict-related publications, 89 (25%) are research papers. Annual volume of research increased over time, from one publication in 2013 to 27 publications in each of 2018 and 2019. Damascus is the most frequently studied governorate (n=33), followed by Aleppo (n=25). Papers used a wide range of research methodologies, primarily quantitative (n=68). The country of institutional affiliation(s) of first and last authors are Syria (n=30, 20 respectively), the United States (n=25, 19 respectively) or the United Kingdom (n=12, 10 respectively). Themes most covered were health status, health system and humanitarian assistance, response or needs (n=38, 32, 26 respectively). Thirty one publications presented field and operational activities and eight publications were reflections or first-hand personal accounts of experiences inside Syria. Authors encountered contextual, methodological and administrative challenges in doing research on health inside Syria. Conclusions. Although it is commonly stated that Syria is among the most documented of recent wars, our analysis shows that a relatively limited number of research studies focused on health or populations inside Syria have been published over the nine years of the conflict. Beyond the need to increase the volume of research, it is important to address the knowledge gaps identified in this review.

scholarly journals A Future for Observational Epidemiology: Clarity, Credibility, Transparency

2019 ◽  
Author(s):  
Sam Harper
Keyword(s):  
Observational Studies ◽  
Health Interventions ◽  
Causal Effects ◽  
Reproducible Research ◽  
Bias Analysis ◽  
Replication Studies ◽  
Research Designs ◽  
Alternative Research ◽  
Quantitative Bias Analysis ◽  
Research Standards

Observational studies are ambiguous, difficult, and necessary for epidemiology. Presently there are concerns that the evidence produced by most observational studies in epidemiology is not credible and contributes to research waste. I argue that observational epidemiology could be improved by focusing greater attention on: 1) defining questions that make clear whether the inferential goal is descriptive or causal; 2) greater utilization of quantitative bias analysis and alternative research designs that aim to decrease the strength of assumptions needed to estimate causal effects; and 3) promoting, experimenting, and perhaps institutionalizing reproducible research standards as well as replication studies to evaluate the fragility of study findings in epidemiology. Greater clarity, credibility, and transparency in observational epidemiology will help to provide reliable evidence that can serve as a basis for making decisions about clinical or population health interventions.

scholarly journals Transcriptomics data availability and reusability in the transition from microarray to next-generation sequencing

2021 ◽  
Author(s):  
Gabriella Rustici ◽  
Eleanor Williams ◽  
Mitra Barzine ◽  
Alvis Brazma ◽  
Roger Bumgarner ◽  
...  
Keyword(s):  
Next Generation Sequencing ◽  
High Throughput ◽  
Transcriptome Analysis ◽  
Data Availability ◽  
Rna Seq ◽  
Data Archives ◽  
Transcriptomics Data ◽  
Persistent Data ◽  
Minimal Information ◽  
Generation Sequencing

ABSTRACTOver the last two decades, molecular biology has been changed by the introduction of high-throughput technologies. Data sharing requirements have prompted the establishment of persistent data archives. A standardized approach for recording and managing these data was first proposed in the Minimal Information About a Microarray Experiment (MIAME) guidelines. The Minimal Information about a high throughput nucleotide Sequencing Experiment (MINSEQE) proposal was introduced in 2008 as a logical extension of the guidelines to next-generation sequencing (NGS) technologies used for transcriptome analysis.We present a historical snapshot of the data-sharing situation focusing on transcriptomics data from both microarray and RNA-sequencing experiments published between 2009 and 2013, a period during which RNA-seq studies became increasingly popular for transcriptome analysis. We assess how much data from RNA-seq based experiments is actually available in persistent data archives, compared to data derived from microarray based experiments, and evaluate how these types of data differ. Based on this analysis, we provide recommendations to improve RNA-seq data availability, reusability, and reproducibility.

Economic Research on the Global Allocation of Scarce Water Resources Needs Better Data

2021 ◽  
pp. 2150013
Author(s):  
Ianna Raissa Moreira Dantas ◽  
Ruth Delzeit ◽  
Gernot Klepper
Keyword(s):  
Water Supply ◽  
Data Availability ◽  
Water Withdrawal ◽  
Economic Research ◽  
Joint Research ◽  
Water Abstraction ◽  
Data Harmonization ◽  
National Statistics ◽  
Data Content ◽  
Global Water

Water sustainability is central to modern political and academic debates. Despite increasing efforts to promote regional and global integrated water management, climate change, population, and economic growth, and increasing consumption of water-intensive goods project higher water deficiency. Robust economic analyses rely on information about water supply and consumption across different production sectors, type of procurement source (public or private water supply), and water prices. Nevertheless, developing current and future economic water assessments and indicators is impeded by the absence of data. Despite the lack of official national statistics on water withdrawal and consumption, a small number of international and global databases have been constructed and attempt to combine available national water information into databases. Water databases do not commonly define and/or distinguish terms such as water use, water consumption, water supply, or water abstraction, and the associated aspects of water scarcity and sustainability. They comprise variable data quality, provided by numerous sources, and estimated values. This paper evaluates the current state of knowledge of national statistics, international and global water databases. We describe the data collection methods, identify basic concepts and definitions of water terms, followed by the criteria of consistent water databases. We inform about data availability across regions, and present the data content and definitions of national, international, and global water databases. The results show inconsistencies of data content and definitions, suggesting no evidence of data harmonization among databases. Therefore, our study cautions researchers to be careful when manipulating and comparing the available water data, especially when deriving policy recommendations or economic conclusions. In the long run, the headway of water research and political assessments depend on political enforcements to refine the meaningfulness of water data and support water collection, reporting, and monitoring. Alternatively, in the short- and medium-run, water data challenges can be addressed by joint research efforts for water data harmonization.

Sign in / Sign up

Export Citation Format

Share Document