scholarly journals Do economics journals foster replicable research?

2017 ◽  
Author(s):  
Sven Vlaeminck ◽  
Ralf Toepfer
Keyword(s):  
Compliance Rate ◽  
Data Availability ◽  
Economic Research ◽  
Reproducible Research ◽  
Necessary Condition ◽  
Scientific Integrity ◽  
Empirical Economics ◽  
Replication Studies ◽  
Data Archives ◽  
Data Policy

Watch the VIDEO here.Replications are pivotal for the credibility of empirical economics. It is widely recognized in economics that replication studies are a necessary condition for scientific integrity. Alarmingly, several studies indicate that a significant share of empirical economics research cannot be replicated. At the same time, the awareness among researchers, that empirically-based research is often based on shaky grounds, has increased in the last years. It becomes more and more evident that there is a need for more replications in economics to regain trust and credibility in empirical economics research.Though established scholarly journals have adopted replication policies in recent years, replication activities only slightly increased. Against this background our talk investigates if and how journals in economics foster replicable research. For this purpose, we will address two aspects:Journals’ data policies and their effective enforcement in economics: The first part of our talk presents the findings of a new study, in which we evaluated almost 600 articles published in 37 well-regarded journals with a data availability policy. First, we highlight the share of articles that fall under the data policy. Subsequently, the talk contrasts for how many of these data-based articles replication files were available in journals’ data archives and/or the supplemental information section of the article. Moreover, the exact requirements of journals’ data policies have been contrasted to the replication files available on journals’ web pages (respectively in their data archives).We developed a ‘compliance rate’ for each journal in our study. The higher the compliance rate the more do journals enforce their data policy. In the first part of our talk, we also discuss the question whether voluntary data policies are effective in fostering replicable research. For this purpose, we compare the compliance rate of journals with a voluntary data policy to their mandatory counterparts.Journals as publication outlets for replication studies: Though researchers agree that replication studies are needed to regain trust and credibility in empirical economic research, replication activities only slightly increased in recent years.  One reason for that finding can be that in the current system replicating other people’s result does not progresses researcher’s career. Another reason is the paucity of publication outlets for such replication studies. In this part of our talk we also discuss whether established journals should implement replication sections or whether a journal which is entirely dedicated to replication, would be a better way to foster the publication of replication studies. As a showcase we will briefly introduce the newly founded “International Journal for Re-Views in Empirical Economics” (IREE).To conclude, we sketch the current and potential future developments in economics when it comes to reproducible research. 

scholarly journals Journals in Economic Sciences: Paying Lip Service to Reproducible Research?

10.29173/iq6 ◽  
2017 ◽  
Vol 41 (1-4) ◽  
pp. 16
Author(s):  
Sven Vlaeminck ◽  
Felix Podkrajac
Keyword(s):  
Research Literature ◽  
Data Availability ◽  
Economic Research ◽  
Reproducible Research ◽  
Research Papers ◽  
Program Code ◽  
Replication Studies ◽  
Data Archives ◽  
Data Policy ◽  
Share Of Articles

The findings of numerous replication studies in economics have raised serious concerns regarding the credibility and reliability of published applied economic research. Literature suggests that economic research often is not replicable because (i) only a small proportion of journals in the field have implemented functional policies on the disclosure of employed datasets and program code, (ii) authors frequently do not comply with these data policies and (iii) editorial offices do not ensure that these policies are enforced. In this paper, we focus on the aspect last mentioned. We empirically evaluate 599 articles published in 37 journals with a data availability policy. We present the share of articles that fall under a data policy, because replication data is needed to verify the published results. Afterwards, we check the journal data archives and supplemental information section of each article for the availability of replication files. For a reduced sub-sample of 245 data-based articles, we check in depth whether the replication files we found are compliant with the requirements of the journal’s respective data policy. Thereby, we are able to determine how much journals in economic sciences enforce their data policies. Our findings suggest a mixed picture: While some journals achieve high compliance rates, a significant share of journals only sporadically provides replication files for data-based research papers. Overall, 47.5% of all articles analysed honour the data policy of the respective journal. Our findings also provide evidence that voluntary data policies are not effective in fostering replicable research.

Benfordʼs Law and Fraud in Economic Research

Benford's Law ◽  
2015 ◽  
Author(s):  
Karl-Heinz Tödter
Keyword(s):  
Scientific Misconduct ◽  
Economic Research ◽  
Benford’S Law ◽  
Empirical Economics ◽  
Benford's Law ◽  
Published Research ◽  
Economics Research

This chapter reviews applications of Benford's law to uncover fraud in macroeconomic data, forecasts, and econometric regression results, as Benford's law provides tools for checking reliability and detecting fraud in science and academia. Scientific misconduct appears in various forms: fabrication, falsification, and plagiarism. Replication is considered the prime strategy against scientific misconduct, but it is seldom performed in empirical economics. Regardless, the potential of Benford's law to enhance the efficiency of replication as a strategy against fraud in published research is discussed in this chapter. In addition, the chapter remarks that routine applications of Benford tests could uncover data anomalies and provide valuable hints of irregularities in empirical economics research.

Factors influencing the data sharing behavior of researchers in sociology and political science

2018 ◽  
Vol 74 (5) ◽  
pp. 1053-1073 ◽  
Author(s):  
Wolfgang Zenk-Möltgen ◽  
Esra Akdeniz ◽  
Alexia Katsanidou ◽  
Verena Naßhoven ◽  
Ebru Balaban
Keyword(s):  
Political Science ◽  
Data Sharing ◽  
Behavioral Control ◽  
Statistical Tests ◽  
Data Availability ◽  
Impact Factors ◽  
Data Set ◽  
Content Type ◽  
Data Policy ◽  
Sharing Behavior

Purpose Open data and data sharing should improve transparency of research. The purpose of this paper is to investigate how different institutional and individual factors affect the data sharing behavior of authors of research articles in sociology and political science. Design/methodology/approach Desktop research analyzed attributes of sociology and political science journals (n=262) from their websites. A second data set of articles (n=1,011; published 2012-2014) was derived from ten of the main journals (five from each discipline) and stated data sharing was examined. A survey of the authors used the Theory of Planned Behavior to examine motivations, behavioral control, and perceived norms for sharing data. Statistical tests (Spearman’s ρ, χ2) examined correlations and associations. Findings Although many journals have a data policy for their authors (78 percent in sociology, 44 percent in political science), only around half of the empirical articles stated that the data were available, and for only 37 percent of the articles could the data be accessed. Journals with higher impact factors, those with a stated data policy, and younger journals were more likely to offer data availability. Of the authors surveyed, 446 responded (44 percent). Statistical analysis indicated that authors’ attitudes, reported past behavior, social norms, and perceived behavioral control affected their intentions to share data. Research limitations/implications Less than 50 percent of the authors contacted provided responses to the survey. Results indicate that data sharing would improve if journals had explicit data sharing policies but authors also need support from other institutions (their universities, funding councils, and professional associations) to improve data management skills and infrastructures. Originality/value This paper builds on previous similar research in sociology and political science and explains some of the barriers to data sharing in social sciences by combining journal policies, published articles, and authors’ responses to a survey.

Replication, Meta-Analysis, and Research Synthesis in Economics

2017 ◽  
Vol 107 (5) ◽  
pp. 56-59 ◽  
Author(s):  
Richard G. Anderson ◽  
Areerat Kichkha
Keyword(s):  
Meta Analysis ◽  
Empirical Studies ◽  
Computer Software ◽  
Research Synthesis ◽  
Economic Research ◽  
Scientific Integrity ◽  
Metadata Analysis

Replicability is essential to assure the scientific integrity of empirical economic research. Some authors have suggested metadata analysis as a complementary tool. This suggestion must be addressed with caution. Empirical studies in most of economics are nonstochastic in the sense that they seldom are created via experiments and, with the data and computer software in-hand, should be reproducible without error.

scholarly journals DIRECTIONS FOR IMPROVEMENT OF GOODS DECLARATION IN CONDITIONS OF INTEGRATION PROCESSES

Management ◽  
2019 ◽  
Vol 28 (2) ◽  
pp. 76-86
Author(s):  
Tetyana Yu. Dudorova
Keyword(s):  
The State ◽  
Economic Research ◽  
Necessary Condition ◽  
Electronic Document ◽  
The European Union ◽  
Expert Assessments ◽  
Import And Export ◽  
And Control ◽  
The Given ◽  
Current Mechanism

Introduction and purpose of the study. Trends in global development in the context of the global economic crisis dictate new challenges, in particular, the simplification of customs and logistics procedures during the import and export of goods to (from) the territory of countries. Today, the maximum approximation of customs procedures in Ukraine to the European and world standards is gradually being made, their maximum simplification at all stages, the reduction of the influence of the human factor on the results of the customs introduction of the elements of the electronic document flow system in the execution of customs clearance and customs control of goods.The hypothesis of scientific research. It is anticipated that in today's conditions of openness of the Ukrainian economy the necessary condition for the organization of the customs business – in general – and customs control, in particular, is the declaration, identification and control of customs value, which has fiscal and regulatory potential, and accordingly characterized by problematic issues that require new theoretical and practical approaches to its evaluation and control. The purpose of the article is to study the current mechanism of customs declaration of goods in Ukraine.Methods of research: in the process of work, the following methods of economic research were used: system-structural, abstract-logical, comparative and balance, expert assessments, etc.Results: The state of the procedure of customs clearance of goods is considered and investigated. The given results of practical introduction of electronic document circulation with the use of electronic digital signature in the customs business of Ukraine.Conclusions. The creation of a multifunctional integrated electronic customs system is aimed at strengthening Ukraine's customs security, further developing and adapting the information systems of the State Customs Service of Ukraine to the standards of the European Union, creating conditions for accelerating customs control and customs clearance procedures, improving the level of services provided by the state in the customs sphere, restriction of the possibility of committing illegal and corrupt acts.

scholarly journals A Future for Observational Epidemiology: Clarity, Credibility, Transparency

2019 ◽  
Author(s):  
Sam Harper
Keyword(s):  
Observational Studies ◽  
Health Interventions ◽  
Causal Effects ◽  
Reproducible Research ◽  
Bias Analysis ◽  
Replication Studies ◽  
Research Designs ◽  
Alternative Research ◽  
Quantitative Bias Analysis ◽  
Research Standards

Observational studies are ambiguous, difficult, and necessary for epidemiology. Presently there are concerns that the evidence produced by most observational studies in epidemiology is not credible and contributes to research waste. I argue that observational epidemiology could be improved by focusing greater attention on: 1) defining questions that make clear whether the inferential goal is descriptive or causal; 2) greater utilization of quantitative bias analysis and alternative research designs that aim to decrease the strength of assumptions needed to estimate causal effects; and 3) promoting, experimenting, and perhaps institutionalizing reproducible research standards as well as replication studies to evaluate the fragility of study findings in epidemiology. Greater clarity, credibility, and transparency in observational epidemiology will help to provide reliable evidence that can serve as a basis for making decisions about clinical or population health interventions.

scholarly journals Transcriptomics data availability and reusability in the transition from microarray to next-generation sequencing

2021 ◽  
Author(s):  
Gabriella Rustici ◽  
Eleanor Williams ◽  
Mitra Barzine ◽  
Alvis Brazma ◽  
Roger Bumgarner ◽  
...  
Keyword(s):  
Next Generation Sequencing ◽  
High Throughput ◽  
Transcriptome Analysis ◽  
Data Availability ◽  
Rna Seq ◽  
Data Archives ◽  
Transcriptomics Data ◽  
Persistent Data ◽  
Minimal Information ◽  
Generation Sequencing

ABSTRACTOver the last two decades, molecular biology has been changed by the introduction of high-throughput technologies. Data sharing requirements have prompted the establishment of persistent data archives. A standardized approach for recording and managing these data was first proposed in the Minimal Information About a Microarray Experiment (MIAME) guidelines. The Minimal Information about a high throughput nucleotide Sequencing Experiment (MINSEQE) proposal was introduced in 2008 as a logical extension of the guidelines to next-generation sequencing (NGS) technologies used for transcriptome analysis.We present a historical snapshot of the data-sharing situation focusing on transcriptomics data from both microarray and RNA-sequencing experiments published between 2009 and 2013, a period during which RNA-seq studies became increasingly popular for transcriptome analysis. We assess how much data from RNA-seq based experiments is actually available in persistent data archives, compared to data derived from microarray based experiments, and evaluate how these types of data differ. Based on this analysis, we provide recommendations to improve RNA-seq data availability, reusability, and reproducibility.

Economic Research on the Global Allocation of Scarce Water Resources Needs Better Data

2021 ◽  
pp. 2150013
Author(s):  
Ianna Raissa Moreira Dantas ◽  
Ruth Delzeit ◽  
Gernot Klepper
Keyword(s):  
Water Supply ◽  
Data Availability ◽  
Water Withdrawal ◽  
Economic Research ◽  
Joint Research ◽  
Water Abstraction ◽  
Data Harmonization ◽  
National Statistics ◽  
Data Content ◽  
Global Water

Water sustainability is central to modern political and academic debates. Despite increasing efforts to promote regional and global integrated water management, climate change, population, and economic growth, and increasing consumption of water-intensive goods project higher water deficiency. Robust economic analyses rely on information about water supply and consumption across different production sectors, type of procurement source (public or private water supply), and water prices. Nevertheless, developing current and future economic water assessments and indicators is impeded by the absence of data. Despite the lack of official national statistics on water withdrawal and consumption, a small number of international and global databases have been constructed and attempt to combine available national water information into databases. Water databases do not commonly define and/or distinguish terms such as water use, water consumption, water supply, or water abstraction, and the associated aspects of water scarcity and sustainability. They comprise variable data quality, provided by numerous sources, and estimated values. This paper evaluates the current state of knowledge of national statistics, international and global water databases. We describe the data collection methods, identify basic concepts and definitions of water terms, followed by the criteria of consistent water databases. We inform about data availability across regions, and present the data content and definitions of national, international, and global water databases. The results show inconsistencies of data content and definitions, suggesting no evidence of data harmonization among databases. Therefore, our study cautions researchers to be careful when manipulating and comparing the available water data, especially when deriving policy recommendations or economic conclusions. In the long run, the headway of water research and political assessments depend on political enforcements to refine the meaningfulness of water data and support water collection, reporting, and monitoring. Alternatively, in the short- and medium-run, water data challenges can be addressed by joint research efforts for water data harmonization.

Data sharing in sociology journals

2014 ◽  
Vol 38 (6) ◽  
pp. 709-722 ◽  
Author(s):  
Wolfgang Zenk-Möltgen ◽  
Greta Lepthien
Keyword(s):  
Data Sharing ◽  
Web Sites ◽  
Data Availability ◽  
Impact Factors ◽  
Data Accessibility ◽  
Content Type ◽  
Data Citation ◽  
Data Policy ◽  
Selection Of ◽  
Sociology Journals

Purpose – Data sharing is key for replication and re-use in empirical research. Scientific journals can play a central role by establishing data policies and providing technologies. The purpose of this paper is to analyses the factors which influence data sharing by investigating journal data policies and the behaviour of authors in sociology. Design/methodology/approach – The web sites of 140 sociology journals were consulted to check their data policy. The results are compared with similar studies from political science and economics. A broad selection of articles published in five selected journals over a period of two years are examined to determine whether authors really cite and share their data and the factors which are related to this. Findings – Although only a few sociology journals have explicit data policies, most journals make reference to a common policy supplied by their association of publishers. Among the journals selected, relatively few articles provide data citations and even fewer make data available – this is true both for journals with and without a data policy. But authors writing for journals with higher impact factors and with data policies are more likely to cite data and to make it really accessible. Originality/value – No study of journal data policies has been undertaken to date for the domain of sociology. A comparison of authors’ behaviours regarding data availability, data citation, and data accessibility for journals with or without a data policy provides useful information about the factors which improve data sharing.

Sign in / Sign up

Export Citation Format

Share Document