scholarly journals Potential limitations in COVID-19 machine learning due to data source variability: A case study in the nCov2019 dataset

2020 ◽  
Author(s):  
Carlos Sáez ◽  
Nekane Romero ◽  
J Alberto Conejero ◽  
Juan M García-Gómez
Keyword(s):  
Machine Learning ◽  
Data Quality ◽  
Data Sharing ◽  
Training Data ◽  
Model Complexity ◽  
Systematic Assessment ◽  
Level Data ◽  
Potential Contributor ◽  
Data Source

Abstract Objective The lack of representative coronavirus disease 2019 (COVID-19) data is a bottleneck for reliable and generalizable machine learning. Data sharing is insufficient without data quality, in which source variability plays an important role. We showcase and discuss potential biases from data source variability for COVID-19 machine learning. Materials and Methods We used the publicly available nCov2019 dataset, including patient-level data from several countries. We aimed to the discovery and classification of severity subgroups using symptoms and comorbidities. Results Cases from the 2 countries with the highest prevalence were divided into separate subgroups with distinct severity manifestations. This variability can reduce the representativeness of training data with respect the model target populations and increase model complexity at risk of overfitting. Conclusions Data source variability is a potential contributor to bias in distributed research networks. We call for systematic assessment and reporting of data source variability and data quality in COVID-19 data sharing, as key information for reliable and generalizable machine learning.

scholarly journals Data Quality Measures and Efficient Evaluation Algorithms for Large-Scale High-Dimensional Data

2021 ◽  
Vol 11 (2) ◽  
pp. 472
Author(s):  
Hyeongmin Cho ◽  
Sangkyun Lee
Keyword(s):  
Machine Learning ◽  
Data Quality ◽  
Large Scale ◽  
High Dimensional Data ◽  
Quality Measures ◽  
Training Data ◽  
Measure Data ◽  
High Dimensional ◽  
Small Scale ◽  
Class Separability

Machine learning has been proven to be effective in various application areas, such as object and speech recognition on mobile systems. Since a critical key to machine learning success is the availability of large training data, many datasets are being disclosed and published online. From a data consumer or manager point of view, measuring data quality is an important first step in the learning process. We need to determine which datasets to use, update, and maintain. However, not many practical ways to measure data quality are available today, especially when it comes to large-scale high-dimensional data, such as images and videos. This paper proposes two data quality measures that can compute class separability and in-class variability, the two important aspects of data quality, for a given dataset. Classical data quality measures tend to focus only on class separability; however, we suggest that in-class variability is another important data quality factor. We provide efficient algorithms to compute our quality measures based on random projections and bootstrapping with statistical benefits on large-scale high-dimensional data. In experiments, we show that our measures are compatible with classical measures on small-scale data and can be computed much more efficiently on large-scale high-dimensional datasets.

scholarly journals Getting the Numbers Right on China's Actual Overseas Investment: The Case of the Netherlands

2017 ◽  
Vol 46 (1) ◽  
pp. 187-209 ◽  
Author(s):  
Piter De Jong ◽  
Mark J. Greeven ◽  
Haico Ebbers
Keyword(s):  
The Netherlands ◽  
Data Quality ◽  
Chinese Firms ◽  
Chamber Of Commerce ◽  
Overseas Investment ◽  
Data Source

This study assesses the quality of Chinese outbound FDI data. In our case study of the Netherlands, we checked the data quality of the often-used Orbis/Amadeus database and its data source, the Dutch Chamber of Commerce (Kamer van Koophandel, KVK), which has one of the oldest and, arguably, one of the better databases within Europe. We analysed Chinese investments in the Netherlands and show that six adjustments are necessary to clean up the data. We also show that not making these adjustments can significantly impact the outcome of research. The cleaned-up data show that sampled Chinese firms are young, small, and private.

Detecting Pressure Anomalies While Drilling Using a Machine Learning Hybrid Approach

2021 ◽  
Author(s):  
Aurore Lafond ◽  
Maurice Ringer ◽  
Florian Le Blay ◽  
Jiaxu Liu ◽  
Ekaterina Millan ◽  
...  
Keyword(s):  
Machine Learning ◽  
Data Quality ◽  
Real Time ◽  
Large Scale ◽  
Hybrid Approach ◽  
Physical Models ◽  
Training Data ◽  
Digital Data ◽  
Machine Learning Techniques ◽  
New System

Abstract Abnormal surface pressure is typically the first indicator of a number of problematic events, including kicks, losses, washouts and stuck pipe. These events account for 60–70% of all drilling-related nonproductive time, so their early and accurate detection has the potential to save the industry billions of dollars. Detecting these events today requires an expert user watching multiple curves, which can be costly, and subject to human errors. The solution presented in this paper is aiming at augmenting traditional models with new machine learning techniques, which enable to detect these events automatically and help the monitoring of the drilling well. Today’s real-time monitoring systems employ complex physical models to estimate surface standpipe pressure while drilling. These require many inputs and are difficult to calibrate. Machine learning is an alternative method to predict pump pressure, but this alone needs significant labelled training data, which is often lacking in the drilling world. The new system combines these approaches: a machine learning framework is used to enable automated learning while the physical models work to compensate any gaps in the training data. The system uses only standard surface measurements, is fully automated, and is continuously retrained while drilling to ensure the most accurate pressure prediction. In addition, a stochastic (Bayesian) machine learning technique is used, which enables not only a prediction of the pressure, but also the uncertainty and confidence of this prediction. Last, the new system includes a data quality control workflow. It discards periods of low data quality for the pressure anomaly detection and enables to have a smarter real-time events analysis. The new system has been tested on historical wells using a new test and validation framework. The framework runs the system automatically on large volumes of both historical and simulated data, to enable cross-referencing the results with observations. In this paper, we show the results of the automated test framework as well as the capabilities of the new system in two specific case studies, one on land and another offshore. Moreover, large scale statistics enlighten the reliability and the efficiency of this new detection workflow. The new system builds on the trend in our industry to better capture and utilize digital data for optimizing drilling.

scholarly journals Like trainer, like bot? Inheritance of bias in algorithmic content moderation

2017 ◽  
Author(s):  
Reuben Binns ◽  
Michael Veale ◽  
Max Van Kleek ◽  
Nigel Shadbolt
Keyword(s):  
Machine Learning ◽  
Training Data ◽  
The Internet ◽  
Men And Women ◽  
Machine Learning Classifiers ◽  
Ethical Choices ◽  
Test Sets ◽  
Networked Publics ◽  
Important Objective

The internet has become a central medium through which 'networked publics' express their opinions and engage in debate. Offensive comments and personal attacks can inhibit participation in these spaces. Automated content moderation aims to overcome this problem using machine learning classifiers trained on large corpora of texts manually annotated for offence. While such systems could help encourage more civil debate, they must navigate inherently normatively contestable boundaries, and are subject to the idiosyncratic norms of the human raters who provide the training data. An important objective for platforms implementing such measures might be to ensure that they are not unduly biased towards or against particular norms of offence. This paper provides some exploratory methods by which the normative biases of algorithmic content moderation systems can be measured, by way of a case study using an existing dataset of comments labelled for offence. We train classifiers on comments labelled by different demographic subsets (men and women) to understand how differences in conceptions of offence between these groups might affect the performance of the resulting models on various test sets. We conclude by discussing some of the ethical choices facing the implementers of algorithmic moderation systems, given various desired levels of diversity of viewpoints amongst discussion participants.

scholarly journals Using Machine Learning Methods to Develop a Short Tree-Based Adaptive Classification Test: Case Study With a High-Dimensional Item Pool and Imbalanced Data

2020 ◽  
Vol 44 (7-8) ◽  
pp. 499-514
Author(s):  
Yi Zheng ◽  
Hyunjung Cheon ◽  
Charles M. Katz
Keyword(s):  
Machine Learning ◽  
Classification Accuracy ◽  
Imbalanced Data ◽  
Training Data ◽  
Adaptive Tests ◽  
Promising Alternative ◽  
Adaptive Classification ◽  
Short Tree ◽  
Classification Test

This study explores advanced techniques in machine learning to develop a short tree-based adaptive classification test based on an existing lengthy instrument. A case study was carried out for an assessment of risk for juvenile delinquency. Two unique facts of this case are (a) the items in the original instrument measure a large number of distinctive constructs; (b) the target outcomes are of low prevalence, which renders imbalanced training data. Due to the high dimensionality of the items, traditional item response theory (IRT)-based adaptive testing approaches may not work well, whereas decision trees, which are developed in the machine learning discipline, present as a promising alternative solution for adaptive tests. A cross-validation study was carried out to compare eight tree-based adaptive test constructions with five benchmark methods using data from a sample of 3,975 subjects. The findings reveal that the best-performing tree-based adaptive tests yielded better classification accuracy than the benchmark method IRT scoring with optimal cutpoints, and yielded comparable or better classification accuracy than the best benchmark method, random forest with balanced sampling. The competitive classification accuracy of the tree-based adaptive tests also come with an over 30-fold reduction in the length of the instrument, only administering between 3 to 6 items to any individual. This study suggests that tree-based adaptive tests have an enormous potential when used to shorten instruments that measure a large variety of constructs.

scholarly journals The Future of PHM Could be Tiny under Cloud: Exploring Potential Application Patterns of TinyML in PHM Scenarios

2021 ◽  
Vol 13 (1) ◽  
Author(s):  
Xingyu Zhou ◽  
Zhuangwei Kang ◽  
Robert Canady ◽  
Shunxing Bao ◽  
Daniel Allen Balasubramanian ◽  
...  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Remaining Useful Life ◽  
Data Driven ◽  
Learning Models ◽  
Level Data ◽  
Data Source ◽  
Single Data ◽  
The Impact ◽  
Machine Learning Models

Deep learning has shown impressive performance acrosshealth management and prognostics applications. Nowadays, an emerging trend of machine learning deployment on resource constraint hardware devices like micro-controllers(MCU) has aroused much attention. Given the distributed andresource constraint nature of many PHM applications, using tiny machine learning models close to data source sensors for on-device inferences would be beneficial to save both time andadditional hardware resources. Even though there has beenpast works that bring TinyML on MCUs for some PHM ap-plications, they are mainly targeting single data source usage without higher-level data incorporation with cloud computing.We study the impact of potential cooperation patterns betweenTinyML on edge and more powerful computation resources oncloud and how this would make an impact on the application patterns in data-driven prognostics. We introduce potential ap-plications where sensor readings are utilized for system health status prediction including status classification and remaining useful life regression. We find that MCUs and cloud com-puting can be adaptive to different kinds of machine learning models and combined in flexible ways for diverse requirement.Our work also shows limitations of current MCU-based deep learning in data-driven prognostics And we hope our work can

scholarly journals Linking Human And Machine Behavior: A New Approach to Evaluate Training Data Quality for Beneficial Machine Learning

2021 ◽  
Author(s):  
Thilo Hagendorff
Keyword(s):  
Machine Learning ◽  
Data Quality ◽  
Training Data ◽  
Supervised Machine Learning ◽  
Ethical Perspective ◽  
New Approach ◽  
Specific Objective ◽  
Machine Learning Applications ◽  
Socially Relevant

AbstractMachine behavior that is based on learning algorithms can be significantly influenced by the exposure to data of different qualities. Up to now, those qualities are solely measured in technical terms, but not in ethical ones, despite the significant role of training and annotation data in supervised machine learning. This is the first study to fill this gap by describing new dimensions of data quality for supervised machine learning applications. Based on the rationale that different social and psychological backgrounds of individuals correlate in practice with different modes of human–computer-interaction, the paper describes from an ethical perspective how varying qualities of behavioral data that individuals leave behind while using digital technologies have socially relevant ramification for the development of machine learning applications. The specific objective of this study is to describe how training data can be selected according to ethical assessments of the behavior it originates from, establishing an innovative filter regime to transition from the big data rationale n = all to a more selective way of processing data for training sets in machine learning. The overarching aim of this research is to promote methods for achieving beneficial machine learning applications that could be widely useful for industry as well as academia.

Sign in / Sign up

Export Citation Format

Share Document