scholarly journals Expert guided natural language processing using one-class classification

2015 ◽  
Vol 22 (5) ◽  
pp. 962-966 ◽  
Author(s):  
Erel Joffe ◽  
Emily J Pettigrew ◽  
Jorge R Herskovic ◽  
Charles F Bearden ◽  
Elmer V Bernstam
Keyword(s):  
Breast Cancer ◽  
Language Processing ◽  
Text Processing ◽  
Binary Classification ◽  
Model Performance ◽  
Imbalanced Data ◽  
Superior Performance ◽  
Support Vector ◽  
Free Text ◽  
One Class Classification

Abstract Introduction Automatically identifying specific phenotypes in free-text clinical notes is critically important for the reuse of clinical data. In this study, the authors combine expert-guided feature (text) selection with one-class classification for text processing. Objectives To compare the performance of one-class classification to traditional binary classification; to evaluate the utility of feature selection based on expert-selected salient text (snippets); and to determine the robustness of these models with respects to irrelevant surrounding text. Methods The authors trained one-class support vector machines (1C-SVMs) and two-class SVMs (2C-SVMs) to identify notes discussing breast cancer. Manually annotated visit summary notes (88 positive and 88 negative for breast cancer) were used to compare the performance of models trained on whole notes labeled as positive or negative to models trained on expert-selected text sections (snippets) relevant to breast cancer status. Model performance was evaluated using a 70:30 split for 20 iterations and on a realistic dataset of 10 000 records with a breast cancer prevalence of 1.4%. Results When tested on a balanced experimental dataset, 1C-SVMs trained on snippets had comparable results to 2C-SVMs trained on whole notes (F = 0.92 for both approaches). When evaluated on a realistic imbalanced dataset, 1C-SVMs had a considerably superior performance (F = 0.61 vs. F = 0.17 for the best performing model) attributable mainly to improved precision (p = .88 vs. p = .09 for the best performing model). Conclusions 1C-SVMs trained on expert-selected relevant text sections perform better than 2C-SVMs classifiers trained on either snippets or whole notes when applied to realistically imbalanced data with low prevalence of the positive class.

scholarly journals Enhanced Changeover Detection in Industry 4.0 Environments with Machine Learning

Sensors ◽  
2021 ◽  
Vol 21 (17) ◽  
pp. 5896
Author(s):  
Eddi Miller ◽  
Vladyslav Borysenko ◽  
Moritz Heusinger ◽  
Niklas Niedner ◽  
Bastian Engelmann ◽  
...  
Keyword(s):  
Machine Learning ◽  
Random Forest ◽  
Binary Classification ◽  
Model Performance ◽  
Support Vector ◽  
Milling Machine ◽  
Vector Machines ◽  
Changeover Times ◽  
Flow Power

Changeover times are an important element when evaluating the Overall Equipment Effectiveness (OEE) of a production machine. The article presents a machine learning (ML) approach that is based on an external sensor setup to automatically detect changeovers in a shopfloor environment. The door statuses, coolant flow, power consumption, and operator indoor GPS data of a milling machine were used in the ML approach. As ML methods, Decision Trees, Support Vector Machines, (Balanced) Random Forest algorithms, and Neural Networks were chosen, and their performance was compared. The best results were achieved with the Random Forest ML model (97% F1 score, 99.72% AUC score). It was also carried out that model performance is optimal when only a binary classification of a changeover phase and a production phase is considered and less subphases of the changeover process are applied.

scholarly journals Evaluation of One-Class Classifiers for Fault Detection: Mahalanobis Classifiers and the Mahalanobis–Taguchi System

Processes ◽  
2021 ◽  
Vol 9 (8) ◽  
pp. 1450
Author(s):  
Seul-Gi Kim ◽  
Donghyun Park ◽  
Jae-Yoon Jung
Keyword(s):  
Fault Detection ◽  
Binary Classification ◽  
Rotating Machinery ◽  
Industrial Robots ◽  
Sensor Data ◽  
Support Vector ◽  
Imbalanced Classification ◽  
Vibration Data ◽  
Binary Classifiers ◽  
One Class Classification

Today, real-time fault detection and predictive maintenance based on sensor data are actively introduced in various areas such as manufacturing, aircraft, and power system monitoring. Many faults in motors or rotating machinery like industrial robots, aircraft engines, and wind turbines can be diagnosed by analyzing signal data such as vibration and noise. In this study, to detect failures based on vibration data, preprocessing was performed using signal processing techniques such as the Hamming window and the cepstrum transform. After that, 10 statistical condition indicators were extracted to train the machine learning models. Specifically, two types of Mahalanobis distance (MD)-based one-class classification methods, the MD classifier and the Mahalanobis–Taguchi system, were evaluated in detecting the faults of rotating machinery. Their performance for fault detection on rotating machinery was evaluated with different imbalanced ratios of data by comparing with binary classification models, which included classical versions and imbalanced classification versions of support vector machine and random forest algorithms. The experimental results showed the MD-based classifiers became more effective than binary classifiers in cases in which there were much fewer defect data than normal data, which is often common in the real-world industrial field.

scholarly journals Intelligent information retrieval for reducing missed cancer and improving the healthcare system

2022 ◽  
Vol 12 (1) ◽  
pp. 0-0
Keyword(s):  
Breast Cancer ◽  
Information Retrieval ◽  
Language Processing ◽  
Missing Values ◽  
Classification Model ◽  
Filter Method ◽  
Breast Cancer Dataset ◽  
Free Text ◽  
Intelligent Information Retrieval ◽  
Intelligent Information

This study presents an intelligent information retrieval system that will effectively extract useful information from breast cancer datasets and utilized that information to build a classification model. The proposed model will reduce the missed cancer rate by providing a comprehensive decision support to the radiologist. The model is built on two datasets, Wisconsin Breast Cancer Dataset (WBCD) and 365 free text mammography reports from a hospital. Effective pre-processing techniques including filling missing values with regression, an effective Natural Language Processing (NLP) Parser is developed to handle free text mammography reports, balancing the dataset with Synthetic Minority Oversampling (SMOTE) was applied to prepare the dataset for learning. Most relevant features were selected with the help of filter method and tf-idf scores. K-NN and SGD classifiers are optimized with optimum value of k for K-NN and hyper tuning the SGD parameters with grid search technique.

scholarly journals BurnoutWords - Detecting Burnout for a Clinical Setting

2021 ◽  
Author(s):  
Sukanya Nath ◽  
Mascha Kurpicz-Briki
Keyword(s):  
Mental Health ◽  
Language Processing ◽  
Workplace Stress ◽  
Support Vector ◽  
Svm Classifier ◽  
Free Text ◽  
Clinical Use ◽  
Real Patient ◽  
Global Pandemic ◽  
Clinical Methods

Burnout, a syndrome conceptualized as resulting from major workplace stress that has not been successfully managed, is a major problem of today's society, in particular in crisis times such as a global pandemic situation. Burnout detection is hard, because the symptoms often overlap with other diseases and syndromes. Typical clinical approaches are using inventories to assess burnout for their patients, even though free-text approaches are considered promising. In research of natural language processing (NLP) applied to mental health, often data from social media is used and not real patient data, which leads to some limitations for the application in clinical use cases. In this paper, we fill the gap and provide a dataset using extracts from interviews with burnout patients containing 216 records. We train a support vector machine (SVM) classifier to detect burnout in text snippets with an accuracy of around 80%, which is clearly higher than the random baseline of our setup. This provides the foundation for a next generation of clinical methods based on NLP.

scholarly journals Automatic detection of actionable radiology reports using bidirectional encoder representations from transformers

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Yuta Nakamura ◽  
Shouhei Hanaoka ◽  
Yukihiro Nomura ◽  
Takahiro Nakao ◽  
Soichiro Miki ◽  
...  
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Short Term Memory ◽  
Binary Classification ◽  
Characteristic Curve ◽  
Gradient Boosting ◽  
Free Text ◽  
Order Information ◽  
Radiology Reports ◽  
Actionable Findings

Abstract Background It is essential for radiologists to communicate actionable findings to the referring clinicians reliably. Natural language processing (NLP) has been shown to help identify free-text radiology reports including actionable findings. However, the application of recent deep learning techniques to radiology reports, which can improve the detection performance, has not been thoroughly examined. Moreover, free-text that clinicians input in the ordering form (order information) has seldom been used to identify actionable reports. This study aims to evaluate the benefits of two new approaches: (1) bidirectional encoder representations from transformers (BERT), a recent deep learning architecture in NLP, and (2) using order information in addition to radiology reports. Methods We performed a binary classification to distinguish actionable reports (i.e., radiology reports tagged as actionable in actual radiological practice) from non-actionable ones (those without an actionable tag). 90,923 Japanese radiology reports in our hospital were used, of which 788 (0.87%) were actionable. We evaluated four methods, statistical machine learning with logistic regression (LR) and with gradient boosting decision tree (GBDT), and deep learning with a bidirectional long short-term memory (LSTM) model and a publicly available Japanese BERT model. Each method was used with two different inputs, radiology reports alone and pairs of order information and radiology reports. Thus, eight experiments were conducted to examine the performance. Results Without order information, BERT achieved the highest area under the precision-recall curve (AUPRC) of 0.5138, which showed a statistically significant improvement over LR, GBDT, and LSTM, and the highest area under the receiver operating characteristic curve (AUROC) of 0.9516. Simply coupling the order information with the radiology reports slightly increased the AUPRC of BERT but did not lead to a statistically significant improvement. This may be due to the complexity of clinical decisions made by radiologists. Conclusions BERT was assumed to be useful to detect actionable reports. More sophisticated methods are required to use order information effectively.

Using Machine Learning to Capture Quality Metrics from Natural Language: A Case Study of Diabetic Eye Exams

2021 ◽  
Author(s):  
Allan Fong ◽  
Nicholas Scoulios ◽  
H. Joseph Blumenthal ◽  
Ryan E. Anderson
Keyword(s):  
Logistic Regression ◽  
Natural Language ◽  
Language Processing ◽  
Synthesis Process ◽  
Support Vector ◽  
Free Text ◽  
Svm Model ◽  
Eye Exams ◽  
The Impact

Abstract Background and Objective The prevalence of value-based payment models has led to an increased use of the electronic health record to capture quality measures, necessitating additional documentation requirements for providers. Methods This case study uses text mining and natural language processing techniques to identify the timely completion of diabetic eye exams (DEEs) from 26,203 unique clinician notes for reporting as an electronic clinical quality measure (eCQM). Logistic regression and support vector machine (SVM) using unbalanced and balanced datasets, using the synthetic minority over-sampling technique (SMOTE) algorithm, were evaluated on precision, recall, sensitivity, and f1-score for classifying records positive for DEE. We then integrate a high precision DEE model to evaluate free-text clinical narratives from our clinical EHR system. Results Logistic regression and SVM models had comparable f1-score and specificity metrics with models trained and validated with no oversampling favoring precision over recall. SVM with and without oversampling resulted in the best precision, 0.96, and recall, 0.85, respectively. These two SVM models were applied to the unannotated 31,585 text segments representing 24,823 unique records and 13,714 unique patients. The number of records classified as positive for DEE using the SVM models ranged from 667 to 8,935 (2.7–36% out of 24,823, respectively). Unique patients classified as positive for DEE ranged from 3.5 to 41.8% highlighting the potential utility of these models. Discussion We believe the impact of oversampling on SVM model performance to be caused by the potential of overfitting of the SVM SMOTE model on the synthesized data and the data synthesis process. However, the specificities of SVM with and without SMOTE were comparable, suggesting both models were confident in their negative predictions. By prioritizing to implement the SVM model with higher precision over sensitivity or recall in the categorization of DEEs, we can provide a highly reliable pool of results that can be documented through automation, reducing the burden of secondary review. Although the focus of this work was on completed DEEs, this method could be applied to completing other necessary documentation by extracting information from natural language in clinician notes. Conclusion By enabling the capture of data for eCQMs from documentation generated by usual clinical practice, this work represents a case study in how such techniques can be leveraged to drive quality without increasing clinician work.

scholarly journals Natural Language Processing Based Instrument for Classification of Free Text Medical Records

2016 ◽  
Vol 2016 ◽  
pp. 1-10 ◽  
Author(s):  
Manana Khachidze ◽  
Magda Tsintsadze ◽  
Maia Archuadze
Keyword(s):  
Language Processing ◽  
Medical Records ◽  
Nearest Neighbor ◽  
Health Management ◽  
Support Vector ◽  
Free Text ◽  
Features Selection ◽  
Second Stage ◽  
History Of

According to the Ministry of Labor, Health and Social Affairs of Georgia a new health management system has to be introduced in the nearest future. In this context arises the problem of structuring and classifying documents containing all the history of medical services provided. The present work introduces the instrument for classification of medical records based on the Georgian language. It is the first attempt of such classification of the Georgian language based medical records. On the whole 24.855 examination records have been studied. The documents were classified into three main groups (ultrasonography, endoscopy, and X-ray) and 13 subgroups using two well-known methods: Support Vector Machine (SVM) andK-Nearest Neighbor (KNN). The results obtained demonstrated that both machine learning methods performed successfully, with a little supremacy of SVM. In the process of classification a “shrink” method, based on features selection, was introduced and applied. At the first stage of classification the results of the “shrink” case were better; however, on the second stage of classification into subclasses 23% of all documents could not be linked to only one definite individual subclass (liver or binary system) due to common features characterizing these subclasses. The overall results of the study were successful.

scholarly journals A Framework for the Selection of Binarization Techniques on Palm Leaf Manuscripts Using Support Vector Machine

2015 ◽  
Vol 2015 ◽  
pp. 1-7 ◽  
Author(s):  
Rapeeporn Chamchong ◽  
Chun Che Fung
Keyword(s):  
Selection Process ◽  
Text Processing ◽  
Imbalanced Data ◽  
Support Vector ◽  
Local Contrast ◽  
Text Documents ◽  
Image Characteristics ◽  
Vector Machines ◽  
High Degree ◽  
Palm Leaf

Challenges for text processing in ancient document images are mainly due to the high degree of variations in foreground and background. Image binarization is an image segmentation technique used to separate the image into text and background components. Although several techniques for binarizing text documents have been proposed, the performance of these techniques varies and depends on the image characteristics. Therefore, selecting binarization techniques can be a key idea to achieve improved results. This paper proposes a framework for selecting binarizing techniques of palm leaf manuscripts using Support Vector Machines (SVMs). The overall process is divided into three steps: (i) feature extraction: feature patterns are extracted from grayscale images based on global intensity, local contrast, and intensity; (ii) treatment of imbalanced data: imbalanced dataset is balanced by using Synthetic Minority Oversampling Technique as to improve the performance of prediction; and (iii) selection: SVM is applied in order to select the appropriate binarization techniques. The proposed framework has been evaluated with palm leaf manuscript images and benchmarking dataset from DIBCO series and compared the performance of prediction between imbalanced and balanced datasets. Experimental results showed that the proposed framework can be used as an integral part of an automatic selection process.

scholarly journals Evaluation of multi-target deep neural network models for compound potency prediction under increasingly challenging test conditions

2021 ◽  
Author(s):  
Raquel Rodríguez-Pérez ◽  
Jürgen Bajorath
Keyword(s):  
Large Scale ◽  
Model Performance ◽  
Network Models ◽  
Data Partitioning ◽  
Superior Performance ◽  
Support Vector ◽  
Neural Network Models ◽  
Multiple Targets ◽  
Single Target ◽  
Qsar Models

AbstractMachine learning (ML) enables modeling of quantitative structure–activity relationships (QSAR) and compound potency predictions. Recently, multi-target QSAR models have been gaining increasing attention. Simultaneous compound potency predictions for multiple targets can be carried out using ensembles of independently derived target-based QSAR models or in a more integrated and advanced manner using multi-target deep neural networks (MT-DNNs). Herein, single-target and multi-target ML models were systematically compared on a large scale in compound potency value predictions for 270 human targets. By design, this large-magnitude evaluation has been a special feature of our study. To these ends, MT-DNN, single-target DNN (ST-DNN), support vector regression (SVR), and random forest regression (RFR) models were implemented. Different test systems were defined to benchmark these ML methods under conditions of varying complexity. Source compounds were divided into training and test sets in a compound- or analog series-based manner taking target information into account. Data partitioning approaches used for model training and evaluation were shown to influence the relative performance of ML methods, especially for the most challenging compound data sets. For example, the performance of MT-DNNs with per-target models yielded superior performance compared to single-target models. For a test compound or its analogs, the availability of potency measurements for multiple targets affected model performance, revealing the influence of ML synergies.

scholarly journals Natural language processing for cognitive therapy: Extracting schemas from thought records

PLoS ONE ◽  
2021 ◽  
Vol 16 (10) ◽  
pp. e0257832
Author(s):  
Franziska Burger ◽  
Mark A. Neerincx ◽  
Willem-Paul Brinkman
Keyword(s):  
Mental Health ◽  
Neural Networks ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Recurrent Neural Networks ◽  
Support Vector ◽  
Free Text ◽  
Cognitive Approach ◽  
Text Input

The cognitive approach to psychotherapy aims to change patients’ maladaptive schemas, that is, overly negative views on themselves, the world, or the future. To obtain awareness of these views, they record their thought processes in situations that caused pathogenic emotional responses. The schemas underlying such thought records have, thus far, been largely manually identified. Using recent advances in natural language processing, we take this one step further by automatically extracting schemas from thought records. To this end, we asked 320 healthy participants on Amazon Mechanical Turk to each complete five thought records consisting of several utterances reflecting cognitive processes. Agreement between two raters on manually scoring the utterances with respect to how much they reflect each schema was substantial (Cohen’s κ = 0.79). Natural language processing software pretrained on all English Wikipedia articles from 2014 (GLoVE embeddings) was used to represent words and utterances, which were then mapped to schemas using k-nearest neighbors algorithms, support vector machines, and recurrent neural networks. For the more frequently occurring schemas, all algorithms were able to leverage linguistic patterns. For example, the scores assigned to the Competence schema by the algorithms correlated with the manually assigned scores with Spearman correlations ranging between 0.64 and 0.76. For six of the nine schemas, a set of recurrent neural networks trained separately for each of the schemas outperformed the other algorithms. We present our results here as a benchmark solution, since we conducted this research to explore the possibility of automatically processing qualitative mental health data and did not aim to achieve optimal performance with any of the explored models. The dataset of 1600 thought records comprising 5747 utterances is published together with this article for researchers and machine learning enthusiasts to improve upon our outcomes. Based on our promising results, we see further opportunities for using free-text input and subsequent natural language processing in other common therapeutic tools, such as ecological momentary assessments, automated case conceptualizations, and, more generally, as an alternative to mental health scales.

Sign in / Sign up

Export Citation Format

Share Document