scholarly journals Clinical Validation of a Deep Learning Algorithm for Detection of Pneumonia on Chest Radiographs in Emergency Department Patients with Acute Febrile Respiratory Illness

2020 ◽  
Vol 9 (6) ◽  
pp. 1981
Author(s):  
Jae Hyun Kim ◽  
Jin Young Kim ◽  
Gun Ha Kim ◽  
Donghoon Kang ◽  
In Jung Kim ◽  
...  
Keyword(s):  
Emergency Department ◽  
Deep Learning ◽  
Performance Test ◽  
Respiratory Illness ◽  
Chest Radiographs ◽  
Observer Performance ◽  
Operating Characteristics ◽  
Predictive Values ◽  
Febrile Respiratory Illness ◽  
Sensitivity Specificity

Early identification of pneumonia is essential in patients with acute febrile respiratory illness (FRI). We evaluated the performance and added value of a commercial deep learning (DL) algorithm in detecting pneumonia on chest radiographs (CRs) of patients visiting the emergency department (ED) with acute FRI. This single-centre, retrospective study included 377 consecutive patients who visited the ED and the resulting 387 CRs in August 2018–January 2019. The performance of a DL algorithm in detection of pneumonia on CRs was evaluated based on area under the receiver operating characteristics (AUROC) curves, sensitivity, specificity, negative predictive values (NPVs), and positive predictive values (PPVs). Three ED physicians independently reviewed CRs with observer performance test to detect pneumonia, which was re-evaluated with the algorithm eight weeks later. AUROC, sensitivity, and specificity measurements were compared between “DL algorithm” vs. “physicians-only” and between “physicians-only” vs. “physicians aided with the algorithm”. Among 377 patients, 83 (22.0%) had pneumonia. AUROC, sensitivity, specificity, PPV, and NPV of the algorithm for detection of pneumonia on CRs were 0.861, 58.3%, 94.4%, 74.2%, and 89.1%, respectively. For the detection of ‘visible pneumonia on CR’ (60 CRs from 59 patients), AUROC, sensitivity, specificity, PPV, and NPV were 0.940, 81.7%, 94.4%, 74.2%, and 96.3%, respectively. In the observer performance test, the algorithm performed better than the physicians for pneumonia (AUROC, 0.861 vs. 0.788, p = 0.017; specificity, 94.4% vs. 88.7%, p < 0.0001) and visible pneumonia (AUROC, 0.940 vs. 0.871, p = 0.007; sensitivity, 81.7% vs. 73.9%, p = 0.034; specificity, 94.4% vs. 88.7%, p < 0.0001). Detection of pneumonia (sensitivity, 82.2% vs. 53.2%, p = 0.008; specificity, 98.1% vs. 88.7%; p < 0.0001) and ‘visible pneumonia’ (sensitivity, 82.2% vs. 73.9%, p = 0.014; specificity, 98.1% vs. 88.7%, p < 0.0001) significantly improved when the algorithm was used by the physicians. Mean reading time for the physicians decreased from 165 to 101 min with the assistance of the algorithm. Thus, the DL algorithm showed a better diagnosis of pneumonia, particularly visible pneumonia on CR, and improved diagnosis by ED physicians in patients with acute FRI.

Deep Learning for Computer-aided Diagnosis of Pneumoconiosis

2021 ◽  
Author(s):  
Zheng Wang ◽  
Qingjun Qian ◽  
Jianfang Zhang ◽  
Caihong Duo ◽  
Wen He ◽  
...  
Keyword(s):  
Deep Learning ◽  
Computer Aided Diagnosis ◽  
Chest Radiographs ◽  
Diagnostic Sensitivity ◽  
Junior Doctors ◽  
Diagnostic Efficiency ◽  
Test Set ◽  
Computer Aided ◽  
Sensitivity Specificity ◽  
Aided Diagnosis

Abstract Background: The diagnosis of pneumoconiosis relies primarily on chest radiographs and exhibits significant variability between physicians. Computer-aided diagnosis (CAD) can improve the accuracy and consistency of these diagnoses. However, CAD based on machine learning requires extensive human intervention and time-consuming training. As such, deep learning has become a popular tool for the development of CAD models. In this study, the clinical applicability of CAD based on deep learning was verified for pneumoconiosis patients.Methods: Chest radiographs were collected from 5424 occupational health examiners who met the inclusion criteria. The data were divided into training, validation, and test sets. The CAD algorithm was then trained and applied to processing of the validation set, while the test set was used to evaluate diagnostic efficacy. Three junior and three senior physicians provided independent diagnoses using images from the test set and a comprehensive diagnosis for comparison with the CAD results. A receiver operating characteristic (ROC) curve was used to evaluate the diagnostic efficiency of the proposed CAD system. A McNemar test was used to evaluate diagnostic sensitivity and specificity for pneumoconiosis, both before and after the use of CAD. A kappa consistency test was used to evaluate the diagnostic consistency for both the algorithm and the clinicians.Results: ROC results suggested the proposed CAD model achieved high accuracy in the diagnosis of pneumoconiosis, with a kappa value of 0.90. The sensitivity, specificity, and kappa values for the junior doctors increased from 0.86 to 0.98, 0.68 to 0.86, and 0.54 to 0.84, respectively (p<0.05), when CAD was applied. However, metrics for the senior doctors were not significantly different.Conclusion: DL-based CAD can improve the diagnostic sensitivity, specificity, and consistency of pneumoconiosis diagnoses, particularly for junior physicians.

scholarly journals Development and Validation of a Deep Learning–based Automatic Detection Algorithm for Active Pulmonary Tuberculosis on Chest Radiographs

2018 ◽  
Vol 69 (5) ◽  
pp. 739-747 ◽  
Author(s):  
Eui Jin Hwang ◽  
Sunggyun Park ◽  
Kwang-Nam Jin ◽  
Jung Im Kim ◽  
So Young Choi ◽  
...  
Keyword(s):  
Deep Learning ◽  
Pulmonary Tuberculosis ◽  
Performance Test ◽  
Automatic Detection ◽  
High Sensitivity ◽  
High Specificity ◽  
Roc Curves ◽  
Chest Radiographs ◽  
Automated System ◽  
Active Pulmonary Tuberculosis

Abstract Background Detection of active pulmonary tuberculosis on chest radiographs (CRs) is critical for the diagnosis and screening of tuberculosis. An automated system may help streamline the tuberculosis screening process and improve diagnostic performance. Methods We developed a deep learning–based automatic detection (DLAD) algorithm using 54c221 normal CRs and 6768 CRs with active pulmonary tuberculosis that were labeled and annotated by 13 board-certified radiologists. The performance of DLAD was validated using 6 external multicenter, multinational datasets. To compare the performances of DLAD with physicians, an observer performance test was conducted by 15 physicians including nonradiology physicians, board-certified radiologists, and thoracic radiologists. Image-wise classification and lesion-wise localization performances were measured using area under the receiver operating characteristic (ROC) curves and area under the alternative free-response ROC curves, respectively. Sensitivities and specificities of DLAD were calculated using 2 cutoffs (high sensitivity [98%] and high specificity [98%]) obtained through in-house validation. Results DLAD demonstrated classification performance of 0.977–1.000 and localization performance of 0.973–1.000. Sensitivities and specificities for classification were 94.3%–100% and 91.1%–100% using the high-sensitivity cutoff and 84.1%–99.0% and 99.1%–100% using the high-specificity cutoff. DLAD showed significantly higher performance in both classification (0.993 vs 0.746–0.971) and localization (0.993 vs 0.664–0.925) compared to all groups of physicians. Conclusions Our DLAD demonstrated excellent and consistent performance in the detection of active pulmonary tuberculosis on CR, outperforming physicians, including thoracic radiologists.

Deep-learning for predicting C-shaped canals in mandibular second molars on panoramic radiographs

2021 ◽  
pp. 20200513
Author(s):  
Su-Jin Jeon ◽  
Jong-Pil Yun ◽  
Han-Gyeol Yeom ◽  
Woo-Sang Shin ◽  
Jong-Hyun Lee ◽  
...  
Keyword(s):  
Deep Learning ◽  
Cone Beam Ct ◽  
Area Under The Curve ◽  
Roc Curves ◽  
Learning System ◽  
Operating Characteristics ◽  
Panoramic Radiographs ◽  
Test Sets ◽  
Sensitivity Specificity ◽  
Deep Learning Model

Objective: The aim of this study was to evaluate the use of a convolutional neural network (CNN) system for predicting C-shaped canals in mandibular second molars on panoramic radiographs. Methods: Panoramic and cone beam CT (CBCT) images obtained from June 2018 to May 2020 were screened and 1020 patients were selected. Our dataset of 2040 sound mandibular second molars comprised 887 C-shaped canals and 1153 non-C-shaped canals. To confirm the presence of a C-shaped canal, CBCT images were analyzed by a radiologist and set as the gold standard. A CNN-based deep-learning model for predicting C-shaped canals was built using Xception. The training and test sets were set to 80 to 20%, respectively. Diagnostic performance was evaluated using accuracy, sensitivity, specificity, and precision. Receiver-operating characteristics (ROC) curves were drawn, and the area under the curve (AUC) values were calculated. Further, gradient-weighted class activation maps (Grad-CAM) were generated to localize the anatomy that contributed to the predictions. Results: The accuracy, sensitivity, specificity, and precision of the CNN model were 95.1, 92.7, 97.0, and 95.9%, respectively. Grad-CAM analysis showed that the CNN model mainly identified root canal shapes converging into the apex to predict the C-shaped canals, while the root furcation was predominantly used for predicting the non-C-shaped canals. Conclusions: The deep-learning system had significant accuracy in predicting C-shaped canals of mandibular second molars on panoramic radiographs.

Emergency Department Management of Febrile Respiratory Illness in Children

2016 ◽  
Vol 32 (7) ◽  
pp. 429-434 ◽  
Author(s):  
Sonal Shah ◽  
Florence Bourgeois ◽  
Rebekah Mannix ◽  
Kyle Nelson ◽  
Richard Bachur ◽  
...  
Keyword(s):  
Emergency Department ◽  
Respiratory Illness ◽  
Emergency Department Management ◽  
Febrile Respiratory Illness ◽  
Department Management

scholarly journals Nowcast Deep Learning Models For Constraining Zero-Day Pathogen Attacks – Application on Chest Radiographs to Covid-19

2020 ◽  
Author(s):  
Michael D. Kuo ◽  
Wan Hang Keith Chiu ◽  
Varut Vardhanabhuti ◽  
Dymtro Poplavskiy ◽  
Philip LH Yu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Sensitivity And Specificity ◽  
Detection System ◽  
Chest Radiographs ◽  
Sufficient Information ◽  
Operating Characteristics ◽  
Rt Pcr ◽  
Chain Reaction ◽  
Prospective Cohorts ◽  
Polymerase Chain

Abstract Outbreaks due to emergent pathogens like Covid-19 are difficult to contain as the time to gather sufficient information to develop a detection system is outpaced by the speed of transmission. Here we develop a general pneumonia (PNA) CXR Deep Learning (DL) model (MAIL1.0) follow by a second-generation DL model (MAIL2.0) for detection of Covid-19 on chest radiographs (CXR). We validate the models on two prospective cohorts of high-risks patients screened for Covid-19 reverse transcriptase-polymerase chain reaction (RT-PCR). MAIL1.0 has an Area Under the Receiver Operating Characteristics (AUC) of 0.93, sensitivity and specificity of 90.5% and 76.7% in detection of visible pneumonia and MAIL2.0 has an AUC of 0.81, sensitivity and specificity of 84.7% and 71.6%, significantly outperforming radiologists, especially amongst asymptomatic and patients presenting with early symptoms. Nowcast DL models may be an effective tool in helping to constrain the outbreak, particularly in resource-stretched healthcare systems.

scholarly journals Development and validation of a deep learning algorithm detecting 10 common abnormalities on chest radiographs

2020 ◽  
pp. 2003061
Author(s):  
Ju Gang Nam ◽  
Minchul Kim ◽  
Jongchan Park ◽  
Eui Jin Hwang ◽  
Jong Hyuk Lee ◽  
...  
Keyword(s):  
Emergency Department ◽  
Deep Learning ◽  
Learning Algorithm ◽  
External Validation ◽  
Chest Radiographs ◽  
Reading Tests ◽  
Deep Learning Algorithm ◽  
The Mean ◽  
Emergency Department Patients ◽  
Development And Validation

We aimed to develop a deep-learning algorithm detecting 10 common abnormalities (DLAD-10) on chest radiographs and to evaluate its impact in diagnostic accuracy, timeliness of reporting, and workflow efficacy.DLAD-10 was trained with 146 717 radiographs from 108 053 patients using a ResNet34-based neural network with lesion-specific channels for 10 common radiologic abnormalities (pneumothorax, mediastinal widening, pneumoperitoneum, nodule/mass, consolidation, pleural effusion, linear atelectasis, fibrosis, calcification, and cardiomegaly). For external validation, the performance of DLAD-10 on a same-day CT-confirmed dataset (normal:abnormal, 53:147) and an open-source dataset (PadChest; normal:abnormal, 339:334) was compared to that of three radiologists. Separate simulated reading tests were conducted on another dataset adjusted to real-world disease prevalence in the emergency department, consisting of four critical, 52 urgent, and 146 non-urgent cases. Six radiologists participated in the simulated reading sessions with and without DLAD-10.DLAD-10 exhibited areas under the receiver-operating characteristic curves (AUROCs) of 0.895–1.00 in the CT-confirmed dataset and 0.913–0.997 in the PadChest dataset. DLAD-10 correctly classified significantly more critical abnormalities (95.0% [57/60]) than pooled radiologists (84.4% [152/180]; p=0.01). In simulated reading tests for emergency department patients, pooled readers detected significantly more critical (70.8% [17/24] versus 29.2% [7/24]; p=0.006) and urgent (82.7% [258/312] versus 78.2% [244/312]; p=0.04) abnormalities when aided by DLAD-10. DLAD-10 assistance shortened the mean time-to-report critical and urgent radiographs (640.5±466.3 versus 3371.0±1352.5 s and 1840.3±1141.1 versus 2127.1±1468.2, respectively; p-values<0.01) and reduced the mean interpretation time (20.5±22.8 versus 23.5±23.7 s; p<0.001).DLAD-10 showed excellent performance, improving radiologists' performance and shortening the reporting time for critical and urgent cases.

scholarly journals Image and structured data analysis for prognostication of health outcomes in patients presenting to the Emergency Department during the COVID-19 pandemic

2021 ◽  
Author(s):  
Liam Butler ◽  
Ibrahim Karabayir ◽  
Mohammad Samie Tootooni ◽  
Majid Afshar ◽  
Ari Goldberg ◽  
...  
Keyword(s):  
Machine Learning ◽  
Emergency Department ◽  
Intensive Care ◽  
Deep Learning ◽  
Clinical Data ◽  
Chest Radiographs ◽  
Gradient Boosting ◽  
Learning Models ◽  
Icu Admission ◽  
Light Gradient

Background: Patients admitted to the emergency department (ED) with COVID-19 symptoms are routinely required to have chest radiographs and computed tomography (CT) scans. COVID-19 infection has been directly related to development of acute respiratory distress syndrome (ARDS) and severe infections lead to admission to intensive care and can also lead to death. The use of clinical data in machine learning models available at time of admission to ED can be used to assess possible risk of ARDS, need for intensive care unit (ICU) admission as well as risk of mortality. In addition, chest radiographs can be inputted into a deep learning model to further assess these risks. Purpose: This research aimed to develop machine and deep learning models using both structured clinical data and image data from the electronic health record (EHR) to adverse outcomes following ED admission. Materials and Methods: Light Gradient Boosting Machines (LightGBM) was used as the main machine learning algorithm using all clinical data including 42 variables. Compact models were also developed using 15 the most important variables to increase applicability of the models in clinical settings. To predict risk of the aforementioned health outcome events, transfer learning from the CheXNet model was implemented on our data as well. This research utilized clinical data and chest radiographs of 3571 patients 18 years and older admitted to the emergency department between 9th March 2020 and 29th October 2020 at Loyola University Medical Center. Main Findings: Our research results show that we can detect COVID-19 infection (AUC = 0.790 (0.746-0.835)) and predict the risk of developing ARDS (AUC = 0.781 (0.690-0.872), ICU admission (AUC = 0.675 (0.620-0.713)), and mortality (AUC = 0.759 (0.678-0.840)) at moderate accuracy from both chest X-ray images and clinical data. Principal Conclusions: The results can help in clinical decision making, especially when addressing ARDS and mortality, during the assessment of patients admitted to the ED with or without COVID-19 symptoms.

Sign in / Sign up

Export Citation Format

Share Document