scholarly journals Agreement and Reliability Analysis of Machine Learning Scaling and Wireless Monitoring in the Assessment of Acute Proximal Weakness by Experts and Non-Experts: A Feasibility Study

2022 ◽  
Vol 12 (1) ◽  
pp. 20
Author(s):  
Eunjeong Park ◽  
Kijeong Lee ◽  
Taehwa Han ◽  
Hyo Suk Nam
Keyword(s):  
Machine Learning ◽  
Reliability Analysis ◽  
Health Management ◽  
Point Of Care ◽  
Correlation Coefficients ◽  
Assessment System ◽  
Neurological Deficits ◽  
Neurological Intensive Care Unit ◽  
Neurological Intensive Care ◽  
Percent Agreement

Assessing the symptoms of proximal weakness caused by neurological deficits requires the knowledge and experience of neurologists. Recent advances in machine learning and the Internet of Things have resulted in the development of automated systems that emulate physicians’ assessments. The application of those systems requires not only accuracy in the classification but also reliability regardless of users’ proficiency in the real environment for the clinical point-of-care and the personalized health management. This study provides an agreement and reliability analysis of using a machine learning-based scaling of Medical Research Council (MRC) proximal scores to evaluate proximal weakness by experts and non-experts. The system trains an ensemble learning model using the signals from sensors attached to the limbs of patients in a neurological intensive care unit. For the agreement analysis, we investigated the percent agreement of MRC proximal scores and Bland-Altman plots of kinematic features between the expert- and non-expert scaling. We also analyzed the intra-class correlation coefficients (ICCs) of kinematic features and Krippendorff’s alpha of the observers’ scaling for the reliability analysis. The mean percent agreement between the expert- and the non-expert scaling was 0.542 for manual scaling and 0.708 for autonomous scaling. The ICCs of kinematic features measured using sensors ranged from 0.742 to 0.850, whereas the Krippendorff’s alpha of manual scaling for the three observers was 0.275. The autonomous assessment system can be utilized by the caregivers, paramedics, or other observers during an emergency to evaluate acute stroke patients.

Agreement and Reliability Analysis of Machine Learning Scaling in the Assessment of Acute Proximal Weakness by Experts and Non-Experts: Clinical Feasibility Study (Preprint)

2021 ◽  
Author(s):  
Eunjeong Park ◽  
Kijeong Lee ◽  
Taehwa Han ◽  
Hyo Suk Nam
Keyword(s):  
Machine Learning ◽  
Reliability Analysis ◽  
Gold Standard ◽  
Assessment Tool ◽  
Expert Knowledge ◽  
Hybrid Approach ◽  
Assessment Model ◽  
Lower Limbs ◽  
Neurological Deficits ◽  
Percent Agreement

BACKGROUND Assessing the symptoms of proximal weakness caused by neurological deficits requires expert knowledge and experienced neurologists. Recent advances in artificial intelligence and the Internet of Things have resulted in the development of automated systems that emulate physicians’ assessments. OBJECTIVE This study provides an agreement and reliability analysis of using an automated scoring system to evaluate proximal weakness by experts and non-experts. METHODS We collected 144 observations from acute stroke patients in a neurological intensive care unit to measure the symptom of proximal weakness of upper and lower limbs. A neurologist performed a gold standard assessment and two medical students performed identical tests as non-expert assessments for manual and machine learning-based scaling of Medical Research Council (MRC) proximal scores. The system collects signals from sensors attached on patients’ limbs and trains a machine learning assessment model using the hybrid approach of data-level and algorithm-level methods for the ordinal and imbalanced classification in multiple classes. For the agreement analysis, we investigated the percent agreement of MRC proximal scores and Bland-Altman plots of kinematic features between the expert- and non-expert scaling. In the reliability analysis, we analysed the intra-class correlation coefficients (ICCs) of kinematic features and Krippendorff’s alpha of the three observers’ scaling. RESULTS The mean percent agreement between the gold standard and the non-expert scaling was 0.542 for manual scaling and 0.708 for IoT-assisted machine learning scaling, with 30.63% enhancement. The ICCs of kinematic features measured using sensors ranged from 0.742 to 0.850, whereas the Krippendorff’s alpha of manual scaling for the three observers was 0.275. The Krippendorff’s alpha of machine learning scaling increased to 0.445, with 61.82% improvement. CONCLUSIONS Automated scaling using sensors and machine learning provided higher inter-rater agreement and reliability in assessing acute proximal weakness. The enhanced assessment supported by the proposed system can be utilized as a reliable assessment tool for non-experts in various emergent environments.

scholarly journals Femtomolar SARS-CoV-2 Antigen Detection Using the Microbubbling Digital Assay with Smartphone Readout Enables Antigen Burden Quantitation and Dynamics Tracking

2021 ◽  
Author(s):  
Hui Chen ◽  
Zhao Li ◽  
Sheng Feng ◽  
Anni Wang ◽  
Melissa Richard-Greenblatt ◽  
...  
Keyword(s):  
Machine Learning ◽  
Computer Vision ◽  
Nucleic Acid ◽  
Symptom Onset ◽  
Limit Of Detection ◽  
Point Of Care ◽  
High Sensitivity ◽  
Percent Agreement ◽  
Potential Marker ◽  
Antigen Assay

AbstractBackgroundLittle is known about the dynamics of SARS-CoV-2 antigen burden in respiratory samples in different patient populations at different stages of infection. Current rapid antigen tests cannot quantitate and track antigen dynamics with high sensitivity and specificity in respiratory samples.MethodsWe developed and validated an ultra-sensitive SARS-CoV-2 antigen assay with smartphone readout using the Microbubbling Digital Assay previously developed by our group, which is a platform that enables highly sensitive detection and quantitation of protein biomarkers. A computer vision-based algorithm was developed for microbubble smartphone image recognition and quantitation. A machine learning-based classifier was developed to classify the smartphone images based on detected microbubbles. Using this assay, we tracked antigen dynamics in serial swab samples from COVID patients hospitalized in ICU and immunocompromised COVID patients.ResultsThe limit of detection (LOD) of the Microbubbling SARS-CoV-2 Antigen Assay was 0.5 pg/mL (10.6 fM) recombinant nucleocapsid (N) antigen or 4000 copies/mL inactivated SARS-CoV-2 virus in nasopharyngeal (NP) swabs, comparable to many rRT-PCR methods. The assay had high analytical specificity towards SARS-CoV-2. Compared to EUA-approved rRT-PCR methods, the Microbubbling Antigen Assay demonstrated a positive percent agreement (PPA) of 97% (95% confidence interval (CI), 92-99%) in symptomatic individuals within 7 days of symptom onset and positive SARS-CoV-2 nucleic acid results, and a negative percent agreement (NPA) of 97% (95% CI, 94-100%) in symptomatic and asymptomatic individuals with negative nucleic acid results. Antigen positivity rate in NP swabs gradually decreased as days-after-symptom-onset increased, despite persistent nucleic acid positivity of the same samples. The computer vision and machine learning-based automatic microbubble image classifier could accurately identify positives and negatives, based on microbubble counts and sizes. Total microbubble volume, a potential marker of antigen burden, correlated inversely with Ct values and days-after-symptom-onset. Antigen was detected for longer periods of time in immunocompromised patients with hematologic malignancies, compared to immunocompetent individuals. Simultaneous detectable antigens and nucleic acids may indicate the presence of replicating viruses in patients with persistent infections.ConclusionsThe Microbubbling SARS-CoV-2 Antigen Assay enables sensitive and specific detection of acute infections, and quantitation and tracking of antigen dynamics in different patient populations at various stages of infection. With smartphone compatibility and automated image processing, the assay is well-positioned to be adapted for point-of-care diagnosis and to explore the clinical implications of antigen dynamics in future studies.

A Comparison of Point-of-Care Instruments Designed for Monitoring Oral Anticoagulation with Standard Laboratory Methods

2000 ◽  
Vol 83 (05) ◽  
pp. 698-703 ◽  
Author(s):  
Robert Gosselin ◽  
Richard White ◽  
Rose Hutchinson ◽  
Jennifer Branch ◽  
Kathy Mahackian ◽  
...  
Keyword(s):  
Oral Anticoagulation ◽  
Clinical Laboratory ◽  
Point Of Care ◽  
Oral Anticoagulant Therapy ◽  
Correlation Coefficients ◽  
Laboratory Method ◽  
Assessment System ◽  
Standard Laboratory ◽  
Standardization Method ◽  
Laboratory Methods

SummaryOur study compared point-of-care (POC) device monitoring with traditional clinical laboratory methods device of patients on oral anticoagulant therapy. The POC devices used in the study were Coumatrak, CoaguChek, CoaguChek Plus, Thrombolytic Assessment System (TAS) PT-One, TAS PTNC, TAS PT, Hemachron Jr. Signature, Protime Microcoagulation System, and Medtronics ACT II. The clinical laboratory method used thromboplastins with different ISI values: Innovin and Thromboplastin C Plus (TPC). All POC INRs showed strong correlation with both laboratory methods, with correlation coefficients of >0.900. All POC methods demonstrated a significant (p <0.05) difference in INR values, except the TAS PTNC and ACT II INRs (p: 0.12 and 0.71 respectively) when compared with Innovin INRs. All POC INRs were significantly different from TPC generated INRs (p <0.05). Comparisons of the POC INRs to the group mean of the POC methods, show higher correlation (R>0.93), but there were still significant (p<0.05) differences noted between the POC group INR mean and CoaguChek Plus, ACT II, TAS PT-One, TAS PTNC, and Hemachron Jr Signature INRs. These data indicate that POC INR biases exist between laboratory methods and POC devices. Until a suitable whole blood INR standardization method is available, we conclude that clinicians using point-of-care anticoagulation monitoring should be aware of differences between POC and parent laboratory values.

scholarly journals Predictors of Outcome in Clinically Diagnosed Viral Encephalitis Patients: A 5-Year Prospective Study

2020 ◽  
Vol 2020 ◽  
pp. 1-9
Author(s):  
Guibo Feng ◽  
Lunqin Zhou ◽  
Feng Li ◽  
Yida Hu ◽  
Xuefeng Wang ◽  
...  
Keyword(s):  
Hospital Stay ◽  
Patient Outcomes ◽  
Viral Encephalitis ◽  
Length Of Hospital Stay ◽  
Neurological Deficits ◽  
High Morbidity ◽  
Neurological Intensive Care Unit ◽  
Poor Outcome ◽  
Neurological Intensive Care ◽  
Medical University

Background. Viral encephalitis is the most common infectious disease of the central nervous system and is associated with high morbidity, mortality, and disability. The objective of this study was to analyze the clinical characteristics, auxiliary examinations, therapeutic management, and outcomes of patients clinically diagnosed with viral encephalitis and identify the outcome predictors. Methods. We conducted a prospective observational study by collecting information from patients clinically diagnosed with viral encephalitis at the First Affiliated Hospital of Chongqing Medical University and Yongchuan Hospital of Chongqing Medical University from January 2013 to December 2018. Univariate and multivariate analyses were performed to identify factors that influenced good patient outcomes (mRS<3) and poor patient outcomes (mRS≥3) at discharge. Results. In total, 216 patients were enrolled in the study. The multivariate analysis suggested that the following factors were associated with a poor outcome: Glasgow Coma Scale (GCS) score (OR 0.154, 95% CI (0.078-0.302), and P<0.001), focal neurological deficits (OR 9.403, 95% CI (1.581-55.928), and P=0.014), and total length of hospital stay (OR 1.119, 95% CI (1.002-1.250), and P=0.045). However, neurological intensive care unit (NICU) treatment, status epilepticus, and abnormal electroencephalogram (EEG) findings did not influence the prognosis of patients. Conclusion. Our study suggests that low GCS scores at admission, focal neurological deficits at admission, and a prolonged total hospital stay are predictors of a poor outcome at discharge in clinically diagnosed viral encephalitis patients. Whether early and effective neurological rehabilitation can improve the prognosis of viral encephalitis patients with focal neurological deficits remains to be confirmed in further studies.

scholarly journals Cost-Effective Machine Learning Based Clinical Pre-Test Probability Strategy for DVT Diagnosis in Neurological Intensive Care Unit

2021 ◽  
Vol 27 ◽  
pp. 107602962110086
Author(s):  
Li Luo ◽  
Ran Kou ◽  
Yuquan Feng ◽  
Jie Xiang ◽  
Wei Zhu
Keyword(s):  
Machine Learning ◽  
Roc Analysis ◽  
Prediction Models ◽  
Cost Effective ◽  
Training Data ◽  
Diagnostic Process ◽  
Neurological Intensive Care Unit ◽  
Neurological Intensive Care ◽  
Predictive Strength ◽  
Test Probability

In order to overcome the shortage of the current costly DVT diagnosis and reduce the waste of valuable healthcare resources, we proposed a new diagnostic approach based on machine learning pre-test prediction models using EHRs. We examined the sociodemographic and clinical factors in the prediction of DVT with 518 NICU admitted patients, including 189 patients who eventually developed DVT. We used cross-validation on the training data to determine the optimal parameters, and finally, the applied ROC analysis is adopted to evaluate the predictive strength of each model. Two models (GLM and SVM) with the strongest ROC were selected for DVT prediction, based on which, we optimized the current intervention and diagnostic process of DVT and examined the performance of the proposed approach through simulations. The use of machine learning based pre-test prediction models can simplify and improve the intervention and diagnostic process of patients in NICU with suspected DVT, and reduce the valuable healthcare resource occupation/usage and medical costs.

Intelligent system of English composition scoring model based on improved machine learning algorithm

2020 ◽  
pp. 1-11
Author(s):  
Jie Liu ◽  
Lin Lin ◽  
Xiufang Liang
Keyword(s):  
Machine Learning ◽  
Evaluation System ◽  
Intelligent System ◽  
Learning Algorithm ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Assessment System ◽  
English Composition ◽  
Region Extraction ◽  
Constraint Model

The online English teaching system has certain requirements for the intelligent scoring system, and the most difficult stage of intelligent scoring in the English test is to score the English composition through the intelligent model. In order to improve the intelligence of English composition scoring, based on machine learning algorithms, this study combines intelligent image recognition technology to improve machine learning algorithms, and proposes an improved MSER-based character candidate region extraction algorithm and a convolutional neural network-based pseudo-character region filtering algorithm. In addition, in order to verify whether the algorithm model proposed in this paper meets the requirements of the group text, that is, to verify the feasibility of the algorithm, the performance of the model proposed in this study is analyzed through design experiments. Moreover, the basic conditions for composition scoring are input into the model as a constraint model. The research results show that the algorithm proposed in this paper has a certain practical effect, and it can be applied to the English assessment system and the online assessment system of the homework evaluation system algorithm system.

scholarly journals Thoracic Point-of-Care Ultrasound: A SARS-CoV-2 Data Repository for Future Artificial Intelligence and Machine Learning

2021 ◽  
pp. 155335062110186
Author(s):  
Abdel-Moneim Mohamed Ali ◽  
Emran El-Alali ◽  
Adam S. Weltz ◽  
Scott T. Rehrig
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Point Of Care ◽  
Prognostic Indicators ◽  
Data Repository ◽  
Point Of Care Ultrasound ◽  
Ultrasound Images ◽  
Clinical Protocol ◽  
Improved Outcomes ◽  
Current Experience

Current experience suggests that artificial intelligence (AI) and machine learning (ML) may be useful in the management of hospitalized patients, including those with COVID-19. In light of the challenges faced with diagnostic and prognostic indicators in SARS-CoV-2 infection, our center has developed an international clinical protocol to collect standardized thoracic point of care ultrasound data in these patients for later AI/ML modeling. We surmise that in the future AI/ML may assist in the management of SARS-CoV-2 patients potentially leading to improved outcomes, and to that end, a corpus of curated ultrasound images and linked patient clinical metadata is an invaluable research resource.

Predicted Number of Pregnant Women in Aichi Prefecture, Japan: Estimation by Machine Learning Database Construction for Disaster Preparation

2021 ◽  
pp. 1-9
Author(s):  
Kanetoshi Hattori ◽  
Ritsuko Hattori
Keyword(s):  
Machine Learning ◽  
Pregnant Women ◽  
Disaster Preparedness ◽  
Correlation Coefficients ◽  
Database Construction ◽  
Aichi Prefecture ◽  
Cascade Correlation ◽  
Correlation Learning ◽  
Disaster Preparation ◽  
Area Data

Abstract Aichi prefecture, Japan is predicted to be hit by Mega-earthquake. Aichi Prefectural Association of Midwives has been making efforts to improve disaster preparedness for pregnant women. This project aims to acquire area data of pregnant women for simulated studies of rescue activities. Number of women in census survey areas in Nagoya City was acquired from nationwide data of pregnant women by machine learning (Cascade-Correlation Learning Architecture). Quite high correlation coefficients between actual data and estimation data were observed. Rescue simulations have been carried out based on the data acquired by this study.

scholarly journals Assessment of reliability and validity of the 5-scale grading system of the point-of-care immunoassay for tear matrix metalloproteinase-9

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Minjeong Kim ◽  
Ja Young Oh ◽  
Seon Ha Bae ◽  
Seung Hyeun Lee ◽  
Won Jun Lee ◽  
...  
Keyword(s):  
Matrix Metalloproteinase ◽  
Calibration Curve ◽  
Point Of Care ◽  
Interobserver Reliability ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Correlation Coefficients ◽  
Grading System ◽  
Intraclass Correlation Coefficients ◽  
The Difference

AbstractWe evaluated the reliability and validity of the 5-scale grading system to interpret the point-of-care immunoassay for tear matrix metalloproteinase (MMP)-9. Six observers graded red bands of photographs of the readout window in MMP-9 immunoassay kit (InflammaDry) two times with 2-week interval based on the 5-scale grading system (i.e. grade 0–4). Interobserver and intraobserver reliability were evaluated using intraclass correlation coefficients. The interobserver agreements were analyzed according to the severity of tear MMP-9 expression. To validate the system, a concentration calibration curve was made using MMP-9 solutions with reference concentrations, then the distribution of MMP-9 concentrations was analyzed according to the 5-scale grading system. Both intraobserver and interobserver reliability was excellent. The readout grades were significantly correlated with the quantified colorimetric densities. The interobserver variance of readout grades had no correlation with the severity of the measured densities. The band density continued to increase up to a maximal concentration (i.e. 5000 ng/mL) according to the calibration curve. The difference of grades reflected the change of MMP-9 concentrations sensitively, especially between grade 2 and 4. Together, our data indicate that the subjective 5-scale grading system in the point-of-care MMP-9 immunoassay is an easy and reliable method with acceptable accuracy.

Sign in / Sign up

Export Citation Format

Share Document