Development of a Semi-Automated Chart Review for Assessing the Development of Radiation Pneumonitis: using Natural Language Processing (Preprint)

2021 ◽  
Author(s):  
Jordan McKenzie ◽  
Rasika Rajapakshe ◽  
Hua Shen ◽  
Shan Rajapakshe ◽  
Angela Lin
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Chart Review ◽  
Radiation Pneumonitis ◽  
Efficient Manner ◽  
Time Investment ◽  
Curative Radiotherapy ◽  
Cancer Agency ◽  
Manual Review

BACKGROUND Health research frequently requires manual chart review to identify patients for the study-specific cohort and examine their clinical outcomes. Manual chart review is a labour-intensive process requiring significant time investment for clinical researchers. OBJECTIVE This study aimed to evaluate the feasibility and accuracy of an assisted chart review program, using an in-house natural language processing (NLP) program, to identify patients who developed radiation pneumonitis (RP) after receiving curative radiotherapy. METHODS A retrospective manual chart review was completed for patients who received curative radiotherapy for stage II-III lung cancer from January 1, 2013 to December 31, 2015 at BC Cancer Kelowna. In the manual chart review, RP diagnosis and grading were recorded using Common Terminology Criteria for Adverse Events (CTCAE) v5.0. From the charts of 50 sample patients, a total of 1413 clinical documents were extracted for review from the Cancer Agency Information System (CAIS). The NLP program was built using the Natural Language Toolkit Python platform. Python version 3.7.2. was used to run the NLP program. The output of the NLP program is a list of the full sentences containing the key terms, the document ID’s and dates from which these sentences were extracted. The result from the manual review was used as the gold standard in this study, with which the result of the NLP program was compared. RESULTS Twenty-five out of the 50 sample patients developed RP grade 1 or greater; the NLP program was able to ascertain 23 out of these 25 patients (sensitivity = 0.92, 95%CI:0.74-0.99; specificity = 0.36, 95%CI:0.18-0.57). Furthermore, the NLP program was able to correctly identify all 9 patients with RP grade 2 or greater, which are patients with clinically significant symptoms (sensitivity = 1.0, 95%CI: 0.66-1.0; specificity = 0.27, 95%CI:0.14-0.43). The NLP program was useful in distinguishing patients with RP from those without RP. The NLP program in this study would avoid unnecessary manual review of 22% of the sample patients (n=11), as these patients were identified as RP grade 0 and will not require further manual review in subsequent studies. CONCLUSIONS This feasibility study showed that the NLP program was able to assist with the identification of patients who developed RP after curative radiotherapy. The NLP program streamlines the manual chart review further by identifying key sentences of interest. This work has a potential to improve future clinical research, as the NLP program shows promise in performing chart review in a more time efficient manner, compared to the traditional labor-intensive manual chart review.

scholarly journals Measuring Adoption of Patient Priorities-Aligned Care Using Natural Language Processing

2020 ◽  
Vol 4 (Supplement_1) ◽  
pp. 183-183
Author(s):  
Javad Razjouyan ◽  
Jennifer Freytag ◽  
Edward Odom ◽  
Lilian Dindo ◽  
Aanand Naik
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Chart Review ◽  
Group Analysis ◽  
Intervention Group ◽  
Multiple Chronic Conditions ◽  
Free Text ◽  
Term Care

Abstract Patient Priorities Care (PPC) is a model of care that aligns health care recommendations with priorities of older adults with multiple chronic conditions. Social workers (SW), after online training, document PPC in the patient’s electronic health record (EHR). Our goal is to identify free-text notes with PPC language using a natural language processing (NLP) model and to measure PPC adoption and effect on long term services and support (LTSS) use. Free-text notes from the EHR produced by trained SWs passed through a hybrid NLP model that utilized rule-based and statistical machine learning. NLP accuracy was validated against chart review. Patients who received PPC were propensity matched with patients not receiving PPC (control) on age, gender, BMI, Charlson comorbidity index, facility and SW. The change in LTSS utilization 6-month intervals were compared by groups with univariate analysis. Chart review indicated that 491 notes out of 689 had PPC language and the NLP model reached to precision of 0.85, a recall of 0.90, an F1 of 0.87, and an accuracy of 0.91. Within group analysis shows that intervention group used LTSS 1.8 times more in the 6 months after the encounter compared to 6 months prior. Between group analysis shows that intervention group has significant higher number of LTSS utilization (p=0.012). An automated NLP model can be used to reliably measure the adaptation of PPC by SW. PPC seems to encourage use of LTSS that may delay time to long term care placement.

scholarly journals Natural Language Processing Performance for the Identification of Venous Thromboembolism in an Integrated Healthcare System

2021 ◽  
Vol 27 ◽  
pp. 107602962110131
Author(s):  
Bela Woller ◽  
Austin Daw ◽  
Valerie Aston ◽  
Jim Lloyd ◽  
Greg Snow ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Venous Thromboembolism ◽  
Natural Language ◽  
Medical Record ◽  
Electronic Medical Record ◽  
Ct Angiography ◽  
Language Processing ◽  
Predictive Value ◽  
Chart Review ◽  
Imaging Studies

Real-time identification of venous thromboembolism (VTE), defined as deep vein thrombosis (DVT) and pulmonary embolism (PE), can inform a healthcare organization’s understanding of these events and be used to improve care. In a former publication, we reported the performance of an electronic medical record (EMR) interrogation tool that employs natural language processing (NLP) of imaging studies for the diagnosis of venous thromboembolism. Because we transitioned from the legacy electronic medical record to the Cerner product, iCentra, we now report the operating characteristics of the NLP EMR interrogation tool in the new EMR environment. Two hundred randomly selected patient encounters for which the imaging report assessed by NLP that revealed VTE was present were reviewed. These included one hundred imaging studies for which PE was identified. These included computed tomography pulmonary angiography—CTPA, ventilation perfusion—V/Q scan, and CT angiography of the chest/ abdomen/pelvis. One hundred randomly selected comprehensive ultrasound (CUS) that identified DVT were also obtained. For comparison, one hundred patient encounters in which PE was suspected and imaging was negative for PE (CTPA or V/Q) and 100 cases of suspected DVT with negative CUS as reported by NLP were also selected. Manual chart review of the 400 charts was performed and we report the sensitivity, specificity, positive and negative predictive values of NLP compared with manual chart review. NLP and manual review agreed on the presence of PE in 99 of 100 cases, the presence of DVT in 96 of 100 cases, the absence of PE in 99 of 100 cases and the absence of DVT in all 100 cases. When compared with manual chart review, NLP interrogation of CUS, CTPA, CT angiography of the chest, and V/Q scan yielded a sensitivity = 93.3%, specificity = 99.6%, positive predictive value = 97.1%, and negative predictive value = 99%.

Coding Neuroradiology Reports for the Northern Manhattan Stroke Study: A Comparison of Natural Language Processing and Manual Review

2000 ◽  
Vol 33 (1) ◽  
pp. 1-10 ◽  
Author(s):  
Jacob S. Elkins ◽  
Carol Friedman ◽  
Bernadette Boden-Albala ◽  
Ralph L. Sacco ◽  
George Hripcsak
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Stroke Study ◽  
Manual Review

scholarly journals Natural Language Processing for Surveillance of Cervical and Anal Cancer and Precancer: Algorithm Development and Split-Validation Study (Preprint)

2020 ◽  
Author(s):  
Carlos R Oliveira ◽  
Patrick Niccolai ◽  
Anette Michelle Ortiz ◽  
Sangini S Sheth ◽  
Eugene D Shapiro ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Human Papillomavirus ◽  
Natural Language ◽  
Language Processing ◽  
Medical Records ◽  
Processing Algorithm ◽  
Accurate Identification ◽  
Pathology Reports ◽  
Manual Review ◽  
Natural Language Processing Algorithm

BACKGROUND Accurate identification of new diagnoses of human papillomavirus–associated cancers and precancers is an important step toward the development of strategies that optimize the use of human papillomavirus vaccines. The diagnosis of human papillomavirus cancers hinges on a histopathologic report, which is typically stored in electronic medical records as free-form, or unstructured, narrative text. Previous efforts to perform surveillance for human papillomavirus cancers have relied on the manual review of pathology reports to extract diagnostic information, a process that is both labor- and resource-intensive. Natural language processing can be used to automate the structuring and extraction of clinical data from unstructured narrative text in medical records and may provide a practical and effective method for identifying patients with vaccine-preventable human papillomavirus disease for surveillance and research. OBJECTIVE This study's objective was to develop and assess the accuracy of a natural language processing algorithm for the identification of individuals with cancer or precancer of the cervix and anus. METHODS A pipeline-based natural language processing algorithm was developed, which incorporated machine learning and rule-based methods to extract diagnostic elements from the narrative pathology reports. To test the algorithm’s classification accuracy, we used a split-validation study design. Full-length cervical and anal pathology reports were randomly selected from 4 clinical pathology laboratories. Two study team members, blinded to the classifications produced by the natural language processing algorithm, manually and independently reviewed all reports and classified them at the document level according to 2 domains (diagnosis and human papillomavirus testing results). Using the manual review as the gold standard, the algorithm’s performance was evaluated using standard measurements of accuracy, recall, precision, and F-measure. RESULTS The natural language processing algorithm’s performance was validated on 949 pathology reports. The algorithm demonstrated accurate identification of abnormal cytology, histology, and positive human papillomavirus tests with accuracies greater than 0.91. Precision was lowest for anal histology reports (0.87, 95% CI 0.59-0.98) and highest for cervical cytology (0.98, 95% CI 0.95-0.99). The natural language processing algorithm missed 2 out of the 15 abnormal anal histology reports, which led to a relatively low recall (0.68, 95% CI 0.43-0.87). CONCLUSIONS This study outlines the development and validation of a freely available and easily implementable natural language processing algorithm that can automate the extraction and classification of clinical data from cervical and anal cytology and histology.

scholarly journals SAT-LB111 Improving Classification of Diabetes Etiology in Electronic Resources Using Phenotype Algorithms and Polygenic Risk Scores

2020 ◽  
Vol 4 (Supplement_1) ◽  
Author(s):  
Lina Sulieman ◽  
Jing He ◽  
Robert Carroll ◽  
Lisa Bastarache ◽  
Andrea Ramirez
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Chart Review ◽  
Risk Scores ◽  
P Value ◽  
Learning Approaches ◽  
Data Types ◽  
Electronic Health

Abstract Electronic Health Records (EHR) contain rich data to identify and study diabetes. Many phenotype algorithms have been developed to identify research subjects with type 2 diabetes (T2D), but very few accurately identify type 1 diabetes (T1D) cases or more rare forms of monogenic and atypical metabolic presentations. Polygenetic risk scores (PRS) quantify risk of a disease using common genomic variants well for both T1D and T2D. In this study, we apply validated phenotyping algorithms to EHRs linked to a genomic biobank to understand the independent contribution of PRS to classification of diabetes etiology and generate additional novel markers to distinguish subtypes of diabetes in EHR data. Using a de-identified mirror of medical center’s electronic health record, we applied published algorithms for T1D and T2D to identify cases, and used natural language processing and chart review strategies to identify cases of maturity onset diabetes of the young (MODY) and other more rare presentations. This novel approach included additional data types such as medication sequencing, ratio and temporality of insulin and non-insulin agents, clinical genetic testing, and ratios of diagnostic codes. Chart review was performed to validate etiology. To calculate PRS, we used genome wide genotyping from our BioBank, the de-identified biobank linking EHR to genomic data using coefficients of 65 published T1D SNPS and 76,996 T2D SNPS using PLINK in Caucasian subjects. In the dataset, we identified 82,238 cases of T2D but only 130 cases of T1D using the most cited published algorithms. Adding novel structured elements and natural language processing identified an additional 138 cases of T1D and distinguished 354 cases as MODY. Among over 90,000 subjects with genotyping data available, we included 72,624 Caucasian subjects since PRS coefficients were generated in Caucasian cohorts. Among those subjects, 248, 6,488, and 21 subjects were identified as T1D, T2D, and MODY subjects respectively in our final PRS cohort. The T1D PRS did significantly discriminate well between cases and controls (Mann-Whitney p-value is 3.4 e-17). The PRS for T2D did not significantly discriminate between cases and controls using published algorithms. The atypical case count was too low to calculate PRS discrimination. Calculation of the PRS score was limited by quality inclusion of variants available, and discrimination may improve in larger data sets. Additionally, blinded physician case review is ongoing to validate the novel classification scheme and provide a gold standard for machine learning approaches that can be applied in validation sets.

scholarly journals Identifying Bladder Cancer Stage And Use Of Chemotherapy In The Electronic Medical Record: How Reliable Is Natural Language Processing?

2021 ◽  
Vol 4 (1) ◽  
Author(s):  
Nirupama Devanathan ◽  
David Haggstrom ◽  
Clint Cary
Keyword(s):  
Bladder Cancer ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Chart Review ◽  
Clinical Stage ◽  
Treatment Algorithm ◽  
Clinical Staging ◽  
Muscle Invasive ◽  
Line Of Therapy

Background: Large automated electronic medical record (EMR) databases, together with natural language processing (NLP) algorithms, have the potential to be valuable tools in studying the patterns and effectiveness of treatment. Therefore, the current study sought to develop novel tools to identify bladder cancer cases, their clinical stage, and the chemotherapy they receive in electronic medical records. Methods: EMR data were obtained from Indiana University Health hospitals from 2008 to 2015.  We developed 2 novel algorithms using natural language processing (NLP) on unstructured data to identify (a) bladder cancer cases and clinical stage, and (b) chemotherapy names and line of chemotherapy. The sensitivity, specificity, PPV, and NPV for the clinical staging and treatment algorithm were calculated against the gold standard of manual chart review Results: A total of 2,559 unique bladder cancer patients were identified and stratified using the clinical staging algorithm, defined as metastatic, muscle invasive, or non-muscle invasive.  We identified 657 metastatic cases, 567 muscle invasive cases, and 604 non-muscle invasive cases. Further, we calculated the PPV for metastatic cases as 69.9%, muscle invasive as 80.4%, and non-muscle invasive as 79.1%. Next, the treatment algorithm was applied to metastatic patients to identify the type of chemotherapy received and 1st or 2nd line of therapy. The PPV for identifying the 1st and 2nd lines were 70.5% and 55.6%, respectively. The PPV for gemcitabine/carboplatin or cisplatin was 57.5%, but for methotrexate, vinblastine, doxorubicin, cisplatin, was 37.5%. Conclusion and Potential Impact: The performance of the algorithm demonstrates the potential for NLP to identify cancer cases, stage, and presence of treatment. While providing meaningful information, the accuracy of the approach suggests that a hybrid method using both NLP algorithms and manual chart review remains the most robust approach. The low performance of the algorithm to identify line of therapy further highlights the need for further NLP development in this area and emphasizes the ongoing need for either human entry or review of structured data.

scholarly journals Natural Language Processing for Surveillance of Cervical and Anal Cancer and Precancer: Algorithm Development and Split-Validation Study

10.2196/20826 ◽  
2020 ◽  
Vol 8 (11) ◽  
pp. e20826
Author(s):  
Carlos R Oliveira ◽  
Patrick Niccolai ◽  
Anette Michelle Ortiz ◽  
Sangini S Sheth ◽  
Eugene D Shapiro ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Human Papillomavirus ◽  
Natural Language ◽  
Language Processing ◽  
Medical Records ◽  
Processing Algorithm ◽  
Accurate Identification ◽  
Pathology Reports ◽  
Manual Review ◽  
Natural Language Processing Algorithm

Background Accurate identification of new diagnoses of human papillomavirus–associated cancers and precancers is an important step toward the development of strategies that optimize the use of human papillomavirus vaccines. The diagnosis of human papillomavirus cancers hinges on a histopathologic report, which is typically stored in electronic medical records as free-form, or unstructured, narrative text. Previous efforts to perform surveillance for human papillomavirus cancers have relied on the manual review of pathology reports to extract diagnostic information, a process that is both labor- and resource-intensive. Natural language processing can be used to automate the structuring and extraction of clinical data from unstructured narrative text in medical records and may provide a practical and effective method for identifying patients with vaccine-preventable human papillomavirus disease for surveillance and research. Objective This study's objective was to develop and assess the accuracy of a natural language processing algorithm for the identification of individuals with cancer or precancer of the cervix and anus. Methods A pipeline-based natural language processing algorithm was developed, which incorporated machine learning and rule-based methods to extract diagnostic elements from the narrative pathology reports. To test the algorithm’s classification accuracy, we used a split-validation study design. Full-length cervical and anal pathology reports were randomly selected from 4 clinical pathology laboratories. Two study team members, blinded to the classifications produced by the natural language processing algorithm, manually and independently reviewed all reports and classified them at the document level according to 2 domains (diagnosis and human papillomavirus testing results). Using the manual review as the gold standard, the algorithm’s performance was evaluated using standard measurements of accuracy, recall, precision, and F-measure. Results The natural language processing algorithm’s performance was validated on 949 pathology reports. The algorithm demonstrated accurate identification of abnormal cytology, histology, and positive human papillomavirus tests with accuracies greater than 0.91. Precision was lowest for anal histology reports (0.87, 95% CI 0.59-0.98) and highest for cervical cytology (0.98, 95% CI 0.95-0.99). The natural language processing algorithm missed 2 out of the 15 abnormal anal histology reports, which led to a relatively low recall (0.68, 95% CI 0.43-0.87). Conclusions This study outlines the development and validation of a freely available and easily implementable natural language processing algorithm that can automate the extraction and classification of clinical data from cervical and anal cytology and histology.

scholarly journals Online Assignment Plagiarism Detector

2021 ◽  
pp. 528-535
Author(s):  
Nikhil Paymode ◽  
Rahul Yadav ◽  
Sudarshan Vichare ◽  
Suvarna Bhoir
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Processing Technique ◽  
The Internet ◽  
Plagiarism Detection ◽  
Efficient Manner ◽  
Detection Approach ◽  
Different Sources ◽  
Student Program

Plagiarism is a big intricacy for companies, Schools, Colleges, and those who published their document on the web. In-Schools and Colleges maximum students write their assignments and experiments by copying other documents. Using this system teachers and examiners can detect the documents and sheets either it is written by a respective student or it is copied from someone else. For checking plagiarism the system takes two or more documents as a input and after using string matching algorithms, NLP ( natural language processing) technique, as well as an NLTK toolkit (natural language toolkit), produces output. In the output, the system returns some score which is an interval of 0 to 1. Where 1 and 0 refer to exactly similar and nothing is similar (Unique) respectively. If a score between 0 to 1 then it shows only some part of the document is similar. The main objective of the system is to find the more accurate plagiarism content in the documents with similar meanings and concepts that are correctly identified in an efficient manner. It is very easy to copy the data from different sources which includes the internet, papers, books over the internet, newspapers, etc. there is a need of detecting plagiarism to increase and improve the learning of students. To solve this problem, a student program plagiarism detection approach is proposed based on Natural Language Processing.

Sign in / Sign up

Export Citation Format

Share Document