Completeness of reporting of clinical prediction models developed using supervised machine learning: a systematic review

Background While many studies have consistently found incomplete reporting of regression-based prediction model studies, evidence is lacking for machine learning-based prediction model studies. We aim to systematically review the adherence of Machine Learning (ML)-based prediction model studies to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Statement. Methods We included articles reporting on development or external validation of a multivariable prediction model (either diagnostic or prognostic) developed using supervised ML for individualized predictions across all medical fields. We searched PubMed from 1 January 2018 to 31 December 2019. Data extraction was performed using the 22-item checklist for reporting of prediction model studies (www.TRIPOD-statement.org). We measured the overall adherence per article and per TRIPOD item. Results Our search identified 24,814 articles, of which 152 articles were included: 94 (61.8%) prognostic and 58 (38.2%) diagnostic prediction model studies. Overall, articles adhered to a median of 38.7% (IQR 31.0–46.4%) of TRIPOD items. No article fully adhered to complete reporting of the abstract and very few reported the flow of participants (3.9%, 95% CI 1.8 to 8.3), appropriate title (4.6%, 95% CI 2.2 to 9.2), blinding of predictors (4.6%, 95% CI 2.2 to 9.2), model specification (5.2%, 95% CI 2.4 to 10.8), and model’s predictive performance (5.9%, 95% CI 3.1 to 10.9). There was often complete reporting of source of data (98.0%, 95% CI 94.4 to 99.3) and interpretation of the results (94.7%, 95% CI 90.0 to 97.3). Conclusion Similar to prediction model studies developed using conventional regression-based techniques, the completeness of reporting is poor. Essential information to decide to use the model (i.e. model specification and its performance) is rarely reported. However, some items and sub-items of TRIPOD might be less suitable for ML-based prediction model studies and thus, TRIPOD requires extensions. Overall, there is an urgent need to improve the reporting quality and usability of research to avoid research waste. Systematic review registration PROSPERO, CRD42019161764.

2020 Pennsylvania Patient Safety Reporting: Updated Acute Care Reporting Rates

In our recent article summarizing 2020 data from acute care facilities in Pennsylvania, reporting rates and fall rates were provided for Q1 and Q2 2020 based on the latest data we had available at the time of publication. Given that 2020 was an unpredictable year in healthcare, any forecasting of rates for Q3 and Q4 2020 would have been unreliable. Therefore, this data snapshot serves to complete reporting rates for 2020 now that all hospital patient days and surgical encounters data from 2020 have been made available for rate calculations.

The influence of biological and lifestyle factors on circulating cell-free DNA in blood plasma

Research and clinical use of circulating cell-free DNA (cirDNA) is expanding rapidly; however, there remain large gaps in our understanding of the influence of lifestyle and biological factors on the amount of cirDNA present in blood. Here, we review 66 individual studies of cirDNA levels and lifestyle and biological factors, including exercise (acute and chronic), alcohol consumption, occupational hazard exposure, smoking, body mass index, menstruation, hypertension, circadian rhythm, stress, biological sex and age. Despite technical and methodological inconsistences across studies, we identify acute exercise as a significant influence on cirDNA levels. Given the large increase in cirDNA induced by acute exercise, we recommend that controlling for physical activity prior to blood collection is routinely incorporated into study design when total cirDNA levels are of interest. We also highlight appropriate selection and complete reporting of laboratory protocols as important for improving the reproducibility cirDNA studies and ability to critically evaluate the results.

1359. Reported Neurologic, Ocular, and Otic Manifestations among Syphilis Cases — 16 States, 2019

Background Syphilis can cause neurologic, ocular, or otic manifestations at any stage, possibly resulting in permanent disability or even death. In 2018, CDC began collecting clinical manifestation data for syphilis cases reported through the National Notifiable Diseases Surveillance System (NNDSS). We present the first estimates of the prevalence of neurologic, ocular, and otic manifestations among syphilis cases in the United States. Methods We reviewed NNDSS data to identify jurisdictions (states + DC) who reported ≥ 70% of their syphilis cases with clinical manifestation data (considered to have “complete reporting”) in 2019. Among these jurisdictions, we determined the prevalence of neurologic, ocular, and otic manifestations (combining verified, likely, and possible clinical manifestations together), stratified by HIV status and by syphilis stage (Unknown/late syphilis vs. Early syphilis [Primary, Secondary, and Early non primary non secondary syphilis]). Results In 2019, 16 states had complete reporting for neurologic, otic, and ocular manifestations. Of the 41,216 syphilis cases reported in these jurisdictions, clinical manifestations were infrequently reported: neurologic (n=445, 1.1%), ocular (n=461, 1.1%), and otic (n=166, 0.4%). Prevalence was higher among HIV-infected persons compared to HIV-negative persons for neurologic (1.4% vs. 0.9%) and ocular manifestations (1.3% vs 1.0%) but was similar for otic manifestations (0.4% vs 0.4%). Prevalence was higher among persons diagnosed with Unknown/late syphilis compared to Early syphilis for neurologic (1.6% vs 0.8%) and ocular manifestations (1.6% vs 0.9%) but similar for otic manifestations (0.5% vs 0.4%); however, 49.4% of cases reported with ≥ 1 of these clinical manifestations were diagnosed with Early syphilis. Conclusion The prevalence of neurologic, ocular, and otic manifestations was low among syphilis cases, but case data likely underestimate the true burden given potential underreporting. The frequency of clinical manifestations, including among HIV-negative persons and persons diagnosed with Early syphilis, emphasizes the importance of evaluating all syphilis cases for clinical signs or symptoms regardless of stage or HIV status. Disclosures All Authors: No reported disclosures

Pothole Detection and Reporting System

Potholes on roads are the major problem for citizens acting as pedestrians as well as drivers. Government bodies which consist of engineers and workers are responsible to detect damages on roads and fix those damages. A recent study stated that every year around 3,597 people die due to potholes. The size and depth of the pothole may vary in a different place. Potholes had to be taken seriously. This system consists of a citizen with a handheld android/ios device with internet and GPS enabled, gathering the data in form of images and reporting to the government along with Geo-location. The study focuses on collecting and analyzing the datasets of potholes that are clicked by the users and detection of a pothole in that image via the TensorFlow Lite Model. The object detection system TensorFlow Lite is used for detecting the potholes, it shows that we can identify potholes from images clicked by the citizens and uploaded by the same application on the server and if a pothole is detected it ensures the complete reporting to the municipal authority along with the location. Index Term: Potholes, Android, GPS, Geo-location, Tensor- Flow Lite, Datasets, Reporting.

How to write a guideline: a proposal for a manuscript template that supports the creation of trustworthy guidelines

Trustworthy health guidelines should provide recommendations, document the development process, and highlight implementation information. Our objective was to develop a guideline manuscript template to help authors write a complete and useful report. The McMaster Grading of Recommendations Assessment, Development and Evaluation (GRADE) centre collaborated with the American Society of Hematology (ASH) to develop guidelines for the management of venous thromboembolism. A template for reporting the guidelines was developed based on prior approaches and refined using input from other key stakeholders. The proposed guideline manuscript template includes: 1) title for guideline identification; 2) abstract, including a summary of key recommendations; 3) overview of all recommendations [executive summary]; 4) the main text, providing sufficient detail on the entire process including objectives, background, and methodological decisions from panel selection and conflict of interest management to criteria for updating, as well as supporting information such as links to online (interactive) tables. The template further allows for tailoring to the specific topic, using examples. Initial experience with the ASH guideline manuscript template was positive, and challenges included drafting descriptions of recommendations involving multiple management pathways, tailoring the template for a specific guideline, and choosing key recommendations to highlight. Feedback from a larger group of guideline authors and users will be needed to evaluate its usefulness and refine. The proposed guideline manuscript template is the first detailed template for transparent and complete reporting of guidelines. Consistent application of the template may simplify preparing an evidence-based guideline manuscript and facilitate its use.

Assessment of reporting quality in randomised controlled clinical trial abstracts of dental implantology published from 2014 to 2016

ObjectivesAccess to full texts of randomised controlled clinical trials (RCTs) is often limited, so brief summaries of studies play a pivotal role. In 2008, a checklist was provided to ensure the transparency and completeness of abstracts. The aim of this investigation was to estimate adherence to the reporting guidelines of the Consolidated Standards of Reporting Trials (CONSORT) criteria for abstracts (CONSORT-A) in RCT publications.Primary endpointAssessment according to the percentage of compliance with the 16 CONSORT-A criteria per study.Materials and methodsThis study is based on a full survey (212 RCT abstracts in dental implantology, PubMed search, publication period 2014–2016, 45 journals, median impact factor: 2.328). In addition to merely documenting ‘adherence’ to criteria, the authors also assessed the ‘complete implementation’ of the requested information where possible. The collection of data was performed independently by two dentists, and a final consensus was reached. The primary endpoint was evaluated by medians and quartiles. Additionally, a Poisson regression was conducted to detect influencing factors.ResultsA median of 50% (Q1–Q3: 44%–63%) was documented for the 16 criteria listed in the CONSORT-A statement. Nine of the 16 criteria were considered in fewer than 50% of the abstracts. ‘Correct implementation’ was achieved for a median of 43% (Q1–Q3: 31%–50%) of the criteria. An additional application of Poisson regression revealed that the number of words used had a locally significant impact on the number of reported CONSORT criteria for abstracts (incidence rate ratio 1.001, 95% CI 1.001 to 1.002).ConclusionTransparent and complete reporting in abstracts appears problematic. A limited word count seems to result in a reduction in necessary information. As current scientific knowledge is often not readily available in the form of publications, abstracts constitute the primary basis for decision making in clinical practice and research. This is why journals should refrain from limiting the number of words too strictly in order to facilitate comprehensive reporting in abstracts.

Incidence of adverse events following immunization (AEFI) among infants after immunization with primary doses of scheduled vaccines in district Srinagar (J&K)

Immunization is the process whereby a person is made immune or resistant to an infectious disease, typically by the administration of a vaccine. However, like other medicinal products, vaccines are not free from adverse reactions. AEFI is any untoward medical occurrence which follows immunization and which does not necessarily have a causal relationship with the usage of the vaccine. The reporting of AEFI’s from the routine system is inadequate due to many reasons. Hence this study was conducted to throw some light and provide the baseline data. To find out the incidence of adverse events following immunization among infants in District Srinagar. It was a prospective study in which parents of infants receiving vaccines were contacted telephonically after specified time intervals to verify the occurrence of adverse events. The children were followed till 30 days of the administration of vaccines up to measles rubella vaccine. The incidence of AEFI reported in this study was 23.03% with 95% CI (22.24% to 23.85%). The most frequently reported AEFI was Fever (54.90%, n=1322), followed by Diarrhea (8.30%, n=200) and Vomiting (8.14%, n=196). AEFI were more in frequency during first week of receiving vaccine and most of the parents of children did not report AEFI after 7 days after vaccination. This study reveals that most of the vaccines associated adverse reactions were of mild and non-serious type and rarely of serious nature, yet proper monitoring of vaccine associated adverse reactions; is too essential. Proper and complete reporting of AEFI’s by field workers needs to be encouraged.

Completeness of reporting of clinical prediction models developed using supervised machine learning: A systematic review

ABSTRACT Objective. While many studies have consistently found incomplete reporting of regression-based prediction model studies, evidence is lacking for machine learning-based prediction model studies. Our aim is to systematically review the adherence of Machine Learning (ML)-based prediction model studies to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Statement. Study design and setting: We included articles reporting on development or external validation of a multivariable prediction model (either diagnostic or prognostic) developed using supervised ML for individualized predictions across all medical fields (PROSPERO, CRD42019161764). We searched PubMed from 1 January 2018 to 31 December 2019. Data extraction was performed using the 22-item checklist for reporting of prediction model studies (www.TRIPOD-statement.org). We measured the overall adherence per article and per TRIPOD item. Results: Our search identified 24 814 articles, of which 152 articles were included: 94 (61.8%) prognostic and 58 (38.2%) diagnostic prediction model studies. Overall, articles adhered to a median of 38.7% (IQR 31.0-46.4) of TRIPOD items. No articles fully adhered to complete reporting of the abstract and very few reported the flow of participants (3.9%, 95% CI 1.8 to 8.3), appropriate title (4.6%, 95% CI 2.2 to 9.2), blinding of predictors (4.6%, 95% CI 2.2 to 9.2), model specification (5.2%, 95% CI 2.4 to 10.8), and model's predictive performance (5.9%, 95% CI 3.1 to 10.9). There was often complete reporting of source of data (98.0%, 95% CI 94.4 to 99.3) and interpretation of the results (94.7%, 95% CI 90.0 to 97.3). Conclusion. Similar to studies using conventional statistical techniques, the completeness of reporting is poor. Essential information to decide to use the model (i.e. model specification and its performance) is rarely reported. However, some items and sub-items of TRIPOD might be less suitable for ML-based prediction model studies and thus, TRIPOD requires extensions. Overall, there is an urgent need to improve the reporting quality and usability of research to avoid research waste.

Taking the Pulse of Hospitals’ Response to the New Price Transparency Rule

As of January 1, 2021, most U.S. hospitals are required to publish pricing information on their website to promote more informed decision making by consumers regarding their care. In a nationally representative sample of 470 hospitals, we analyzed whether hospitals met price transparency information reporting requirements and the extent to which complete reporting was associated with ownership status, bed size category, system affiliation, and location in a metropolitan area. Fewer than one quarter of sampled hospitals met the price transparency information requirements of the new rule, which include five types of standard charges in machine-readable form and the consumer-shoppable display of 300 shoppable services. Our analyses of hospital reporting by organizational and market attributes revealed limited differences, with some exceptions for nonprofit and system-member hospitals demonstrating greater responsiveness with respect to the consumer-shoppable aspects of the rule.