Efficiently calculating ROC curves, AUC, and uncertainty from 2AFC studies with finite samples

2020 ◽  
Author(s):  
Dylan H. Shekter ◽  
Frank W. Samuelson
Keyword(s):  
Roc Curves ◽  
Finite Samples

Comparison of diagnostic accuracy of 18F-FDG PET, 123I-IMT- and 99mTc-MIBI SPECT

2006 ◽  
Vol 45 (01) ◽  
pp. 49-56 ◽  
Author(s):  
N. Özdemir-Sahin ◽  
P. Hipp ◽  
W. Mier ◽  
M. Eisenhut ◽  
J. Debus ◽  
...  
Keyword(s):  
Diagnostic Accuracy ◽  
Fdg Pet ◽  
Tumour Progression ◽  
Roc Curves ◽  
Low Grade ◽  
Who Grade ◽  
Pet Tracer ◽  
Patient Groups ◽  
99Mtc Mibi ◽  
Mibi Spect

Summary Aim was to evaluates the diagnostic accuracy of the SPECTtracers 3-123I-α-methyl-L-tyrosine (IMT) and 99mTc(I)- hexakis(2-methoxyisobutylisonitrile) (MIBI) as well as the PET-tracer 2-18F-2-deoxyglucose (FDG) for detecting tumour progression in irradiated low grade astrocytomas (LGA). Patients, methods: We examined 91 patients (56 males; 35 females; 44.7 ± 11.5 years), initially suffering from histologically proven LGAs (mean WHO grade II) and treated by stereotactic radiotherapy (59.0 ± 4.6 Gy). On average 21.9 ± 11.2 months after radiotherapy, patients presented new Gd-DTPA enhancing lesions on MRI, which did not allow a differentiation between progressive tumour (PT) and non-PT (nPT) at this point of time. PET scans (n=82) were acquired 45 min after injection of 208 ± 32 MBq FDG. SPECT scans started 10 min after injection of 269 ± 73 MBq IMT (n=68) and 15 min after injection of 706 ± 63 MBq MIBI (n=34). Lesions were classified as PT and nPT based on prospective follow-up (clinically, MRI) for 17.2 ± 9.9 months after PET/SPECT. Lesion-to-normal ratios (L/N) were calculated using contra lateraly mirrored reference regions for the SPECT examinations and reference regions in the contra lateral grey (GM) and white matter (WM) for FDG PET. Ratios were evaluated by Receiver Operating Characteristic (ROC) analysis. Results: In the patient groups nPT and PT, L/N ratios for FDG (GS) were 0.6 ± 0.3 vs. 1.2 ± 0.5 (p = 0.003), for FDG (WS) 1.2 ± 0.4 vs. 2.6 ± 0.4 (p <0.001), for IMT 1.1 ± 0.1 vs. 1.8 ± 0.4 (p <0.001) and for MIBI 1.6 ± 0.7 vs. 2.6 ± 2.2 (p = 0.554). Areas under the non-parametric ROC-curves were: 0.738 ± 0.059 for FDG (GS), 0.790 ± 0.057 for FDG (WS), 0.937 ± 0.037 for IMT and 0.564 ± 0.105 for MIBI. Conclusion: MIBI-SPECT examinations resulted in a low accuracy and especially in a poor sensitivity even at modest specificity values. A satisfying diagnostic accuracy was reached with FDG PET. Using WM as reference region for FDG PET, a slightly higher AUC as compared to GM was calculated. IMT yielded the best ROC characteristics and the highest diagnostic accuracy for differentiating between PT and nPT in irradiated LGA.

57Co-Bleomycin Scintigraphy in Staging of Carcinoma of the Cervix

1985 ◽  
Vol 24 (01) ◽  
pp. 48-51 ◽  
Author(s):  
K. Dursk ◽  
T. Pertyński
Keyword(s):  
Cervical Cancer ◽  
Roc Curves ◽  
Selection Of

Summary65 patients with cervical cancer at various stages of the disease were studied scintigraphically after 57Co-bleomycin administration, active urine being flushed out from the bladder continuously. The patients were submitted to other auxiliary tests and the interpretation of scintigraphic images could thus be verified. Scintigraphic data were classified semiquantitatively by scoring in the range from zero to 5. Results were evaluated in terms of ROC-curves. This method of analysis permitted selection of cut-off levels for binary decisions that yielded satisfactory sensitivity and acceptable specificity of the test.

Use of Receiver Operating Characteristic (ROC) Curves to Evaluate Computer Confidence Threshold and Clinical Performance in the Diagnosis of Appendicitis

1978 ◽  
Vol 17 (03) ◽  
pp. 157-161 ◽  
Author(s):  
F. T. De Dombal ◽  
Jane C. Horrocks
Keyword(s):  
Receiver Operating Characteristic ◽  
Operating Characteristic ◽  
Clinical Performance ◽  
Roc Curves ◽  
Final Diagnosis ◽  
Confidence Threshold ◽  
Computer Aided ◽  
Computer Confidence ◽  
Overall Performance ◽  
Receiver Operating

This paper uses simple receiver operating characteristic (ROC) curves (i) to study the effect of varying computer confidence of threshold levels and (ii) to evaluate clinical performance in the diagnosis of acute appendicitis. Over 1300 patients presenting to five centres with abdominal pain of short duration were studied in varying detail. Clinical and computer-aided diagnostic predictions were compared with the »final« diagnosis. From these studies it is concluded the simplistic setting of a 50/50 confidence threshold for the computer program is as »good« as any other. The proximity of a computer-aided system changed clinical behaviour patterns; a higher overall performance level was achieved and clinicians performance levels became associated with the »mildly conservative« end of the computers ROC curve. Prior forecasts of over-confidence or ultra-caution amongst clinicians using the computer-aided system have not been fulfilled.

Evaluation of the clot burden score (cbs) for acute ischemic stroke (ais) in intent-to-treat patients.

2019 ◽  
Vol 1 (1) ◽  
pp. 11-15 ◽  
Author(s):  
Sarah Yaziz ◽  
Ahmad Sobri Muda ◽  
Wan Asyraf Wan Zaidi ◽  
Nik Azuan Nik Ismail
Keyword(s):  
Ischemic Stroke ◽  
Clinical Outcome ◽  
Acute Ischemic Stroke ◽  
Predictive Value ◽  
Area Under The Curve ◽  
Rank Correlation ◽  
Treatment Decision ◽  
Roc Curves ◽  
Significant Negative Correlation ◽  
Clot Burden

Background : The clot burden score (CBS) is a scoring system used in acute ischemic stroke (AIS) to predict patient outcome and guide treatment decision. However, CBS is not routinely practiced in many institutions. This study aimed to investigate the feasibility of CBS as a relevant predictor of good clinical outcome in AIS cases. Methods:  A retrospective data collection and review of AIS patients in a teaching hospital was done from June 2010 until June 2015. Patients were selected following the inclusion and exclusion criteria. These patients were followed up after 90 days of discharge. The Modified Rankin scale (mRS) was used to assess their outcome (functional status). Linear regression Spearman Rank correlation was performed between the CBS and mRS. The quality performance of the correlations was evaluated using Receiver operating characteristic (ROC) curves. Results: A total of 89 patients with AIS were analysed, 67.4% (n=60) male and 32.6% (n=29) female. Twenty-nine (29) patients (33.7%) had a CBS ?6, 6 patients (6.7%) had CBS <6, while 53 patients (59.6%) were deemed clot free. Ninety (90) days post insult, clinical assessment showed that 57 (67.6%) patients were functionally independent, 27 (30.3%) patients functionally dependent, and 5 (5.6%) patients were deceased. Data analysis reported a significant negative correlation (r= -0.611, p<0.001). ROC curves analysis showed an area under the curve of 0.81 at the cut-off point of 6.5. This showed that a CBS of more than 6 predicted a good mRS clinical outcome in AIS patients; with sensitivity of 98.2%, specificity of 53.1%, positive predictive value (PPV) of 76%, and negative predictive value (NPV) of 21%. Conclusion: CBS is a useful additional variable for the management of AIS cases, and should be incorporated into the routine radiological reporting for acute ischemic stroke (AIS) cases.

Conscious access to fear-relevant information is mediated by threshold

2011 ◽  
Vol 42 (2) ◽  
pp. 56-64 ◽  
Author(s):  
Remigiusz Szczepanowski
Keyword(s):  
Present Report ◽  
Intrinsic Property ◽  
Roc Curves ◽  
Relevant Information ◽  
Perceptual Processing ◽  
High Threshold ◽  
Operating Characteristics ◽  
Emotional Perception ◽  
Fearful Faces ◽  
The Subject

Conscious access to fear-relevant information is mediated by thresholdThe present report proposed a model of access consciousness to fear-relevant information according to which there is a threshold for emotional perception beyond that the subject makes hits with no false alarm. The model was examined by having the participants performed a confidence-ratings masking task with fearful faces. Measures of the thresholds for conscious access were taken by looking at the receiver operating characteristics (ROC) curves generated from a three-state low- and high-threshold (3-LHT) model by Krantz. Indeed, the analysis of the masking data revealed that the ROCs had threshold-like-nature (a two-limb shape) rather continuous (a curvilinear shape) challenging in this fashion the classical signal-detection view on perceptual processing. Moreover, the threshold ROC curve exhibited the specific y-intercepts relevant to conscious access performance. The study suggests that the threshold can be an intrinsic property of conscious access, mediating emotional contents between perceptual states and consciousness.

scholarly journals MicroRNAs-1299, -126-3p and -30e-3p as Potential Diagnostic Biomarkers for Prediabetes

Diagnostics ◽  
2021 ◽  
Vol 11 (6) ◽  
pp. 949
Author(s):  
Cecil J. Weale ◽  
Don M. Matshazi ◽  
Saarah F. G. Davids ◽  
Shanel Raghubeer ◽  
Rajiv T. Erasmus ◽  
...  
Keyword(s):  
Receiver Operating Characteristic ◽  
Operating Characteristic ◽  
Characteristic Curve ◽  
Predictive Ability ◽  
Roc Curves ◽  
Tolerance Test ◽  
Cross Sectional Study ◽  
Oral Glucose ◽  
Cross Sectional ◽  
Receiver Operating

This cross-sectional study investigated the association of miR-1299, -126-3p and -30e-3p with and their diagnostic capability for dysglycaemia in 1273 (men, n = 345) South Africans, aged >20 years. Glycaemic status was assessed by oral glucose tolerance test (OGTT). Whole blood microRNA (miRNA) expressions were assessed using TaqMan-based reverse transcription quantitative-PCR (RT-qPCR). Receiver operating characteristic (ROC) curves assessed the ability of each miRNA to discriminate dysglycaemia, while multivariable logistic regression analyses linked expression with dysglycaemia. In all, 207 (16.2%) and 94 (7.4%) participants had prediabetes and type 2 diabetes mellitus (T2DM), respectively. All three miRNAs were significantly highly expressed in individuals with prediabetes compared to normotolerant patients, p < 0.001. miR-30e-3p and miR-126-3p were also significantly more expressed in T2DM versus normotolerant patients, p < 0.001. In multivariable logistic regressions, the three miRNAs were consistently and continuously associated with prediabetes, while only miR-126-3p was associated with T2DM. The ROC analysis indicated all three miRNAs had a significant overall predictive ability to diagnose prediabetes, diabetes and the combination of both (dysglycaemia), with the area under the receiver operating characteristic curve (AUC) being significantly higher for miR-126-3p in prediabetes. For prediabetes diagnosis, miR-126-3p (AUC = 0.760) outperformed HbA1c (AUC = 0.695), p = 0.042. These results suggest that miR-1299, -126-3p and -30e-3p are associated with prediabetes, and measuring miR-126-3p could potentially contribute to diabetes risk screening strategies.

scholarly journals Impulse oscillometry for detection of small airway dysfunction in subjects with chronic respiratory symptoms and preserved pulmonary function

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Liang-Yuan Li ◽  
Tian-Sheng Yan ◽  
Jing Yang ◽  
Yu-Qi Li ◽  
Lin-Xi Fu ◽  
...  
Keyword(s):  
Pulmonary Function ◽  
Respiratory Symptoms ◽  
Early Stage ◽  
Roc Curves ◽  
Diagnostic Value ◽  
Small Airway ◽  
Impulse Oscillometry ◽  
Additional Method ◽  
Cutoff Values ◽  
Higher Sensitivity

Abstract Background Subjects with chronic respiratory symptoms and preserved pulmonary function (PPF) may have small airway dysfunction (SAD). As the most common means to detect SAD, spirometry needs good cooperation and its reliability is controversial. Impulse oscillometry (IOS) may complete the deficiency of spirometry and have higher sensitivity. We aimed to explore the diagnostic value of IOS to detect SAD in symptomatic subjects with PPF. Methods The evaluation of symptoms, spirometry and IOS results in 209 subjects with chronic respiratory symptoms and PPF were assessed. ROC curves of IOS to detect SAD were analyzed. Results 209 subjects with chronic respiratory symptoms and PPF were included. Subjects who reported sputum had higher R5–R20 and Fres than those who didn’t. Subjects with dyspnea had higher R5, R5–R20 and AX than those without. CAT and mMRC scores correlated better with IOS parameters than with spirometry. R5, R5–R20, AX and Fres in subjects with SAD (n = 42) significantly increased compared to those without. Cutoff values for IOS parameters to detect SAD were 0.30 kPa/L s for R5, 0.015 kPa/L s for R5–R20, 0.30 kPa/L for AX and 11.23 Hz for Fres. Fres has the largest AUC (0.665, P = 0.001) among these parameters. Compared with spirometry, prevalence of SAD was higher when measured with IOS. R5 could detect the most SAD subjects with a prevalence of 60.77% and a sensitivity of 81% (AUC = 0.659, P = 0.002). Conclusion IOS is more sensitive to detect SAD than spirometry in subjects with chronic respiratory symptoms and PPF, and it correlates better with symptoms. IOS could be an additional method for SAD detection in the early stage of diseases.

scholarly journals Diagnosis of left ventricular hypertrophy using non-ECG-gated 15O-water PET

2021 ◽  
Author(s):  
Jens Sörensen ◽  
Jonny Nordström ◽  
Tomasz Baron ◽  
Stellan Mörner ◽  
Sven-Olof Granstam ◽  
...  
Keyword(s):  
Method Development ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Roc Curves ◽  
Left Ventricular ◽  
Intraclass Correlation Coefficients ◽  
Concentric Hypertrophy ◽  
2D Echocardiography ◽  
Positron Emission ◽  
Septal Wall Thickness

Abstract Aim To develop a method for diagnosing left ventricular (LV) hypertrophy from cardiac perfusion 15O-water positron emission tomography (PET). Methods We retrospectively pooled data from 139 subjects in four research cohorts. LV remodeling patterns ranged from normal to severe eccentric and concentric hypertrophy. 15O-water PET scans (n = 197) were performed with three different PET devices. A low-end scanner (66 scans) was used for method development, and remaining scans with newer devices for a blinded evaluation. Dynamic data were converted into parametric images of perfusable tissue fraction for semi-automatic delineation of the LV wall and calculation of LV mass (LVM) and septal wall thickness (WT). LVM and WT from PET were compared to cardiac magnetic resonance (CMR, n = 47) and WT to 2D-echocardiography (2DE, n = 36). PET accuracy was tested using linear regression, Bland–Altman plots, and ROC curves. Observer reproducibility were evaluated using intraclass correlation coefficients. Results High correlations were found in the blinded analyses (r ≥ 0.87, P < 0.0001 for all). AUC for detecting increased LVM and WT (> 12 mm and > 15 mm) was ≥ 0.95 (P < 0.0001 for all). Reproducibility was excellent (ICC ≥ 0.93, P < 0.0001). Conclusion 15O-water PET might detect LV hypertrophy with high accuracy and precision.

scholarly journals TIRADS, SRE and SWE in INDETERMINATE thyroid nodule characterization: Which has better diagnostic performance?

2021 ◽  
Author(s):  
Ilaria Celletti ◽  
Daniele Fresilli ◽  
Corrado De Vito ◽  
Marco Bononi ◽  
Sara Cardaccio ◽  
...  
Keyword(s):  
Thyroid Nodule ◽  
Preoperative Staging ◽  
Shear Wave Elastography ◽  
Roc Curves ◽  
Strain Ratio ◽  
Multicenter Studies ◽  
Increased Sensitivity ◽  
Bonferroni Test ◽  
Indeterminate Cytology ◽  
Sensitivity Specificity

Abstract Purpose To assess Strain Ratio (SRE) and Shear Wave Elastography (SWE) accuracy alone and with TIRADS classification, for the risk stratification of indeterminate thyroid nodules. Materials and methods 128 Patients with 128 indeterminate nodules candidates for thyroidectomy underwent preoperative staging neck ultrasound and were classified according to K-TIRADS score. After TIRADS evaluation, semi-quantitative (SRE) and quantitative (SWE expressed in kPa) elastosonography were performed and relative diagnostic performances, alone and in combination, were compared through ROC curves analysis. In order to maximize the SRE and SWE sensitivity and specificity, their cut-off values were calculated using the Liu test. Bonferroni test was used to evaluate statistically significant differences with a p value < 0.05. Results Sensitivity, specificity, PPV and NPV were, respectively, 71.4%, 82.4%, 62.5%, 87.5% for K-TIRADS baseline US, 85.7%, 94.1%, 85.7%, 94.1% for SRE and 57.1%, 79.4%, 53.3%, 81.8% for SWE (kPa expressed). SRE evaluation showed the best diagnostic accuracy compared to the SWE (kPa expressed) (p < 0.05) and to the K-TIRADS (p > 0.05). The association of SRE with conventional ultrasound with K-TIRADS score increased sensitivity (92.9% vs 71.4%) but decreased the specificity than conventional US alone (76.5% vs 82.4%). Conclusion Strain Elastosonography can be associated with K-TIRADS US examination in the thyroid nodule characterization with indeterminate cytology; in fact, adding the SRE to K-TIRADS assessment significantly increases its sensitivity and negative predictive value. However, further multicenter studies on larger population are warranted.

Sign in / Sign up

Export Citation Format

Share Document