Joint inference about sensitivity and specificity at the optimal cut-off point associated with Youden index

2014 ◽  
Vol 77 ◽  
pp. 1-13 ◽  
Author(s):  
Jingjing Yin ◽  
Lili Tian
2021 ◽  
Author(s):  
Zhongjing Jiang ◽  
Qile Gao ◽  
Mingxing Tang ◽  
Hongqi Zhang ◽  
Yanbing Li ◽  
...  

Abstract Background: The ability of T-SPOT.TB to differentiate Mycobacterium tuberculosis infection of the spine from other infections is little known. This study quantified the efficiency, sensitivity, and specificity of the T-SPOT.TB assay to distinguish between spinal tuberculosis (STB) caused by M. tuberculosis and other infections of the spine and evaluated whether diagnostic performance was improved by adjusting the T-SPOT.TB assay criteria. Methods: From January 2010 to May 2020, 147 patients with spinal infections were recruited. Peripheral blood mononuclear cells were collected, and the number of spot-forming cells was observed. Patients’ white blood cell (WBC) counts, C-reactive protein (CRP), erythrocyte sedimentation rate (ESR), procalcitonin (PCT), and TB antibodies were recorded. Specimen/tissue bacteriological culture was the reference standard for sensitivity and specificity. Results: There were 77 (52.4%) participants with confirmed TB and 70 (47.6%) with other infections. The groups were comparable in T-SPOT.TB assay results, age, sex, lesions in the segments, WBC count, CRP, procalcitonin, ESR, and TB antibodies. The sensitivity and specificity of the T-SPOT.TB assay for identifying STB was 88.3% and 40.0%, respectively. On the basis of Relative operating characteristic curve (ROC) analysis and the Youden index, when we adjusted the T-SPOT.TB assay’s diagnostic criteria, ESAT-6>12 or CFP-10>19,the sensitivity and specificity of the T-SPOT.TB assay for identifying STB was 83.1% and 64.3%, respectively. Conclusion: The T-SPOT.TB assay has great sensitivity to distinguish STB from other spinal infections; however, the specificity is extremely low. Specificity can be significantly improved while sensitivity is guaranteed by adjusting the diagnostic criteria.


EP Europace ◽  
2020 ◽  
Vol 22 (Supplement_1) ◽  
Author(s):  
S Raab ◽  
L Roten ◽  
M Branca ◽  
N Nozica ◽  
M Wilhelm ◽  
...  

Abstract Background Structural disarray of hypertrophied myocytes and interstitial fibrosis characterize hypertrophic cardiomyopathy (HCM). These morphological changes also affect atrial myocytes and, together with hemodynamic alterations because of HCM, may lead to atrial cardiomyopathy.  Purpose To investigate the incremental value of P-wave parameters to differentiate left ventricular hypertrophy (LVH) because of HCM from LVH in hypertensive heart disease (HHD) and athletes heart.  Methods In a prospective study, we compared electrocardiographic (including signal-averaged ECG of the P wave) and echocardiographic data of patients with HCM, HHD and athletes heart. We developed a predictive model with a simple scoring system to identify HCM. Results We compared data of 27 patients with HCM (70% males, 49.8 ± 14.5 years), 324 patients with HHD (52% males, 74.8 ± 5.5 years), and 215 subjects with athletes heart (72% males, 42.3 ± 7.5). The table shows the significant differences among the 3 groups. We included the following parameters into a predictive score to differentiate HCM from other forms of LVH: QRS width (>88ms = 1 point), P-wave integral (>688µVs = 1 point) and septum thickness (>12mm = 2 points). A score >2 (Youden index 0.626) correctly classified HCM in 81% of the cases with a sensitivity and specificity of 82% an 81%, respectively.  Conclusion Differentiation of HCM from other forms of LVH is improved by including atrial parameters. A simple scoring system including septum thickness, QRS width and P wave integral allowed identification of patients with HCM with a sensitivity and specificity of >80%. This score needs to be validated prospectively. Table 1 HCM HHD Athletes P-value HCM vs HHD* HCM vs Athletes* 95%-CI P-value 95%-CI P-value P-wave duration [ms] 152.7 ± 25.8 143.9 ± 16.5 133.5 ± 14.2 <0.001 -16.9 -24.6 to -9.1 <0.001 -16.3 -22.7 to -9.9 <0.001 P-wave integral [µVs] 850.4 ± 272.4 672.0 ± 235.4 773.1 ± 260.1 <0.001 -198.6 -320.8 to -76.3 0.002 -68.2 -169.7 to 33.2 0.187 QRS [ms] 110.3 ± 27.3 96.9 ± 20.3 95.1 ± 9.8 <0.001 -16.4 -24.7 to -8.1 <0.001 -13.8 -20.8 to -6.9 <0.001 QTc [ms] 447.9 ± 27.2 438.6 ± 24.5 414.0 ± 22.9 <0.001 -21.1 -32.7 to -9.5 <0.001 -30.8 -40.5 to -21.2 <0.001 LVMMI [g/m2] 153.6 ± 55.5 133.5 ± 30.3 98.6 ± 19.7 <0.001 -15.3 -29.7 to -0.9 0.038 -56.1 -67.7 to -44.6 <0.001 IVS [ms] 16.8 ± 4.2 11.8 ± 2.2 10.3 ± 1.5 <0.001 -5.2 -6.3 to -4.1 <0.001 -6.4 -7.3 to -5.6 <0.001 LAVI [ml/m2] 43.2 ± 13.9 30.5 ± 9.7 30.8 ± 9.5 <0.001 -14.6 -20.0 to -9.3 <0.001 -12.2 -16.6 to -7.9 <0.001 The table shows the study result after univariate and multivariate (*; adjusting for age and sex) analysis. Abstract Figure 1


2013 ◽  
Vol 17 (4) ◽  
pp. 861-869 ◽  
Author(s):  
Carolina Avila Vianna ◽  
Rogério da Silva Linhares ◽  
Renata Moraes Bielemann ◽  
Eduardo Coelho Machado ◽  
David Alejandro González-Chica ◽  
...  

AbstractObjectiveTo evaluate the adequacy and accuracy of cut-off values currently recommended by the WHO for assessment of cardiovascular risk in southern Brazil.DesignPopulation-based study aimed at determining the predictive ability of waist circumference for cardiovascular risk based on the use of previous medical diagnosis for hypertension, diabetes mellitus and/or dyslipidaemia. Descriptive analysis was used for the adequacy of current cut-off values of waist circumference, receiver operating characteristic curves were constructed and the most accurate criteria according to the Youden index and points of optimal sensitivity and specificity were identified.SettingPelotas, southern Brazil.SubjectsIndividuals (n2112) aged ≥20 years living in the city were selected by multistage sampling, since these individuals did not report the presence of previous myocardial infarction, angina pectoris or stroke.ResultsThe cut-off values currently recommended by WHO were more appropriate in men than women, with overestimation of cardiovascular risk in women. The area under the receiver operating characteristic curve showed moderate predictive ability of waist circumference in men (0·74, 95 % CI 0·71, 0·76) and women (0·75, 95 % CI 0·73, 0·77). The method of optimal sensitivity and specificity showed better performance in assessing the accuracy, identifying the values of 95 cm in men and 87 cm in women as the best cut-off values of waist circumference to assess cardiovascular risk.ConclusionsThe cut-off values currently recommended for waist circumference are not suitable for women. Longitudinal studies should be conducted to evaluate the consistency of the findings.


2015 ◽  
Vol 26 (3) ◽  
pp. 1429-1442 ◽  
Author(s):  
Leonidas E Bantis ◽  
Christos T Nakas ◽  
Benjamin Reiser ◽  
Daniel Myall ◽  
John C Dalrymple-Alford

The three-class approach is used for progressive disorders when clinicians and researchers want to diagnose or classify subjects as members of one of three ordered categories based on a continuous diagnostic marker. The decision thresholds or optimal cut-off points required for this classification are often chosen to maximize the generalized Youden index (Nakas et al., Stat Med 2013; 32: 995–1003). The effectiveness of these chosen cut-off points can be evaluated by estimating their corresponding true class fractions and their associated confidence regions. Recently, in the two-class case, parametric and non-parametric methods were investigated for the construction of confidence regions for the pair of the Youden-index-based optimal sensitivity and specificity fractions that can take into account the correlation introduced between sensitivity and specificity when the optimal cut-off point is estimated from the data (Bantis et al., Biomet 2014; 70: 212–223). A parametric approach based on the Box–Cox transformation to normality often works well while for markers having more complex distributions a non-parametric procedure using logspline density estimation can be used instead. The true class fractions that correspond to the optimal cut-off points estimated by the generalized Youden index are correlated similarly to the two-class case. In this article, we generalize these methods to the three- and to the general k-class case which involves the classification of subjects into three or more ordered categories, where ROC surface or ROC manifold methodology, respectively, is typically employed for the evaluation of the discriminatory capacity of a diagnostic marker. We obtain three- and multi-dimensional joint confidence regions for the optimal true class fractions. We illustrate this with an application to the Trail Making Test Part A that has been used to characterize cognitive impairment in patients with Parkinson’s disease.


2020 ◽  
Author(s):  
Yuri Kiso ◽  
Yoko Matsuda ◽  
Shikine Esaka ◽  
Yuri Hamashima ◽  
Hiroto Shirahata ◽  
...  

Abstract Background: Cytological diagnosis of pancreatic specimens obtained by endoscopic ultrasound-guided fine needle aspiration (EUS-FNA) is often challenging because of the small amount of sample or well-differentiated adenocarcinoma with weak cytological atypia. Therefore, the sensitivity and specificity of cytological diagnosis for pancreatic cancer should be improved. Hence, we aimed to evaluate the indices, which are used to distinguish malignant from benign lesions for the cytological diagnosis of pancreatic EUS-FNA specimens. Methods: Seven reviewers, including 3 cytotechnologists and 4 medical doctors, evaluated 20 morphological indices in pancreatic specimens obtained by EUS-FNA (malignant, n=111; benign, n=31). Statistical analyses were performed using Fisher’s exact test, logistic regression analysis, area under the receiver operating characteristic curve, and Youden Index.Results: Among the 20 indices, there was a high incidence rate (>40%) of the following 13 indices in malignant cases: structural atypia, hyperchromatic nucleus, irregular cell polarity, unclear cell boundary, nuclear membrane thickening, anisonucleosis, overlapping, irregular nuclei, high nuclear/cytoplasmic ratio, binding decline, simultaneous appearance of malignant and benign cells, enlarged nucleoli, and background necrosis. When we diagnosed pancreatic specimens using these 13 cytological indices, the cutoff value of 8/9 showed the highest Youden index (0.950) as well as high sensitivity and specificity in distinguishing malignant from benign specimens (98% and 97%, respectively). Conclusion: Thirteen cytological indices showed high sensitivity and specificity in differentiating malignant and benign lesions using pancreatic EUS-FNA samples. Further validation or prospective studies are necessary to establish criteria for the cytological diagnosis of pancreatic cancer.


2021 ◽  
Author(s):  
Shubin Li ◽  
Yang Gao ◽  
Mingqiu Hu ◽  
Yuehuan Li ◽  
Xu Meng

Abstract Background:This study aimed to evaluate the role of SAMe-TT2R2 score in the prediction of anticoagulation control after mechanical mitral valve replacement.Methods and Results:We retrospectively reviewed clinical data of 161 patients who received mechanical mitral valve replacement at Beijing Anzhen Hospital from January 2013 to December 2013. Collected data included general information of patients, medication and smoking, postoperative embolism due to anticoagulant, bleeding complications and death information. In the SAMe-TT2R2 score results, the lowest score was 2 points (6.3%), the highest score was 7 points (0.7%). The number of people with 4 points was the largest ,60 people (41.4%).When the cut-off value of SAMe-TT2R2 score was set to ≥4, the sensitivity and specificity of predicting TTR≥65% were 69.8% and 93.1%, respectively. The Youden index was 0.629. If the cut-off value of SAMe-TT2R2 score was set to ≤4, the sensitivity and specificity of predicting TTR≥65% were 93.0% and 44.1%, respectively, and the Youden index was 0.371.ROC curve evaluates the predictive power of the SAMe-TT2R2 score for TTR≥65%. The figure showed that when the cut-off point≥4, the best combination of sensitivity and specificity was shown (69.8% and 93.1%, respectively).The area under the curve AUC was 0.854.Conclusions:After mechanical mitral valve replacement, the SAME-TT2R2 model can effectively predict the level of TTR during the course of using oral warfarin anticoagulation, and the SAMe-TT2R2 score ≥4 can be used to predict TTR<65%.


2021 ◽  
Vol 108 (Supplement_4) ◽  
Author(s):  
M Schneider ◽  
I Labgaa ◽  
D Vrochides ◽  
A Zerbi ◽  
G Nappo ◽  
...  

Abstract Objective Lymph node ratio (LNR, positive lymph nodes/collected lymph nodes during surgery) was identified as an important prognostic factor of survival in resected pancreatic cancer. Several nomograms based on LNR were recently proposed to predict survival after pancreatoduodenectomy (PD). The present study aimed to externally validate 3 published nomograms using an international cohort. Methods Consecutive patients with ductal adenocarcinoma of the pancreatic head who underwent PD without neoadjuvant treatment from 6 tertiary centers in Europe and the USA were retrospectively collected from 2000 to 2017. Patients with metastases at diagnosis, R2 resection, missing data regarding LNR, and who died within 90 postoperative days were excluded. The 3 selected nomograms were the updated Amsterdam nomogram (including LNR, adjuvant therapy, margin status, and tumor grade), the nomogram by Pu et al. (including LNR, age, tumor grade, and T stage) and the nomogram by Li et al. (including LNR, age, tumor location, grade, size, and TNM stage). Overall survivals (OS) were calculated using Kaplan-Meier method. For the validation, calibration (Hosmer-Lemeshow test), discrimination capacity (ROC curves for 3-year OS), and clinical utility (sensitivity and specificity at the value of Youden index) were assessed. Results After exclusion of 95 patients with metastases, R2 resection, and who died within 90 postoperative days, 1167 patients were included. Median OS of the entire cohort was 23 months (95% confidence interval: 21-24). For the 3 nomograms, Kaplan-Meier curves showed significant diminution of OS with increasing scores (p &lt; 0.01 for the 3 nomograms). All nomograms showed good calibration (non significant Hosmer-Lemeshow goodness-of-fit tests). For the updated Amsterdam nomogram, the area under the ROC curve (AUROC) for 3-year OS was 0.66. Sensitivity and specificity were 73% and 50%. Regarding the nomogram by Pu et al., the AUROC was 0.67. Sensitivity and specificity were 65% and 60%. For the nomogram by Li et al., the AUROC was 0.67, while sensitivity and specificity were 56% and 71%. Conclusion The 3 selected nomograms were validated using an external international cohort and displayed interesting and comparable predictive values. Those nomograms may be used in clinical practice to estimate survival after PD for ductal adenocarcinoma.


2007 ◽  
Vol 215 (2) ◽  
pp. 122-131 ◽  
Author(s):  
Heinz Holling ◽  
Dankmar Böhning ◽  
Walailuck Böhning

Abstract. This paper considers meta-analysis of binary data that use a dichotomized continuous score. Classification into two categories, e.g., qualified or not qualified, is often based upon a threshold or cut-off value. This threshold might vary between studies since intentionally different values are used. However, conventional meta-analysis methodology analyzing sensitivity and specificity separately might then be confounded by a potentially unknown variation of the cut-off value. In order to cope with varying thresholds, an overall estimate of the misclassification error is suggested instead, which is equivalent to the well-known Youden index. It is argued that this index is less prone to between-study variation of cut-off values. To adjust for potential study effects a Mantel-Haenszel estimator of the overall misclassification error is suggested. Arguments are illustrated using, as an example, the diagnosis of alcoholism using the Alcohol Use Disorders Identification Test (AUDIT).


2021 ◽  
Vol 19 (1) ◽  
Author(s):  
Simon T. Schäfer ◽  
Anne-Christine Otto ◽  
Alice-Christin Acevedo ◽  
Klaus Görlinger ◽  
Steffen Massberg ◽  
...  

Abstract Background DOAC detection is challenging in emergency situations. Here, we demonstrated recently, that modified thromboelastometric tests can reliably detect and differentiate dabigatran and rivaroxaban. However, whether all DOACs can be detected and differentiated to other coagulopathies is unclear. Therefore, we now tested the hypothesis that a decision tree-based thromboelastometry algorithm enables detection and differentiation of all direct Xa-inhibitors (DXaIs), the direct thrombin inhibitor (DTI) dabigatran, as well as vitamin K antagonists (VKA) and dilutional coagulopathy (DIL) with high accuracy. Methods Following ethics committee approval (No 17–525-4), and registration by the German clinical trials database we conducted a prospective observational trial including 50 anticoagulated patients (n = 10 of either DOAC/VKA) and 20 healthy volunteers. Blood was drawn independent of last intake of coagulation inhibitor. Healthy volunteers served as controls and their blood was diluted to simulate a 50% dilution in vitro. Standard (extrinsic coagulation assay, fibrinogen assay, etc.) and modified thromboelastometric tests (ecarin assay and extrinsic coagulation assay with low tissue factor) were performed. Statistical analyzes included a decision tree analyzes, with depiction of accuracy, sensitivity and specificity, as well as receiver-operating-characteristics (ROC) curve analysis including optimal cut-off values (Youden-Index). Results First, standard thromboelastometric tests allow a good differentiation between DOACs and VKA, DIL and controls, however they fail to differentiate DXaIs, DTIs and VKAs reliably resulting in an overall accuracy of 78%. Second, adding modified thromboelastometric tests, 9/10 DTI and 28/30 DXaI patients were detected, resulting in an overall accuracy of 94%. Complex decision trees even increased overall accuracy to 98%. ROC curve analyses confirm the decision-tree-based results showing high sensitivity and specificity for detection and differentiation of DTI, DXaIs, VKA, DIL, and controls. Conclusions Decision tree-based machine-learning algorithms using standard and modified thromboelastometric tests allow reliable detection of DTI and DXaIs, and differentiation to VKA, DIL and controls. Trial registration Clinical trial number: German clinical trials database ID: DRKS00015704.


2020 ◽  
Vol 79 (Suppl 1) ◽  
pp. 899.2-899
Author(s):  
R. Pineda-Sic ◽  
D. Vega-Morales ◽  
L. Santoyo-Fexas ◽  
M. M. Castañeda-Martínez ◽  
I. D. J. Hernandez-Galarza ◽  
...  

Background:Rheumatoid Factor (RF) is used for the classification of Rheumatoid arthritis (RA) in patients whose main complaint is hand arthritis but is well known that other conditions have positive RF. Differences in analytical performance between assays were found in previous studies, there was a poor numerical agreement, and using the cut-off proposed by the manufacturer, variation in sensitivity and specificity can be seen.2Objectives:To determine the cut-off value for RF with the best diagnostic performance for RA in patients with hand arthritis compared with their main differential diagnoses.Methods:Descriptive, cross-sectional, diagnostic test study. Sampling size was calculated 2:1, with the area under the ROC curve (AUC): 0.708, power 80% and significance 0.05. Patients with RA, Systemic Lupus Erythematosus (SLE), Osteoarthritis (OA), fibromyalgia (FBM), Polyarticular Gout (PG), Psoriatic Arthritis(Psa), Sjögren Syndrome (SS), Arthralgia suspicious for progression to RA (ASP) and healthy subjects, previously classified with the current criteria by certified rheumatologist were included. EUROINMMUN ® assays for RF isotypes IgA, IgM, IgG were used. ROC curves and Youden index for determining the cut-off with the best sensitivity and specificity for each RF isotype comparing RA with other diseases were used. Statistical analyses were performed with SPSS V.24.Results:Three hundred and thirty-two serum samples from patients were included, in the other rheumatic diseases different from RA, we found positive RF IgM in 141 (45.5%), 82 (26.5%) IgA and 25 (8.1%) IgG. For each clinical scenario the cut-off with the best sensitivity and specificity was different from the manufacturer proposed, and isotypes with the best diagnostic performance was not the same for each disease, Table 1. We present the ROC curve for RF isotypes in Figure 1.Conclusion:In order to improve the diagnosis of Rheumatoid Arthritis, different cut-off points of RF and isotypes should be used.References:[1] Van Hoovels L, Jacobs J, Vander Cruyssen B, Van Den Bremt S, Verschueren P, Bossuyt X. Performance characteristics of rheumatoid factor and anti-cyclic citrullinated peptide antibody assays may impact ACR/EULAR classification of rheumatoid arthritis. Ann Rheum Dis. 2018;77(5):667-677. doi:10.1136/annrheumdis-2017-212365[2] Van Boekel MAM, Vossenaar ER, Van Den Hoogen FHJ, Van Venrooij WJ. Autoantibody systems in rheumatoid arthritis: Specificity, sensitivity and diagnostic value. Arthritis Res. 2002;4(2):87-93. doi:10.1186/ar395.Disclosure of Interests:None declared


Sign in / Sign up

Export Citation Format

Share Document