Increasing the Reliability of Reliability Diagrams

2007 ◽  
Vol 22 (3) ◽  
pp. 651-661 ◽  
Author(s):  
Jochen Bröcker ◽  
Leonard A. Smith
Keyword(s):  
Quantitative Evaluation ◽  
Visual Evaluation ◽  
Probability Paper ◽  
Reliability Diagram ◽  
Probabilistic Forecasts ◽  
Resampling Method

Abstract The reliability diagram is a common diagnostic graph used to summarize and evaluate probabilistic forecasts. Its strengths lie in the ease with which it is produced and the transparency of its definition. While visually appealing, major long-noted shortcomings lie in the difficulty of interpreting the graph visually; for the most part, ambiguities arise from variations in the distributions of forecast probabilities and from various binning procedures. A resampling method for assigning consistency bars to the observed frequencies is introduced that allows for immediate visual evaluation as to just how likely the observed relative frequencies are under the assumption that the predicted probabilities are reliable. Further, an alternative presentation of the same information on probability paper eases quantitative evaluation and comparison. Both presentations can easily be employed for any method of binning.

scholarly journals Visual and Quantitative Evaluation of Amyloid Brain PET Image Synthesis with Generative Adversarial Network

2020 ◽  
Vol 10 (7) ◽  
pp. 2628 ◽  
Author(s):  
Hyeon Kang ◽  
Jang-Sik Park ◽  
Kook Cho ◽  
Do-Young Kang
Keyword(s):  
Quantitative Evaluation ◽  
Data Augmentation ◽  
Image Synthesis ◽  
Turing Test ◽  
Training Data ◽  
Generative Adversarial Network ◽  
Visual Evaluation ◽  
Image Distribution ◽  
Adversarial Network ◽  
Quantitative Measurements

Conventional data augmentation (DA) techniques, which have been used to improve the performance of predictive models with a lack of balanced training data sets, entail an effort to define the proper repeating operation (e.g., rotation and mirroring) according to the target class distribution. Although DA using generative adversarial network (GAN) has the potential to overcome the disadvantages of conventional DA, there are not enough cases where this technique has been applied to medical images, and in particular, not enough cases where quantitative evaluation was used to determine whether the generated images had enough realism and diversity to be used for DA. In this study, we synthesized 18F-Florbetaben (FBB) images using CGAN. The generated images were evaluated using various measures, and we presented the state of the images and the similarity value of quantitative measurement that can be expected to successfully augment data from generated images for DA. The method includes (1) conditional WGAN-GP to learn the axial image distribution extracted from pre-processed 3D FBB images, (2) pre-trained DenseNet121 and model-agnostic metrics for visual and quantitative measurements of generated image distribution, and (3) a machine learning model for observing improvement in generalization performance by generated dataset. The Visual Turing test showed similarity in the descriptions of typical patterns of amyloid deposition for each of the generated images. However, differences in similarity and classification performance per axial level were observed, which did not agree with the visual evaluation. Experimental results demonstrated that quantitative measurements were able to detect the similarity between two distributions and observe mode collapse better than the Visual Turing test and t-SNE.

scholarly journals A Combined Pulmonary-Radiology Workshop for Visual Evaluation of COPD: Study Design, Chest CT Findings and Concordance with Quantitative Evaluation

2012 ◽  
Vol 9 (2) ◽  
pp. 151-159 ◽  
Author(s):  
COPDGene CT Workshop Group: R. Grah Barr ◽  
Eugene A. Berkowitz ◽  
Francesca Bigazzi ◽  
Frederick Bode ◽  
Jessica Bon ◽  
...  
Keyword(s):  
Quantitative Evaluation ◽  
Study Design ◽  
Chest Ct ◽  
Visual Evaluation ◽  
Ct Findings

scholarly journals Feasibility study of amyloid PET-only quantification; a comparison with visual interpretation.

2020 ◽  
Author(s):  
Natsumi Shimokawa ◽  
Go Akamatsu ◽  
Miyako Kadosaki ◽  
Masayuki Sasaki
Keyword(s):  
Alzheimer’S Disease ◽  
Alzheimer's Disease ◽  
Quantitative Evaluation ◽  
Quantitative Methods ◽  
Standardized Uptake Value ◽  
Amyloid Pet ◽  
Visual Interpretation ◽  
Visual Evaluation ◽  
Quantification Method ◽  
Sensitivity Specificity

Abstract Objective Visual evaluation is the standard for amyloid positron emission tomography (PET) examination, though the result depends upon the physician’s subjective review of the images. Therefore, objective quantitative evaluation is expected to be useful for image interpretation. In this study, we examined the usefulness of the quantitative evaluation of amyloid PET using a PET-only quantification method in comparison with visual evaluation. Methods A total of 166 individuals, including 58 cognitively normal controls, 62 individuals with mild cognitive impairment, and 46 individuals with early Alzheimer’s disease, were retrospectively investigated. They underwent 11C-Pittsburgh compound-B (PiB) PET examination through the Japanese-Alzheimer’s disease neuroimaging initiative (J-ADNI). Amyloid accumulation in cerebral cortices was assessed using visual and quantitative methods. The quantitative evaluation was performed using the adaptive template method and empirically PiB-prone region of interest, and the standardized uptake value ratio (SUVR) in each area was obtained. Results Visual evaluation and SUVR were significantly correlated in the cerebral cortices (ρ = 0.85–0.87; p < 0.05). In visual evaluation, the sensitivity, specificity, and accuracy were 78%, 76%, and 77%, respectively. Meanwhile, for quantitative evaluation, the sensitivity, specificity, and accuracy were 78%, 76%, and 77%, in mean cortical SUVR (mcSUVR) and 79%, 79%, and 79% in maximum SUVR (maxSUVR), respectively. Conclusion The PET-only quantification method resulted in a concordant result with visual evaluation and was considered to be useful for amyloid PET.

Differentiation of Renal Oncocytoma From Renal Cell Carcinoma Using 99mTc-Sestamibi SPECT/CT.

2021 ◽  
Author(s):  
Antonios Tzortzakakis ◽  
Thomas Papathomas ◽  
Ove Gustafsson ◽  
Stefan Gabrielson ◽  
Kiril Trpkov ◽  
...  
Keyword(s):  
Computed Tomography ◽  
Renal Cell Carcinoma ◽  
Cell Carcinoma ◽  
Quantitative Evaluation ◽  
Renal Cell ◽  
Renal Parenchyma ◽  
Renal Oncocytoma ◽  
Visual Evaluation ◽  
Ct Examination ◽  
Renal Tumours

Abstract Background: 99mTc-Sestamibi Single Photon Emission Computed Tomography/Computed Tomography (SPECT/CT) contributes to the non-invasive differentiation of renal oncocytoma (RO) from renal cell carcinoma (RCC). We investigated whether standard uptake value (SUV) SPECT, has a beneficial role in differentiating renal oncocytoma (RO) from renal cell carcinoma (RCC) besides visual assessment. As a secondary aim, we evaluated the mitochondrial content of 19 oncocytic tumours arranged in a tissue microarray, by immunohistochemistry, using succinate dehydrogenase complex subunit B (SDHB) protein expression. In addition to visual evaluation of 99mTc-Sestamibi uptake by characterizing renal tumours as Sestamibi positive or Sestamibi negative regarding their uptake compared to the non-tumoral renal parenchyma, SUVmean and SUVmax measurements were performed in the renal tumour and the non-tumoral renal parenchyma. Intra Class Correlation calculated to assess the intra-reader reliability of SUV measurements. ROC-analysis demonstrated an optimal cut off SUV value that differentiates RO from RCC. SDHB score was analysed using Cochran–Armitage test for trend.Results: 57 renal tumours from 52 patients were evaluated. Visual evaluation of 99mTc-Sestamibi SPECT/CT examination resulted in a sensitivity of 83%, whereas quantitative evaluation showed a sensitivity of 64% regarding the differential of RO from RCC. A significant trend (p=0,0328) of increased SDHB score found in the Sestamibi positive group.Conclusion: Quantitative evaluation with SUV SPECT measurements did not improve the performance of 99mTc-Sestamibi SPECT/CT in differentiating RO from RCC. 99mTc- Sestamibi SPECT/CT identified a larger Sestamibi-positive tumour group containing RO, Hybrid Oncocytic Chromophobe Tumours and the majority of chromophobe RCCs. Thus, Sestamibi-negative renal tumours, that are possibly malignant, should be considered for surgery. Patients with Sestamibi-positive tumours can be suited for biopsy and follow up according to active surveillance protocols. Low Grade Oncoytic Tumour, a provisional renal entity with no proven recurrence or metastatic potential, appears to be positive on 99mTc- Sestamibi SPECT/CT examination.

Knowledge and Skills for Evaluation of Velopharyngeal Physiology for Speech

2014 ◽  
Vol 24 (2) ◽  
pp. 67-76 ◽  
Author(s):  
Thomas Watterson ◽  
Lynn Marty Grames
Keyword(s):  
Visual Evaluation ◽  
Swallowing Function ◽  
Knowledge And Skills ◽  
Speech Language Pathologist ◽  
Speech Evaluation ◽  
The Relationship

The American Speech-Language-Hearing Association (ASHA) has developed Knowledge and Skills (KAS) recommendations for evaluation of the larynx and swallowing function but the evaluation of velopharyngeal (VP) function has never been addressed. This article will review previous documents that have addressed general endoscopic knowledge and skills and develop a case for a new KAS that specifically addresses visualization and evaluation of the VP mechanism. The new KAS document will delineate and explain the relationship between speech evaluation and visual evaluation of VP physiology. The unique skills required of the speech-language pathologist for this kind of evaluation will be discussed.

Quantitative evaluation of -ray spectra from scintillation measurements

Physica ◽  
1952 ◽  
Vol 18 (2) ◽  
pp. 1147-1150
Author(s):  
D MAEDER ◽  
V WINTERSTEIGER
Keyword(s):  
Quantitative Evaluation

Visual Evaluation of Computer-Generated Textures

1994 ◽  
Author(s):  
George A. Geri ◽  
Don R. Lyon ◽  
Yehoshua Y. Zeevi
Keyword(s):  
Visual Evaluation
Sign in / Sign up

Export Citation Format

Share Document