Evaluation metrics for bone segmentation in ultrasound

2015 ◽  
Author(s):  
Matthew Lougheed ◽  
Gabor Fichtinger ◽  
Tamas Ungi
Keyword(s):  
Evaluation Metrics ◽  
Bone Segmentation

scholarly journals Preliminary Planning for a Multi-institutional Database for Ultrasound Bone Segmentation

10.29007/m1ll ◽  
2019 ◽  
Author(s):  
Prashant Pandey ◽  
Hridayi Patel ◽  
Pierre Guy ◽  
Ilker Hacihalilogu ◽  
Antony J. Hodgson
Keyword(s):  
International Collaboration ◽  
Image Database ◽  
Evaluation Metrics ◽  
Computer Assisted ◽  
Bone Surface ◽  
Bone Segmentation ◽  
Preliminary Planning ◽  
Research Centres ◽  
Ultrasound Bone

Ultrasound (US) bone segmentation is a key component in many US-based computer assisted orthopaedic systems. Although numerous US bone segmentations techniques exist, there remains no direct way of comparing their performances. This is primarily due to the lack of an accessible US bone image database, and secondly due to a lack of standard vali- dation practices. To address this issue, we are beginning a multi-institutional international collaboration across multiple research centres with the aim of creating an open database for US bone segmentation consisting of several thousand US images and corresponding bone surface segmentations. Our collaboration also aims to address outstanding issues in US bone segmentation, such as determining the reliability of manual segmentations and establishing a set of evaluation metrics which should be reported in future segmentation studies. Finally, we strongly encourage interested researchers to join and contribute to this project as this will help to create a more diverse database and knowledgeable collaboration.

Artificial Intelligence for Skin Cancer Detection: A Scoping Review (Preprint)

2020 ◽  
Author(s):  
Abdulrahman Takiddin ◽  
Jens Schneider ◽  
Yin Yang ◽  
Alaa Abd-Alrazaq ◽  
Mowafa Househ
Keyword(s):  
Artificial Intelligence ◽  
Skin Cancer ◽  
Scoping Review ◽  
Performance Metrics ◽  
Data Extraction ◽  
Characteristic Curve ◽  
Evaluation Metrics ◽  
Cancer Type ◽  
Computer Algorithms ◽  
Scoping Reviews

BACKGROUND Skin cancer is the most common cancer type affecting humans. Traditional skin cancer diagnosis methods are costly, require a professional physician, and take time. Hence, to aid in diagnosing skin cancer, Artificial Intelligence (AI) tools are being used, including shallow and deep machine learning-based techniques that are trained to detect and classify skin cancer using computer algorithms and deep neural networks. OBJECTIVE The aim of this study is to identify and group the different types of AI-based technologies used to detect and classify skin cancer. The study also examines the reliability of the selected papers by studying the correlation between the dataset size and number of diagnostic classes with the performance metrics used to evaluate the models. METHODS We conducted a systematic search for articles using IEEE Xplore, ACM DL, and Ovid MEDLINE databases following the PRISMA Extension for Scoping Reviews (PRISMA-ScR) guidelines. The study included in this scoping review had to fulfill several selection criteria; to be specifically about skin cancer, detecting or classifying skin cancer, and using AI technologies. Study selection and data extraction were conducted by two reviewers independently. Extracted data were synthesized narratively, where studies were grouped based on the diagnostic AI techniques and their evaluation metrics. RESULTS We retrieved 906 papers from the 3 databases, but 53 studies were eligible for this review. While shallow techniques were used in 14 studies, deep techniques were utilized in 39 studies. The studies used accuracy (n=43/53), the area under receiver operating characteristic curve (n=5/53), sensitivity (n=3/53), and F1-score (n=2/53) to assess the proposed models. Studies that use smaller datasets and fewer diagnostic classes tend to have higher reported accuracy scores. CONCLUSIONS The adaptation of AI in the medical field facilitates the diagnosis process of skin cancer. However, the reliability of most AI tools is questionable since small datasets or low numbers of diagnostic classes are used. In addition, a direct comparison between methods is hindered by a varied use of different evaluation metrics and image types.

scholarly journals An empirical evaluation of cost-based federated SPARQL query processing engines

Semantic Web ◽  
2021 ◽  
pp. 1-26
Author(s):  
Umair Qudus ◽  
Muhammad Saleem ◽  
Axel-Cyrille Ngonga Ngomo ◽  
Young-Koo Lee
Keyword(s):  
Query Processing ◽  
Detailed Analysis ◽  
Performance Metrics ◽  
Empirical Evaluation ◽  
Sparql Query ◽  
Evaluation Metrics ◽  
Future Cost ◽  
Query Plan ◽  
Fine Grained ◽  
Runtime Performance

Finding a good query plan is key to the optimization of query runtime. This holds in particular for cost-based federation engines, which make use of cardinality estimations to achieve this goal. A number of studies compare SPARQL federation engines across different performance metrics, including query runtime, result set completeness and correctness, number of sources selected and number of requests sent. Albeit informative, these metrics are generic and unable to quantify and evaluate the accuracy of the cardinality estimators of cost-based federation engines. To thoroughly evaluate cost-based federation engines, the effect of estimated cardinality errors on the overall query runtime performance must be measured. In this paper, we address this challenge by presenting novel evaluation metrics targeted at a fine-grained benchmarking of cost-based federated SPARQL query engines. We evaluate five cost-based federated SPARQL query engines using existing as well as novel evaluation metrics by using LargeRDFBench queries. Our results provide a detailed analysis of the experimental outcomes that reveal novel insights, useful for the development of future cost-based federated SPARQL query processing engines.

Sign in / Sign up

Export Citation Format

Share Document