scholarly journals Robust Hierarchical Density Estimation and Regression for Re-stained Histological Whole Slide Image Co-registration

2019 ◽  
Author(s):  
Jun Jiang ◽  
Nicholas B. Larson ◽  
Naresh Prodduturi ◽  
Thomas J. Flotte ◽  
Steven N. Hart
Keyword(s):  
Image Registration ◽  
Density Estimation ◽  
Digital Pathology ◽  
Tissue Samples ◽  
Registration Errors ◽  
Slide Image ◽  
Specific Proteins ◽  
Histological Images ◽  
Hematoxylin And Eosin ◽  
Whole Slide Images

AbstractFor many disease conditions, tissue samples are colored with multiple dyes and stains to add contrast and location information for specific proteins to accurately identify and diagnose disease. This presents a computational challenge for digital pathology, as whole-slide images (WSIs) need to be properly overlaid (i.e. registered) to identify co-localized features. Traditional image registration methods sometimes fail due to the high variation of cell density and insufficient texture information in WSIs – particularly at high magnifications. In this paper, we proposed a robust image registration strategy to align re-stained WSIs precisely and efficiently. This method is applied to 30 pairs of immunohistochemical (IHC) stains and their hematoxylin and eosin (H&E) counterparts. Our approach advances the existing methods in three key ways. First, we introduce refinements to existing image registration methods. Second, we present an effective weighting strategy using kernel density estimation to mitigate registration errors. Third, we account for the linear relationship across WSI levels to improve accuracy. Our experiments show significant decreases in registration errors when on matching IHC and H&E pairs, enabling subcellular-level analysis on stained and re-stained histological images. We also provide a tool to allow users to develop their own registration benchmarking experiments.

scholarly journals A Digital Pathology Solution to Resolve the Tissue Floater Conundrum

2020 ◽  
Author(s):  
Liron Pantanowitz ◽  
Pamela Michelow ◽  
Scott Hazelhurst ◽  
Shivam Kalra ◽  
Charles Choi ◽  
...  
Keyword(s):  
Digital Pathology ◽  
Primary Diagnosis ◽  
Image Search ◽  
Molecular Tests ◽  
Tissue Identification ◽  
Digital Database ◽  
Glass Slides ◽  
Search Tool ◽  
Hematoxylin And Eosin ◽  
Whole Slide Images

Context.— Pathologists may encounter extraneous pieces of tissue (tissue floaters) on glass slides because of specimen cross-contamination. Troubleshooting this problem, including performing molecular tests for tissue identification if available, is time consuming and often does not satisfactorily resolve the problem. Objective.— To demonstrate the feasibility of using an image search tool to resolve the tissue floater conundrum. Design.— A glass slide was produced containing 2 separate hematoxylin and eosin (H&E)-stained tissue floaters. This fabricated slide was digitized along with the 2 slides containing the original tumors used to create these floaters. These slides were then embedded into a dataset of 2325 whole slide images comprising a wide variety of H&E stained diagnostic entities. Digital slides were broken up into patches and the patch features converted into barcodes for indexing and easy retrieval. A deep learning-based image search tool was employed to extract features from patches via barcodes, hence enabling image matching to each tissue floater. Results.— There was a very high likelihood of finding a correct tumor match for the queried tissue floater when searching the digital database. Search results repeatedly yielded a correct match within the top 3 retrieved images. The retrieval accuracy improved when greater proportions of the floater were selected. The time to run a search was completed within several milliseconds. Conclusions.— Using an image search tool offers pathologists an additional method to rapidly resolve the tissue floater conundrum, especially for those laboratories that have transitioned to going fully digital for primary diagnosis.

scholarly journals PathFlow-MixMatch for Whole Slide Image Registration: An Investigation of a Segment-Based Scalable Image Registration Method

2020 ◽  
Author(s):  
Joshua J. Levy ◽  
Christopher R. Jackson ◽  
Christian C. Haudenschild ◽  
Brock C. Christensen ◽  
Louis J. Vaickus
Keyword(s):  
Image Registration ◽  
Large Scale ◽  
Digital Pathology ◽  
Superior Performance ◽  
Registration Method ◽  
Target Registration Error ◽  
Alignment Algorithms ◽  
Slide Image ◽  
Segment Matching ◽  
The Impact

AbstractImage registration involves finding the best alignment between different images of the same object. In these tasks, the object in question is viewed differently in each of the images (e.g. different rotation or light conditions, etc.). In digital pathology, image registration aligns correspondent regions of tissue from different stereotactic viewpoints (e.g. subsequent deeper sections of the same tissue). These comparisons are important for histological analysis and can facilitate previously unavailable manipulations, such as 3D tissue reconstruction and cell-level alignment of immunohistochemical (IHC) and special stains. Several benchmarks have been established for evaluating image registration techniques for histological tissue; however, little work has evaluated the impact of scaling registration techniques to Giga-Pixel Whole Slide Images (WSI), which are large enough for significant memory limitations, and contain recurrent patterns and deformations that hinder traditional alignment algorithms. Furthermore, as tissue sections often contain multiple, discrete, smaller tissue fragments, it is unnecessary to align an entire image when the bulk of the image is background whitespace and tissue fragments’ orientations are often agnostic of each other. We present a methodology for circumventing large-scale image registration issues in histopathology and accompanying software. By removing background pixels, parsing the slide into discrete tissue segments, and matching, orienting and registering smaller segment pairs, we recovered registrations with lower Target Registration Error (TRE) when compared to utilizing the unmanipulated WSI. We tested our technique by having a pathologist annotate landmarks from 13 pairs of differently stained liver biopsy slides, performing WSI and segment-based registration techniques, and comparing overall TRE. Preliminary results demonstrate superior performance of registering segment pairs versus registering WSI (difference of median TRE of 44 pixels, p<0.001). Segment matching within WSI is an effective solution for histology image registration but requires further testing and validation to ensure its viability for stain translation and 3D histology analysis.

scholarly journals Tissue Type Recognition in Whole Slide Histological Images

2021 ◽  
Author(s):  
Alexander Khvostikov ◽  
Andrey Krylov ◽  
Ilya Mikhailov ◽  
Pavel Malkov ◽  
Natalya Danilova
Keyword(s):  
Digestive Tract ◽  
Lamina Propria ◽  
Gastric Adenocarcinoma ◽  
Digital Pathology ◽  
Tissue Type ◽  
Depth Of Invasion ◽  
Tubular Structures ◽  
Urgent Task ◽  
Histological Images ◽  
Whole Slide Images

Automatic layers recognition of the wall of the stomach and colon on whole slide images is an extremely urgent task in digital pathology as it can be used for automatic determining the depth of invasion of the digestive tract tumors. In this paper we propose a new CNN-based method of automatic tissue type recognition on whole slide histological images. We also describe an effective pipeline of training that uses 2 different training datasets. The proposed method of automatic tissue type recognition achieved 0.929 accuracy and 0.903 balanced accuracy on CRC-VAL-HE-7K dataset for 9-types classification and 0.98 accuracy and 0.926 balanced accuracy on the test subset of whole slide images from PATH-DT- MSU dataset for 5-types classification. The developed method makes it possible to classify the areas corresponding to the gastric own mucous glands in the lamina propria and also to distinguish the tubular structures of a highly differentiated gastric adenocarcinoma with normal glands.

scholarly journals CAD systems for colorectal cancer from WSI are still not ready for clinical acceptance

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Sara P. Oliveira ◽  
Pedro C. Neto ◽  
João Fraga ◽  
Diana Montezuma ◽  
Ana Monteiro ◽  
...  
Keyword(s):  
Colorectal Cancer ◽  
Imaging Techniques ◽  
Digital Pathology ◽  
Incidence Rates ◽  
Cancer Type ◽  
Tissue Samples ◽  
Digital Era ◽  
Processing Techniques ◽  
Aided Diagnosis ◽  
Whole Slide Images

AbstractMost oncological cases can be detected by imaging techniques, but diagnosis is based on pathological assessment of tissue samples. In recent years, the pathology field has evolved to a digital era where tissue samples are digitised and evaluated on screen. As a result, digital pathology opened up many research opportunities, allowing the development of more advanced image processing techniques, as well as artificial intelligence (AI) methodologies. Nevertheless, despite colorectal cancer (CRC) being the second deadliest cancer type worldwide, with increasing incidence rates, the application of AI for CRC diagnosis, particularly on whole-slide images (WSI), is still a young field. In this review, we analyse some relevant works published on this particular task and highlight the limitations that hinder the application of these works in clinical practice. We also empirically investigate the feasibility of using weakly annotated datasets to support the development of computer-aided diagnosis systems for CRC from WSI. Our study underscores the need for large datasets in this field and the use of an appropriate learning methodology to gain the most benefit from partially annotated datasets. The CRC WSI dataset used in this study, containing 1,133 colorectal biopsy and polypectomy samples, is available upon reasonable request.

scholarly journals Defining the area of mitoses counting in invasive breast cancer using whole slide image

2021 ◽  
Author(s):  
Asmaa Ibrahim ◽  
Ayat G. Lashen ◽  
Ayaka Katayama ◽  
Raluca Mihai ◽  
Graham Ball ◽  
...  
Keyword(s):  
Breast Cancer ◽  
Practical Method ◽  
Relative Increase ◽  
Significant Rise ◽  
Mitotic Count ◽  
Morphological Criteria ◽  
Slide Image ◽  
Hematoxylin And Eosin ◽  
Mitotic Counting ◽  
Whole Slide Images

AbstractAlthough counting mitoses is part of breast cancer grading, concordance studies showed low agreement. Refining the criteria for mitotic counting can improve concordance, particularly when using whole slide images (WSIs). This study aims to refine the methodology for optimal mitoses counting on WSI. Digital images of 595 hematoxylin and eosin stained sections were evaluated. Several morphological criteria were investigated and applied to define mitotic hotspots. Reproducibility, representativeness, time, and association with outcome were the criteria used to evaluate the best area size for mitoses counting. Three approaches for scoring mitoses on WSIs (single and multiple annotated rectangles and multiple digital high-power (×40) screen fields (HPSFs)) were evaluated. The relative increase in tumor cell density was the most significant and easiest parameter for identifying hotspots. Counting mitoses in 3 mm2 area was the most representative regarding saturation and concordance levels. Counting in area <2 mm2 resulted in a significant reduction in mitotic count (P = 0.02), whereas counting in area ≥4 mm2 was time-consuming and did not add a significant rise in overall mitotic count (P = 0.08). Using multiple HPSF, following calibration, provided the most reliable, timesaving, and practical method for mitoses counting on WSI. This study provides evidence-based methodology for defining the area and methodology of visual mitoses counting using WSI. Visual mitoses scoring on WSI can be performed reliably by adjusting the number of monitor screens.

scholarly journals Developing image analysis pipelines of whole-slide images: Pre- and post-processing

2020 ◽  
pp. 1-11
Author(s):  
Byron Smith ◽  
Meyke Hermsen ◽  
Elizabeth Lesser ◽  
Deepak Ravichandar ◽  
Walter Kremers
Keyword(s):  
Image Analysis ◽  
Deep Learning ◽  
Digital Pathology ◽  
Disruptive Technology ◽  
Post Processing ◽  
Artifact Detection ◽  
The Press ◽  
Slide Image ◽  
Processing Order ◽  
Whole Slide Images

Abstract Deep learning has pushed the scope of digital pathology beyond simple digitization and telemedicine. The incorporation of these algorithms in routine workflow is on the horizon and maybe a disruptive technology, reducing processing time, and increasing detection of anomalies. While the newest computational methods enjoy much of the press, incorporating deep learning into standard laboratory workflow requires many more steps than simply training and testing a model. Image analysis using deep learning methods often requires substantial pre- and post-processing order to improve interpretation and prediction. Similar to any data processing pipeline, images must be prepared for modeling and the resultant predictions need further processing for interpretation. Examples include artifact detection, color normalization, image subsampling or tiling, removal of errant predictions, etc. Once processed, predictions are complicated by image file size – typically several gigabytes when unpacked. This forces images to be tiled, meaning that a series of subsamples from the whole-slide image (WSI) are used in modeling. Herein, we review many of these methods as they pertain to the analysis of biopsy slides and discuss the multitude of unique issues that are part of the analysis of very large images.

scholarly journals Deep Learning Based Retrieval System for Gigapixel Histopathology Cases and Open Access Literature

2018 ◽  
Author(s):  
Sebastian Otálora ◽  
Roger Schaer ◽  
Oscar Jimenez-del-Toro ◽  
Manfredo Atzori ◽  
Henning Müller
Keyword(s):  
Artificial Intelligence ◽  
Clinical Practice ◽  
Retrieval System ◽  
Digital Pathology ◽  
Image Features ◽  
Retrieval Performance ◽  
Slide Image ◽  
Text Information ◽  
Computation Algorithms ◽  
Whole Slide Images

ABSTRACTClinical practice is getting increasingly stressful for pathologists due to increasing complexity and time constraints. Histopathology is slowly shifting to digital pathology, thus creating opportunities to allow pathologists to improve reading quality or save time using Artificial Intelligence (AI)-based applications. We aim to enhance the practice of pathologists through a retrieval system that allows them to simplify their workflow, limit the need for second opinions, while also learning in the process. In this work, an innovative retrieval system for digital pathology is integrated within a Whole Slide Image (WSI) viewer, allowing to define regions of interest in images as queries for finding visually similar areas using deep representations. The back-end similarity computation algorithms are based on a multimodal approach, allowing to exploit both text information and content-based image features. Shallow and deep representations of the images were evaluated, the later showed a better overall retrieval performance in a set of 112 whole slide images from biopsies. The system was also tested by pathologists, highlighting its capabilities and suggesting possible ways to improve it and make it more usable in clinical practice. The retrieval system developed can enhance the practice of pathologists by enabling them to use their experience and knowledge to properly control artificial intelligence tools for navigating repositories of images for decision support purposes.

scholarly journals Characterizing Immune Responses in Whole Slide Images of Cancer With Digital Pathology and Pathomics

2020 ◽  
Vol 8 (4) ◽  
pp. 133-148
Author(s):  
Rajarsi Gupta ◽  
Han Le ◽  
John Van Arnam ◽  
David Belinsky ◽  
Mahmudul Hasan ◽  
...  
Keyword(s):  
Lung Cancer ◽  
Digital Pathology ◽  
Tumor Infiltrating Lymphocytes ◽  
Screening Tests ◽  
Cancer Tissue ◽  
Tissue Samples ◽  
Wide Range ◽  
Novel Biomarkers ◽  
Infiltrating Lymphocytes ◽  
Whole Slide Images

Abstract Purpose of Review Our goal is to show how readily available Pathomics tissue analytics can be used to study tumor immune interactions in cancer. We provide a brief overview of how Pathomics complements traditional histopathologic examination of cancer tissue samples. We highlight a novel Pathomics application, Tumor-TILs, that quantitatively measures and generates maps of tumor infiltrating lymphocytes in breast, pancreatic, and lung cancer by leveraging deep learning computer vision applications to perform automated analyses of whole slide images. Recent Findings Tumor-TIL maps have been generated to analyze WSIs from thousands of cases of breast, pancreatic, and lung cancer. We report the availability of these tools in an effort to promote collaborative research and motivate future development of ensemble Pathomics applications to discover novel biomarkers and perform a wide range of correlative clinicopathologic research in cancer immunopathology and beyond. Summary Tumor immune interactions in cancer are a fascinating aspect of cancer pathobiology with particular significance due to the emergence of immunotherapy. We present simple yet powerful specialized Pathomics methods that serve as powerful clinical research tools and potential standalone clinical screening tests to predict clinical outcomes and treatment responses for precision medicine applications in immunotherapy.

scholarly journals Two Ensemble-CNN Approaches for Colorectal Cancer Tissue Type Classification

2021 ◽  
Vol 7 (3) ◽  
pp. 51
Author(s):  
Emanuela Paladini ◽  
Edoardo Vantaggiato ◽  
Fares Bougourzi ◽  
Cosimo Distante ◽  
Abdenour Hadid ◽  
...  
Keyword(s):  
Colorectal Cancer ◽  
Deep Learning ◽  
Digital Pathology ◽  
Texture Features ◽  
Tissue Type ◽  
Cancer Tissue ◽  
Learning Methods ◽  
Feature Based ◽  
Type Classification ◽  
Whole Slide Images

In recent years, automatic tissue phenotyping has attracted increasing interest in the Digital Pathology (DP) field. For Colorectal Cancer (CRC), tissue phenotyping can diagnose the cancer and differentiate between different cancer grades. The development of Whole Slide Images (WSIs) has provided the required data for creating automatic tissue phenotyping systems. In this paper, we study different hand-crafted feature-based and deep learning methods using two popular multi-classes CRC-tissue-type databases: Kather-CRC-2016 and CRC-TP. For the hand-crafted features, we use two texture descriptors (LPQ and BSIF) and their combination. In addition, two classifiers are used (SVM and NN) to classify the texture features into distinct CRC tissue types. For the deep learning methods, we evaluate four Convolutional Neural Network (CNN) architectures (ResNet-101, ResNeXt-50, Inception-v3, and DenseNet-161). Moreover, we propose two Ensemble CNN approaches: Mean-Ensemble-CNN and NN-Ensemble-CNN. The experimental results show that the proposed approaches outperformed the hand-crafted feature-based methods, CNN architectures and the state-of-the-art methods in both databases.

scholarly journals A deep-learning-based prognostic nomogram integrating microscopic digital pathology and macroscopic magnetic resonance images in nasopharyngeal carcinoma: a multi-cohort study

2020 ◽  
Vol 12 ◽  
pp. 175883592097141
Author(s):  
Fan Zhang ◽  
Lian-Zhen Zhong ◽  
Xun Zhao ◽  
Di Dong ◽  
Ji-Jin Yao ◽  
...  
Keyword(s):  
Deep Learning ◽  
Magnetic Resonance ◽  
Nasopharyngeal Carcinoma ◽  
Treatment Failure ◽  
Digital Pathology ◽  
Imaging Features ◽  
Multi Scale ◽  
External Test ◽  
Whole Slide Images ◽  
Independent Cohort

Background: To explore the prognostic value of radiomics-based and digital pathology-based imaging biomarkers from macroscopic magnetic resonance imaging (MRI) and microscopic whole-slide images for patients with nasopharyngeal carcinoma (NPC). Methods: We recruited 220 NPC patients and divided them into training ( n = 132), internal test ( n = 44), and external test ( n = 44) cohorts. The primary endpoint was failure-free survival (FFS). Radiomic features were extracted from pretreatment MRI and selected and integrated into a radiomic signature. The histopathological signature was extracted from whole-slide images of biopsy specimens using an end-to-end deep-learning method. Incorporating two signatures and independent clinical factors, a multi-scale nomogram was constructed. We also tested the correlation between the key imaging features and genetic alternations in an independent cohort of 16 patients (biological test cohort). Results: Both radiomic and histopathologic signatures presented significant associations with treatment failure in the three cohorts (C-index: 0.689–0.779, all p < 0.050). The multi-scale nomogram showed a consistent significant improvement for predicting treatment failure compared with the clinical model in the training (C-index: 0.817 versus 0.730, p < 0.050), internal test (C-index: 0.828 versus 0.602, p < 0.050) and external test (C-index: 0.834 versus 0.679, p < 0.050) cohorts. Furthermore, patients were stratified successfully into two groups with distinguishable prognosis (log-rank p < 0.0010) using our nomogram. We also found that two texture features were related to the genetic alternations of chromatin remodeling pathways in another independent cohort. Conclusion: The multi-scale imaging features showed a complementary value in prognostic prediction and may improve individualized treatment in NPC.

Sign in / Sign up

Export Citation Format

Share Document