MuTILs: explainable, multiresolution computational scoring of Tumor-Infiltrating Lymphocytes in breast carcinomas using clinical guidelines

Tumor-Infiltrating Lymphocytes (TILs) have strong prognostic and predictive value in breast cancer, but their visual assessment is subjective. We present MuTILs, a convolutional neural network architecture specifically optimized for the assessment of TILs in whole-slide image scans in accordance with clinical scoring recommendations. MuTILs is a concept bottleneck model, designed to be explainable and to encourage sensible predictions at multiple resolutions. Our computational scores match visual scores and have independent prognostic value in invasive breast cancers from the TCGA dataset.

Integrating morphologic and molecular histopathological features through whole slide image registration and deep learning

Background Modern molecular pathology workflows in neuro-oncology heavily rely on the integration of morphologic and immunohistochemical patterns for analysis, classification, and prognostication. However, despite the recent emergence of digital pathology platforms and artificial intelligence-driven computational image analysis tools, automating the integration of histomorphologic information found across these multiple studies is challenged by large files sizes of whole slide images (WSIs) and shifts/rotations in tissue sections introduced during slide preparation. Methods To address this, we develop a workflow that couples different computer vision tools including scale-invariant feature transform (SIFT) and deep learning to efficiently align and integrate histopathological information found across multiple independent studies. We highlight the utility and automation potential of this workflow in the molecular subclassification and discovery of previously unappreciated spatial patterns in diffuse gliomas. Results First, we show how a SIFT-driven computer vision workflow was effective at automated WSI alignment in a cohort of 107 randomly selected surgical neuropathology cases (97/107 (91%) showing appropriate matches, AUC = 0.96). This alignment allows our AI-driven diagnostic workflow to not only differentiate different brain tumor types, but also integrate and carry out molecular subclassification of diffuse gliomas using relevant immunohistochemical biomarkers (IDH1-R132H, ATRX). To highlight the discovery potential of this workflow, we also examined spatial distributions of tumors showing heterogenous expression of the proliferation marker MIB1 and Olig2. This analysis helped uncovered an interesting and unappreciated association of Olig2 positive and proliferative areas in some gliomas (r = 0.62). Conclusion This efficient neuropathologist-inspired workflow provides a generalizable approach to help automate a variety of advanced immunohistochemically compatible diagnostic and discovery exercises in surgical neuropathology and neuro-oncology.

An Open-Source Whole Slide Image Registration Workflow at Cellular Precision Using Fiji, QuPath and Elastix

Image analysis workflows for Histology increasingly require the correlation and combination of measurements across several whole slide images. Indeed, for multiplexing, as well as multimodal imaging, it is indispensable that the same sample is imaged multiple times, either through various systems for multimodal imaging, or using the same system but throughout rounds of sample manipulation (e.g. multiple staining sessions). In both cases slight deformations from one image to another are unavoidable, leading to an imperfect superimposition Redundant and thus a loss of accuracy making it difficult to link measurements, in particular at the cellular level. Using pre-existing software components and developing missing ones, we propose a user-friendly workflow which facilitates the nonlinear registration of whole slide images in order to reach sub-cellular resolution level. The set of whole slide images to register and analyze is at first defined as a QuPath project. Fiji is then used to open the QuPath project and perform the registrations. Each registration is automated by using an elastix backend, or semi-automated by using BigWarp in order to interactively correct the results of the automated registration. These transformations can then be retrieved in QuPath to transfer any regions of interest from an image to the corresponding registered images. In addition, the transformations can be applied in QuPath to produce on-the-fly transformed images that can be displayed on top of the reference image. Thus, relevant data can be combined and analyzed throughout all registered slides, facilitating the analysis of correlative results for multiplexed and multimodal imaging.

Potential quality pitfalls of digitalized whole slide image of breast pathology in routine practice

Using digitalized whole slide images (WSI) in routine histopathology practice is a revolutionary technology. This study aims to assess the clinical impacts of WSI quality and representation of the corresponding glass slides. 40,160 breast WSIs were examined and compared with their corresponding glass slides. The presence, frequency, location, tissue type, and the clinical impacts of missing tissue were assessed. Scanning time, type of the specimens, time to WSIs implementation, and quality control (QC) measures were also considered. The frequency of missing tissue ranged from 2% to 19%. The area size of the missed tissue ranged from 1–70%. In most cases (>75%), the missing tissue area size was <10% and peripherally located. In all cases the missed tissue was fat with or without small entrapped normal breast parenchyma. No missing tissue was identified in WSIs of the core biopsy specimens. QC measures improved images quality and reduced WSI failure rates by seven-fold. A negative linear correlation between the frequency of missing tissue and both the scanning time and the image file size was observed (p < 0.05). None of the WSI with missing tissues resulted in a change in the final diagnosis. Missing tissue on breast WSI is observed but with variable frequency and little diagnostic consequence. Balancing between WSI quality and scanning time/image file size should be considered and pathology laboratories should undertake their own assessments of risk and provide the relevant mitigations with the appropriate level of caution.

Defining the area of mitoses counting in invasive breast cancer using whole slide image

Although counting mitoses is part of breast cancer grading, concordance studies showed low agreement. Refining the criteria for mitotic counting can improve concordance, particularly when using whole slide images (WSIs). This study aims to refine the methodology for optimal mitoses counting on WSI. Digital images of 595 hematoxylin and eosin stained sections were evaluated. Several morphological criteria were investigated and applied to define mitotic hotspots. Reproducibility, representativeness, time, and association with outcome were the criteria used to evaluate the best area size for mitoses counting. Three approaches for scoring mitoses on WSIs (single and multiple annotated rectangles and multiple digital high-power (×40) screen fields (HPSFs)) were evaluated. The relative increase in tumor cell density was the most significant and easiest parameter for identifying hotspots. Counting mitoses in 3 mm2 area was the most representative regarding saturation and concordance levels. Counting in area <2 mm2 resulted in a significant reduction in mitotic count (P = 0.02), whereas counting in area ≥4 mm2 was time-consuming and did not add a significant rise in overall mitotic count (P = 0.08). Using multiple HPSF, following calibration, provided the most reliable, timesaving, and practical method for mitoses counting on WSI. This study provides evidence-based methodology for defining the area and methodology of visual mitoses counting using WSI. Visual mitoses scoring on WSI can be performed reliably by adjusting the number of monitor screens.

Performance-efficient distributed transfer and transformation of big spatial histopathology datasets in the cloud

Whole Slide Image (WSI) datasets are giga-pixel resolution, unstructured histopathology datasets that consist of extremely big files (each can be as large as multiple GBs in compressed format). These datasets have utility in a wide range of diagnostic and investigative pathology applications. However, the datasets present unique challenges: The size of the files, propriety data formats, and lack of efficient parallel data access libraries limit the scalability of these applications. Commercial clouds provide dynamic, cost-effective, scalable infrastructure to process these datasets, however, we lack the tools and algorithms that will transfer/transform them onto the cloud seamlessly, providing faster speeds and scalable formats. In this study, we present novel algorithms that transfer these datasets onto the cloud while at the same time transforming them into symmetric scalable formats. Our algorithms use intelligent file size distribution, and pipelining transfer and transformation tasks without introducing extra overhead to the underlying system. The algorithms, tested in the Amazon Web Services (AWS) cloud, outperform the widely used transfer tools and algorithms, and also outperform our previous work. The data access to the transformed datasets provides better performance compared to the related work. The transformed symmetric datasets are fed into three different analytics applications: a distributed implementation of a content-based image retrieval (CBIR) application for prostate carcinoma datasets, a deep convolutional neural network application for classification of breast cancer datasets, and to show that the algorithms can work with any spatial dataset, a Canny Edge Detection application on satellite image datasets. Although different in nature, all of the applications can easily work with our new symmetric data format and performance results show near-linear speed-ups as the number of processors increases.