Protocol for applying Machine Learning models for the transformation of conventional fluorescence images to super-resolution

Machine Learning offers the opportunity to visualize the invisible in conventional fluorescence microscopy images by improving their resolution while preserving and enhancing image details. This protocol describes the application of GAN-based Machine Learning models to transform the resolution of conventional fluorescence microscopy images to a resolution comparable with super-resolution. It provides a flexible environment using a modern app functioning on both desktop and mobile computers. This approach can be extended for use on other types of microscopy images empowering life science researchers with modern analytical tools.

Application of convolutional neural networks towards nuclei segmentation in localization-based super-resolution fluorescence microscopy images

Background Automated segmentation of nuclei in microscopic images has been conducted to enhance throughput in pathological diagnostics and biological research. Segmentation accuracy and speed has been significantly enhanced with the advent of convolutional neural networks. A barrier in the broad application of neural networks to nuclei segmentation is the necessity to train the network using a set of application specific images and image labels. Previous works have attempted to create broadly trained networks for universal nuclei segmentation; however, such networks do not work on all imaging modalities, and best results are still commonly found when the network is retrained on user specific data. Stochastic optical reconstruction microscopy (STORM) based super-resolution fluorescence microscopy has opened a new avenue to image nuclear architecture at nanoscale resolutions. Due to the large size and discontinuous features typical of super-resolution images, automatic nuclei segmentation can be difficult. In this study, we apply commonly used networks (Mask R-CNN and UNet architectures) towards the task of segmenting super-resolution images of nuclei. First, we assess whether networks broadly trained on conventional fluorescence microscopy datasets can accurately segment super-resolution images. Then, we compare the resultant segmentations with results obtained using networks trained directly on our super-resolution data. We next attempt to optimize and compare segmentation accuracy using three different neural network architectures. Results Results indicate that super-resolution images are not broadly compatible with neural networks trained on conventional bright-field or fluorescence microscopy images. When the networks were trained on super-resolution data, however, we attained nuclei segmentation accuracies (F1-Score) in excess of 0.8, comparable to past results found when conducting nuclei segmentation on conventional fluorescence microscopy images. Overall, we achieved the best results utilizing the Mask R-CNN architecture. Conclusions We found that convolutional neural networks are powerful tools capable of accurately and quickly segmenting localization-based super-resolution microscopy images of nuclei. While broadly trained and widely applicable segmentation algorithms are desirable for quick use with minimal input, optimal results are still found when the network is both trained and tested on visually similar images. We provide a set of Colab notebooks to disseminate the software into the broad scientific community (https://github.com/YangLiuLab/Super-Resolution-Nuclei-Segmentation).

Application of Convolutional Neural Networks Towards Nuclei Segmentation in Localization-based Super-Resolution Fluorescence Microscopy Images

Background Automated segmentation of nuclei in microscopic images has been conducted to enhance throughput in pathological diagnostics and biological research. Segmentation accuracy and speed has been significantly enhanced with the advent of convolutional neural networks. A barrier in the broad application of neural networks to nuclei segmentation is the necessity to train the network using a set of application specific images and image labels. Previous works have attempted to create broadly trained networks for universal nuclei segmentation; however, such networks do not work on all imaging modalities, and best results are still commonly found when the network is retrained on user specific data. Stochastic optical reconstruction microscopy (STORM) based super-resolution fluorescence microscopy has opened a new avenue to image nuclear architecture at nanoscale resolutions. Due to the large size and discontinuous features typical of super-resolution images, automatic nuclei segmentation can be difficult. In this study, we apply commonly used networks (Mask R-CNN and UNet architectures) towards the task of segmenting super-resolution images of nuclei. First, we assess whether networks broadly trained on conventional fluorescence microscopy datasets can accurately segment super-resolution images. Then, we compare the resultant segmentations with results obtained using networks trained directly on our super-resolution data. We next attempt to optimize and compare segmentation accuracy using three different neural network architectures. Results Results indicate that super-resolution images are not broadly compatible with neural networks trained on conventional bright-field or fluorescence microscopy images. When the networks were trained on super-resolution data, however, we attained nuclei segmentation accuracies (F1-Score) in excess of 0.8, comparable to past results found when conducting nuclei segmentation on conventional fluorescence microscopy images. Overall, we achieved the best results utilizing the Mask R-CNN architecture. Conclusions We found that convolutional neural networks are powerful tools capable of accurately and quickly segmenting localization-based super-resolution microscopy images of nuclei. While broadly trained and widely applicable segmentation algorithms are desirable for quick use with minimal input, optimal results are still found when the network is both trained and tested on visually similar images. We provide a set of Colab notebooks to disseminate the software into the broad scientific community (https://github.com/YangLiuLab/Super-Resolution-Nuclei-Segmentation).

Phase separation at the nanoscale quantified by dcFCCS

Liquid–liquid phase separation, driven by multivalent macromolecular interactions, causes formation of membraneless compartments, which are biomolecular condensates containing concentrated macromolecules. These condensates are essential in diverse cellular processes. Formation and dynamics of micrometer-scale phase-separated condensates are examined routinely. However, limited by commonly used methods which cannot capture small-sized free-diffusing condensates, the transition process from miscible individual molecules to micrometer-scale condensates is mostly unknown. Herein, with a dual-color fluorescence cross-correlation spectroscopy (dcFCCS) method, we captured formation of nanoscale condensates beyond the detection limit of conventional fluorescence microscopy. In addition, dcFCCS is able to quantify size and growth rate of condensates as well as molecular stoichiometry and binding affinity of client molecules within condensates. The critical concentration to form nanoscale condensates, identified by our experimental measurements and Monte Carlo simulations, is at least several fold lower than the detection limit of conventional fluorescence microscopy. Our results emphasize that, in addition to micrometer-scale condensates, nanoscale condensates are likely to play important roles in various cellular processes and dcFCCS is a simple and powerful quantitative tool to examine them in detail.

Single particle-resolution fluorescence microscopy of nanoplastics

Understanding of nanoplastic prevalence and toxicology is limited by imaging challenges resulting from their small size. Fluorescence microscopy is widely applied to track and identify microplastics in laboratory studies and environmental samples. However, conventional fluorescence microscopy, due to diffraction, lacks the resolution to precisely localize nanoplastics in tissues, distinguish them from free dye, or quantify them in environmental samples. To address these limitations, we developed techniques to label nanoplastics for imaging with Stimulated Emission Depletion (STED) microscopy to achieve resolution at an order of magnitude superior to conventional fluorescence microscopy. These techniques include (1) passive sorption; (2) swell incorporation; and (3) covalent coupling of STED-compatible fluorescence dyes to nanoplastics. We demonstrate that our labeling techniques, combined with STED microscopy, can be used to resolve nanoplastics of different shapes and compositions as small as 50 nm. We also test STED imaging of nanoplastics in exposure experiments with the model worm C. elegans. These techniques will allow more precise localization and quantification of nanoplastics in complex matrices.

How to quantitatively measure, assess and correct the fluorescence crosstalk in the wide-field microscopy

During the multi-color fluorescence imaging, a fluorophore can be detected in the non-corresponding channel due to the broad spectra of the fluorophore and filters, resulting in the artefact called the crosstalk (crossover or bleed-through). Unfortunately, the fluorescence crosstalk is ubiquitous for the conventional fluorescence microscopy. Significant crosstalk can distort the image and lead to incorrect interpretation. Here, we introduce a simple quantitative metric, the crosstalk factor, to measure the crosstalk effect of a fluorophore on a channel in the wide-field fluorescence microscopy. We describe a cell biologist friendly protocol which should be easily implemented by cell biologists using commonly available image processing software such as ImageJ.