FLUORESCENCE MICROSCOPY IMAGES SEGMENTATION AND ANALYSIS USING MACHINE LEARNING

Microscopy image analysis can provide substantial information for clinical study and understanding of the biological structure. Two-photon microscopy is a type of fluorescence microscopy that can visualize deep into tissue with near-infrared excitation light. Large 3D image volumes of complex subcellular are often produced, which calls for automatic image analysis techniques. Automatic methods that can obtain nuclei quantity in microscopy image volumes are needed for biomedical research and clinical diagnosis. In general, several challenges exist for counting nuclei in 3D image volumes. These include “crowding” and touching of nuclei, overlapping of two or more nuclei, and shape and size variances of the nuclei. In this thesis, a 3D nuclei counter using two different generative adversarial networks (GAN) is proposed and evaluated. Synthetic data that resembles real microscopy image is generated with a GAN. The synthetic data is used to train another 3D GAN network that counts the number of nuclei. Our approach is evaluated with respect to the number of ground truth nuclei and compared with common ways of counting used in the biological research.Fluorescence microscopy 3D image volumes of rat kidneys are used to test our 3D nuclei counter. The evaluation of both networks shows that the proposed technique is successful for counting nuclei in 3D. Then, a 3D segmentation and classification method to segment and identify individual nuclei in fluorescence microscopy volumes without having ground truth volumes is introduced. Three dimensional synthetic data is generated using the Recycle-GAN with the Hausdorff distance loss introduced into preserve the shape of individual nuclei. Realistic microscopy image volumes with nuclei segmentation mask and nucleus boundary ground truth volumes are generated.A subsequent 3D CNN with a regularization term that discourage detection out of nucleus boundary is used to detect and segment nuclei. Nuclei boundary refinement is then performed to enhance nuclei segmentation. Experimental results on our rat kidney dataset show the proposed method is competitive with respect to several state-of-the-art methods. A Distributed and Networked Analysis of Volumetric Image Data (DINAVID) system is developed to enable remote analysis of microscopy images for biologists. There are two main functions integrated in the system, a 3D visualization tool and a remote computing tool for nuclei segmentation. The 3D visualization enables real-time rendering of large volumes of microscopy data. The segmentation tool provides fast inferencing of pre-trained deep learning models trained with 5 different types of microscopy data.