Artificial Second Harmonic Generation Image Feature Tuning with Generative Models and SeFa Semantic Analysis.

Abstract

Collagen fibers within the extracellular matrix undergo morphological changes in the presence of many diseases including cancers and connective tissue disorders. Second Harmonic Generation (SHG) microscopy is ideal for visualizing this morphology by its intrinsic contrast and 3D imaging capabilities of fibrillar collagen. To build upon the success of deep learning-based methods analyzing microscopy datasets, in this study, we adapted the generative adversarial network model StyleGAN specifically for use with SHG data. Here we trained our model StyleGAN2-ADA-SHG on a dataset of 1,319 SHG images to extract further information content. When comparing evaluation metrics between the baseline dataset and the generator, we found that the model is well suited to generating novel images that appear perceptually similar to those from our small set of training data. Using Semantic Factorization (SeFa) on the generated images to examine the 10 highest scoring eigenvectors, we found two semantics with strong correlation to fiber morphological features that were quantified via analysis using CT-Fire, CurveAlign, and other fiber-based metrics. We integrated analysis tools into the SeFa GUI to support an extensible framework for future analyses and image editing. Additionally, we tested two image projection methods for editing SHG images in the GAN's latent space using the identified semantics. The encoder-based approach significantly outperformed the Learned Perceptual Image Patch Similarity (LPIPS) method. With further refinement, these tools can be used to create SHG images of collagen fibers with variable morphology based on disease types for additional analysis purposes.