Toward a thinking microscope: deep learning-enabled computational microscopy and sensing (Conference Presentation)

Deep learning is a class of machine learning techniques that uses multi-layered artificial neural networks for automated analysis of signals or data. The name comes from the general structure of deep neural networks, which consist of several layers of artificial neurons, each performing a nonlinear operation, stacked over each other. Beyond its main stream applications such as the recognition and labeling of specific features in images, deep learning holds numerous opportunities for revolutionizing image formation, reconstruction and sensing fields. In fact, deep learning is mysteriously powerful and has been surprising optics researchers in what it can achieve for advancing optical microscopy, and introducing new image reconstruction and transformation methods. From physics-inspired optical designs and devices, we are moving toward data-driven designs that will holistically change both optical hardware and software of next generation microscopy and sensing, blending the two in new ways. Today, we sample an image and then act on it using a computer. Powered by deep learning, next generation optical microscopes and sensors will understand a scene or an object and accordingly decide on how and what to sample based on a given task – this will require a perfect marriage of deep learning with new optical microscopy hardware that is designed based on data. For such a thinking microscope, unsupervised learning would be the key to scale up its impact on various areas of science and engineering, where access to labeled image data might not be immediately available or very costly, difficult to acquire. In this presentation, I will provide an overview of some of our recent work on the use of deep neural networks in advancing computational microscopy and sensing systems, also covering their biomedical applications.