Using Deep Learning with Large Dataset of Microscope Images to Develop an Automated Embryo Grading System

The assessment of embryo viability for in vitro fertilization (IVF) is mainly based on subjective visual analysis, with the limitation of intra- and inter-observer variation and a time-consuming task. In this study, we used deep learning with large dataset of microscopic embryo images to develop an automated grading system for embryo assessment. This study included a total of 171,239 images from 16,201 embryos of 4,146 IVF cycles at Stork Fertility Center (https://www.e-stork.com.tw) from March 6, 2014 to April 13, 2018. The images were captured by inverted microscope (Zeiss Axio Observer Z1) at 112 to 116 hours (Day 5) or 136 to 140 hours (Day 6) after fertilization. Using a pre-trained network trained on the ImageNet dataset as convolution base, we applied Convolutional Neural Network (CNN) on embryo images, using ResNet50 architecture to fine-tune ImageNet parameters. The predicted grading results was compared with the grading results from trained embryologists to evaluate the model performance. The images were labeled by trained embryologists, based on Gardner’s grading system: blastocyst development ranking from 3–6, ICM quality as A, B, or C; and TE quality as a, b, or c. After pre-processing, the images were divided into training, validation, and test groups, in which 60% were allocated to the training group, 20% to the validation group, and 20% to the test group. The ResNet50 algorithm was trained on the 60% images allocated to the training group, and the algorithm’s performance was evaluated using the 20% images allocated to the test group. The results showed an average predictive accuracy of 75.36% for the all three grading categories: 96.24% for blastocyst development, 91.07% for ICM quality, and 84.42% for TE quality. To the best of our knowledge, this is the first study of an automatic embryo grading system using large dataset from Asian population. Combing the promising results obtained in this study with time-lapse microscope system integrated with IVF Electronic Medical Record platform, a fully automated and non-invasive pipeline for embryo assessment will be achieved.