O-004 Multi-center, external validation of a novel artificial intelligence (AI) model that predicts blastocyst PGT-A results from mature oocytes

Abstract

Study question Is a model developed to predict euploid blastocyst development from mature oocytes generalizable across varying geographic locations? Summary answer A non-invasive AI model predicts euploid blastocyst development from mature oocytes with an AUC of 0.68 on a large dataset from 5 clinics (4 countries). What is known already MAGENTA is an AI-based model that assesses mature oocyte images and provides a score (0-10) correlated to its likelihood of developing to a blastocyst-stage embryo. An additional model has been recently developed to further predict the likelihood of euploid blastocyst development from the same oocyte images yet also incorporates oocyte age and MAGENTA’s assessments as key features. This Ploidy-AI model provides additional insight reflective of the oocyte’s potential chromosomal complement–enhancing the clinical utility of assessments. With the development of a new AI model, external validation of its performance is necessary to ensure generalizability across different geographies and patient demographics. Study design, size, duration This is a retrospective study that included 13,307 images of mature oocytes obtained from 5 clinics in 4 countries (1603 patients, 1949 cycles) including Argentina (C1; mean age 32.6±7.0, BMI unavailable), Brazil (C2; mean age 38.1±3.6, BMI 38.3), Spain (C3; mean age 38.7±3.8, BMI 20.4 and C4; mean age 38.9±3.4, BMI 21.8), and USA (C5; mean age 37.2±4.1; BMI 27). Images were obtained immediately post-ICSI from Embryoscope or GERI Time-Lapse incubators between the years 2020-2024. Participants/materials, setting, methods 13,307 oocyte images were assessed by MAGENTA and the Ploidy-AI model to predict each oocyte’s likelihood of developing into a euploid blastocyst (0-100%). Oocytes that did not develop into a blastocyst (n = 7385) or those that developed into an aneuploid blastocyst (n = 3534) were labelled the negative outcome, whereas those that developed into euploid blastocysts (n = 2388) were labeled the positive outcome. Untested or mosaic blastocysts were excluded. Main results and the role of chance On 13,307 mature oocytes, the Ploidy-AI model achieved an AUC of 0.68, sensitivity 0.54, and specificity 0.71. Oocytes that failed blastulation or developed into an aneuploid blastocyst had significantly lower median model-predicted euploid probability (n = 10,919, 0.20) than those that developed into an euploid blastocyst (n = 2388, 0.28) by Mann-Whitney U-test (p < 0.001). Additionally, model-predicted euploid probabilities were divided into quartiles (Q) according to the distribution within this dataset—Q1 (n = 3327), Q2 (n = 3327), Q3 (n = 3326), Q4 (n = 3327). A significant, stepwise positive increase in true euploid development rate for oocytes within each quartile of model-predicted probabilities was observed by pairwise-proportions test with Bonferroni correction (all p < 0.001): Q1(6%), Q2(14%), Q3(22%), and Q4(30%). Subgroup analysis by Clinic revealed consistent performance across all 5 clinics; C1 (n = 1643) – AUC 0.66, sensitivity 0.70, specificity 0.54; C2 (n = 7239) – AUC 0.68, sensitivity 0.51, specificity 0.72; C3 (n = 2442) – AUC 0.72, sensitivity 0.49, specificity 0.80; C4 (n = 802) – AUC 0.68, sensitivity 0.46, specificity 0.76; C5 (n = 1181) – AUC 0.66, sensitivity 0.61, specificity 0.62. The Ploidy-AI model performance was significantly higher on C3 than C1 (p < 0.001), C2 (p < 0.01), C5 (p < 0.01), and the overall dataset (p < 0.01) by DeLong’s test; however, no significant differences were observed in the other clinic-to-clinic or clinic-to-overall dataset comparisons. Limitations, reasons for caution The model displayed significantly higher performance on C3 compared to three other clinics, although model performance on the remaining clinics was similar and comparable to the overall dataset AUC. Further validating the model in additional geographies may ensure greater application. This study was retrospective in nature, prospective evaluation is warranted. Wider implications of the findings External validation of newly developed AI models is critical prior to clinical utilization. Large, diverse datasets, as in this study, ensure model generalization. This study presents a robust validation of a model that predicts blastocyst ploidy development from mature oocytes and is consistent across various clinic locations in different countries. Trial registration number No