Suboptimal bioinformatic predictions of antimicrobial resistance from whole-genome sequences in multidrug-resistant Corynebacterium isolates

Abstract

Herein, we combined different bioinformatics tools and databases (BV-BRC, ResFinder, RAST, and KmerResistance) to perform a prediction of antimicrobial resistance (AMR) in the genomic sequences of 107 Corynebacterium striatum isolates for which trustable antimicrobial susceptibility (AST) phenotypes could be retrieved. Then, the reliabilities of the AMR predictions were evaluated by different metrics: area under the ROC curve (AUC); Major Error Rates (MERs) and Very Major Error Rates (VMERs); Matthews Correlation Coefficient (MCC); F1-Score; and Accuracy. Out of 15 genes that were reliably detected in the C. striatum isolates, only tetW yielded predictive values for tetracycline resistance that were acceptable considering Food and Drug Administration (FDA)’s criteria for quality (MER < 3.0% and VMER with a 95% C.I. ≤1.5–≤7.5); this was accompanied by a MCC score higher than 0.9 for this gene. Noteworthy, our results indicate that other commonly used metrics (AUC, F1-score, and Accuracy) may render overoptimistic evaluations of AMR-prediction reliabilities on imbalanced datasets. Accordingly, out of 10 genes tested by PCR on additional multidrug-resistant Corynebacterium spp. isolates (n = 18), the tetW gene rendered the best agreement values with AST profiles (94.11%). Overall, our results indicate that genome-based AMR prediction can still be challenging for MDR clinical isolates of emerging Corynebacterium spp.