Reliability Prediction of Self-Adaptive Systems Managing Uncertain AI Black-Box Components

Advances in Artificial Intelligence (AI) are associated with a growing complexity of AI models, at the expense of transparency and comprehensibility. The black-box nature of AI components is of particular concern in safety-critical applications, as it can not be guaranteed whether a prediction is correct or not. Incorrect predictions, however, can have serious consequences, e.g., fatal collisions in autonomous driving. Therefore, we propose a novel method for safeguarding AI black-box components based on monitoring input data by using Self-Adaptive Systems (SAS). The presented concepts serve not only as a starting point for runtime approaches (e.g., models at runtime), but also for design-time approaches. As second contribution, we propose an approach for the validation of reconfiguration strategies of SAS's managing uncertain AI black-box components w.r.t. reliability objectives at design-time. We demonstrate the applicability of our approach by a proof-of-concept.