Machine Learning Operations Model Store: Optimizing Model Selection for AI as a Service

With the emergence of Artificial Intelligence-as-a-Service (AIaaS), end users have access to an ever-growing number of pre-trained, fine-tuned, and evaluated Machine Learning (ML) models. These models are typically stored in a model store, an online repository that is part of Machine Learning Operations (MLOps) frameworks. Unfortunately, selecting the optimal ML model from a plethora of available options is challenging. The problem becomes even more pronounced when the micro-location of the user and the dynamic aspects of the environment are taken into account. To this end, we propose a dynamic approach for selecting the most appropriate ML model using Deep Reinforcement Learning (DRL). We demonstrate its effectiveness in a specific use case of energy consumption forecasting. We validate the proposed solution using the energy consumption dataset HUE and demonstrate that it outperforms heuristic methods for model selection by over ten percent while remaining cost-effective.