Analysing the energy impact of different optimisations for machine learning models

Abstract

Nowadays, there is an increasing use of artificial intelligence algorithms in the software applications we use in our daily lives. This allows us to effectively and efficiently solve a wide range of problems, but it is important to pay attention to the environmental impact they may have. This kind of algorithms often demand intensive use of several computational resources, including energy consumption, which means that more and more attention must be paid to the design and parametrization of machine learning algorithms in order to consider their energy efficiency, along with their functionality. With a proper assessment of energy consumption, developers gain the ability to take energy efficiency as a requirement for developing a machine learning model. As an illustrative example, in this paper we analyze the energy impact of changing the optimization method of a machine learning model based on logistic regression. We used three versions of the same logistic regression model using the Scikit-Learn python package, with the only difference between each version being the solver they use (SAG, Newton-CG, LBFGS), and measured their energy consumption for processing a dataset for detecting fraudulent credit card transactions. Our results reveal a major difference in consumption between the solver with least consumption (LBFGS, 961.36 W/s) and the most (Newton-CG, 2,761.71 W/s), while their difference in accuracy is only 0.016 percent points. This confirms the usefulness of evaluating the energy impact of the optimization choices of algorithms, so that developers can adequately consider the trade-off between the traditional quality measures (e.g. precision, recall, etc.) and energy consumption.