On The Irrelevance of Machine Learning Algorithms and the Importance of Relativity

Abstract

Information explosion has brought us a wide range of data formats and machine learning keeps in constant evolution to develop mechanisms to extract knowledge from them. Modern models in the Deep Learning space have proven to be very successful in multiple applications, yet in the tabular space they fail to provide consistent competitive performance. However, in this work we claim model selection can become irrelevant as the key tends to lie in data processing. In this paper we introduce the concept of relativity in feature engineering, a powerful methodology to boost any classifier performance and we provide over 30 different configurations of models and feature engineering designs to prove we can bias any result to help an arbitrary model score best. Our results attribute 600% more value to feature engineering than model selection. In order to validate the effectiveness of our approach, we submitted our work to a live machine learning competition with outstanding results regardless of our model of choice.