Feature importance analysis of optimized machine learning modeling for predicting customers satisfaction at the United States Airlines

Abstract

Customer experience is crucial in the airline industry, as understanding passenger satisfaction helps airlines improve service quality. This study evaluates hyperparameter optimization and feature interpretability in machine learning models for predicting airline passenger satisfaction. Support Vector Machine (SVM) and Multilayer Perceptron (MLP) models were tested for binary classification, labeling passengers as ‘Satisfied’ or ‘Neutral or Dissatisfied’ using a Kaggle dataset with ∼104,000 training and ∼26,000 test records. Hyperparameter tuning used grid search with 10-fold cross-validation. For SVM, the optimal setup included the RBF kernel, C = 10, and gamma = ‘auto’, achieving a mean score of 0.9606. For MLP, the best configuration used no regularization, "he" initialization, ReLU activation, 30 epochs, batch size of 32, two hidden layers with 32 neurons each, and a learning rate of 0.001, yielding a mean score of 0.9556. Performance metrics included accuracy, precision, recall, and F1-Score, with SVM achieving a test accuracy of 0.96, precision of 0.97, and F1-Score of 0.95, slightly outperforming MLP by <1 %, though MLP was faster at 0.3 s versus SVM’s 18 s. Both models surpassed baseline models and prior studies, benefiting from robust preprocessing and a large dataset. Permutation importance analysis identified Type of Travel, Inflight Wi-Fi Service, Customer Type, and Online Boarding as key predictors, emphasizing passenger needs for digital connectivity and personalized services. These insights guide airlines to prioritize reliable Wi-Fi and efficient online boarding to enhance satisfaction, loyalty, and competitive positioning.