Prediction accuracy in maritime simulator training performance assessment with varying data frequency

This study investigates how varying data sampling frequencies affect the classification accuracy of Machine Learning (ML) models when predicting student performance in maritime simulator training. ML-driven performance prediction is an essential part of Predictive Learning Analytics (PLA). If acceptable prediction accuracy can be achieved by using lower frequency data with larger time intervals between recorded data points, valuable resources in terms of data storage, handling, and computational cost, can be potentially saved. This study utilizes simulator log data from navigation students performing a Williamson Turn in both Ballast and Loaded ship conditions on a desktop simulator. Data frequencies ranging from 01 to 09 second intervals are examined. Results are evaluated by Area Under the Curve (AUC), Accuracy, Log Loss, Precision, Recall, and F1 Scores. The eXtreme Gradient Boosted Trees, variants of Keras Residual Neural Network, and Light Gradient Boosted Trees are trained on 87.5 % and tested on 12.5 % of the data. The best accuracy measurement scores are achieved on the 1-s frequency intervals in both ballast and loaded condition analysis. Further, the 1-s frequency intervals models are also the fastest and require less Random Access Memory (RAM). With reducing data frequency intervals, the model evaluation metrics deteriorate.