Evaluating machine learning performance using python for neural network models in urban transportation in New York city case study

Abstract

This study investigates neural network performance optimization for New York City taxi trip duration prediction to address critical gaps in transportation machine learning, where reproducibility, comprehensive diagnostics, and computational efficiency remain underreported. The research addresses limitations in prior literature that emphasize accuracy without standardized preprocessing, leakage prevention, or systematic cost-performance analysis. The objective was to develop a unified, reproducible framework combining an auditable, from-scratch NumPy neural network with production-grade Keras MLPs, systematically benchmarked against classical models under identical preprocessing and data splits. Methodology encompasses four independent phases: theoretical validation using XOR classification, statistical benchmarking through rigorous cross-validation on 1.3 million NYC taxi records, systematic architecture optimization across small/medium/large configurations, and advanced optimization achieving state-of-the-art performance. Key results demonstrate perfect XOR convergence validation (loss reduction from 0.7065 to 0.0198), competitive baseline performance against Random Forest (93.3 %±0.013 vs 90.5 %±0.044 accuracy), optimal medium architecture achieving 0.459 RMSLE, and final proposed model reaching 0.3092 RMSLE—representing 31.8 % improvement over Random Forest (0.4536) and 27.4 % over enhanced Keras baselines (0.4261). The framework incorporates comprehensive residual diagnostics, feature importance analysis, and computational profiling with statistical significance testing. Results establish new benchmarks for NYC taxi duration prediction while providing a methodologically replicable framework for future urban mobility analytics and operational ETA systems.