BERT for Aviation Text Classification

Abstract

The advent of transformer-based models pre-trained on large-scale text corpora has rev-olutionized Natural Language Processing (NLP) in recent years. Models such as BERT (Bidirectional Encoder Representations from Transformers) offer powerful tools for understanding contextual information and have achieved impressive results in numerous language understanding tasks. However, their application in the aviation domain remains relatively unexplored. This study discusses the challenges of applying multi-label classification problems on aviation text data. A custom aviation domain specific BERT model (Aviation-BERT) is compared against BERT-base-uncased for anomaly event classification in the Aviation Safety Reporting System (ASRS) data. Aviation-BERT is shown to have superior performance based on multiple metrics. By focusing on the potential of NLP in advancing complex aviation safety report analysis, the present work offers a comprehensive evaluation of BERT on aviation domain datasets and discusses its strengths and weaknesses. This research highlights the significance of domain-specific NLP models in improving the accuracy and efficiency of safety report classification and analysis in the aviation industry.