An Effective Approach of Named Entity Recognition for Cyber Threat Intelligence

Abstract

Traditional methods of domain named entity recognition (NER) rely on manually-defined feature templates and domain experience. Aiming at domain NER task of unstructured cyber threat intelligence (CTI), this paper proposed an approach based on BiLSTM-CRF model and domain dictionary matching correction. This approach utilizes bi-directional Long Short-Term Memory (BiLSTM) to automatically capture features of context, Conditional Random Fields (CRF) to learn label constraint rule, and an ontology-based domain dictionary for matching correction. Due to the lack of available domain dataset, this paper adopts the pre-processed unstructured CTI text as dataset for domain NER experiment. The experimental results show that the proposed approach reaches 85% in F1 score, and can significantly reduce reliance on manually-defined features.