EntailClass: A Classification Approach to EntailSum and End-to-End Document Extraction, Identification, and Evaluation

Abstract

The novelty of zero-shot text classification can address the fundamental challenge of the lack of labeled training data. With the current plethora of multidisciplinary, unstandardized text data, scalable classification models favor unsupervised methods over their supervised counterparts. Overall, the aim is to automate the labelling of each sentence in an input document consisting of section titles and section text. We propose an end-to-end pipeline that includes a document parser, a text classification model called EntailClass, and finally an evaluator to determine balanced accuracy. The suggested pipeline employs a zero-shot approach to classify text within any desired set of aspects. Moreover, text sentences are paired with their section titles and chronological order is maintained within sentences of the same aspect. The proposed automated, three-step pipeline represents a step towards solving the challenge of text classification without the need for an individual dataset for each aspect. It also offers the potential for seamless integration into existing workflows. This zero-shot, generalizable pipeline has achieved 87.2% accuracy and outperformed other state-of-the-art models when applied to supervisory documents.