A Fine-grained Classification Method for Cross-domain Policy Texts Based on Instruction Tuning

Abstract

The well-organized structure of Policy Texts (PTs) is fundamental to intelligent governance, yet most PTs lack fine-grained category labels. PTs from different domains follow different classification systems, and traditional encoder-only models cannot directly handle scenarios where the label spaces of the source and target domains differ significantly, as their output layer typically is a fixed-dimensional classification head. Therefore, we propose a Cross-Domain Policy Text Classification (CDPTC) task. We introduce a method for the task called InstructCDPTC. This method, within an instruction tuning framework, transforms the classification task into a generation task, using the decoder-only model BigBird to predict masked tokens. We wrap the original PT within an instruction template containing a task description, a label description, and a mask sequence, which serve as input to BigBird. During training, we use the names of gold categories as the prediction targets for masked positions. During inference, we determine the final predicted category by computing the semantic distance between the averaged representations of the mask predictions and each candidate label. We constructed a dataset of 20,189 labeled policy texts from five different policy domains to evaluate InstructCDPTC. Experimental results demonstrate that InstructCDPTC achieves an F1 score of 0.824 under conditions where the sample distribution and label space of the target domain are entirely unseen, surpassing other baselines.