HITSZ TMG at ICASSP 2023 SPGC Shared Task: Leveraging Pre-Training and Distillation Method for Title Generation with Limited Resource

In this paper, we present our proposed method for the shared task of the ICASSP 2023 Signal Processing Grand Challenge (SPGC). We participate in Topic Title Generation (TTG), Track 3 of General Meeting Understanding and Generation (MUG) [1] in SPGC. The primary objective of this task is to generate a title that effectively summarizes the given topic segment. With the constraints of limited model size and external dataset availability, we propose a method as Pre-training - Distillation / Fine-tuning (PDF), which can efficiently leverage the knowledge from large model and corpus. Our method achieves first place during preliminary and final contests in ICASSP2023 MUG Challenge Track 3.