IDS-Extract:Downsizing Deep Learning Model For Question and Answering

Abstract

In recent years, Question-answering systems are extensively used in human-computer systems, and the accuracy rate on a large scale is increasing. However, in actual deployment, a large number of parameters are often accompanied by a large amount of memory and long-term processing requirements. Therefore, compressing the data of the model, reducing training time, memory, becomes more and more urgent. we aim to resolve issues: IDS-Extract dynamically sized data to support models and devices with different memory. The proposed technique does efficient data extraction, segments that are not meaningful for model learning on the original dataset and output multiple datasets of adaptive size followed by target training based on model size. We leverage techniques in IG(Integration Gradient), DPR, and SBERT to improve localization performance for answer positions. We compare the model performance of SQuAD and the data set reduced by the IDS extraction technique, and the results prove that our technique can train the model more targeted and obtain higher performance evaluation. We prove that this method has successfully passed the sanity check, and can be directly applied to emotion recognition, two-classification, and multi-classification fields.