Removement of MisLeading Data in Transfer Learning Using BERT

Abstract

When using machine learning to solve natural language processing tasks, the domains in which the model is trained and the domains in which the learned model is applied are different. Domain shift problem reduces model performance. Transfer learning using Bidirectional Encoder Representations from Transformers(BERT) is an effective method used for solving this problem. However, even with this method, we face the problem known as “negative transfer”, which occurs when some source labeled data adversely affect the learning in the target domain. In this study, we propose a for removing misleading data, causing negative transfer, for document classification tasks. We demonstrated the effectiveness of our proposed method in an experiment using the Webis-CLS-10 dataset.