Shallow learning for MTL in end-to-end RNN for basic sequence tagging

Multitask learning (MTL) has been successfully applied for sequence tagging tasks in many machine learning approaches. The idea of MTL is employed to obtain better performance by sharing information among the tasks compared to a single task. However, most of the MTL in end-to-end sequence tagging models perform by sharing the entire set of parameters resulting in overwriting of parameters and need retraining of all the parameters. In the paper, a novel pipeline architecture has been proposed that effectively combines two RNN based sub-networks for sequence tagging tasks with minimal training overhead and truly acts as an end-to-end system. The proposed architecture performs Named entity recognition (NER) tagging as the primary sequence tagging task along with phrase tagging that works as the assistance sequence tagging task. To utilize the learning from the assisted network, we modify the base network by adding fully connected (FC) layers for NER. Our MTL approach tries to retain the learning parameters from individual tasks and, retraining is done only to the FC(shallow) layers of the modified base network. To validate the proposed MTL settings, we train on CoNLL 2003 corpus and compare the result with previously well established end-to-end based NER tagging models.