Disease Classification on Admission and on Discharge with Residual CNN-Transformer

Clinical professionals perform disease classification on both admission and discharge of a patient, but previous works ignore the former. Physicians make a preliminary diagnosis based solely on current observations such as chief complaint and present illness at the admission time. Only limited information is available to decide which examination or treatment to make afterward. On discharge, complete medical records during hospitalization are available for deciding the International Classification of Diseases (ICD) code. Either occasion should be covered in a comprehensive disease classification system to meet the reality. Besides, from the technical perspective, previous state-of-the-art models employ the per-label attention mechanism to aggregate the contextualized vectors, less capable of handling the multi-label classification task up to 8,921 codes. In this paper, we conduct a comprehensive study on disease classification on both the admission and the discharge of patients. Furthermore, we propose a novel multi-head label decoding method that can replace the per-label attention module adopted by previous works. Experimental results show that our model achieves state-of-the-art performance in both admission and discharge scenarios.