A systematic study of parameter sharing strategies in multi-task learning for drug synergy and sensitivity prediction

Multi-task learning (MTL) is a useful approach for modeling related prediction tasks by sharing representations while allowing each task to keep its unique features. In computational drug discovery, drug synergy and drug sensitivity predictions are closely linked, but it is still unclear where and how to share model parameters between these tasks. In this study, we introduce a multi-task learning framework that predicts both drug synergy and drug sensitivity, with a main goal of improving synergy prediction by using sensitivity prediction as a supporting task. Our model uses molecular descriptors and drug induced gene expression signatures to describe pharmaceuticals, while untreated cancer cell lines are described by their baseline gene expression profiles. We use a mutual attention mechanism to capture complex interactions between drugs and cells. With this shared feature extraction base, we systematically test different parameter sharing strategies, looking at both hard and soft sharing at various network depths. Our experiments show that the success of parameter sharing depends on both the sharing strategy and the network layer where sharing happens. Hard parameter sharing works best at deeper layers, while soft parameter sharing allows for more stable and flexible knowledge transfer between tasks. Among the soft sharing methods we tested, dense sharing based on Neural Discriminative Dimensionality Reduction (NDDR) consistently outperforms cross stitch and sluice networks, especially when used in early network layers. Extending dense sharing to multiple layers further improves shared representations and leads to better performance. An updated NDDR version with nonlinear bottleneck transformations and residual connections achieves the highest overall performance. In summary, this work gives clear insight about where and how information should be shared in multi-task learning for drug synergy and sensitivity prediction. Our results show that both the sharing level and the way sharing is done strongly affect performance. Multi-level and depth-aware sharing strategies lead to better predictions. These findings give practical guidance for building effective multi-task models in computational drug discovery.