The Effect of Within-Bag Sampling on End-to-End Multiple Instance Learning

End-to-end multiple instance learning (MIL) is an important concept with a wide range of applications. It is gaining increased popularity in the (bio)medical imaging community since it may provide a possibility to, while relying only on weak labels assigned to large regions, obtain more fine-grained information. However, processing very large bags in end-to-end MIL is problematic due to computer memory constraints. We propose within-bag sampling as one way of applying end-to-end MIL methods on very large data. We explore how different levels of sampling affect the performance of a well-known high-performing end-to-end attention-based MIL method, to understand the conditions when sampling can be utilized. We compose two new datasets tailored for the purpose of the study, and propose a strategy for sampling during MIL inference to arrive at reliable bag labels as well as instance level attention weights. We perform experiments without and with different levels of sampling, on the two publicly available datasets, and for a range of learning settings. We observe that in most situations the proposed bag-level sampling can be applied to end-to-end MIL without performance loss, supporting its confident usage to enable end-to-end MIL also in scenarios with very large bags. We share the code as open source at https://github.com/MIDA-group/SampledABMIL