Towards foundation-style models for energy-frontier heterogeneous neutrino detectors via self-supervised pretraining

Abstract

Accelerator-based neutrino physics is entering an energy-frontier regime in which interactions reach the tera-electronvolt scale and produce exceptionally dense, overlapping detector signatures. Event interpretation is impractical for conventional reconstruction and challenging for supervised models trained from scratch, particularly when labelled data are scarce and analyses span diverse objectives. Here we present a sparse Vision Transformer framework for learning reusable representations from heterogeneous detector data. Self-supervised pretraining combines masked-autoencoder reconstruction with relational voxel-level objectives for hierarchy, ghost and particle identification, followed by joint fine-tuning across classification and regression tasks. On simulated events from the proposed FASER calorimeter (FASERCAL) concept, pretraining improves neutrino-flavour and charm-quark identification, momentum regression and vertex reconstruction, with relational objectives providing further gains in topologically complex channels. Attribution and representation diagnostics show a more structured latent space, while detector-subsystem ablations recover physically plausible channel-dependent roles. With roughly 103 labelled events, the pretrained encoder matches the flavour-classification performance of scratch training with an order of magnitude more data. Cross-domain transfer is observed on public plastic-scintillator and liquid-argon benchmarks. A matched alternative-generator stress test shows stable flavour and kinematic performance while exposing charm tagging as generator sensitive. These results support self-supervised multimodal pretraining as a route towards reusable detector representations. We use ‘foundation-style’ in this restricted sense, rather than claiming a completed general-purpose detector foundation model. Alonso-Monsalve et al. demonstrate that self-supervised pretraining helps deep learning models to interpret complex neutrino detector events, improving classification, reconstruction and data efficiency while enabling transfer across different neutrino detector technologies.