Deep learning methods are becoming increasingly common in particle physics. In this talk, I will discuss foundation models -- models trained on data without any labels. Specifically, I'll present PolarBERT: a foundation model for the IceCube Neutrino Observatory that learns to predict detector responses from unlabeled neutrino event data. The model can then be fine-tuned for downstream tasks such as neutrino direction reconstruction.
I will also discuss the broader context of whether lessons from Large Language Models are applicable to theoretical and experimental particle physics.
Ankita Budhraja, Juraj Klaric, Johannes Michel, Maria Laura Piscopo