Speaker
Sam Klein
(University of Geneva)
Description
The Bert pretraining paradigm has proven to be highly effective in many domains including natural language processing, image processing and biology. To apply the Bert paradigm the data needs to be described as a set of tokens, and each token needs to be labelled. To date the Bert paradigm has not been explored in the context of HEP. The samples that form the data used in HEP can be described as a set of particles (tokens) where each particle is represented as a continuous vector. We explore different approaches for discretising/labelling particles such that the Bert pretraining can be performed and demonstrate the utility of the resulting pretrained models on common downstream HEP tasks.
Primary authors
Dr
John Raine
(University of Geneva)
Prof.
Lukas Heinrich
(Technische Universitat Munchen)
Mr
Matthew Leigh
(University of Geneva)
Dr
Michael Kagan
(SLAC National Accelerator Laboratory)
Prof.
Rita Osadchy
(University of Haifa)
Sam Klein
(University of Geneva)
Prof.
Tobias Golling
(University of Geneva)
