Searches for new phenomena at the Large Hadron Collider at CERN usually boil down to performing a statistical hypothesis test for the presence of a new signal over a background of known Standard Model physics. Due to the high dimensionality of the feature space, these tests are usually done with the help of machine learning classifiers. Methods for doing this are well-established when one has access to reliable samples from both the signal and background distributions. However, when one or both of these samples are unreliable or unavailable, the commonly used methods may lose sensitivity or not be applicable. In this talk, I will give an overview of our recent work on model-agnostic searches for new physics in high-dimensional feature spaces. The overarching goal is to develop powerful tests for the presence of a signal that make weak assumptions about the signal, the background or both. Along the way, I will draw connections to high-dimensional two-sample testing, anomaly detection, transfer learning and simulation-based inference.