The Coffea (Columnar Object Framework for Effective Analysis) framework continues to evolve as a cornerstone tool for high-energy physics data analysis, providing physicists with efficient, scalable solutions for processing complex event data. This talk presents the current status of Coffea, highlighting significant recent developments and their impact on the HEP analysis community.
A major...
The migration from TTree to RNTuple as the preferred format for data storage is now in full force. Uproot, the IO package in Scikit-HEP, has stayed at the forefront of this migration. It now has 100% bi-directional support between RNTuples and Awkward arrays, supports schema extensions, and defaults to writing RNTuples instead of TTrees. I will be discussing the status of RNTuple support, some...
This talk presents Pepper, a coffea-based, general purpose, CMS-endorsed analysis framework. Pepper is designed to run on the CMS NanoAOD dataset without intermediate ntuples, aiming to allow full analyses with fast time-to-insight. Its features include substantial book-keeping and histogram-filling capabilities, implementation of standard CMS objects, problematic file handling, resumption...
FCCAnalyses is the common analysis framework for the Future Circular Collider (FCC) study, built on ROOT's RDataFrame and the EDM4hep event data model. Its performance-critical processing is written in C++, but the framework is designed and driven from Python end-to-end: users configure analyses, manage dataset metadata, and run their processing on anything from a laptop to a grid site through...
This talk aims to present coffea-workflow package - workflow manager for coffea HEP analyses. This HEP-specific extension for coffea analyses does not replace existing workflow managers (Snakemake, LAW, …) - instead it focuses on three things coffea alone does not provide out of the box:
- Partial results - split your fileset into independently cached chunks; if some fail you keep the...
The "Integration Challenge" is the latest IRIS-HEP project exploring physics analysis pipelines for the HL-LHC. Compared to earlier iterations, this one focuses on capturing the detailed needs of experiments. The contribution presents the analysis pipeline built for the ATLAS version of the project, which is based on a variety of Scikit-HEP and IRIS-HEP projects (including uproot, awkward,...
Analysis workflows for the High-Luminosity LHC (HL-LHC) will need to handle far more data than current approaches were designed for. To prepare for this, CMS analysis pipelines are being built on the scientific Python stack, with HEP-specific tools filling the gaps where general-purpose software falls short. This talk gives an update on that effort through the lens of the Integration...
OmniFold gives you per-event weights instead of a histogram, so you can build whatever distribution you want afterward. But that makes results hard to share. A weight file is just a few hundred unlabeled columns, and readers are left guessing which one is the main result, which columns belong to the same uncertainty estimate, how to combine them, and whether the rows even still match the...
Large language models are rapidly becoming useful assistants for software development, analysis, and documentation. In practice, however, the limiting factor is rarely the model itself. The hard part is giving the model reliable, secure access to the tools, data, and domain knowledge that scientists already use.
This talk introduces the Model Context Protocol (MCP) from the perspective of...
Following the recent increase in LLM capabilities, there are many exploratory projects ongoing to identify the most promising use cases for HEP. In this contribution I report on two examples in the context of statistical model building. The first example focuses on testing the capabilities of an automated support system for complex software. The second investigates to what extent it may be...
Combine is used for most statistical results in CMS, but its many methods, options, and error messages make it hard to learn and lead to many repeated support questions. We present an AI assistant for Combine built around a large‑language‑model agent that answers using the tool's own trusted sources and cites the material it relies on. The assistant can also reproduce and diagnose a user's...
Modern high-energy physics analyses rely on complex software workflows that must evolve alongside changing libraries, data formats, execution environments, and computing technologies. Experience with FAST-HEP has shown that declarative analysis descriptions improve readability, but do not by themselves make the complete analysis unambiguous, replaceable, or reproducible.
This talk introduces...
Born for first-hand user experience of using Luigi based workflows for an HEP analyses, the ParaO Workflow Python package addresses several pain points that arise when developing and executing an analysis.
This contribution will introduce the core concepts, while relating to principles already found in Luigi, and will showcase the new features that attempt to address perceived shortcomings in Luigi.
This contribution will show how ParaO is used to conduct an analysis and related first hand experiences.
Statistical modeling is central to discovery in particle physics, yet the tools commonly used to define, share, and evaluate these models are often complex, fragmented, or tightly coupled to legacy systems. In parallel, the scientific Python community has developed a variety of statistical modeling tools that have been widely adopted for their performance and ease of use, but remain...
SysVar is a Python package offering a complete workflow for handling and propagating systematic uncertainties in high-energy physics analyses that rely on templates built from simulated data.
Propagating systematic uncertainties from correction weights into templates, while keeping the correlations in the signal extraction variables intact, requires increased bookkeeping as analyses grow in...
evermore is a python package that provides building blocks for building and evaluating (binned) likelihood functions in high-energy physics using JAX. The key concepts of evermore are performance and differentiability. JAX provides automatic differentiation, just-in-time (jit) compilation, and vectorization capabilities, which can be leveraged to improve the performance of statistical...
everwillow is a statistical inference library for high energy physics built on JAX pytrees. It is deliberately open-world: there is no model specification format and no assumption about where the likelihood comes from. Any negative log-likelihood written as a JAX function works, whether it comes from a binned template fit, an unbinned parametric model, or another JAX-based package, and...
The packaging of high energy physics software with robust, yet flexible, distribution methods is a complicated problem that has been met with multiple approaches by the community. The HEP Packaging Coordination community project expands packaging of the HEP software ecosystem through building and distributing language-agnostic conda packages on...
After 3 years, Pixi is widely adopted in the scientific Python ecosystem, let me show you why!
Scientific Python has specific challenges that Pixi can solve well; a lot of our beloved packages contain C, C++, Rust, CUDA or even Fortran code. With Pixi, a single tool can install compilers, different Python versions and other build tools in one go, thanks to piggybacking on the years of...
