Speaker
Jonas Eppelt
Description
Reproducible, fault-tolerant analysis pipelines are essential to modern HEP computing, and workflow managers provide the backbone for meeting these demands. In this talk, I present an overview of workflow management in HEP, with a focus on Luigi—a Python-based framework widely used in the community—and its HEP-specific extensions, b2luigi and Law. I discuss their design philosophies, key features, and ongoing development efforts. I then broaden the scope with a brief comparison to alternative workflow management tools from outside HEP, including Snakemake, Apache Airflow, Dask/Spark, and emerging commercial platforms, highlighting their relative strengths and trade-offs in the context of typical HEP analysis workflows.
Primary author
Jonas Eppelt
Co-authors
Alexander Heidelbach
(Ludwig-Maximilians-Universität München)
Giacomo de Pietro
