Description

Technical info

Data Format

The data are stored in various hdf5 files, each hdf file has several tables which can be loaded as pandas dataframes using their key:

import pandas as pd
df = pd.read_hdf( <filename>, key=<key> )

the tables are:

• event_info: values of the eight observables for each event

  • column names:['gen_nch', 'gen_mass', 'gen_sphericity', 'gen_transverseSpherocity','gen_thrust', 'gen_transverseThrust', 'gen_broadening', 'gen_isotropy','generated_weight']

• pseudodata_event_info: values of the eight observables for each event for the pseudodata truth (when such a sample exists)

  • column names:['gen_nch', 'gen_mass', 'gen_sphericity', 'gen_transverseSpherocity','gen_thrust', 'gen_transverseThrust', 'gen_broadening', 'gen_isotropy','generated_weight']

• nominal_weight_info: values of the weights for nominal unfolding result

  • column names: ['sample', 'stage', 'weight']

• syst_template_weight_info: values of the weights for 1 standard deviation shifts of the uncertainty sources

  • column names: ['sample', 'stage', 'weight']

• stat_toy_weight_info: values of the weights for toy unfolding samples used to estimate the statistical uncertainties

  • column names: ['sample', 'stage', 'weight']

• syst_toy_weight_info: values of the weights for toy unfolding samples used to estimate the systematic uncertainty

  • column names: ['sample', 'stage', 'weight']

• frequentist_toy_weight_info: values of weights for frequentist toy unfolding samples used to estimate bias and coverage

  • column names: ['sample', 'stage', 'weight']

The events and weights are stored in separate trees, but match one-to-one by their index values, such that one can get all relevant information by joining the dataframes.  The pseudodata and toy related tables do not exist for all files. For example:

import pandas as pd
dfe = pd.read_hdf('unfolded_info_data.h5', key='event_info')
dfw = pd.read_hdf('unfolded_info_data.h5', key='nominal_weight_info')
df = dfe.join( dfw )

will produce a single dataframe with all of the relevant event info and weight info per sample. Note that the weight dataframes contain multiple entries per event, corresponding to different samples, for the same event the same index is always used.

Technical info

Files

The file unfolded_info_data.h5 contains the unfolded event data as well as all toy samples used for producing the uncertainty from the main analysis.

The files unfolded_info_pseudodata_test_<X>.h5 contain validation data where one simulated Monte Carlo sample has been used to unfold the other. Each of these files use one Pythia tune to unfold another, where the name of the Pythia tune is indicated in the filename and in the `sample` field of the relevant tables.

The file unfolded_info_pseudodata_test_CUEP8M1withA3.h5  contains the frequentist_toy_weight_info table for bias and coverage studies. 

Technical info

Sample Code

We provide some basic code for running over the data to produce histograms in yoda format, including estimation of errors from the toy datasets or from templates. Some code is also provided for running some closure, bias, and coverage checks on one of the pseudodata samples (CP5 unfolded with A3), where frequentist toys have been produced in order to check the bias and coverage properties.

The sample code can be found in the git repository: 

https://gitlab.cern.ch/cms-analysis/smp/smp-23-008

Any important updates to the code are stored as tags in the repository. Additional details on running the code are in the README of the repository.

This code uses the input to produce yoda histograms as well as run some validation tests such as closure and coverage.

The primary analysis results as stored in hep-data can be reproduced using:

python make_yoda_histograms.py -d [path to data hdf file] --toys-syst --toys-stat

This produces a yoda file with the histograms of the results of the unfolding as well as files storing each of the histograms produced for the toy datasets used to estimate the uncertainties.

Because of the size of the data, one might need to use `ulimit -s unlimited` before running, or producing the toy histograms separately first with:

python make_yoda_histograms.py -d [path to data hdf file] --toys-syst
python make_yoda_histograms.py -d [path to data hdf file] --toys-stat
python make_yoda_histograms.py -d [path to data hdf file] --hists-syst [path to toy histograms from step 1] --hists-stat [path to toy histograms from step 2] 

When running any changes such as different binnings or combinations of observables, it is always recommend to run closure, coverage and bias tests with the provided pseudodata samples in order to estimate the reliability of the unfolding performance for the given setup.