Update yll binning #650

davidwalter2 · 2026-01-16T21:57:24Z

Harmonizing the dataset resources

Dataset names are given by "dataset_era" for example "Zmumu_2016postVFP", "Zmumu_2017" etc.
The dataset names for data are changed to "primaryDataset_era" e.g. "SingleMuon_2018A" instead of just "Data" for similar reasons.
This allows to process datasets from different eras in one histmaker e.g. in the gen histmaker
The cross sections are now defined in common to have them at a single place

Update on binning

Switching to use 20 quantiles in yll for Z reco level fit, updating the quantile file correspondingly
Updating the angular coefficients file to the updated binning with different binning between W and Z. But using the same file now for all analyses (except theory agnostic).
- Verified to get the same results for the W (when using 2016postVFP samples only)
- The new file include the 2017 and 2018 samples, leading to small differences also for the W
Removed duplicated binning definition and tried to harmonize small differences
Update theory corrections with fine binning for the Z

Requires: https://gitlab.cern.ch/cms-wmass/wremnants-data/-/merge_requests/95

… unfolding and alphaS; update wremnants-data with this file

…convention for dataset names '<sample name>_<era>' for MC and '<primary dataset>_<era> for data'

…city corrections

…; in dyturbo always set bins with qT<1GeV to 0

into 260116_updBinning

kdlong · 2026-01-27T15:18:43Z

.github/workflows/main.yml

          $WREMNANTS_OUTDIR/WMass_eta_pt_charge_poiAsNoi/WMass_absEtaGen_ptGen_qGen_theoryfit/fitresults.hdf5
          --grouping max --config 'utilities/styles/styles.py' --postfix poiAsNoi
-          -o $WEB_DIR/$PLOT_DIR/unfolding_mw -n 50 --otherExtensions pdf png --showNumbers --oneSidedImpacts -s absimpact --scale 1.5
+          -o $WEB_DIR/$PLOT_DIR/unfolding_mw -n 50 --otherExtensions pdf png --showNumbers --oneSidedImpacts -s absimpact --scaleImpacts 1.5


I don't think you have to implement it for this PR, but it should be possible to take this from the pdfInfo to avoid it having to be manually synchronised.

I think we want to keep the rabbit scripts analysis independent so I don't see how the pdfInfo can be used here. Maybe through the specified config file which. But I agree to do this at some later time.

scripts/corrections/corrs_by_helicity/make_alphaS_gen_hists.py

kdlong · 2026-01-27T15:21:27Z

scripts/corrections/corrs_by_helicity/make_alphaS_gen_hists.py

+            -o {args.outdir} --maxFiles -1 -j 300 --addHelicityAxis --pdf {pdf} --aggregateGroups Zmumu Wmunu
+            """
        print(f"Running command: {command}")
        os.system(command)


os.system from a python script to call a python script is kind of criminal...

First of all, I didn't add this script. But I find it useful. If the histmaker scripts were written with a main function etc. it could be possible to import and run ... I would leave this for future work.

@kdlong, David speaks the truth: he bears no more guilt than you, it is I who is to be blamed for this terrible sin. Punish me as you see fit

kdlong · 2026-01-27T15:24:40Z

scripts/corrections/make_theory_corr.py

+                    fig, ax = plt.subplots(figsize=(6, 6))
+                    icorrh.plot(ax=ax, cmin=0.5, cmax=1.5)
+
+                    plot_name = f"corr2D_{generator}_MiNNLO_{proc}{suffix}"


Why are all these plots useful vs. just having one plot differential in Q?

For debugging. There is a break statement at the end of the loop so it will only plot the first one, but it can be commented. This also fixes the case the minnloh hist has variations, which were previously summed up

scripts/corrections/corrs_by_helicity/make_alphaS_gen_hists.py

scripts/corrections/corrs_by_helicity/make_pdf_from_corr_gen_hists.py

scripts/corrections/corrs_by_helicity/make_pdf_gen_hists.py

scripts/corrections/make_theory_corr.py

scripts/histmakers/w_z_gen_dists.py

scripts/rabbit/setupRabbit.py

wremnants/theory_corrections.py

into 260116_updBinning

davidwalter2 · 2026-01-28T15:18:34Z

The PR is ready to be merged from my side, the CI also looks reasonable:
This PR: https://cmsmwbot.web.cern.ch/WMassAnalysis/PRValidation/PR650/2026_01_28/lowPU/
Reference: https://cmsmwbot.web.cern.ch/WMassAnalysis/PRValidation/ReferenceRuns/2026_01_28_a04e38f/lowPU/

davidwalter2 added 9 commits January 16, 2026 16:16

Use 20 quantiles for yll instead of 10

55436a1

Update wremnants-data

150189d

Merge branch 'main' of github.com:WMass/WRemnants into 260116_updBinning

90eb410

Use new quantile file

c947d3f

Synchronize sample paths; Treat DY low mass similar to Zmumu

50b474f

Some more harminizations in datasets; common xsec definitions

eff3a82

Use single file for angular coefficients shared across main analysis,…

650b069

… unfolding and alphaS; update wremnants-data with this file

Fixes to previous commits

162950d

Update wremnants-data with new angular cofficient file

e486a7c

davidwalter2 force-pushed the 260116_updBinning branch from 101074d to 995126f Compare January 18, 2026 20:19

fix ci

c6c2197

davidwalter2 force-pushed the 260116_updBinning branch from 995126f to c6c2197 Compare January 18, 2026 23:59

davidwalter2 added 7 commits January 19, 2026 16:02

Rename samples ending with 'PostVFP' to '_2016PostVFP' and introduce …

de9dc1a

…convention for dataset names '<sample name>_<era>' for MC and '<primary dataset>_<era> for data'

Further unify binning; adapt 'make_theory_corr' to use different eras

db06e92

Change logic to maintain backwards compatibility with previous byHeli…

a388aaf

…city corrections

Fix typos

e0ece9d

Fix CI

9558dcb

Revert correcting DYLowMass dataset

56485e3

Update wremnants-data with new theory corrections

17db1fa

davidwalter2 force-pushed the 260116_updBinning branch from fe20c46 to 17db1fa Compare January 20, 2026 15:23

davidwalter2 added 3 commits January 20, 2026 16:20

Add fine binning in ptVgen and absYVGen option for theory corrections…

35cb7d0

…; in dyturbo always set bins with qT<1GeV to 0

Add reference to new BSM sample

f81bbb2

Use more fine binning for byHelicity corrections and uncertainties

e6750b9

davidwalter2 force-pushed the 260116_updBinning branch from b3625c6 to e6750b9 Compare January 21, 2026 18:14

davidwalter2 and others added 6 commits January 25, 2026 10:15

Adapt latest changes of iterative unfolding

97c6bd9

Add betavar templates

37e25ca

Update rabbit version, pass metadata to rabbit from setupCombine

1872570

Fix accidental change of submodules

6730177

Update theory corrections by helicity and helper scripts

443c3ad

Merge branch 'rabbitFileMetadata' of https://github.com/kdlong/WRemnants

e4d23c2

into 260116_updBinning

davidwalter2 added 2 commits January 26, 2026 19:54

Adapt CI to new version of rabbit

2f2eb45

Adapt CI to new version of rabbit

6fc0aef

kdlong reviewed Jan 27, 2026

View reviewed changes

scripts/corrections/corrs_by_helicity/make_alphaS_gen_hists.py Outdated Show resolved Hide resolved

kdlong reviewed Jan 27, 2026

View reviewed changes

lucalavezzo reviewed Jan 27, 2026

View reviewed changes

kdlong and others added 11 commits January 27, 2026 11:00

Set rabbit to the latest version

c25296f

Harmonize dataset dicts

0d7d1ba

Update theory correction scripts

0e95401

Add option to constrain NOI variations

ff7bfdb

update rabbit, exclude BB profiling from external postfit in CI

1876c9b

Merge branch 'rabbitFileMetadata' of https://github.com/kdlong/WRemnants

ff355ad

into 260116_updBinning

Update submodules rabbit and wremnant-sdata

782057f

Fix extended dataset option for lowPU

6d32ed2

Change CI back to previous logic for externalPostfit

798d145

Fix lowPU

a94b163

Fix low PU in CI and also fix plotting low PU unfolded distributions

9b07326

lucalavezzo merged commit ce1a03b into WMass:main Jan 28, 2026
19 checks passed

Update yll binning #650

Update yll binning #650

Uh oh!

Conversation

davidwalter2 commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kdlong Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

davidwalter2 Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kdlong Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

davidwalter2 Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

lucalavezzo Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

kdlong Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

davidwalter2 Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

davidwalter2 commented Jan 28, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

davidwalter2 commented Jan 16, 2026 •

edited

Loading