Machinery for systematics studies in Qn::StatCalculate

[eugene274]

(draft based on discussion with Ilya)

To evaluate systematic uncertainties we compare different estimations (Q-vector components, reference subevent, cuts, etc).
Currently Qn::StatCalculate provides machinery to evaluate only statistical uncertainties.
To evaluate systematic uncertainties one has to operate either on exported TGraph or play with DataContainer and develop own structures to keep variations and maintain consistency during various transformations.

Ilya proposed to extend current Qn::StatCalculate with internal field (structure) Systematics which will keep systematic variations and calculate significance of variation.
Variations (V) are added to 'reference' (R) estimation (it is usually all components and all references combined), so I propose method: Qn::StatCalculate::AddVariation (const Qn::StatCalculate& variation)

Internal structure Systematics contains list of systematic variations each of them consists of:

• Variation itself (Qn::StatCalculate?)
• Sample-by-sample difference between variation (V) and the (R) reference. It could be Qn::StatCalculare object (R-V).

For QA purposes I propose to keep (or calculate ad-hoc) in Systematics a matrix with correlations:

      ||       R       |      V1       |      V2       | ...
|  R  ||       1       |  cor(R,V1)    | cor(R,V2)     | ...
|  V1 || cor(V1,R)     |         1     | cor(V1,V2)    | ... 

Where correlation coefficient is obtained as follows

cor(R,V) = sum_{b-st. samples} (R_s - <R>) (V_s - <V>)/Var(R)/Var(V)

Using this matrix one can obtain 'independent' (uncorrelated) sources of systematic uncertainty by changing basis.
See:
https://en.wikipedia.org/wiki/Principal_component_analysis and
https://root.cern.ch/doc/master/classTPrincipal.html

[eugene274]

This feature request is open for discussion

[kreisl]

Good suggestion! I am very much in favour of implementing such a feature!
I have some rather technical questions:

1. If variations are part of the StatCalculate object and are therefore propagated through the calculations, we need to make sure that either all StatCalculate objects which are part of the calculation of the observable include this variations, or it falls back to the reference if the variation is not found (e.g. when calculating a ratio and the variation is only present in the numerator).

2. I guess it would be good to have a new base class so we do not have a recursive relation in the StatsCalculate object. I am not sure if a variation of a variation makes sense.

3. We need to think about how to add new methods to DataContainer<StatsCalculate> objects. Right now we add the functionality by hand through template specialisation in the DataContainer header file. We might need many methods in future ... I haven't really though about how to solve this problem.

[eugene274]

1. I don't think we need to propagate systematics through the arithmetic operations (maybe for unary only). I guess (hope) we'll use this machinery on flow coefficients which are kinda final-state objects and shouldn't be modified anymore. If some operation is needed, user shall perform this operation upon all variations and then take care on systematics.
2. I agree with you statement on 'variation of variation', but making the new entity requires many auxiliary objects to be introduced: all ToTGraph-s, conversions from/to StatCalculate, etc.
3. Are you talking about polymorphic DataContainer<T> interface which depends on T?

[kreisl]

1. For example: <Q_A2, Q_2B> / R{Q_2B}, where the numerator is the observed flow and the denominator is the resolution correction. Would the systematic variations be added before calculation of this ratio or afterward.
   If they are added before, we would need to treat it correctly when dividing. For example, a case could be where a systematic variation only affects the sub event A and not sub event B. If this is not included it should prevent calculations from happening after variations are added.
   I would be in favour of including the "propagation of the variations" this functionality, as it would cover most of the systematic variations that I encountered.
   I am open to discussing this in more detail.

2. Yes that is true. Let's think what would make more sense. Maybe we could visualise the structure of the classes to help us decide.

3. I have to think about it I don't know how to solve it. I am open to ideas. I am referring to the template specialisations for example the lines starting at:

   QnTools/include/base/QnTools/DataContainer.hpp

   Line 945 in c6afd8f

   // Template specializations for visualisation //