BUG: Treatment of scalars in input data varies by input format

[MImmesberger]

Is your feature request related to a problem?

When users provide a dataframe and a mapper, scalar values are broadcast to arrays matching the length of p_id. But when users provide the exact same data as an input tree, we do not broadcast; instead, we partially apply the scalars inside the TT function.

Describe the solution you would like to see

Since the partial-application feature does not work well yet, we should keep the current behavior of the dataframe + mapper pipeline; otherwise, casual users will run into errors that are nearly impossible to debug.

So either:

1. we find a way to fix the partial-application issue for all inputs, or
2. (preferred in the short run) broadcast scalar inputs for input trees as well, and require users who want to rely on partial-application to supply their inputs via processed_data instead.

[hmgaudecker]

Can you remind me what does not work well with partialling?

[MImmesberger]

1. We can't partial if the column feeds directly into a function that is defined on the array level (e.g. p_id_elternteil_1 because we need it in bg_id).
2. Inputs of derived functions cannot be partialled (e.g. partialling einkommensteuer__einkünfte__aus_selbstständiger_tätigkeit__betrag_m does not work; ...__betrag_y works), as laid out in Illustrate partialling errors of derived inputs in tt_function #54. (Will transform this to an issue soon, seems like we won't tackle that this year).

[hmgaudecker]

For now, documenting this here should be enough. After all, casual users should not use the pytree version. I think having different behaviour here is very natural; a tree simply is a different animal than a dataframe.