Table-driven live data interpretation?

[aaeegg]

Currently, we're using C code to decode Volvo live data. It might be nice to instead use a table-driven approach, with a "little language" such as algebraic expressions. We could drop in a small, suitably licensed postfix expression parser. For J1979, we're already using table-driven decoding, but it's too simplistic because it only has a scale and offset for each PID - can't handle bit fields, can't handle multiple values in one response, any "multiple choice" values need individual formatter functions like format_fuel().

Table entries might look something like this:

{
"bat", /* identifier, for friendly CLI commands instead of using numeric address */
"Battery voltage", /* description */
0x7a, NS_LIVEDATA, 0x0300, /* ECU, namespace, address */
"%s", /* overall format string */
{{"x0*29750/8250*5/255", /* scaling formula, output is always double-precision float */
format_printf, "%.1f V" /* formatting function and template */
}, NULL}
},

{
"atfv",
"ATF temperature sensor voltage",
0x6e, NS_LIVEDATA, 0x0c00,
"%s",
{{"(x0*256+x1)*5/1023", /* decoding a two-byte value */
format_printf, "%.2f V"
}, NULL
},

{
"ect",
"Engine Coolant Temperature",
0x7a, NS_LIVEDATA, 0x0200,
"%s",
{{"x1-80",
format_temperature, /* displays C and/or F according to configuration */
(void *)0 /* digits after the decimal point for format_temperature */
}, NULL}
},

{
"modesw",
"Mode selector",
0x6e, NS_LIVEDATA, 0x0500,
"MS1 %s, MS2 %s, switch position %s", /* Multiple values decoded together */
{{"x0&1",
format_enum,
(char *[]){"low","high", NULL} /* enumeration formatting from integer portion of scaled value */
},{"(x0/2)&1", /* bitfield extraction */
format_enum,
(char *[]){"low","high", NULL}
},{"x0",
format_enum,
(char *[]){"Open","S","E","W","Unknown", NULL} /* Out-of-range values fold to last table entry */
}, NULL}
},

NULL

Or we could embed a Lua interpreter and use Lua expressions for scaling and formatting - although this is kind of going full circle back to code:

{
"bat",
"Battery voltage",
0x7a, NS_LIVEDATA, 0x0300,
"v[0]",
{{"x[1]*29750/8250*5/255",
"string.format('%.1f V',x)"
}, NULL}
}

This would also apply to scaling for other vehicles we might add in the future. Also, table-driven decoding would make it easier to display in a GUI, return in an API, or log to CSV files if we ever add any of those things.

[fenugrec]

I'm not opposed to this, indeed - when merging your latest D2 decoding additions, I was thinking "hmm this looks like it may soon need a table or something, like J1979".
I'm not entirely happy with the J1979 decoding, for the reasons you stated and others.

Lua sounds like total overkill, but I'm open to ideas. I imagine if some fields need conditional formatting based on e.g. bit fields, something more elaborate may be needed.

We could drop in a small, suitably licensed postfix expression parser.

I think you are right on track there, one thing I don't want is a reinvented homemade wheel.

[aaeegg]

For fields that need conditional formatting that can't be expressed in a formula, we could put in an escape hatch to call a C function instead of using a formula.

One thing I'm not entirely satisfied with in my previously proposed table format is how it combines decoding/scaling with presentation. If we were adding other frontends like a dashboard app, then we'd want a way to get speed as a number, not a formatted string. So maybe we want to split into one table for scaling and one for CLI:

struct scaling foo[]={
{
"coolant_temperature", /* internal parameter name */
0x7a, NS_LIVEDATA, 0x0200, /* ECU, namespace, address */
scale_algebraic, /* standard or custom scaling function */
"x1-80", /* formula */
"Celsius", /* units; formatter recognizes unit name and displays as C and/or F */
0, /* digits of precision after the decimal point */
},
{
"ms1",
0x6e, NS_LIVEDATA, 0x0500,
scale_algebraic,
"x0&1",
"bool",
0
},
{
"driving_mode",
0x6e, NS_LIVEDATA, 0x0500,
scale_algebraic,
"x1",
"enum",
0
}, NULL
};

struct presentation bar[]={
{"ect", /* identifier for CLI */
"Engine Coolant Temperature: $coolant_temperature"
}, NULL
};

[fenugrec]

For fields that need conditional formatting that can't be expressed in a formula, we could put in an escape hatch to call a C function instead of using a formula.

sure, that might work. Some difficulties arise when you want to render a value to a fixed width field though - I mean how to convey that width accross layers.

One thing I'm not entirely satisfied with in my previously proposed table format is how it combines decoding/scaling with presentation

I ran into a similar challenge for the "diag_cfgi", where I wanted to abstract configuration items like int, bool, string... ended up with a not very satisfactory union and an enum to indicate the type.
Same thing will happen here for e.g. binary flags or even values like " number of DTCs pending" where a float is perhaps not an adequate generic representation.

Another thing to add to the numeric parameters, that GUIs often like, is an expected min/max range. Nobody wants a temperature gauge that reads from -1E37 to 1E37 : )