Refactor mixin with new grafonnet

[skl]

This mixin currently depends on a deprecated version of grafonnet. At Grafana Labs, we're looking to contribute a refactor to the latest version.

Please note the compatibility discussion from a similar effort here:

• Refactor mixin with grafonnet etcd-io/etcd#16246

Asking early if this is likely to be an acceptable contribution?

EDIT this is happenening here:

• Grafana angular deprecation in grafana 11 #898

[jon-rei]

I would really like to see this as I am having some problems viewing data in panels where there are a lot of values. The migration seems to fix this.
Is there any work being done on this?

[skl]

#898

[povilasv]

I think if we go same route of supporting older version with same grafana7x field, I would be happy to review / merge.

If possible smaller PRs, so it's easier to review .

Question did ectd mixin change break kube-prometheus / kube-prometheus-stack or did it all work fine?

[skl]

Maybe the author @v-zhuravlev can answer the above?

[v-zhuravlev]

There was an unintentional regression with etcd that should be fixed soon. Other than that it should be ok.

P.S. We are currently testing more decomposed format(modular observ lib) based on grafonnet lib. There, targets, dashboards, panels, templated variables, annotations etc - all defined separately, so it is very easy to reuse, filter, or modify separate panels,alerts, targets before generating the final result, even if there are no config flags for this alteration in the mixin.

Also, we suggest that each 'observ lib' should have arguments to help instantiating mixin multiple times in the same grafana without alerts or dashboard overlapping.

Variable selectors should be configurable by providing two groups of labels (groupLabels (identifies group of instances, like 'cluster') and instanceLabels (that indentifies single entityt (could pod or instance).

Examples:
Hello world lib (with structure similar to used in etcd, but updated): grafana/jsonnet-libs#1084
Windows observ lib(using same structure, but for actual feature rich windows mixin): grafana/jsonnet-libs#1050
Logs lib(the one we use to conditionally attach logs dashboard to any other mixin): https://github.com/grafana/jsonnet-libs/tree/master/logs-lib (and here #866)

For example, here variables and panels are modified by acessing them directly and applying grafonnet mixins, before getting final results;

local g = import 'github.com/grafana/grafonnet/gen/grafonnet-latest/main.libsonnet';
local logslib = import 'github.com/grafana/jsonnet-libs/logs-lib/logs/main.libsonnet';


local linuxFilterSelector = 'unit!=""';
local linuxLabels = ['job', 'instance', 'unit', 'level'];

// pick one of Loki's parsers to use: i.e. logfmt, json.
// | __error__=`` is appended automatically
// https://grafana.com/docs/loki/latest/logql/log_queries/#parser-expression
// set null or do not provide at all if parsing is not required.
local formatParser = 'unpack';

// 2. create and export systemd logs dashboard
local systemdLogs =
  logslib.new('Linux systemd logs',
                    datasourceRegex='',
                    filterSelector=linuxFilterSelector,
                    labels=linuxLabels,
                    formatParser=formatParser,
                    showLogsVolume=true)
  // override panels or variables using grafonnet
  {
    panels+:
      {
        logs+:
          g.panel.logs.options.withEnableLogDetails(false),
      },
    variables+:
      {
        regex_search+:
          g.dashboard.variable.textbox.new('regex_search', default='error'),
      },
  };
// export logs dashboard
systemdLogs.dashboards.logs

Once it gets more mature, I think this format can helpful for refactoring kubernetes mixin as well.

[nissessenap]

Any update on this?
I found this discussion after creating #898, and I read through the changes that were done in etcd mixins and what wasn't fixed in #884.

I do not understand why the community should strive to support backwards compatible to this degree as a community. Supporting a version so old as Grafana 7 is a bit crazy in my mind.
Backwards compatibility is great and all, but not to the degree that we can't support new features without breaking our back.

Grafana supports 2 version at the time, right now it's Grafana 10 and 9, and within a few weeks during Grafana Con, I assume Grafana 11 will get released.
If the community thinks that it's too much to ask to update Grafana once a year, even though Kubernetes only supports versions for 1 year, increase then number of supported Grafana versions. For example 3 versions instead of 2.

And if someone is using grafana 7 they can always use older versions of mixins, just like what was done with kube-state-metric compatibility.

But these are my two cents, and I'm not in a position where I know jsonnet, so I can't contribute any changes (which also makes me stuck with Grafana 10 unless this gets resolved). So no matter what you decide, thank you for working on mixins.

[skl]

Grafana will be able to migrate certain panels, please see this blog post for more information:

• Removal of AngularJS support in Grafana: what you need to know

[nissessenap]

I know, but for those of us that manages dashboards though code like terraform or grafana-operator it would help a lot of this was solved as code.
Instead of first having to apply the json to grafana and then have to export it again with the new formatted panels by grafana.

[skl]

Thank you for sharing your use case 🙂

[povilasv]

To be fair I think maybe Im wrong for asking to support Grafana 7, but there was a license change there and maybe some folks don't want to run with new license?

But to be honest would review and take any PR that wants to migrate and support new Grafana format.

[skl]

Closing discussion as the work is nearly complete over in:

• Grafana angular deprecation in grafana 11 #898