feature request: grafana pyroscope

[Cluas]

https://github.com/grafana/phlare

[Cluas]

I would be interested to know what the result of trying Phlare with Qryn is. Have you tried to run it? Does it not work with Loki already out of the box or via grafana agent?
…
On Mon, Nov 14, 2022, 22:14 Cluas @.> wrote: https://github.com/grafana/phlare — Reply to this email directly, view it on GitHub <#249>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLWWEXBZP3NST2M4I2DVYLWIMEZRANCNFSM6AAAAAASAQBDIU . You are receiving this because you are subscribed to this thread.Message ID: @.>

https://grafana.com/docs/phlare/latest/operators-guide/configure-agent/about-the-agent/

In the future, the agent will be integrated into the Grafana Agent, which will remove the needs to run a standalone agent if you’re already running the Grafana Agent.

api can be found hear. https://github.com/grafana/phlare/blob/main/proto/push/v1/push.proto

[Cluas]

We now use https://github.com/parca-dev/parca, We modified its code to replace the underlying storage with clickhouse, but parca's grafana plugin is not pretty 😂

[Cluas]

[lmangani]

@Cluas this is most definitely going to be supported, and we'll begin integration as soon as Grafana documents the APIs, since ingestion is only half of the picture. We have no time to reverse engineer the client API right now but if you do, please share the endpoints and requests Grafana uses to accelerate the integration process.

[Cluas]

@Cluas this is most definitely going to be supported, and we'll begin integration as soon as Grafana documents the APIs, since ingestion is only half of the picture. We have no time to reverse engineer the client API right now but if you do, please share the endpoints and requests Grafana uses to accelerate the integration process.

https://github.com/grafana/phlare/blob/main/pkg/openapiv2/gen/phlare.swagger.json

I think this is what we need.

[adubovikov]

We now use https://github.com/parca-dev/parca, We modified its code to replace the underlying storage with clickhouse, but parca's grafana plugin is not pretty joy

do you insert the data directly to clickhouse or thru own gRPC server ?

[Cluas]

We now use https://github.com/parca-dev/parca, We modified its code to replace the underlying storage with clickhouse, but parca's grafana plugin is not pretty joy

do you insert the data directly to clickhouse or thru own gRPC server ?

For direct insertion, I implemented the Querier and MetadataStore interfaces, abstracting the Ingester interface. metastore flag adds the type clickhouse, a new profilestore flag with the option of frostdb or clickhouse.

[lmangani]

https://github.com/grafana/phlare/blob/main/pkg/openapiv2/gen/phlare.swagger.json

I think this is what we need.

Indeed, if you like puzzles! I think as we did for Loki & Co, we need to observe a actual integration and enumerate the MVP APIs we need to support to get started and to establish analogies with our existing underlying formats. We're willing to try this, but we need samples. Any material is super welcome to push this integration forward.

[tomershafir]

@lmangani hey again :) key grafana issues to push forward for qryn integration by pyroscope slack thread:

1. [DOC] Document API for Push and Querrier services grafana/pyroscope#2156
2. Merge grafana/phlare repo into grafana/pyroscope grafana/pyroscope#2010
   btw pyroscope cloud is now GA

I can probably help contributing, though I would prefer the go version =)

[lmangani]

The proto files for ingestion and query (the only two function this seems to take) are on github so the task is mostly about creating a new table and the required handlers. I would guess this integration should be structurally similar to the tempo handlers, and probably much simpler in terms of complexity being only about a couple endpoints with few parameters.

https://github.com/grafana/pyroscope/blob/next/api/querier/v1/querier.proto
https://github.com/grafana/pyroscope/blob/next/api/ingester/v1/ingester.proto

[tomershafir]

write path edge is the distributor, and read path edge is indeed the querier, so these are the apis we need:
https://github.com/grafana/pyroscope/blob/main/api/querier/v1/querier.proto
https://github.com/grafana/pyroscope/blob/main/api/push/v1/push.proto

they released v1.0 btw, here is their api stability note

[lmangani]

... and thanks to the amazing @tomershafir and @akvlad this idea eventually became a reality! @Cluas let us know what you think if you get around trying this, or feel free to help us add more ingestion protocols in otel-collector 😉