diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 848eb10..4064419 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,4 +1,10 @@ -# Contributing your recipes +# 🍚 Contributing your recipes -Do you have a cool recipe to contribute? That's great! Just open a PR with your -configuration YAML, along with a description on the readme file. +Do you have a cool recipe to contribute? That's great! Here's what I look for when reviewing a contribution: + +- Keep a narrow focus and demonstrate one concept or integration +- The recipe has to be reproducible for everyone +- Specify the versions of the software you used for testing the recipe +- It should be self-contained. It might reference other recipes, but everything needed to succeed is included in the recipe itself +- Unless you are demonstrating a specific exporter, export the telemetry to Grafana Cloud for consistency with the other recipes +- Recipes featuring other commercial vendors are OK as long as they can be reproduced by everyone with a free account. If the main purpose of the recipe is to show how to export data to this vendor, propose one that is similar to [`grafana-cloud`](./grafana-cloud/) diff --git a/README.md b/README.md index 3b5447e..a636cab 100644 --- a/README.md +++ b/README.md @@ -1,34 +1,38 @@ -# OpenTelemetry Collector Recipes by jpkrohling +# 🧑🏼‍🍳🍲 OpenTelemetry Collector Cookbook This repository has a personal collection of OpenTelemetry Collector recipes curated by [@jpkrohling](https://github.com/jpkrohling). -**Note** that only the recipes referenced in the README files were tested. The others are provided as-is, and may not work. +## 📔 Recipes -## Recipes +Each directory in this repository is a recipe and has an appropriate readme file with instructions. -The recipes are organized in the following categories: +This repository grew organically based on tests that I needed to perform in order to verify a bug report, create an example configuration for a Grafana Labs customer, or prepare for a presentation. As such, quite a few recipes lack good descriptions and running instructions. Those recipes are found under ["ratatouille"](./ratatouille/). -* [Simple](recipes/simple/README.md#simple-recipes): simple recipes that can be used as a starting point for more complex scenarios -* [Grafana](recipes/grafana/): recipes to use with Grafana products and projects, including Loki, Tempo, Mimir, and Grafana Cloud -* [Kubernetes](recipes/kubernetes/): recipes to use on Kubernetes, including the Operator -* [Load balancing](recipes/load-balancing/): recipes to use with the load balancing exporter -* [OTTL](recipes/ottl/): recipes that use components featuring OpenTelemetry Transformation Language (OTTL) -* [Routing](recipes/routing/): recipes to use with the routing processor -* [Tail-sampling](recipes/tail-sampling/): recipes to use with the tail-sampling processor +# 🥢 Tools used -# Pre-requisites +- [`telemetrygen`](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/cmd/telemetrygen#installing) is used a lot in this repository to send telemetry data to our Collector instance +- [`otelcol-contrib`](https://github.com/open-telemetry/opentelemetry-collector-releases/releases) is used as well, both in binary format for local examples, and as container image in examples using Kubernetes +- [`k3d`](https://k3d.io) is used in the Kubernetes recipes, in order to create a local Kubernetes cluster +- [OpenTelemetry Operator](https://github.com/open-telemetry/opentelemetry-operator) is used in most Kubernetes recipes. See the setup instructions below +- `kubens` from the [`kubectx`](https://github.com/ahmetb/kubectx) project -Most of the recipes send data to the Collector using the [`telemetrygen`](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/cmd/telemetrygen#installing) command-line tool. +## 🍴 OpenTelemetry Operator -Unless otherwise specified, the recipes will work with the contrib distribution of the collector, which can be started like this: +To get a working instance of the OpenTelemetry Operator, [follow the official instructions](https://github.com/open-telemetry/opentelemetry-operator?tab=readme-ov-file#getting-started) from the project, but here's a quick summary of what's needed for our purposes: -```commandline -otelcol-contrib --config recipes/simple/simplest.yaml +```terminal +k3d cluster create + +kubectl apply -f https://github.com/cert-manager/cert-manager/releases/latest/download/cert-manager.yaml +kubectl wait --for=condition=Available deployments/cert-manager -n cert-manager + +kubectl apply -f https://github.com/open-telemetry/opentelemetry-operator/releases/latest/download/opentelemetry-operator.yaml +kubectl wait --for=condition=Available deployments/opentelemetry-operator-controller-manager -n opentelemetry-operator-system ``` -# Bugs +# 🪳 Bugs -Did you find a bug? Is a recipe confusing, or not working at all? Please [open an issue](https://github.com/jpkrohling/otelcol-configs/issues/new). Make sure to include: +Did you find a bug? Is a recipe confusing, or not working at all? Please [open an issue](https://github.com/jpkrohling/otelcol-cookbook/issues/new). Make sure to include: - the recipe name - the command you used to run the recipe diff --git a/grafana-cloud-from-kubernetes/README.md b/grafana-cloud-from-kubernetes/README.md new file mode 100644 index 0000000..4b277d9 --- /dev/null +++ b/grafana-cloud-from-kubernetes/README.md @@ -0,0 +1,54 @@ +# 🍜 Recipe: Grafana Cloud from Kubernetes + +This recipe shows how to send telemetry data to Grafana Cloud with the OpenTelemetry Collector running on a Kubernetes Cluster. This is very similar to the ["grafana-cloud"](../grafana-cloud/) recipe, with extra steps to make it run on Kubernetes. You are encouraged to try that one first if you are not familiar with the Collector or sending data to Grafana Cloud via OTLP. + +One interesting aspect of this recipe is that it keeps the credentials on a secret, mounting them as environment variables. This way, the secrets aren't exposed in plain-text. + +## 🧄 Ingredients + +- OpenTelemetry Operator, see the main [`README.md`](../README.md) for instructions +- The `telemetrygen` tool, or any other application that is able to send OTLP data to our collector +- The `otelcol-cr.yaml` file from this directory +- A `GRAFANA_CLOUD_USER` environment variable, also known as Grafana Cloud instance ID, found under the instructions for "OpenTelemetry" on your Grafana Cloud stack. +- A `GRAFANA_CLOUD_TOKEN` environment variable, which can be generated under the instructions for "OpenTelemetry" on your Grafana Cloud stack. +- The endpoint for your stack + +## 🥣 Preparation + +1. Create and switch to a namespace for our recipe + ```terminal + kubectl create ns grafana-cloud-from-kubernetes + kubens grafana-cloud-from-kubernetes + ``` + +2. Create a secret with the credentials: + ```terminal + kubectl create secret generic grafana-cloud-credentials --from-literal=GRAFANA_CLOUD_USER="..." --from-literal=GRAFANA_CLOUD_TOKEN="..." + ``` + +3. Change the `endpoint` parameter for the `otlphttp` exporter to point to your stack's endpoint + +4. Install the OTel Collector custom resource + ```terminal + kubectl apply -f grafana-cloud-from-kubernetes/otelcol-cr.yaml + ``` + +5. Open a port-forward to the Collector: + ```terminal + kubectl port-forward svc/grafana-cloud-from-kubernetes-collector 4317 + ``` + +6. Send some telemetry to your Collector + ```terminal + telemetrygen traces --traces 2 --otlp-insecure --otlp-attributes='recipe="grafana-cloud-from-kubernetes"' + ``` + +7. Open your Grafana instance, go to Explore, and select the appropriate datasource, such as "...-traces". If used the command above, click "Search" and you should see two traces listed, each with two spans. + +## 😋 Executed last time with these versions + +The most recent execution of this recipe was done with these versions: + +- OpenTelemetry Operator v0.100.1 +- OpenTelemetry Collector Contrib v0.101.0 +- `telemetrygen` v0.101.0 diff --git a/grafana-cloud-from-kubernetes/otelcol-cr.yaml b/grafana-cloud-from-kubernetes/otelcol-cr.yaml new file mode 100644 index 0000000..93979c2 --- /dev/null +++ b/grafana-cloud-from-kubernetes/otelcol-cr.yaml @@ -0,0 +1,42 @@ +apiVersion: opentelemetry.io/v1beta1 +kind: OpenTelemetryCollector +metadata: + name: grafana-cloud-from-kubernetes +spec: + image: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib:0.101.0 + envFrom: + - secretRef: + name: grafana-cloud-credentials + config: + extensions: + basicauth: + client_auth: + username: "${env:GRAFANA_CLOUD_USER}" + password: "${env:GRAFANA_CLOUD_TOKEN}" + + receivers: + otlp: + protocols: + grpc: {} + + exporters: + otlphttp: + endpoint: https://otlp-gateway-prod-eu-west-2.grafana.net/otlp + auth: + authenticator: basicauth + + service: + extensions: [ basicauth ] + pipelines: + traces: + receivers: [ otlp ] + processors: [ ] + exporters: [ otlphttp ] + logs: + receivers: [ otlp ] + processors: [ ] + exporters: [ otlphttp ] + metrics: + receivers: [ otlp ] + processors: [ ] + exporters: [ otlphttp ] diff --git a/grafana-cloud/README.md b/grafana-cloud/README.md new file mode 100644 index 0000000..1811742 --- /dev/null +++ b/grafana-cloud/README.md @@ -0,0 +1,36 @@ +# 🍜 Recipe: Grafana Cloud + +This recipe shows how to send telemetry data to Grafana Cloud with the OpenTelemetry Collector. The recommended way to send data for most use-cases is via the OTLP endpoint, given that the Collector already has an internal representation on that format. It's also better to send OTLP Logs via the OTLP endpoint to Grafana Cloud Logs, as it would appropriately place some values, such as the trace ID and span ID, on its new metadata storage, which isn't part of the message body or index. + +## 🧄 Ingredients + +- The `telemetrygen` tool, or any other application that is able to send OTLP data to our collector +- The `otelcol.yaml` file from this directory +- A `GRAFANA_CLOUD_USER` environment variable, also known as Grafana Cloud instance ID, found under the instructions for "OpenTelemetry" on your Grafana Cloud stack. +- A `GRAFANA_CLOUD_TOKEN` environment variable, which can be generated under the instructions for "OpenTelemetry" on your Grafana Cloud stack. +- The endpoint for your stack + +## 🥣 Preparation + +1. Change the `endpoint` parameter for the `otlphttp` exporter to point to your stack's endpoint + +2. Export the environment variables `GRAFANA_CLOUD_USER` and `GRAFANA_CLOUD_TOKEN` + +3. Run a Collector distribution with the provided configuration file + ```terminal + otelcol-contrib --config otelcol.yaml + ``` + +4. Send some telemetry to your Collector + ```terminal + telemetrygen traces --traces 2 --otlp-insecure --otlp-attributes='recipe="grafana-cloud"' + ``` + +5. Open your Grafana instance, go to Explore, and select the appropriate datasource, such as "...-traces". If used the command above, click "Search" and you should see two traces listed, each with two spans. + +## 😋 Executed last time with these versions + +The most recent execution of this recipe was done with these versions: + +- OpenTelemetry Collector Contrib v0.101.0 +- `telemetrygen` v0.101.0 diff --git a/recipes/grafana/grafana-cloud-otlp-gw-us-central2.yaml b/grafana-cloud/otelcol.yaml similarity index 60% rename from recipes/grafana/grafana-cloud-otlp-gw-us-central2.yaml rename to grafana-cloud/otelcol.yaml index ca2ac4a..93626ca 100644 --- a/recipes/grafana/grafana-cloud-otlp-gw-us-central2.yaml +++ b/grafana-cloud/otelcol.yaml @@ -4,9 +4,6 @@ extensions: username: "${env:GRAFANA_CLOUD_USER}" password: "${env:GRAFANA_CLOUD_TOKEN}" -processors: - batch: - receivers: otlp: protocols: @@ -14,7 +11,7 @@ receivers: exporters: otlphttp: - endpoint: https://otlp-gateway-prod-us-central-0.grafana.net/otlp + endpoint: https://otlp-gateway-prod-eu-west-2.grafana.net/otlp auth: authenticator: basicauth @@ -25,3 +22,11 @@ service: receivers: [ otlp ] processors: [ ] exporters: [ otlphttp ] + logs: + receivers: [ otlp ] + processors: [ ] + exporters: [ otlphttp ] + metrics: + receivers: [ otlp ] + processors: [ ] + exporters: [ otlphttp ] diff --git a/kafka-on-kubernetes/README.md b/kafka-on-kubernetes/README.md new file mode 100644 index 0000000..e86c6d0 --- /dev/null +++ b/kafka-on-kubernetes/README.md @@ -0,0 +1,72 @@ +# 🍜 Recipe: Kafka on Kubernetes + +This recipe uses a Kafka cluster between two layers of Collectors, showing how to scale a collection pipeline that is able to absorb spikes in traffic without putting extra pressure on the backend. The idea is that the first layer might scale according to the demand, sending data to Kafka, which is consumed by a static set of Collectors. This architecture is suitable for scaling Collectors when the backend can eventually catch-up with the traffic, as is usually the case with sudden spikes. + +Note that we've used the `transform` processor to add the current timestamp to all spans at both layers (`published_at` and `consumed_at`), so that we know when we sent something to the queue, and when we've consumed it from the queue. + +## 🧄 Ingredients + +- OpenTelemetry Operator, see the main [`README.md`](../README.md) for instructions +- A Kafka Cluster and one topic for each telemetry data type (metric, logs, traces) +- The `telemetrygen` tool, or any other application that is able to send OTLP data to our collector +- The `otelcol-cr.yaml` file from this directory +- A `GRAFANA_CLOUD_USER` environment variable, also known as Grafana Cloud instance ID, found under the instructions for "OpenTelemetry" on your Grafana Cloud stack. +- A `GRAFANA_CLOUD_TOKEN` environment variable, which can be generated under the instructions for "OpenTelemetry" on your Grafana Cloud stack. +- The endpoint for your stack + +## 🥣 Preparation + +1. Install [Strimzi](https://strimzi.io), a Kubernetes Operator for Kafka + ```terminal + kubectl create ns kafka + kubens kafka + + kubectl create -f 'https://strimzi.io/install/latest?namespace=kafka' + kubectl wait --for=condition=Available deployments/strimzi-cluster-operator --timeout=300s + ``` + +2. Install the Kafka cluster and topics for our recipe + ```terminal + kubectl apply -f kafka-for-otelcol.yaml + kubectl wait kafka/kafka-for-otelcol --for=condition=Ready --timeout=300s + ``` + +3. Create and switch to a namespace for our recipe + ```terminal + kubectl create ns kafka-on-kubernetes + kubens kafka-on-kubernetes + ``` + +4. Create a secret with the credentials: + ```terminal + kubectl create secret generic grafana-cloud-credentials --from-literal=GRAFANA_CLOUD_USER="..." --from-literal=GRAFANA_CLOUD_TOKEN="..." + ``` + +5. Create the OTel Collector custom resources that publishes to and consumes from the Kafka topic + ```terminal + kubectl apply -f otelcol-pub.yaml + kubectl apply -f otelcol-sub.yaml + kubectl wait --for=condition=Available deployments/otelcol-pub-collector + kubectl wait --for=condition=Available deployments/otelcol-sub-collector + ``` + +6. Open a port-forward to the Collector that is publishing to Kafka: + ```terminal + kubectl port-forward svc/otelcol-pub-collector 4317 + ``` + +7. Send some telemetry to your Collector + ```terminal + telemetrygen traces --traces 2 --otlp-insecure --otlp-attributes='recipe="kafka-on-kubernetes"' + ``` + +8. Open your Grafana instance, go to Explore, and select the appropriate datasource, such as "...-traces". If used the command above, click "Search" and you should see two traces listed, each with two spans. + +## 😋 Executed last time with these versions + +The most recent execution of this recipe was done with these versions: + +- Strimzi v0.41.0 +- OpenTelemetry Operator v0.100.1 +- OpenTelemetry Collector Contrib v0.101.0 +- `telemetrygen` v0.101.0 diff --git a/kafka-on-kubernetes/kafka-for-otelcol.yaml b/kafka-on-kubernetes/kafka-for-otelcol.yaml new file mode 100644 index 0000000..2791da8 --- /dev/null +++ b/kafka-on-kubernetes/kafka-for-otelcol.yaml @@ -0,0 +1,63 @@ +apiVersion: kafka.strimzi.io/v1beta2 +kind: KafkaNodePool +metadata: + name: dual-role + labels: + strimzi.io/cluster: kafka-for-otelcol +spec: + replicas: 1 + roles: + - controller + - broker + storage: + type: jbod + volumes: + - id: 0 + type: persistent-claim + size: 100Gi + deleteClaim: false + kraftMetadata: shared +--- +apiVersion: kafka.strimzi.io/v1beta2 +kind: Kafka +metadata: + name: kafka-for-otelcol + annotations: + strimzi.io/node-pools: enabled + strimzi.io/kraft: enabled +spec: + kafka: + version: 3.7.0 + metadataVersion: 3.7-IV4 + config: + offsets.topic.replication.factor: 1 + transaction.state.log.replication.factor: 1 + transaction.state.log.min.isr: 1 + default.replication.factor: 1 + min.insync.replicas: 1 + listeners: + - name: plain + port: 9092 + type: internal + tls: false +--- +apiVersion: kafka.strimzi.io/v1beta2 +kind: KafkaTopic +metadata: + name: otlp-spans + labels: + strimzi.io/cluster: kafka-for-otelcol +--- +apiVersion: kafka.strimzi.io/v1beta2 +kind: KafkaTopic +metadata: + name: otlp-metrics + labels: + strimzi.io/cluster: kafka-for-otelcol +--- +apiVersion: kafka.strimzi.io/v1beta2 +kind: KafkaTopic +metadata: + name: otlp-logs + labels: + strimzi.io/cluster: kafka-for-otelcol diff --git a/kafka-on-kubernetes/otelcol-pub.yaml b/kafka-on-kubernetes/otelcol-pub.yaml new file mode 100644 index 0000000..4028c13 --- /dev/null +++ b/kafka-on-kubernetes/otelcol-pub.yaml @@ -0,0 +1,34 @@ +apiVersion: opentelemetry.io/v1beta1 +kind: OpenTelemetryCollector +metadata: + name: otelcol-pub +spec: + image: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib:0.101.0 + config: + receivers: + otlp: + protocols: + http: {} + grpc: {} + + processors: + transform: + error_mode: ignore + trace_statements: + - context: span + statements: + - set(attributes["published_at"], UnixMilli(Now())) + + exporters: + kafka: + protocol_version: 2.0.0 + brokers: kafka-for-otelcol-kafka-brokers.kafka.svc.cluster.local:9092 + topic: otlp-spans + + service: + pipelines: + traces: + receivers: [ otlp ] + processors: [ transform ] + exporters: [ kafka ] + diff --git a/kafka-on-kubernetes/otelcol-sub.yaml b/kafka-on-kubernetes/otelcol-sub.yaml new file mode 100644 index 0000000..d6af99c --- /dev/null +++ b/kafka-on-kubernetes/otelcol-sub.yaml @@ -0,0 +1,44 @@ +apiVersion: opentelemetry.io/v1beta1 +kind: OpenTelemetryCollector +metadata: + name: otelcol-sub +spec: + image: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib:0.101.0 + envFrom: + - secretRef: + name: grafana-cloud-credentials + config: + extensions: + basicauth: + client_auth: + username: "${env:GRAFANA_CLOUD_USER}" + password: "${env:GRAFANA_CLOUD_TOKEN}" + + receivers: + kafka: + protocol_version: 2.0.0 + brokers: kafka-for-otelcol-kafka-brokers.kafka.svc.cluster.local:9092 + topic: otlp-spans + + processors: + transform: + error_mode: ignore + trace_statements: + - context: span + statements: + - set(attributes["consumed_at"], UnixMilli(Now())) + + exporters: + otlphttp: + endpoint: https://otlp-gateway-prod-eu-west-2.grafana.net/otlp + auth: + authenticator: basicauth + + service: + extensions: [ basicauth ] + pipelines: + traces: + receivers: [ kafka ] + processors: [ transform ] + exporters: [ otlphttp ] + diff --git a/own-telemetry/README.md b/own-telemetry/README.md new file mode 100644 index 0000000..7a5eeb3 --- /dev/null +++ b/own-telemetry/README.md @@ -0,0 +1,62 @@ +# 🍜 Recipe: Own telemetry + +This recipe shows how to explore the Collector own's telemetry. Note that we are using the `debug` exporter here, so that we don't get confused about what's telemetry processed by the Collector and what's telemetry from the Collector. + +## 🧄 Ingredients + +- The `telemetrygen` tool, or any other application that is able to send OTLP data to our collector +- The `otelcol.yaml` file from this directory +- A `GRAFANA_CLOUD_USER` environment variable, also known as Grafana Cloud instance ID, found under the instructions for "OpenTelemetry" on your Grafana Cloud stack. +- A `GRAFANA_CLOUD_TOKEN` environment variable, which can be generated under the instructions for "OpenTelemetry" on your Grafana Cloud stack. +- The combination of user and password as an HTTP basic auth header +- The endpoint for your stack + +## 🥣 Preparation + +1. Generate a basic auth HTTP header + ```terminal + echo -n "$GRAFANA_CLOUD_USER:$GRAFANA_CLOUD_TOKEN" | base64 -w0 + ``` + +2. Use the output from the command above as the value for the basic auth header on `otelcol.yaml` + +3. Change the `endpoint` parameter for the `otlp` exporter within the `telemetry` node to point to your stack's endpoint, plus the path `/v1/traces` + +4. Run a Collector distribution with the provided configuration file + ```terminal + otelcol-contrib --config otelcol.yaml + ``` + +5. Send some telemetry to your Collector + ```terminal + telemetrygen traces --traces 2 --otlp-insecure --otlp-attributes='recipe="own-telemetry"' + ``` + +6. At console where you started the Collector, you should see log entries reporting the lifecycle of the Collector, eventually reaching a log line similar to the following: + ``` + 2024-05-28T14:26:13.135+0200 info service@v0.101.0/telemetry.go:103 Serving metrics {"address": ":8888", "level": "Normal"} + 2024-05-28T14:26:13.136+0200 info service@v0.101.0/service.go:169 Starting otelcol-contrib... {"Version": "0.101.0", "NumCPU": 16} + 2024-05-28T14:26:13.136+0200 info service@v0.101.0/service.go:195 Everything is ready. Begin running and processing data. + ``` + +7. Open `localhost:8888/metrics` and explore the available metrics there, especially the following ones: + ```prometheus + # HELP otelcol_exporter_sent_spans Number of spans successfully sent to destination. + # TYPE otelcol_exporter_sent_spans counter + otelcol_exporter_sent_spans{exporter="debug",service_instance_id="890d4346-b65e-4ad9-a071-0ada4fb43143",service_name="otelcol-contrib",service_version="0.101.0"} 4 + + # HELP otelcol_receiver_accepted_spans Number of spans successfully pushed into the pipeline. + # TYPE otelcol_receiver_accepted_spans counter + otelcol_receiver_accepted_spans{receiver="otlp",service_instance_id="890d4346-b65e-4ad9-a071-0ada4fb43143",service_name="otelcol-contrib",service_version="0.101.0",transport="grpc"} 4 + ``` + +8. Open your Grafana instance, go to Explore, and select the traces datasource. You should see one trace with three spans, representing the incoming gRPC connection, the receiver operation, and the exporter operation + +9. Switch to your metrics datasource, and check that you have metrics like `receiver_accepted_spans_total` and `exporter_sent_spans_total`. It's normal if they take a couple of minutes to show up there. + +## 😋 Executed last time with these versions + +The most recent execution of this recipe was done with these versions: + +- OpenTelemetry Collector Contrib v0.101.0 +- `telemetrygen` v0.101.0 diff --git a/own-telemetry/otelcol.yaml b/own-telemetry/otelcol.yaml new file mode 100644 index 0000000..68d5fba --- /dev/null +++ b/own-telemetry/otelcol.yaml @@ -0,0 +1,41 @@ +receivers: + otlp: + protocols: + http: + grpc: + +exporters: + debug: + +service: + pipelines: + traces: + receivers: [otlp] + exporters: [debug] + metrics: + receivers: [otlp] + exporters: [debug] + logs: + receivers: [otlp] + exporters: [debug] + telemetry: + traces: + processors: + - batch: + schedule_delay: 1000 + exporter: + otlp: + endpoint: https://otlp-gateway-prod-eu-west-2.grafana.net/otlp/v1/traces + protocol: http/protobuf + headers: + Authorization: "Basic OTQyNzM4OmdsY19leUp2SWpvaU5USTVNemN4SWl3aWJpSTZJbk4wWVdOckxUazBNamN6T0MxdmRHeHdMWGR5YVhSbExXOTBiSEF0TWpBeU5DMHdOUzB5TnlJc0ltc2lPaUl6YldReGJEVTFOMDgzT0RCWVRXUTFkSGxGYldwRFdUZ2lMQ0p0SWpwN0luSWlPaUp3Y205a0xXVjFMWGRsYzNRdE1pSjlmUT09" + metrics: + level: detailed + readers: + - periodic: + exporter: + otlp: + endpoint: https://otlp-gateway-prod-eu-west-2.grafana.net/otlp/v1/metrics + protocol: http/protobuf + headers: + Authorization: "Basic OTQyNzM4OmdsY19leUp2SWpvaU5USTVNemN4SWl3aWJpSTZJbk4wWVdOckxUazBNamN6T0MxdmRHeHdMWGR5YVhSbExXOTBiSEF0TWpBeU5DMHdOUzB5TnlJc0ltc2lPaUl6YldReGJEVTFOMDgzT0RCWVRXUTFkSGxGYldwRFdUZ2lMQ0p0SWpwN0luSWlPaUp3Y205a0xXVjFMWGRsYzNRdE1pSjlmUT09" diff --git a/recipes/grafana/loki-receiver-auth.yaml b/ratatouille/grafana/loki-receiver-auth.yaml similarity index 100% rename from recipes/grafana/loki-receiver-auth.yaml rename to ratatouille/grafana/loki-receiver-auth.yaml diff --git a/recipes/kubernetes/client-side-load-balancing/README.md b/ratatouille/kubernetes/client-side-load-balancing/README.md similarity index 100% rename from recipes/kubernetes/client-side-load-balancing/README.md rename to ratatouille/kubernetes/client-side-load-balancing/README.md diff --git a/recipes/kubernetes/client-side-load-balancing/resources.yaml b/ratatouille/kubernetes/client-side-load-balancing/resources.yaml similarity index 100% rename from recipes/kubernetes/client-side-load-balancing/resources.yaml rename to ratatouille/kubernetes/client-side-load-balancing/resources.yaml diff --git a/recipes/kubernetes/k8sevents/k8sevents.yaml b/ratatouille/kubernetes/k8sevents/k8sevents.yaml similarity index 100% rename from recipes/kubernetes/k8sevents/k8sevents.yaml rename to ratatouille/kubernetes/k8sevents/k8sevents.yaml diff --git a/recipes/kubernetes/k8sevents/workload.yaml b/ratatouille/kubernetes/k8sevents/workload.yaml similarity index 100% rename from recipes/kubernetes/k8sevents/workload.yaml rename to ratatouille/kubernetes/k8sevents/workload.yaml diff --git a/recipes/kubernetes/manual-sidecar.yaml b/ratatouille/kubernetes/manual-sidecar.yaml similarity index 100% rename from recipes/kubernetes/manual-sidecar.yaml rename to ratatouille/kubernetes/manual-sidecar.yaml diff --git a/recipes/kubernetes/otelcol.yaml b/ratatouille/kubernetes/otelcol.yaml similarity index 100% rename from recipes/kubernetes/otelcol.yaml rename to ratatouille/kubernetes/otelcol.yaml diff --git a/recipes/kubernetes/sidecar-cr.yaml b/ratatouille/kubernetes/sidecar-cr.yaml similarity index 100% rename from recipes/kubernetes/sidecar-cr.yaml rename to ratatouille/kubernetes/sidecar-cr.yaml diff --git a/recipes/kubernetes/sidecar-workload.yaml b/ratatouille/kubernetes/sidecar-workload.yaml similarity index 100% rename from recipes/kubernetes/sidecar-workload.yaml rename to ratatouille/kubernetes/sidecar-workload.yaml diff --git a/recipes/kubernetes/target-allocator.yaml b/ratatouille/kubernetes/target-allocator.yaml similarity index 100% rename from recipes/kubernetes/target-allocator.yaml rename to ratatouille/kubernetes/target-allocator.yaml diff --git a/recipes/kubernetes/workload.yaml b/ratatouille/kubernetes/workload.yaml similarity index 100% rename from recipes/kubernetes/workload.yaml rename to ratatouille/kubernetes/workload.yaml diff --git a/recipes/load-balancing-exporter/README.md b/ratatouille/load-balancing-exporter/README.md similarity index 100% rename from recipes/load-balancing-exporter/README.md rename to ratatouille/load-balancing-exporter/README.md diff --git a/recipes/load-balancing-exporter/cr-load-balancer.yaml b/ratatouille/load-balancing-exporter/cr-load-balancer.yaml similarity index 100% rename from recipes/load-balancing-exporter/cr-load-balancer.yaml rename to ratatouille/load-balancing-exporter/cr-load-balancer.yaml diff --git a/recipes/ottl/routing.yml b/ratatouille/ottl/routing.yml similarity index 100% rename from recipes/ottl/routing.yml rename to ratatouille/ottl/routing.yml diff --git a/recipes/ottl/sensitive-log-body.yaml b/ratatouille/ottl/sensitive-log-body.yaml similarity index 100% rename from recipes/ottl/sensitive-log-body.yaml rename to ratatouille/ottl/sensitive-log-body.yaml diff --git a/recipes/ottl/tail-sampling.yaml b/ratatouille/ottl/tail-sampling.yaml similarity index 100% rename from recipes/ottl/tail-sampling.yaml rename to ratatouille/ottl/tail-sampling.yaml diff --git a/recipes/routing/conn-tenants.yaml b/ratatouille/routing/conn-tenants.yaml similarity index 100% rename from recipes/routing/conn-tenants.yaml rename to ratatouille/routing/conn-tenants.yaml diff --git a/recipes/routing/proc-tenants.yaml b/ratatouille/routing/proc-tenants.yaml similarity index 100% rename from recipes/routing/proc-tenants.yaml rename to ratatouille/routing/proc-tenants.yaml diff --git a/recipes/routing/two-matching-routes.yaml b/ratatouille/routing/two-matching-routes.yaml similarity index 100% rename from recipes/routing/two-matching-routes.yaml rename to ratatouille/routing/two-matching-routes.yaml diff --git a/recipes/simple/README.md b/ratatouille/simple/README.md similarity index 100% rename from recipes/simple/README.md rename to ratatouille/simple/README.md diff --git a/recipes/simple/auth.yml b/ratatouille/simple/auth.yml similarity index 100% rename from recipes/simple/auth.yml rename to ratatouille/simple/auth.yml diff --git a/recipes/simple/blocking.yaml b/ratatouille/simple/blocking.yaml similarity index 100% rename from recipes/simple/blocking.yaml rename to ratatouille/simple/blocking.yaml diff --git a/recipes/simple/complete-auth.yaml b/ratatouille/simple/complete-auth.yaml similarity index 100% rename from recipes/simple/complete-auth.yaml rename to ratatouille/simple/complete-auth.yaml diff --git a/recipes/simple/complete-oauth.yaml b/ratatouille/simple/complete-oauth.yaml similarity index 100% rename from recipes/simple/complete-oauth.yaml rename to ratatouille/simple/complete-oauth.yaml diff --git a/recipes/simple/contrib-9086-jaeger-receiver-with-remote-sampling.yaml b/ratatouille/simple/contrib-9086-jaeger-receiver-with-remote-sampling.yaml similarity index 100% rename from recipes/simple/contrib-9086-jaeger-receiver-with-remote-sampling.yaml rename to ratatouille/simple/contrib-9086-jaeger-receiver-with-remote-sampling.yaml diff --git a/recipes/simple/count-before-sampling.yaml b/ratatouille/simple/count-before-sampling.yaml similarity index 100% rename from recipes/simple/count-before-sampling.yaml rename to ratatouille/simple/count-before-sampling.yaml diff --git a/recipes/simple/debug-26557.yaml b/ratatouille/simple/debug-26557.yaml similarity index 100% rename from recipes/simple/debug-26557.yaml rename to ratatouille/simple/debug-26557.yaml diff --git a/recipes/simple/include-debug-exporter.yaml b/ratatouille/simple/include-debug-exporter.yaml similarity index 100% rename from recipes/simple/include-debug-exporter.yaml rename to ratatouille/simple/include-debug-exporter.yaml diff --git a/recipes/simple/include.yaml b/ratatouille/simple/include.yaml similarity index 100% rename from recipes/simple/include.yaml rename to ratatouille/simple/include.yaml diff --git a/recipes/simple/issue5610.yaml b/ratatouille/simple/issue5610.yaml similarity index 100% rename from recipes/simple/issue5610.yaml rename to ratatouille/simple/issue5610.yaml diff --git a/recipes/simple/jaegerremotesampling.yaml b/ratatouille/simple/jaegerremotesampling.yaml similarity index 100% rename from recipes/simple/jaegerremotesampling.yaml rename to ratatouille/simple/jaegerremotesampling.yaml diff --git a/recipes/simple/logs-to-metrics.yaml b/ratatouille/simple/logs-to-metrics.yaml similarity index 100% rename from recipes/simple/logs-to-metrics.yaml rename to ratatouille/simple/logs-to-metrics.yaml diff --git a/recipes/simple/oidc-auth-agent.yml b/ratatouille/simple/oidc-auth-agent.yml similarity index 100% rename from recipes/simple/oidc-auth-agent.yml rename to ratatouille/simple/oidc-auth-agent.yml diff --git a/recipes/simple/oidc-auth-collector.yml b/ratatouille/simple/oidc-auth-collector.yml similarity index 100% rename from recipes/simple/oidc-auth-collector.yml rename to ratatouille/simple/oidc-auth-collector.yml diff --git a/recipes/simple/own-telemetry.yaml b/ratatouille/simple/own-telemetry.yaml similarity index 100% rename from recipes/simple/own-telemetry.yaml rename to ratatouille/simple/own-telemetry.yaml diff --git a/recipes/simple/receiver-auth.yml b/ratatouille/simple/receiver-auth.yml similarity index 100% rename from recipes/simple/receiver-auth.yml rename to ratatouille/simple/receiver-auth.yml diff --git a/recipes/simple/receiver-tls.yml b/ratatouille/simple/receiver-tls.yml similarity index 100% rename from recipes/simple/receiver-tls.yml rename to ratatouille/simple/receiver-tls.yml diff --git a/recipes/simple/rejekts-local.yaml b/ratatouille/simple/rejekts-local.yaml similarity index 100% rename from recipes/simple/rejekts-local.yaml rename to ratatouille/simple/rejekts-local.yaml diff --git a/recipes/simple/rejekts.yaml b/ratatouille/simple/rejekts.yaml similarity index 100% rename from recipes/simple/rejekts.yaml rename to ratatouille/simple/rejekts.yaml diff --git a/recipes/simple/resilient-log-pipeline.yaml b/ratatouille/simple/resilient-log-pipeline.yaml similarity index 100% rename from recipes/simple/resilient-log-pipeline.yaml rename to ratatouille/simple/resilient-log-pipeline.yaml diff --git a/recipes/simple/simplest.yaml b/ratatouille/simple/simplest.yaml similarity index 100% rename from recipes/simple/simplest.yaml rename to ratatouille/simple/simplest.yaml diff --git a/recipes/tail-sampling/README.md b/ratatouille/tail-sampling/README.md similarity index 100% rename from recipes/tail-sampling/README.md rename to ratatouille/tail-sampling/README.md diff --git a/recipes/tail-sampling/always-on.yaml b/ratatouille/tail-sampling/always-on.yaml similarity index 100% rename from recipes/tail-sampling/always-on.yaml rename to ratatouille/tail-sampling/always-on.yaml diff --git a/recipes/tail-sampling/and.yaml b/ratatouille/tail-sampling/and.yaml similarity index 100% rename from recipes/tail-sampling/and.yaml rename to ratatouille/tail-sampling/and.yaml diff --git a/recipes/tail-sampling/multiple-ottl.yaml b/ratatouille/tail-sampling/multiple-ottl.yaml similarity index 100% rename from recipes/tail-sampling/multiple-ottl.yaml rename to ratatouille/tail-sampling/multiple-ottl.yaml diff --git a/recipes/tail-sampling/multiple.yaml b/ratatouille/tail-sampling/multiple.yaml similarity index 100% rename from recipes/tail-sampling/multiple.yaml rename to ratatouille/tail-sampling/multiple.yaml diff --git a/recipes/tail-sampling/not-vip.yaml b/ratatouille/tail-sampling/not-vip.yaml similarity index 100% rename from recipes/tail-sampling/not-vip.yaml rename to ratatouille/tail-sampling/not-vip.yaml diff --git a/recipes/tail-sampling/only-2-percent-successful-specific-endpoint-otherwise-everything.yaml b/ratatouille/tail-sampling/only-2-percent-successful-specific-endpoint-otherwise-everything.yaml similarity index 100% rename from recipes/tail-sampling/only-2-percent-successful-specific-endpoint-otherwise-everything.yaml rename to ratatouille/tail-sampling/only-2-percent-successful-specific-endpoint-otherwise-everything.yaml diff --git a/recipes/tail-sampling/probabilistic.yaml b/ratatouille/tail-sampling/probabilistic.yaml similarity index 94% rename from recipes/tail-sampling/probabilistic.yaml rename to ratatouille/tail-sampling/probabilistic.yaml index 8bd79bb..b4e2420 100644 --- a/recipes/tail-sampling/probabilistic.yaml +++ b/ratatouille/tail-sampling/probabilistic.yaml @@ -18,7 +18,7 @@ processors: ] exporters: - logging: + debug: connectors: @@ -30,4 +30,4 @@ service: processors: - tail_sampling exporters: - - logging + - debug diff --git a/recipes/tail-sampling/root-longer-than-decisionwait.yaml b/ratatouille/tail-sampling/root-longer-than-decisionwait.yaml similarity index 100% rename from recipes/tail-sampling/root-longer-than-decisionwait.yaml rename to ratatouille/tail-sampling/root-longer-than-decisionwait.yaml diff --git a/recipes/tail-sampling/tail-sampling-with-spanmetrics.yaml b/ratatouille/tail-sampling/tail-sampling-with-spanmetrics.yaml similarity index 100% rename from recipes/tail-sampling/tail-sampling-with-spanmetrics.yaml rename to ratatouille/tail-sampling/tail-sampling-with-spanmetrics.yaml diff --git a/recipes/tail-sampling/vip.yaml b/ratatouille/tail-sampling/vip.yaml similarity index 100% rename from recipes/tail-sampling/vip.yaml rename to ratatouille/tail-sampling/vip.yaml diff --git a/recipes/bytevalue/payload.json b/recipes/bytevalue/payload.json deleted file mode 100644 index 03a52fe..0000000 --- a/recipes/bytevalue/payload.json +++ /dev/null @@ -1,51 +0,0 @@ -{ - "resourceSpans": [ - { - "resource": { - "attributes": [ - { - "key": "service.name", - "value": { - "stringValue": "my.service" - } - } - ] - }, - "scopeSpans": [ - { - "scope": { - "name": "my.library", - "version": "1.0.0", - "attributes": [ - { - "key": "my.scope.attribute", - "value": { - "stringValue": "some scope attribute" - } - } - ] - }, - "spans": [ - { - "traceId": "5B8EFFF798038103D269B633813FC60C", - "spanId": "EEE19B7EC3C1B174", - "parentSpanId": "EEE19B7EC3C1B173", - "name": "I'm a server span", - "startTimeUnixNano": 1544712660000000000, - "endTimeUnixNano": 1544712661000000000, - "kind": 2, - "attributes": [ - { - "key": "my.span.attr", - "value": { - "bytesValue": "YWJjMTIzCg==" - } - } - ] - } - ] - } - ] - } - ] -} \ No newline at end of file diff --git a/recipes/grafana/grafana-cloud.yaml b/recipes/grafana/grafana-cloud.yaml deleted file mode 100644 index d72104a..0000000 --- a/recipes/grafana/grafana-cloud.yaml +++ /dev/null @@ -1,33 +0,0 @@ -extensions: - basicauth/traces: - client_auth: - username: "239237" - password: "..." - -processors: - batch: - -receivers: - otlp: - protocols: - grpc: - -exporters: - debug: - verbosity: detailed - otlphttp: - endpoint: https://otlp-gateway-prod-us-central-0.grafana.net/otlp - auth: - authenticator: basicauth/traces - -service: - extensions: [ basicauth/traces ] - pipelines: - traces: - receivers: [ otlp ] - processors: [ ] - exporters: [ otlphttp ] - metrics: - receivers: [ otlp ] - processors: [ ] - exporters: [ debug, otlphttp ] diff --git a/recipes/grafana/journald-to-grafana-cloud-logs.yaml b/recipes/grafana/journald-to-grafana-cloud-logs.yaml deleted file mode 100644 index 75e1492..0000000 --- a/recipes/grafana/journald-to-grafana-cloud-logs.yaml +++ /dev/null @@ -1,23 +0,0 @@ -extensions: - basicauth/logs: - client_auth: - username: "${LOGS_USER_ID}" - password: "${TOKEN}" - -receivers: - journald: - directory: /var/log/journal - -exporters: - loki: - endpoint: "${LOGS_URL}" - auth: - authenticator: basicauth/logs - -service: - extensions: [ basicauth/logs ] - pipelines: - logs: - receivers: [ journald ] - processors: [ ] - exporters: [ loki ] diff --git a/recipes/grafana/local-loki.yaml b/recipes/grafana/local-loki.yaml deleted file mode 100644 index 0148dcd..0000000 --- a/recipes/grafana/local-loki.yaml +++ /dev/null @@ -1,40 +0,0 @@ -receivers: - journald: - directory: /var/log/journal/a04e3a44cdd740f88d6a7ae3bb8c70cf - -exporters: - logging: - loglevel: debug - loki: - endpoint: http://localhost:3100/loki/api/v1/push - tls: - insecure: true - labels: - attributes: - host.name: host_name - format: json - tenant: - source: attributes - value: tenant - -processors: - attributes: - actions: - - action: insert - key: host.name - value: guarana - resource: - attributes: - - action: insert - key: tenant - value: globex - -extensions: - -service: - extensions: - pipelines: - logs: - receivers: [journald] - processors: [attributes, resource] - exporters: [logging, loki] diff --git a/recipes/grafana/multiple-pipelines.yaml b/recipes/grafana/multiple-pipelines.yaml deleted file mode 100644 index 1da3bcb..0000000 --- a/recipes/grafana/multiple-pipelines.yaml +++ /dev/null @@ -1,40 +0,0 @@ -receivers: - otlp: - endpoint: localhost:4317 - - otlp/2: - endpoint: localhost:4417 - -exporters: - otlp: - endpoint: localhost:4417 - headers: - TenanID: acme - header_source: - key: tenant - from_context: ... - - otlp/grafanacloud: - endpoint: http://loki.cloud.grafana.net:3100/loki/api/v1/push - auth: - authenticator: multitenancy - key: X-Scope-ID - from_context: TenantID - -extensions: - -service: - extensions: - pipelines: - traces: - receivers: [otlp] - processors: [] - exporters: [otlp] - - traces/with-tenant: - receivers: [otlp/2] - processors: [] - exporters: [otlp/grafanacloud] - - -## tracegen -otlp-endpoint localhost:4317 -otlp-insecure \ No newline at end of file diff --git a/recipes/grafana/own-telemetry.yaml b/recipes/grafana/own-telemetry.yaml deleted file mode 100644 index cdd475a..0000000 --- a/recipes/grafana/own-telemetry.yaml +++ /dev/null @@ -1,42 +0,0 @@ -extensions: - basicauth/otlp: - client_auth: - username: 239237 - password: eyJrIjoiMGJmOTNlZDc2ZDZlM2I3NjYzYWQ2ODY2ZTNiY2U3M2MwM2I5ZTVhYiIsIm4iOiJsb2NhbC1jb2xsZWN0b3Itc2VsZi1zY3JhcGluZyIsImlkIjo1MjkzNzF9 - -receivers: - otlp: - protocols: - grpc: - prometheus/self: - config: - scrape_configs: - - job_name: otel-collector-otlp - scrape_interval: 5s - static_configs: - - targets: [localhost:8888] - -exporters: - logging/traces: - logging/metrics: - verbosity: detailed - - otlphttp: - auth: - authenticator: basicauth/otlp - endpoint: https://otlp-gateway-prod-us-central-0.grafana.net/otlp - -processors: - -service: - extensions: [basicauth/otlp] - pipelines: - traces: - receivers: [otlp] - processors: - exporters: [logging/traces] - - metrics: - receivers: [prometheus/self] - processors: - exporters: [logging/metrics, otlphttp] diff --git a/recipes/grafana/remote-loki.yaml b/recipes/grafana/remote-loki.yaml deleted file mode 100644 index c47cbbe..0000000 --- a/recipes/grafana/remote-loki.yaml +++ /dev/null @@ -1,39 +0,0 @@ -receivers: - journald: - directory: /var/log/journal/a04e3a44cdd740f88d6a7ae3bb8c70cf - -exporters: - logging: - loglevel: debug - loki: - endpoint: https://logs-prod-us-central1.grafana.net/loki/api/v1/push - headers: - authorization: Basic ODc5NDA6ZXlKcklqb2labVpsWWpsaE56SXpPRGRsTW1aalpXRTBOVGN3TVRkbU16VXlNakl4TUdJellqYzBNbVEzTVNJc0ltNGlPaUpqYjJ4c1pXTjBiM0lpTENKcFpDSTZOVEk1TXpjeGZRPT0= - labels: - attributes: - host.name: host_name - format: json - tenant: - source: attributes - value: tenant - -processors: - attributes: - actions: - - action: insert - key: host.name - value: guarana - resource: - - action: insert - key: tenant - value: acme - -extensions: - -service: - extensions: - pipelines: - logs: - receivers: [journald] - processors: [attributes, resource] - exporters: [logging, loki] diff --git a/recipes/grafana/scale-article.yaml b/recipes/grafana/scale-article.yaml deleted file mode 100644 index d27fc63..0000000 --- a/recipes/grafana/scale-article.yaml +++ /dev/null @@ -1,45 +0,0 @@ -extensions: - basicauth/traces: - client_auth: - username: "${TRACES_USER_ID}" - password: "${TOKEN}" - -receivers: - otlp: - protocols: - grpc: - endpoint: localhost:4317 - otlp/2: - protocols: - grpc: - endpoint: localhost:5317 - -processors: - batch: - send_batch_size: 256 - send_batch_max_size: 256 - -exporters: - otlp/2: - endpoint: localhost:5317 - tls: - insecure: true - otlp/tempo: - endpoint: tempo-us-central1.grafana.net:443 - auth: - authenticator: basicauth/traces - -service: - extensions: [ basicauth/traces ] - pipelines: - traces: - receivers: [ otlp ] - processors: [ ] - exporters: [ otlp/2 ] - traces/2: - receivers: [ otlp/2 ] - processors: [ batch ] - exporters: [ otlp/tempo ] - telemetry: - metrics: - level: detailed diff --git a/recipes/grafana/tempo.yaml b/recipes/grafana/tempo.yaml deleted file mode 100644 index 30553b4..0000000 --- a/recipes/grafana/tempo.yaml +++ /dev/null @@ -1,40 +0,0 @@ -extensions: - basicauth/traces: - client_auth: - username: "${TRACES_USER_ID}" - password: "${TOKEN}" - - basicauth/logs: - client_auth: - username: "${LOGS_USER_ID}" - password: "${TOKEN}" - -processors: - batch: - -receivers: - otlp: - protocols: - grpc: - -exporters: - otlp/gct: - endpoint: tempo-us-central1.grafana.net:443 - auth: - authenticator: basicauth/traces - loki/gcl: - endpoint: "${LOGS_URL}" - auth: - authenticator: basicauth/logs - -service: - extensions: [ basicauth/traces, basicauth/logs ] - pipelines: - traces: - receivers: [ otlp ] - processors: [ ] - exporters: [ otlp/gct ] - logs: - receivers: [ otlp ] - processors: [ ] - exporters: [ loki/gcl ]