changed images to custom distros, small doc fixes

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

README.md

@@ -10,6 +10,7 @@ This repository grew organically based on tests that I needed to perform in orde
 
 # 🥢 Tools used
 
+- Custom Collector images from [my collection of distributions](https://github.com/jpkrohling/otelcol-distributions). Anywhere a custom image is being used, you can use contrib if you prefer.
 - [`telemetrygen`](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/cmd/telemetrygen#installing) is used a lot in this repository to send telemetry data to our Collector instance
 - [`otelcol-contrib`](https://github.com/open-telemetry/opentelemetry-collector-releases/releases) is used as well, both in binary format for local examples, and as container image in examples using Kubernetes
 - [`k3d`](https://k3d.io) is used in the Kubernetes recipes, in order to create a local Kubernetes cluster

client-side-load-balancing/README.md

@@ -2,6 +2,8 @@
 
 This recipe shows how to configure a Collector to load balance its exporter requests to another layer of Collectors. While the solution itself is very simple, comprising of two lines of configuration, verifying that it works is trickier: we want to start with a fixed number of server instances, likely higher than 1, and scale up. If it works, we'll see that all instances will have equivalent load over time, with the rate of requests going down as more server collectors are available. 
 
+For a successful load-balancing, we need clients that are able to retrieve a list of available endpoints, and we need the server to force connections to be closed at a specific amount of time. While the client configuration can be reused as is, the server configuration in this example requires some attention before being used in production: make sure to adjust the max connection age to a value that makes sense to you.
+
 We are discarding the telemetry data that we are generating, as we are only interested in assessing this behavior by observing the Collector's own metrics.
 
 ## 🧄 Ingredients
@@ -51,9 +53,9 @@ We are discarding the telemetry data that we are generating, as we are only inte
    telemetrygen traces --rate 100 --duration 10m --otlp-insecure --otlp-attributes='recipe="client-side-load-balancing"'
    ```
 
-9. After a few minutes, update the `otelcol-server.yaml` to have 10 replicas instead of the original 5. 
+9. After a few minutes, update the `otelcol-server.yaml` to have 10 replicas instead of the original 5.
 
-10. Open your Grafana instance, go to Explore, and select the metrics datasource ("...-prom") and a run a query like: `rate(receiver_accepted_spans_total[$__rate_interval])`
+10. Open your Grafana instance, go to Explore, and select the metrics datasource ("...-prom") and a run a query like: `rate(receiver_accepted_spans_total[$__rate_interval])`. You can also remove the `rate` function to see the totals, which should show the first collectors having similar numbers among themselves, while the newer collectors would have lower numbers, but still equivalent among each other.
 
 ## 😋 Executed last time with these versions
 

client-side-load-balancing/otelcol-server.yaml

@@ -3,13 +3,16 @@ kind: OpenTelemetryCollector
 metadata:
   name: otelcol-server
 spec:
-  image: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib:0.101.0
+  image: ghcr.io/jpkrohling/otelcol-distributions/otelcol-otlp:0.101.5
   replicas: 5
   config:
     receivers:
       otlp:
         protocols:
-          grpc: {}
+          grpc:
+            keepalive:
+              server_parameters:
+                max_connection_age: 1m
 
     exporters:
       nop: {}

grafana-cloud-from-kubernetes/otelcol-cr.yaml

@@ -3,7 +3,7 @@ kind: OpenTelemetryCollector
 metadata:
   name: grafana-cloud-from-kubernetes
 spec:
-  image: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib:0.101.0
+  image: ghcr.io/jpkrohling/otelcol-distributions/otelcol-otlp:0.101.5
   envFrom:
   - secretRef: 
       name: grafana-cloud-credentials

kafka-on-kubernetes/otelcol-pub.yaml

@@ -3,7 +3,7 @@ kind: OpenTelemetryCollector
 metadata:
   name: otelcol-pub
 spec:
-  image: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib:0.101.0
+  image: ghcr.io/jpkrohling/otelcol-distributions/otelcol-kafka:0.101.5
   config:
     receivers:
       otlp:

kafka-on-kubernetes/otelcol-sub.yaml

@@ -3,7 +3,7 @@ kind: OpenTelemetryCollector
 metadata:
   name: otelcol-sub
 spec:
-  image: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib:0.101.0
+  image: ghcr.io/jpkrohling/otelcol-distributions/otelcol-kafka:0.101.5
   envFrom:
   - secretRef: 
       name: grafana-cloud-credentials

profiling-the-collector/README.md

@@ -5,6 +5,8 @@ This recipe shows how to use Pyroscope to profile the OpenTelemetry Collector ru
 - ["grafana-cloud"](../grafana-cloud/)
 - ["grafana-cloud-from-kubernetes"](../grafana-cloud-from-kubernetes/)
 
+We are discarding the telemetry data that we are generating, as we are only interested in assessing this behavior by observing the Collector's profiles.
+
 ## 🧄 Ingredients
 
 - OpenTelemetry Operator, see the main [`README.md`](../README.md) for instructions
@@ -25,47 +27,40 @@ This recipe shows how to use Pyroscope to profile the OpenTelemetry Collector ru
     kubens profiling-the-collector
    ```
 
-2. Create a secret with the credentials:
-   ```terminal
-    kubectl create secret generic grafana-cloud-credentials --from-literal=GRAFANA_CLOUD_USER="$GRAFANA_CLOUD_USER" --from-literal=GRAFANA_CLOUD_TOKEN="$GRAFANA_CLOUD_TOKEN"
-   ```
-
-3. Change the `endpoint` parameter for the `otlphttp` exporter to point to your stack's endpoint
-
-4. Install the OTel Collector custom resource
+2. Install the OTel Collector custom resource
    ```terminal
    kubectl apply -f otelcol-cr.yaml
    ```
 
-5. Open a port-forward to the Collector
+3. Open a port-forward to the Collector
    ```terminal
    kubectl port-forward svc/profiling-the-collector-collector 4317
    ```
 
-6. Send 10 traces per second for 30 minutes to our Collector
+4. Send 500 traces per second for 30 minutes to our Collector
    ```terminal
-   telemetrygen traces --duration 30m --rate 10 --otlp-insecure --otlp-attributes='recipe="profiling-the-collector"'
+   telemetrygen traces --duration 30m --rate 500 --otlp-insecure --otlp-attributes='recipe="profiling-the-collector"'
    ```
 
-7. Install a custom service exposing the `pprof` port
+5. Install a custom service exposing the `pprof` port
    ```terminal
    kubectl apply -f otelcol-pprof-svc.yaml
    ```
 
-8. Add the Grafana Helm repository
+6. Add the Grafana Helm repository
    ```terminal
    helm repo add grafana https://grafana.github.io/helm-charts
    helm repo update
    ```
 
-9.  Replace the user, token, and URL on the values.yaml to reflect the values from your environment variables
+7.  Replace the URL on the values.yaml to reflect your Pyroscope connection details
 
-10. Install Grafana Agent with the Pyroscope eBPF Profiler
+8.  Install Grafana Agent with the Pyroscope eBPF Profiler
    ```terminal
    helm install pyroscope-ebpf grafana/grafana-agent -f values.yaml --set username=$GRAFANA_CLOUD_PROFILES_USER --set password=$GRAFANA_CLOUD_PROFILES_TOKEN
    ```
 
-11. Open your Grafana instance, go to Explore, and select the profiles datasource.
+9.  Open your Grafana instance, go to Explore, and select the profiles datasource.
 
 
 ## 😋 Executed last time with these versions

profiling-the-collector/otelcol-cr.yaml

@@ -3,42 +3,32 @@ kind: OpenTelemetryCollector
 metadata:
   name: profiling-the-collector
 spec:
-  image: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib:0.101.0
-  envFrom:
-  - secretRef: 
-      name: grafana-cloud-credentials
+  image: ghcr.io/jpkrohling/otelcol-distributions/otelcol-otlp:0.101.5
   config:
     extensions:
       pprof:
         endpoint: 0.0.0.0:1777
-      basicauth:
-        client_auth:
-          username: "${env:GRAFANA_CLOUD_USER}"
-          password: "${env:GRAFANA_CLOUD_TOKEN}"
 
     receivers:
       otlp:
         protocols:
           grpc: {}
 
     exporters:
-      otlphttp:
-        endpoint: https://otlp-gateway-prod-eu-west-2.grafana.net/otlp
-        auth:
-          authenticator: basicauth
+      nop: {}
 
     service:
-      extensions: [ pprof, basicauth ]
+      extensions: [ pprof ]
       pipelines:
         traces:
           receivers:  [ otlp ]
           processors: [  ]
-          exporters:  [ otlphttp ]
+          exporters:  [ nop ]
         logs:
           receivers:  [ otlp ]
           processors: [  ]
-          exporters:  [ otlphttp ]
+          exporters:  [ nop ]
         metrics:
           receivers:  [ otlp ]
           processors: [  ]
-          exporters:  [ otlphttp ]
+          exporters:  [ nop ]