add auth to xds grpc

[lgadban]

No description provided.

[yuval-k]

more context:

Envoy side

On the envoy side, we need to send an auth token. We can create a projected service account jwt for the gateway deployment:

# ... rest of pod spec
        volumeMounts:
        - mountPath: /etc/podinfo/secrets/
          name: xds-svc-acct
          readOnly: true
      volumes:
      - name: xds-svc-acct
        projected:
          defaultMode: 420
          sources:
          - serviceAccountToken:
              audience: k8sgateway-server
              expirationSeconds: 600
              path: token

then modify the xds cluster in the bootstrap config to use it:

  - name: xds_cluster
    load_assignment:
      # ...
    typed_extension_protocol_options:
      envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
        "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
        explicit_http_config:
          http2_protocol_options: {}
        http_filters:
        - name: transform
          typed_config:
            "@type": type.googleapis.com/envoy.api.v2.filter.http.FilterTransformations
            transformations:
            - match:
                  prefix: "/"
              route_transformations:
                request_transformation:
                  transformation_template:
                    headers: { "authorization": {"text": "Bearer {{ trim(data_source(\"token\")) -}}"} }
                    passthrough: {}
                    data_sources:
                      token:
                        filename: "/etc/podinfo/secrets/token"
                        watched_directory:
                          path: "/etc/podinfo/secrets"

        - name: envoy.filters.http.upstream_codec
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.upstream_codec.v3.UpstreamCodec

this will add an authorization header to xDS requests. and when the token is rotated by kubelet, the new token will be used on new requests.

Control plane side

On the kgateway side, we need to validate the JWT (we can follow istio's example on how to do this - we grab the issuer from our own JWT and use it to discover open-id config and validate incoming JWTs)

The JWT will also give us the pod name. we follow the owner-refs all the way up (pod -> RS->Deployment->Gateway) to discovery which Gateway CR the pod belongs to, and associate the pod with that gateway. We would no longer need the "role" metadata we have today.

Follow-up

To have a complete story we can complement this with 1-way TLS to have full e2e security (we don't need mTLS because the client uses the JWT for auth). The only thing envoy would need in this case is the CA for the TLS connection. the CA not secret, so if the user uses a custom CA we can copy it to envoy's bootstrap config or as a config map in the gateway's namespace (similar to what k8s does with kube-root-ca.crt). The last thing we'll need to document is how to handle root CA rotation.