Skip to content

Support for Channel based APIs

Jump to bottom
Daniel Lamando edited this page Feb 10, 2023 · 5 revisions

The Kubernetes API contains a small number of special APIs which allow for bidirectional data flow:

  • Pod Exec
  • Pod Attach
  • Pod Port-forward

These APIs all use a Kubernetes concept called "channels" to expose their streaming functionality.

This library (/x/kubernetes_client) includes experimental support for those channel-based APIs. However, there are multiple significant restrictions, because of various causes which we will get into below.

TIP: For local development, it's very likely that these APIs will not successfully connect.
The most reliable workaround is launching a local authenticating proxy:

kubectl proxy --reject-paths='^-$'

Note that this can have security implications depending on what else has access to your localhost.

Kubernetes Background

Traditionally, the channel-based APIs in Kubernetes are served by apiserver over the SPDY networking protocol. Kubectl commands such as kubectl exec and kubectl port-forward continue to use SPDY to this day. Unfortunately, SPDY became an obsolete protocol, and new software does not support SPDY anymore.

SPDY's useful components were reused when defining the modern HTTP/2 specification. There is no cross-compatibility between the two, though. SPDY and HTTP/2 are two different protocols.

When Kubernetes was faced with the fading compatibility of SPDY, the path forward was adding WebSocket alternatives for the relevant APIs (exec/attach and port-forward). These endpoints are available in all recent Kubernetes control planes.

The WebSocket endpoints offered by Kubernetes have limitations compared to their SPDY ancestors. They are also awkward to connect to from browser-style contexts like Deno.

Limitations with /x/kubernetes_client

TLS authentication is not supported

  • Caused by: Deno limitation
  • Tracking issue: https://github.com/denoland/deno/issues/11846
  • Workarounds:
    • Switch to a header-based authentication scheme such as token auth
    • Connect thru kubectl proxy --reject-paths='^-$' instead of connecting directly to the cluster

Cannot connect to bare IP addresses over HTTPS

  • Caused by: Deno limitation
  • Tracking issue: https://github.com/denoland/deno/issues/7660
  • Workarounds:
    • Add an entry to /etc/hosts (or to actual DNS) for your cluster's IP address
    • Connect thru kubectl proxy --reject-paths='^-$' instead of connecting directly to the cluster

Cannot send EOF for exec/attach stdin

The only way to EOF the remote process over Kubernetes WebSocket is by closing the socket. This means that any final output will likely not be returned to the client.

Cannot open more port forwards on existing connection

  • Caused by: Kubernetes limitation
  • Tracking issue: (none yet)
  • Workarounds: None

Whenever a new tunneled connection is desired for port-forwarding, a whole new WebSocket (and thus TCP socket) must be set up. This is specific to the Kubernetes WebSocket protocol as the SPDY protocol can multiplex efficiently.

Clone this wiki locally