Prevent OOM from malformed snappy payloads by validating decoded length

A specially crafted remote-write request can declare an extremely large
decoded length while providing only a small encoded payload. Prometheus
allocates memory based on the declared decoded size, so a single request
can trigger an allocation of ~2.5 GB. A few such requests are enough to
crash the process with OOM.

Here's the script that can be used to reproduce the issue:

echo
"97eab4890a170a085f5f6e616d655f5f120b746573745f6d6574726963121009000000000000f03f10d48fc9b2a333"
\
  | xxd -r -p \
  | curl -X POST \
      "http://100.123.0.5:8429/api/v1/write" \
      -H "Content-Type: application/x-protobuf" \
      -H "Content-Encoding: snappy" \
      -H "X-Prometheus-Remote-Write-Version: 0.1.0" \
      --data-binary @-

This change adds a hard limit: the requested decoded length must be less
than 32 MB. Requests exceeding the limit are rejected with HTTP 400
before any allocation occurs.

Signed-off-by: Max Kotliar <[email protected]>

Author: makasim

exp/api/remote/remote_api.go

@@ -448,6 +448,8 @@ func WithWriteHandlerMiddlewares(middlewares ...func(http.Handler) http.Handler)
 	}
 }
 
+var maxDecodedSize = 32 * 1024 * 1024
+
 // SnappyDecodeMiddleware returns a middleware that checks if the request body is snappy-encoded and decompresses it.
 // If the request body is not snappy-encoded, it returns an error.
 // Used by default in NewHandler.
@@ -479,6 +481,18 @@ func SnappyDecodeMiddleware(logger *slog.Logger) func(http.Handler) http.Handler
 				return
 			}
 
+			decodedSize, err := snappy.DecodedLen(bodyBytes)
+			if err != nil {
+				logger.Error("Error snappy decoding remote write request", "err", err)
+				http.Error(w, err.Error(), http.StatusBadRequest)
+				return
+			}
+			if decodedSize > maxDecodedSize {
+				logger.Error("Error snappy decoding remote write request: decoded size exceeds maximum allowed", "decodedSize", decodedSize, "maxDecodedSize", maxDecodedSize)
+				http.Error(w, "decoded size exceeds maximum allowed", http.StatusBadRequest)
+				return
+			}
+
 			decompressed, err := snappy.Decode(nil, bodyBytes)
 			if err != nil {
 				// TODO(bwplotka): Add more context to responded error?