Receiving chunk extensions in HTTP/1.1 chunked encoding #3522
Description
Requested functionality
When writing an HTTP server with hyper, I want to be able to receive and read the chunk extensions that clients send when using HTTP/1.1 chunked encoding.
Currently hyper simply discards these chunk extensions.
Motivating use case
Implementing an S3-compatible server using hyper would require this functionality. Amazon S3 Signature Version 4 includes a "streaming mode" where the SHA256 signature of each chunk is included as a chunk extension:
Example Header:
PUT /examplebucket/chunkObject.txt HTTP/1.1\r\n
<Other Headers>
x-amz-content-sha256: STREAMING-AWS4-HMAC-SHA256-PAYLOAD\r\n
Content-Encoding: aws-chunked\r\n
x-amz-decoded-content-length: 66560\r\n
Transfer-Encoding: chunked\r\n
\r\n
<Payload>
Example chunk from payload:
10000;chunk-signature=ad80c730a21e5b8d04586a2213dd63b9a0e99e0e2307b0ade35a65485a288648\r\n
<65536-bytes>
The server is required to validate the signature of each chunk as it is sent, so it needs to be able to extract the chunk signatures from the HTTP chunk extensions.
Idea 1: New frame type
Define a new Frame variant for chunk extensions.
impl Frame<T> {
pub fn chunk_extensions(extensions: HeaderMap) -> Self { ... }
pub fn is_chunk_extensions(&self) -> bool { ... }
pub fn into_chunk_extensions(self) -> Result<HeaderMap, Self> { ... }
pub fn chunk_extensions_ref(&self) -> Option<&HeaderMap> { ... }
pub fn chunk_extensions_mut(&mut self) -> Option<&mut HeaderMap> { ... }
}Then the Incoming body can yield a Frame::chunk_extensions(..) before each Frame::data(..) (or rather, before the first frame in each chunk?).
This seems like the least invasive option, but might not really solve the problem sufficiently (see Limitations below).
Limitations
Ordering
In order to implement the S3 chunked signature, the server has to be able to associate each set of chunk extensions with the chunk that it precedes.
However, the Body trait provides no guarantee about the order of different types of Frames that are yielded. So programmers would have to trust that any Body is going to yield chunk extensions before each chunk. But theoretically somebody could implement Body in a way that subverts that expectation.
Perhaps this is not truly an issue, but rather an "understood quirk" of the Body trait. I say this because a similar problem exists with the chunked trailers. The Body trait does not document any guarantee that you'll receive at most one Frame::trailers(..), nor that that frame will be the last frame yielded. But I'd wager that users of a Body might make one or more of those assumptions.
Sending
It's not clear how the hyper client code should interpret any Frame::chunk_extensions(..) that are yielded by the request Body 🤔 🤔 🤔 .
Idea 2: Callbacks
Maybe the hyper server code could allow callers to specify some sort of callback that the Dispatch will call whenever it encounters chunk extensions. Haven't thought this idea through very much.
Alternatives considered
I can't think of any. The processing of the chunk extensions is completely hidden away in hyper's guts and cannot be customized by end users.
Any thoughts on other ways to expose this functionality?