Skip to content

Commit 07d4d66

Browse files
committed
fix: httpx.Client is slow when sending large data
This commit addresses an issue where sending large data with httpx.Client was extremely slow. The problem was that the entire data payload was being loaded into memory and sent in a single operation. This fix refactors the sending process for iterable bytes types to use a chunked approach with `yield`, which significantly improves performance and reduces memory usage for large payloads. Signed-off-by: xiexianbin <[email protected]>
1 parent 3fee278 commit 07d4d66

File tree

1 file changed

+8
-1
lines changed

1 file changed

+8
-1
lines changed

httpx/_content.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,14 @@ def __iter__(self) -> Iterator[bytes]:
6161
else:
6262
# Otherwise iterate.
6363
for part in self._stream:
64-
yield part
64+
# For bytes types, should use yield to send data in
65+
# chunks. Sending a very large part at once will
66+
# result in extremely slow transmission.
67+
if isinstance(part, bytes):
68+
for i in range(0, len(part), self.CHUNK_SIZE):
69+
yield part[i:i+self.CHUNK_SIZE]
70+
else:
71+
yield part
6572

6673

6774
class AsyncIteratorByteStream(AsyncByteStream):

0 commit comments

Comments
 (0)