Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fine-tune download speed for xet + stream backpressure #1279

Open
coyotte508 opened this issue Mar 12, 2025 · 0 comments
Open

Fine-tune download speed for xet + stream backpressure #1279

coyotte508 opened this issue Mar 12, 2025 · 0 comments
Labels
hub @huggingface/hub related

Comments

@coyotte508
Copy link
Member

coyotte508 commented Mar 12, 2025

Currently we have a mechanism to have 1000 chunks in backpressure. (would love that to be a byte amount but it's not doable easily due to incompatibilities between implementations of ReadableStream, would need per-platform adaptations)

But if the stream is consumed instantly (eg writing to file), we have a bottleneck because we don't make calls in parallel to the CAS service.

Eg someone with a download speed of 1GB/s, each chunk CAS range is at most 64MB compressed, that's 16 HTTP request/second, assuming a 30ms ping, that's a 33% download speed decrease just from doing the request one after the other.

And it can even be worse, with smaller ranges, and higher bandwiths.

So we need to add some parallelism probably, and ideally be able to react to the download speed / latency

Like maybe open multiple read streams in // and join them and add on a new read stream as an old one is consumed (see also https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream/pipeTo)

@coyotte508 coyotte508 added the hub @huggingface/hub related label Mar 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hub @huggingface/hub related
Projects
None yet
Development

No branches or pull requests

1 participant