-
-
Notifications
You must be signed in to change notification settings - Fork 614
Description
Hi ClawHub Team,
We are building an internal ClawHub mirror/proxy for our organization to provide faster and more reliable access to ClawHub skills within our network. We'd be happy to operate this as a third-party community mirror if that's something the project welcomes.
During development, we ran into upstream rate limiting that makes it difficult to keep the proxy functional under normal usage:
- Metadata APIs (search, resolve, skills): 120 requests/min
- Download API: 20 requests/min
The download limit in particular is quite restrictive — when multiple users install different skills concurrently, the proxy quickly hits 429 responses, causing significant delays (8+ seconds per install with retries).
We've already implemented caching with extended TTLs (metadata 30min, downloads 72h), automatic retry with backoff, and stale-cache fallback, but the fundamental bottleneck remains for first-time / uncached requests.
We'd like to ask a few questions:
- Rate limit policy — Is there a way to get elevated rate limits for a mirror/proxy service? For example, via allowlisted IP ranges, an API token, or a dedicated mirror account?
- Full repository sync — Is there a mechanism to do a bulk export or full sync of the skill registry (metadata + packages)? This would allow us to operate entirely offline without depending on clawhub.ai for ongoing requests.
- Official mirror tooling — Does ClawHub provide (or plan to provide) any official mirroring/proxy tool, similar to what other ecosystems offer (e.g., goproxy for Go, Verdaccio for npm, Bandersnatch for PyPI)? If not, would you be open to community contributions in this area?
We're happy to share our proxy implementation or collaborate on an official solution if there's interest.
Thanks for your time.