Skip to content

Third-Party Mirror for ClawHub — Rate Limit & Sync Options Inquiry #518

@89ao

Description

@89ao

Hi ClawHub Team,

We are building an internal ClawHub mirror/proxy for our organization to provide faster and more reliable access to ClawHub skills within our network. We'd be happy to operate this as a third-party community mirror if that's something the project welcomes.

During development, we ran into upstream rate limiting that makes it difficult to keep the proxy functional under normal usage:

  • Metadata APIs (search, resolve, skills): 120 requests/min
  • Download API: 20 requests/min

The download limit in particular is quite restrictive — when multiple users install different skills concurrently, the proxy quickly hits 429 responses, causing significant delays (8+ seconds per install with retries).

We've already implemented caching with extended TTLs (metadata 30min, downloads 72h), automatic retry with backoff, and stale-cache fallback, but the fundamental bottleneck remains for first-time / uncached requests.

We'd like to ask a few questions:

  1. Rate limit policy — Is there a way to get elevated rate limits for a mirror/proxy service? For example, via allowlisted IP ranges, an API token, or a dedicated mirror account?
  2. Full repository sync — Is there a mechanism to do a bulk export or full sync of the skill registry (metadata + packages)? This would allow us to operate entirely offline without depending on clawhub.ai for ongoing requests.
  3. Official mirror tooling — Does ClawHub provide (or plan to provide) any official mirroring/proxy tool, similar to what other ecosystems offer (e.g., goproxy for Go, Verdaccio for npm, Bandersnatch for PyPI)? If not, would you be open to community contributions in this area?

We're happy to share our proxy implementation or collaborate on an official solution if there's interest.

Thanks for your time.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions