[PROPOSAL] MIght be good to add custom rate limit to LLM token input

[hughchaozhang]

Proposal Summary

I ran the demo and one request filled the token rate limit per minute for Claude

Problem Statement

• ran the demo web search and summarize into markdown, hit the rate limit in one shot

Proposed Solution

consider adding custum rate limit for LLM models

Alternatives Considered

No response

Additional Context

No response

[yiwenlu66]

Good catch! Can be particularly relevant to individual developers with lower-tier api keys.