Rate limit
Published Updated
What is Rate limit?
A rate limit is a cap an API provider sets on how many requests or tokens you can use in a given time window. Going over it returns errors until the window resets.
A deliberate restriction put in place by an API provider to cap the number of requests a user or system can make within a specific timeframe.
Rate limiting protects servers from being overwhelmed by too much traffic, whether malicious (DDoS attacks) or accidental (a runaway script). If a client exceeds their allowed quota (e.g., 60 requests per minute), the server returns an HTTP 429 "Too Many Requests" error, forcing the client to wait and retry later.
How it relates to Orbit
Security deep audits can raise rate-limiting findings when the application map and source support the concern.
When you would use it
You architect rate-limiting logic to protect your backend services, and you write error-handling logic in your frontend apps to gracefully handle rate limit rejections from third-party APIs.
Common operations
- Enforcing API quotas to prevent users on a free tier from abusing server resources.
- Implementing exponential backoff in application code when hitting OpenAI rate limits.
Frequently asked questions
What should a client do when it hits a rate limit?
Back off and retry later, ideally honouring the provider’s Retry-After header, or switch to another model.
What does Orbit do when a model hits its limit?
Orbit watches live quota and can fall back to the fallback model set for that tier. See Monitor Quotas & Cost.