How to deal with openai rate limits when generating graph?

[patrickkidd]

Is there a way to slow down the requests? I am hitting this generating the graph for anything other than a two-page markdown file.

Thanks!

backend   | 2025-07-23 14:21:34,232 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,233 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,234 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,235 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,236 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,237 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,238 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,239 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,240 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,241 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,242 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,243 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,244 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,245 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,245 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,246 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,247 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,248 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"
backend   | 2025-07-23 14:21:34,249 - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 429 Too Many Requests"

[RealPeterGriffin]

There isn't out of the box. You would have to use something like LangChain's InMemoryRateLimiter or implement your own.

https://python.langchain.com/api_reference/core/rate_limiters/langchain_core.rate_limiters.InMemoryRateLimiter.html

[jexp]

@patrickkidd was that for a local deployment? We haven't seen that for our public deployment.