-
Notifications
You must be signed in to change notification settings - Fork 121
Open
Description
Description:
I am experiencing an issue where my container crashes or resets immediately after an HTTP POST request logs this debug message:
2025-08-06 10:07:03,466 - httpcore.connection - DEBUG - close.started
2025-08-06 10:07:03,467 - httpcore.connection - DEBUG - close.complete
2025-08-06 10:07:03,468 - httpcore.connection - DEBUG - connect_tcp.started host='us-central1-aiplatform.googleapis.com' port=443 local_address=None timeout=60.0 socket_options=None
2025-08-06 10:07:03,653 - httpcore.connection - DEBUG - connect_tcp.complete return_value=<httpcore._backends.sync.SyncStream object at 0x7bd514187b10>
2025-08-06 10:07:03,654 - httpcore.connection - DEBUG - start_tls.started ssl_context=<ssl.SSLContext object at 0x7bd514731be0> server_hostname='us-central1-aiplatform.googleapis.com' timeout=60.0
2025-08-06 10:07:03,840 - httpcore.connection - DEBUG - start_tls.complete return_value=<httpcore._backends.sync.SyncStream object at 0x7bd62bd80250>
2025-08-06 10:07:03,840 - httpcore.http11 - DEBUG - send_request_headers.started request=<Request [b'POST']>
2025-08-06 10:07:03,841 - httpcore.http11 - DEBUG - send_request_headers.complete
2025-08-06 10:07:03,841 - httpcore.http11 - DEBUG - send_request_body.started request=<Request [b'POST']>
2025-08-06 10:07:04,201 - httpcore.http11 - DEBUG - send_request_body.complete
2025-08-06 10:07:04,202 - httpcore.http11 - DEBUG - receive_response_headers.started request=<Request [b'POST']>
The request headers and body are sent successfully, but right when the client starts receiving the response headers, the container unexpectedly resets.
Additional Info:
- I have implemented timeouts and configured a safe connection pool with a high number of connections to handle load smoothly.
Environment:
- Containerized app running on Docker
- HTTP client using
httpcore
(likely viahttpx
) - Timeouts and connection pool settings already applied
- LLM Async Streaming like this Rare edge case issue causing a thread deadlock with access to _optional_thread_lock in ConnectionPool #990
What I have tried:
- Configured client timeouts to avoid hanging connections.
- Increased connection pool size for high concurrency.
- Checked container logs, but no specific errors aside from the sudden reset.
Request for help:
- Has anyone experienced sudden container restarts at the point of
receive_response_headers.started
? - Could networking or
httpcore
bugs be the culprits despite proper timeout and connection management? - Recommendations for debugging this issue further or any known workarounds would be highly appreciated.
If you want, I can also help you phrase this for a particular platform or add more technical detail — just let me know!
Metadata
Metadata
Assignees
Labels
No labels