Paralegal app hangs due to "No route.." to pod

[yuliiamashkovtseva]

Name and Version

Core 0.25.0

What steps will reproduce the bug?

The paralegal app became unresponsive ( nor reply for a prompt).

The issue was fixed by paralegal pod restart.
Could you please help us identify the cause of the 'No route...' error from the DeploymentPostController, as this issue is likely to occur again.

What is the expected behavior?

No response

What do you see instead?

The following were observed in Chat logs:

message: 'request to http://ENV_URL/openai/deployments/virtual-paralegal-assistant-v3/chat/completions?api-version=2025-01-01-preview failed, reason: socket hang up',

The following errors are observed in Core (not only for Paralegal app, but for other models too. They were working correctly though):

2025-03-31 13:19:10.158 ERROR [vert.x-eventloop-thread-1] [DeploymentPostController] No route. Trace: 5dbe9bbbb93a106f941ce4e1fd7b9fa0. Span: 7551fc7b9f785d55. Project: null. Deployment: gpt-4-omni-mini. User sub: 17524476-4266-46bd-aa13-6d004fcf70e5
2025-03-31 13:19:21.798 ERROR [vert.x-eventloop-thread-1] [DeploymentPostController] No route. Trace: f287523f219b11f763eff961ae50a5d8. Span: d45edd300ac5168e. Project: null. Deployment: gpt-4-omni-mini. User sub: 17524476-4266-46bd-aa13-6d004fcf70e5
2025-03-31 14:47:38.942 ERROR [vert.x-eventloop-thread-1] [DeploymentPostController] No route. Trace: 0dbdc8c52defb1423827d2300f9c6c34. Span: f4e619ea846f8172. Project: null. Deployment: virtual-paralegal-assistant-v3. User sub: fd969a97-1721-45d9-8861-dffdfa9aa6d9

Additional information

No response

[astsiapanay]

Hello @yuliiamashkovtseva

There is a typical use case when an upstream server(it could be LLM adapter or application) returns http error codes 429, 502, 503, 504.

The Core tries to make one more retry and if all retries are failed then the Core returns error code 502 or 429(depending on what error codes upstream server returned during the retries).