[QUESTION]: I am getting grpc.aio._call.AioRpcError when sending concurrent load_partitions requests

[mihailyanchev]

Is there an existing issue for this?

• I have searched the existing issues

What is your question?

I use pymilvus 2.5.4 and a Milvus cluster the same version.

When I have several concurrent requests to load a partition from the async client I get the following:

File "/usr/local/lib/python3.12/site-packages/mle_utilities/persistence/milvus/async_sharded_vectorstore.py", line 298, in _load_partition
    await self.async_client.load_partitions(collection_name=collection_name, partition_names=[partition_name])
  File "/usr/local/lib/python3.12/site-packages/pymilvus/milvus_client/async_milvus_client.py", line 243, in load_partitions
    await conn.load_partitions(collection_name, partition_names, timeout=timeout, **kwargs)
  File "/usr/local/lib/python3.12/site-packages/pymilvus/client/async_grpc_handler.py", line 828, in load_partitions
    response = await self._async_stub.LoadPartitions(request, timeout=timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/grpc/aio/_interceptor.py", line 472, in __await__
    response = yield from call.__await__()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/grpc/aio/_call.py", line 327, in __await__
    raise _create_rpc_error(
grpc.aio._call.AioRpcError: <AioRpcError of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "recvmsg:Connection reset by peer"
	debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2025-02-27T09:02:32.243883236+00:00", grpc_status:14, grpc_message:"recvmsg:Connection reset by peer"}"
>

What might be the reason? Should I just retry until it passes? I see no issues with my milvus cluster.

Anything else?

No response

[stephen37]

Hey @mihailyanchev, it's something that is that @XuanYang-cn is working on at the moment. We should have a solution to that in the future :)

[mihailyanchev]

Hey @mihailyanchev, it's something that is that @XuanYang-cn is working on at the moment. We should have a solution to that in the future :)

@stephen37 thanks! Let me know if this issue is tracked anywhere.