[Question] Is it possible to limit client connections to a shard?

[MENNAELMESALMY]

Hey!

I have a MemoryDB cluster consisting of two shards, I noticed that when connecting to it a connection is established with all the nodes in this cluster (both shards/all nodes), the problem with that is it creates a connection with nodes that it will never interact with because each client will only be able to access a subset of keys residing in a single shard (using hashtags).
Is there a way to make the client only connect to a single shard or a subset of nodes within this shard ( I have a lot of clients so I want to distribute their connections).
To give more context I am running into the limitation of 64k connections per node so I want clients to connect to the least needed amount of nodes.

Thanks!

[mgravell]

That's an interesting scenario! No, we don't currently have a feature for that, but I'm open to supporting such. The simplest way I can think of doing this would be via a lazy connect - i.e. we don't spin connections up until we absolutely need them - but we'd need to do something clever to avoid getting failures with your first few commands. The other way we could approach this is via some kind of node filter; specifying slot numbers is awkward and opaque - I suspect the better approach is, as you mention, via an allow-list of hash-tags (which technically would also allow individual keys to be specified, but the typical use would be: hash-tags). The first option is less to configure; the second option allows more enforced control, i.e. you can be reasonably sure that a suitably configured client will never attempt to talk to a different box. Let me know your thoughts here.

[MENNAELMESALMY]

Thanks for the prompt response!
That would be great to support and will make our life way easier with multitenancy within a single cluster.
I think both approaches are reasonable.
The first approach I would imagine is very good when someone (like me) wants to establish the least amount of connection and only attempt connections when needed (first time connecting or the node failed).
To clarify when you say something clever do you mean maybe retry several times? or maybe have the seed endpoint pointing to the right shard, not the cluster endpoint?

The second approach would be more explicit and more straightforward I would imagine and maybe even more robust on startup and node failures?

Could we perhaps have a way to pass allowed hashtags to force the client not to talk to some server (second approach) but also configure the number of connections that I want to be connected to, let's say two, and if one fails the client can do a lazy connect to another server. (maintain only X open connections at a time)

I am not an expert here though so I am brainstorming 😅 Happy to help with this feature, It will be really useful for us!

[mgravell]

when you say something clever

I simply don't want you to see "no connection was available (blah blah)" while it is spinning up - I am not 100% sure this would happen, I'd need to check.

and if one fails the client can do a lazy connect to another server

we can't really do that unless we have good reason to think that the key ownership has changed, which we'll obtain automatically if we see -MOVED etc

Additional notes:

• either disabling pub/sub or forcing RESP3 (both already available via config) could immediately half connection usage
• we might also want to consider whether to allow primary/replica/both connections

[MENNAELMESALMY]

we can't really do that unless we have good reason to think that the key ownership has changed, which we'll obtain automatically if we see -MOVED etc

I don't think it will change, what I meant was we connect to 2 out of 4 replicas at a time but if a replica fails when you are connected, try another.

Either disabling pub/sub or forcing RESP3 (both already available via config) could immediately half connection usage

already on RESP3

we might also want to consider whether to allow primary/replica/both connections

Is that achievable now? I only pass the flags to PREFER_REPLICA with commands to distribute the load but the connections look like they are open to all nodes regardless

[mgravell]

I don't think it will change, what I meant was we connect to 2 out of 4 replicas at a time but if a replica fails when you are connected, try another.

ah, right; I didn't realise you were focusing on replicas there; I was talking about primary nodes

Is that achievable now? I only pass the flags to PREFER_REPLICA with commands to distribute the load but the connections look like they are open to all nodes regardless

No - this would be a new thing also; I think we should consider horizontal (slots) and vertical (replicas) at the same time, if the objective is to avoid unnecessary connections.