feat(peer-store): introduce libp2p-peer-store

[drHuangMHT]

Description

Introduce libp2p-peer-store for a peer store implementation.

Related: #4103

Notes & open questions

Change checklist

• I have performed a self-review of my own code
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• A changelog entry has been made in the appropriate crates

[drHuangMHT]

This is a very basic implementation, would love to hear more ideas on how to implement this!

[elenaf9]

Thanks for starting this effort @drHuangMHT!

If I understand the current implementation correctly, the purpose of this store is just to track connected peers, and explicitly added addresses. However, I think we have to keep in mind that protocols like kademlia or identify very frequently report all addresses that they learn as NewExternalAddrOfPeer. So with the current MemoryStore implementation, the store would grow unbounded with all of these addresses. I think there needs to be some GC strategy.

Also, after reading #4103 (comment), I think a peer store implementation could also do much more than just track explicitly added addresses, e.g.:

• track how valuable a know address is, by using infos about how the address info was received, if we connected to it already, etc
• implement a TTL for address records
• track other meta data for a peer, e.g. public key

I don't think this needs to be implemented in this PR. But I think we could forward more of the already present info to the Store trait, so that a custom implementation can have more sophisticated logic.

[drHuangMHT]

Oops that use<'a> syntax was stabilized in rust 1.82, the compiler on 1.80 wasn't happy with it.

[jxs]

Thanks for starting this @drHuangMHT, already looking good! left some notes to allow further customization.
cc @elenaf9

[drHuangMHT]

• track how valuable a know address is, by using infos about how the address info was received, if we connected to it already, etc

This can get very complicated very soon, for example how to record the activitiy on the address(there will be some overhead) and how to make it machine-readable(scoring system). The discussion will be quite lengthy.

• implement a TTL for address records

I didn't see a good way to implement TTL(garbage collect) for records, any pointer?

• track other meta data for a peer, e.g. public key

I don't see the remote public key being available through swarm itself, be it in the form of PeerRecord, SignedEnvelope or just PublicKey. But the pubkey is indeed a very important metadata of a peer.

[elenaf9]

• track how valuable a know address is, by using infos about how the address info was received, if we connected to it already, etc

This can get very complicated very soon, for example how to record the activitiy on the address(there will be some overhead) and how to make it machine-readable(scoring system). The discussion will be quite lengthy.

• implement a TTL for address records

I didn't see a good way to implement TTL(garbage collect) for records, any pointer?

• track other meta data for a peer, e.g. public key

I don't see the remote public key being available through swarm itself, be it in the form of PeerRecord, SignedEnvelope or just PublicKey. But the pubkey is indeed a very important metadata of a peer.

I don't think we need to solve these in this PR, or even within this repo for that matter. These were just meant as examples for things that user's might want to use the new libp2p-peer-store for (e.g. when also combining it with other behaviors). Sorry if that wasn't clear!

[drHuangMHT]

I don't think we need to solve these in this PR, or even within this repo for that matter. These were just meant as examples for things that user's might want to use the new libp2p-peer-store for (e.g. when also combining it with other behaviors). Sorry if that wasn't clear!

NP, PeerRecord is also what we need for gossipsub Peer eXchange feature that is currently missing. Conversations like this can be put into the documentation to give users some inspiration, and MemoryStore can be a good example as well.

[elenaf9]

Thank you for the follow ups!
Couple more comments.

[dknopik]

Hey @drHuangMHT!
I tried to use the peer store. I like it! I've added some comments with changes I needed to make.

Also, I agree with @jxs's comment that it would be useful to add a generic parameter to the memory store for user data. I know that it is only a reference implementation and users are free to implement Store themselves, but I feel like custom data would be a rather common use case, and it would be unfortunate if users had to copy-paste the memory store in too many cases.

[dknopik]

Hi @drHuangMHT,

thanks, the changes look great!
Another thing that I found that would be useful is providing a way to iterate over the peers we have in the store. I am not sure though if this should be offered by the Behaviour, or only by the Store implementation. What do you think?

[drHuangMHT]

thanks, the changes look great! Another thing that I found that would be useful is providing a way to iterate over the peers we have in the store. I am not sure though if this should be offered by the Behaviour, or only by the Store implementation. What do you think?

I think it would be better to implement this on the store implementation(not on Behaviour or Store trait itself) so we don't clutter the interfaces, and we already expose the internal store on Behaviour.

[jxs]

Hi @drHuangMHT thanks for driving this.
I had a call Today with @elenaf9 and @dknopik to discuss this PR, as we are using this it to understand how it may help us across two projects where we need a Peer Store.
I would like to avoid adding features that don't have real demand and be as simple as possible in this phase, to then iterate adding them later when needed.
I Left some comments that stemmed from that call, Elena and Daniel feel free to add and correct anything that I missed. DrHuang give your thoughts as well ofc.
Thanks!

[jxs]

I don't know if we need all these methods for now (the CRUD seem duplicated on the MemoryStore), can we just have PeerRecord as a struct and have it in the upper module?