PEX meshing

[anacrolix]

I thought a central place to discuss the PEX meshing concerns would be useful @yarikk. The PEX BEP mentions a rate limiting per-connection, ignoring PEX updates from a peer more frequent than once a minute. I mistook the check to do this once a minute on a Torrent-level as a bug, but @yarikk suggests that it provides superior meshing behaviour. This behaviour was originally added in #446.

If I understand, the meshing behaviour you want to avoid @yarikk is having a client attempt to connect to all available peers in a swarm too quickly? Is this because once a peer has been "added" over PEX, it will never be received again (unless there is an intermediate drop and then add again, which would occur over a course of at least a few minutes).

I think there are two defences against this: first, the half open limit and client dial rate limiter will reduce the rate at which new addrs are attempted. Second, addresses received from PEX wont' be used if the client is already filled up with good peers that have higher peer priorities. It's possible that the peer priorities weren't implemented when PEX first went in to anacrolix/torrent.

Looking at other torrent client implementations, they seem to maintain the pool of peers info even when they're not connected. That is, if tehy attempt to connect to a peer and it fails or drops, it won't be forgotten and need to be obtained from a tracker/PEX/DHT all over again. anacrolix/torrent doesn't do that, although it does allow new peer addresses to be pooled even when they're already connected to (both due to laziness on my part, and as an easy signal that a really does exist and should be tried again if it drops, it can only be added again after it's evicted for being dialed, whether that fails or not).

PEX has been working great, so it's not intention to clobber the existing behaviour. But if I'm correct in understanding per-torrent rate limiting is due to the shortcomings in anacrolix/torrent, it's my preference to fix those. I think the peer-addr-pool-despite-dialing that other clients use is the best approach. In the short term, before that's implemented, I think the per-Torrent rate limiting should be restored with comments saying why it's in the form that it is.

[anacrolix]

I've just checked the torrent-level peer adding logic from 1e13625#r114051594. If we ignore a PEX message from a peer due to a torrent-level wait interval, we never get to process those same peer addresses again from that peer. So I don't actually think it's having any positive effect except to rate limit PEX additions vs. other sources like DHT and trackers.

It could be more correct to add a limit per source to the pending addrs (

torrent/torrent.go

Line 114 in a8ab0ba

peers prioritizedPeers

) than to make any other changes to avoid this.

[yarikk]

The discarded peers are the re-introduced through other peers, that's not a problem; I confirmed that experimentally back when submitted the original PR. We operate swarms which use PEX on too of bootstrap set of peers, and never saw a starving peer with the throttling enabled.

I do recognize it's not ideal, and an alternative throttling may yield similar or better results, but it's hard to prove without rigorous validation/testing. Should an alternative be offered, I'd love to see the peer count charts before/after the change as a proof that it works.

[anacrolix]

Do I have it correct then that the torrent-level rate limiting is to avoid PEX from overwhelming other peer sources?

[yarikk]

PEX exposes every peer to everyone. Unlike other sources like Trackers and DHT which poll their exchanges on fixed schedule (and even cease the polling when number of connections are above the minimum), PEX is unsolicited and can arrive from any peer at any time. That makes it prone to an excessive inter-linking and makes it hard for new clients to join swarms larger than max connections on a single node. The cool-off period is not a BEP-backed approach but it does help to slow down the rate of every peer discovery and establishing excessive amount of connections. A less brutal approach would be to hold the peer info received over PEX into a staging area which then is polled similarly to how Trackers and DHT are polled.