GitLab

* update bootstrapPeers to internally be func() []peer.AddrInfo

* add BootstrapPeersFunc and add test

* fullrt: add experimental FullRT DHT client

The FullRT client periodically crawls the network to fill its local routing table and uses that to perform queries. It supports performing many provides or puts together taking advantage of efficiencies of grouping the DHT operations by where they will occur in the Kademlia space. Additionally, it has a more tunable function for determining when a query is complete that does not require waiting on any individual peer (which could be offline) to respond. This client is experimental and its exposed interfaces should be expected to change and break over time.

* crawler: starting peers with addresses and peers found during a crawl have their addresses extended. logging improved

* dht: move IpfsDHT options to the internal package. Make a breaking change to the filter interfaces to support more DHT implementations

* dht: GetClosestPeers now returns a slice of peers instead of a channel of peers since we have to wait for the query to complete to return the closest peers anyway

* dht: the subscriberNotifiee has been refactored to work more independently of the underlying message sender implementation

test: add test for findpeer query on node that does not qualify to be dialed as part of the query under the queryfilter

dht.msgSender

refactor: extract messaging components from IpfsDHT into its own struct. create a new struct that manages sending DHT messages that can be used independently from the DHT.

This reverts commit c9851849.

This reverts commit df4f28c3.

Ongoing work is being done in feat/dht-hardening-0.7 and this change isn't quite ready.

* Peer Diversity in the DHT

* method for getting bootstrap peers and double the usefulness interval

* rt refresh refactor

* Bootstrap when RT is empty and optimize allocations.
Co-authored-by: Steven Allen <steven@stebalien.com>

Also, fail if the routing tables aren't well-formed.

It turns out there were a couple of issues with this test:

1. It used too many file descriptors.
2. It tried to query the DHT without bootstrapping any nodes. That's never going
to work.

Handle empty keys, both when sent in RPC requests and in the local API.

test: fix TestFindPeerQuery and TestFindPeerQueryMinimal to not try and refresh routing tables for orphaned nodes

When this option is passed, the DHT node will listen on and query _only_ using
the old DHT protocol. Importantly, the node won't even pretend to be a new DHT
node because it's routing table includes V1 peers.

Except deprecated warn calls

* fix Low Peers and Cleanup

* new RT management policy

fix(test): TestFindPeerQuery and TestFindPeerQueryMinimal now refresh DHT routing tables before running. TestFindPeerQuery no longer runs with the race detector because it uses too many goroutines.

* remove disjoint queries
* remove globally queried check
* removed dht rng field and mutex
* removed the Disjoint option
Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>

* feat(query): fully async implementation of Kademlia lookup. peers returned from the lookup are not guaranteed to be alive (i.e. we're only guaranteed to have dialed the closest beta peers to the target), but given stable and correct routing tables the expectation that most of the peers returned are alive is high.
* feat(query): add wrapper lookup followup function to followup after the lookup is completed and ensure that the closest k returned peers from a lookup have been queried even for beta < k
* refactor(query) modified the structure returned from lookups to be a useful subset of the full query state instead of the entire query state
* feat(options): beta parameter exposed as the Resiliency parameter
* feat(routing): do not mark the routing table as updated after a FindPeer query
* feat(routing): FindPeer can return addresses even if not Connected as long as it was either recently connected (CanConnect) or was discovered during the lookup
* feat(bootstrap): bootstrap logic now uses GetClosestPeers instead of FindPeer
* refactor(dht): stopFn no longer takes any state
* fix(test): changed GetClosestPeers test to only assume beta instead of k peers since that is now more appropriate given the query logic changes and that the routing tables in that test are bad, i.e. a ring network with arbitrary peerIDs
Co-authored-by: Petar Maymounkov <petarm@gmail.com>
Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>

* upgraded the protocol id to version 2 (i.e. /kad/2.0.0) and made it so v2 peers running in server mode respond to queries from v1 peers. Note: v2 peers will only send queries using the v2 protocol, will only add v2 peers to their routing tables, and will only tell v1 peers about v2 peers.
* to run a forked network we now use network specific protocol prefixes instead of manually setting protocol IDs. Use the ProtocolPrefix option instead of the Protocols option.
* emit errors during initialization if the user misuses the default protocol prefix by setting parameters inconsistent with the default protocol's network specification
* since the Client option has been deprecated it's been removed from the dht's options. While deprecated it is still available in the dht options package. Setting `Client(false)` now puts the node into ModeAuto.

* feat: move options to main package and make internals private

Rationale:

1. This allows us to make private options for testing.
2. This removes an import for DHT users.
3. This makes options much easier to discover.
4. This makes it possible to make the config/options internals private.

We originally put them in a sub-package to avoid poluting the root namespace,
but that isn't really necessary.

This keeps the old package (for now) to avoid breaking too much.

* created Mode(ModeOpt) option for choosing between auto/client/server modes
* Auto mode internally switches the DHT between client and server modes based on the EvtLocalReachabilityChanged event emitted on the event bus (e.g. by AutoNAT)
* routing table management of peers that switch between client and server mode while we are connected to them (i.e. are in auto mode)
* removed Client(bool) option, becoming a DHT client is specified using Mode(ModeClient) instead

options: KValue and AlphaValue global variables no longer used internally. Concurrency option now sets alpha. DisjointPaths option now sets d. Default number of disjoint paths is now bucketSize/2.