GitLab

fix: set the score ledger on start

It's possible to start receiving and processing messages before we get around to starting.

feat: update for go-libp2p-core 0.7.0 interface changes

fix: guard access to the mock wiretap with a lock

* Add WireTap interface

WireTap interface can be used to access all messages send and received
by Bitswap. This can be used to implement advanced statistics/analysis
logic, which is beyond scope of Bitswap, but can be implemented as IPFS
plugin.

Some examples of potential applications:
 - per CID bandwidth tracker (see: https://gitcoin.co/issue/PinataCloud/apollo/2/100023631)
 - detailed per peer stats
 - intrusion detection system (IDS) implementation

* Add test for WireTap

* Share common code between network/ipfs_impl_test.go tests

Extract the code that is common in TestMessageResendAfterError,
TestMessageSendTimeout and TestMessageSendNotSupportedResponse to a
separate function.

* Make prepareNetwork() return two hosts and two networks

Let prepareNetwork() make simmetric setup with two `ErrHost`s
with two `impl` networks to be sure we test `impl` instances on
both ends.

* Added TestNetworkCounters test to the "network" package

The test shows we have a problem with `MessagesSent` counter.

* Fix: Increment stats.MessagesSent in msgToStream() function

Fixes the bug with incrementing `MessagesSent` counter only in
`SendMessage()` method if `impl`. Now it works for `MessageSender`
too.

* Allow to specify a network event listener for tests

Added `listener network.Notifiee` to the `receiver` structure.
If a listener is specified then `prepareNetwork()` connects it
to the mock network it builds before making any connections.

* Wait for all network streams are closed in testNetworkCounters

Wait for all network streams are closed instead of just using a
timeout. The timeout of 5 s is still used as a deadline (it makes
the test to fail).

* Fix: Close the MessageSender in testNetworkCounters()

The `MessageSender` needs to be closed if we want all streams in
the network to be closed.

* Fix: Close MessageSender in other tests too
Co-authored-by: Paul Wolneykien <manowar@altlinux.org>

* Separate decision engine ledger on two parts: score and the wantlist

This is the first step to make external decision logic (tagging
peers with score values) possible.

The wantlist still resides in the original `ledger` struct while
sent/received byte accounting and scores are extracted to the new
`scoreledger` struct managed by the original `scoreWorker()` logic.
The accounting is integrated into the `Engine` via `ScoreLedger`
interface making it possible to replace the original `scoreWorker()`
with some other logic. The interface, however, doesn't allow a
score logic to directly touch peer tags: the logic may decide about
score values while tagging itself is still under control of Engine.

Note: with this commit it's yet not possible to replace the original
score logic because there is no public methods for that.

* Added "WithScoreLedger" Bitswap option

New `WithScoreLedger(decision.ScoreLedger)` option in the `bitswap`
package is the way to connect a custom `ScoreLedger` implementation
to the decision engine. The `Engine` now has the corresponding
`UseScoreLedger(ScoreLedger)` method.

The `ScoreLedger` and `ScorePeerFunc` types are exposed from the
internal `decision` package to the public one.

Because its options are processed by the `Bitswap` after construction
of its parts but before starting of the engine, the default
`scoreLedger` initialization is moved from `newEngine()` to
`StartWorkers()`.

New `TestWithScoreLedger` test is added. The test checks for start and
stop of the testing score ledger implementation that is specified via
`WithScoreLedger` option.

* Combine score ledger start with initialization of the score function

Having a separate `Init(ScoreFunc)` method seems redundant (thx
@dirkmc for pointing about that). As a bonus, the two-step ledger
starting process is now enclosed in the `startScoreLedger()` function.

* Let's call Stop() to stop a ScoreLedger

The `Close()` method was there to stop the ledger. Let call it
`Stop()` now.

* Get return of the blank Receipt out of conditional block

Explicitly form it as the final resort.
Co-authored-by: Paul Wolneykien <manowar@altlinux.org>

I'm not sure why we set "full" to true here, but this could be the source of a
whole bunch of bidirectional sync issues. That is, if two peers are syncing off
each other, they could repeatedly "reset" each other's wantlist to "empty".

fix want gauge calculation

Fix PeerManager signalAvailabiity() race

fix: avoid taking accessing the peerQueues without taking the lock

fix: update circleci ci-go

Or, really, just avoid accessing it. We don't need it.

This caused a concurrent map access panic under load.

Quick alternative to #407 to fix the main issue.

If peer is first to send a block to session, protect connection

Ensure sessions register with PeerManager

* feat: total wants gauge

* fix: in gauges count wants regardless of which peers they're sent to

* fix: want block gauge calculation

* refactor: simplify peermanagerwants

Improve peer manager performance

Instead of tracking offsets, just create a "new" slice starting with the
broadcast cancel slice. Under the covers, this will just use the same memory
over and over.

fix: return wants from engine.WantlistForPeer()