GitLab

* feat(graphsync): add incoming block hook types

add types (not implemented) for incoming block hook

* feat(asyncloader): identify local loads

add a boolean value to AsyncLoadResult to determine if a request load was local

* fixup! feat(graphsync): add incoming block hook types

* feat(hooks): add block hooks plumbing

* feat(requestmanager): process block hooks

Process block hooks on each block, and connect last response to blocks

* feat(peerresponsemanager): add transactions

add the ability to do multple response operations with a gaurantee they will end up in a single
response

* refactor(lint): fix lint errors

* Updating go-ipld-prime to NodeAssembler era.

The 'testutil/chaintypes' codegen is currently dropped, and I ported
the testchain fixtures to using basicnode instead.  We can restore this
to using codegen soon, but it's easier to complete this set of changes
by temporarily switching to using basicnode.

Tests using 'NewUnencodableSelectorSpec' were discarded -- this more or
less exercised what would happen if an uninitialized zero value got
into a program, and this is now happily statically impossible.

Some tests are not passing; have not sussed yet.

* fix(deps): upgrade ipld-prime with bug fix

use ipld prime with fixed marshalling bug to get tests passing

* fix(lint): fix lint error
Co-authored-by: Eric Myhre <hash@exultant.us>

* feat(graphsync): add update to message

add update to message protobuf. also fix deserialization error with cancel requests

* feat(requestmanager): support updates for response hooks

Support sending request updates in incoming response hooks. Also factor hooks out of request manager

* refactor(hooks): make hooks approach consistent

for ease of use and learning in library, make hooks structure consistent across requestor and
responder

* feat(responsemanager): update request processing

Add the ability to processing update requests as needed

* test(responsemanager): add more tests for update behavior

* test(integration):add update integration test

Add test to verify the whole update flow

* fix(lint): fix lint error

* feat(deps): upgrade IPLD prime to last before refactor

* refactor(responsemanager): refactor traversal

switch to iterative traversal that is more amenable to interruption

* feat(peerresponsemanager): add bytes sent to peer response sender

Add a return value to SendResponse in peer response sender to return the number of block bytes
transmitted. Note: does not count unsent blocks. Also convert block size measurements to uint64

* feat(responsemanager): add response pausing

add first implementation of response pausing in response manager

* refactor(responsemanager): extract classes to simplify response manager

extract hooks as independant functionality

* test(responsemanager): add tests for block hooks and request hooks

* test(graphsync): add pause/resume integration test

* fix(responsemanager): fix directory misname

requesthooks.go directory -> requesthooks

* fix(tests): cleanup races & lint

* test(graphsync): fix flaky test

* feat(graphsync): define hook interfaces, test infrastructure

rename hooks, add outgoing request hook, define chain types for node builder chooser tests

* feat(graphsync): define persistence option

add persistence options to asyncloader & responsemanager + handling

* feat(graphsync): complete persistence / node builder chooser roundtrip

* fix(lint): fix lint errors

* feat(ipldutil): add NodeBuiderChooser to traverse

Add the option to set NodeBuilderChooser on traverse, using default when nil

* feat(responsemanager): set loader and chooser in hooks

Add the ability to set a custom loader and custom node builder chooser in a response hook

* refactor(hooks): refactor default validation as hook, add unregister option

* feat(graphsync): add disable default validation option

* fix(responsemanager): fix mutex  unlocking

cover case where unlocking was not happening

* refactor(tests): introduce testify

Introduce testify, cleanup lots of tests, add channel assertions to simplify channel tests, make
more readable

* refactor(tests): clean up error messages

* fix(deps): remove bitswap

remove accidentally added bitswap

* refactor(tests): put expected in correct position

Make sure for require.equal that the expected value is the first value

* ci(circle): switch to circle

Use configs from bitswap

* fix(deps): update and standardize

* fix(lint): fix lint errors

* fix(tests): relax timings for context cancels

Since we are not relying on context cancellation as anything but a test failsafe, relax timings
significantly for slower machines

* fix(test): fix minor test error

* fix(tests): minor timing fix

* fix(lint): fix lint errors

* fix(tests): relax timings for context cancels

Since we are not relying on context cancellation as anything but a test failsafe, relax timings

Remove all code references to a "bridge", consolodate test utils

remove remaining methods for ipld bridge

Removed mock selector spec, switch to using more real blockchain simulator

Remove the encode and decode on ipldbridge as part of ramp up to removing entirely
Also move selector encode/decode to within protobuf encoding

Add processing of on extensions in responses in the request manager

add the ability to send extensions when creating requests

To reduce future import cycles, and expose more types properly, move the root package to an
interface/types file and move the main implementation into /impl. This also exposes an interface
other implementers can replace.

return a boolean instead of an error if an extention is found on a message

Per https://github.com/ipld/specs/pull/205, metadata is now an extension, and further extensions
will be added soon

Adds callbacks to manage connections, also removes network receiver methods from public graphsync
interface, and updates libp2p to handle go-libp2p-core refactor

Updates IPLD Bridge to use latest go-ipld-prime and runs and actual round trip test using GraphSync,
go-ipld-prime, and real selectors

Successfully execute query on remote system and return results

Improve Error message on missing blocks, and also add missing comments

Move ResponseProgress to the types package and extract some testing functions to testutil

make sure order channels are consumed in repsonse collector doesn't block sends on the other

Make the request manager actually inject responses from the network, and perform a selector
verification

extracts parts of old asyncloader to load attempt queue
expands functionality of asyncloader to handle response cache
moves response cache and unverified block store under async laoder

wrapper around an async load to make a regular ipld.loader for the request manager

Continue to load links synchronously after network request completes

Async loader returns a single channel result type

Create a cache to manager how to track how long to keep unverified blocks around

Create an in memory object for storing blocks received from the server temporarily

Create async loader to handle multiple load attempts as responses come in from network

Convert GraphSyncRequest and GraphSyncResponse to structs, track by value, pass directly into add
methods on GraphSyncMessage

Handle failed request status codes from server and pass on errors

Cleanup request manager test to extract repeated code

Add a channel of asynchronous errors instead of only an initial synchronous one