GitLab

close the node channel in GetMany in the test dag

Extract dagservice and friends from go-ipfs

I considered (well, implemented then threw it away) allowing contexts on all
calls to Batch (Add, Commit, etc). However, really, you should treat a batch
as a single large "operation".

I also went down the road of generalizing batches to sessions. However, it
became immediately obvious that permitting add *and* remove *and* fetch would
require a lot of bookkeeping and that you'd lose a lot of performance. So, we'll
do that separately.

We'll need these for slower/remote datastores.

The caller can just call `node.Cid()` and returning CIDs from `AddMany` requires
allocation.

1. Add is already idempotent. This makes remove match.
2. Generally, all we care about is that the node no longer exists in the DAG. If
two callers remove a node at the same time, they should both succeed. Currently,
we *ignore* the result of Remove in go-ipfs. IMO, it would be better to let
errors be *errors* and only return an error if something goes wrong.
3. This can be significantly faster. It allows us to batch/queue removes (as
long as we guarantee that they'll eventually happen).
4. This matches how most databases/key-value stores operate.

An alternative would be to return `(deleted bool, err error)` but then we don't
get the speedup.

(ipfs/go-ipfs#4296)

1. Modern storage devices (i.e., SSDs) tend to be highly parallel.
2. Allows us to read and write at the same time (avoids pausing while flushing).

fixes https://github.com/ipfs/go-ipfs/issues/898#issuecomment-331849064

It really just doesn't fit. We're working on making this method obsolete
anyways.

Keep merkledag clean.

This will allow many consumers of `DAGService` to take `NodeGetter` instead and
implementing `GetMany` for all `NodeGetter`s is pretty trivial.

We're going to want to do this eventually and we have to refactor anyways so we
might as well do this now.

Fixes ipfs/go-ipfs#4010

We'll just live with this name.

Also:

1. Specify the threading guarantees.
2. Vastly simplify it to use a single channel for synchronization.

This way, not *all* DAGServices need to implement this interface, they can just
implement it as an optimization.

This is a WIP/RFC attempt at extracting DAGService from go-ipfs.

gx: update go-cid, go-multibase, base32

rework block decoding framework

1. Use an interface instead of a map so that we can define fancier block
   decoders.
2. Make DefaultBlockDecoder thread safe.