GitLab

It is rising number of running threads of go-ipfs from 12 to 200+ when
the flatfs datastore works on slowish HDDs.

This is to prevent race that can happen while GCing the repo
Walk will build list of files and one of those files can be removed
by GC, currently it causes a failure which shouldn't be the case.

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

Print formatted string using Errorf

merge ipfs org changes back into jbenet/go-datastore

We have some random crashes there and it would be good
to know what are erros that are causing those crashes.

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

add log statement for retry

vendor

Batching

implement batch ops for different datastore types

rename Transaction to Batch

Revert "add in support for batched writes"

add in benchmarks for put and batchput

move batching into separate interface

address concerns from PR

regrab old code

For background, see https://github.com/golang/go/issues/3366

Here's the failure seen:

    --- FAIL: TestPutOverwrite (0.03s)
    	flatfs_test.go:118: Put fail: rename C:\Users\gates\AppData\Local\Temp\test-datastore-flatfs-679014441\7175\put- 717454915 C:\Users\gates\AppData\Local\Temp\test-datastore-flatfs-679014441/7175/71757578.data: Cannot create a file when that file already exists.

Used to error out with

    flatfs_test.go:66: Put fail: failed to open dir: fsync: The handle is invalid

It seems Windows does not want syscall.Fsync to be called on directory
handles.

Previous commit made flatfs strip first byte of keys (under the
assumption that datastore always makes it be "/").

Backported from commit e965c53780003a01fcbd94af96333ace4b3e2748 in the
vendored copy in https://github.com/ipfs/go-ipfs