GitLab

Ignore non-directories in the top level directory to allow placing a
README or similar file there.

Note: Readdirnames does not return entries for "." or ".." but it a
good idea to skip dirs/files starting with '.' anyway.

filepath.Walk() sorts the names and calls lstat() on each entry, both of
which are unnecessary when all we need are the key names in random order.

For large datastores can improve the performance by about 50%.

It is rising number of running threads of go-ipfs from 12 to 200+ when
the flatfs datastore works on slowish HDDs.

This is to prevent race that can happen while GCing the repo
Walk will build list of files and one of those files can be removed
by GC, currently it causes a failure which shouldn't be the case.

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

We have some random crashes there and it would be good
to know what are erros that are causing those crashes.

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

add log statement for retry

vendor

implement batch ops for different datastore types

rename Transaction to Batch

Revert "add in support for batched writes"

add in benchmarks for put and batchput

move batching into separate interface

address concerns from PR

regrab old code

For background, see https://github.com/golang/go/issues/3366

Here's the failure seen:

    --- FAIL: TestPutOverwrite (0.03s)
    	flatfs_test.go:118: Put fail: rename C:\Users\gates\AppData\Local\Temp\test-datastore-flatfs-679014441\7175\put- 717454915 C:\Users\gates\AppData\Local\Temp\test-datastore-flatfs-679014441/7175/71757578.data: Cannot create a file when that file already exists.

Used to error out with

    flatfs_test.go:66: Put fail: failed to open dir: fsync: The handle is invalid

It seems Windows does not want syscall.Fsync to be called on directory
handles.

Backported from commit e965c53780003a01fcbd94af96333ace4b3e2748 in the
vendored copy in https://github.com/ipfs/go-ipfs

Sadly untestable without low-level fault injection or extensive
mocking.

Sadly untestable without low-level fault injection or extensive
mocking.