GitLab

Store the sharding function used in the file "SHARDING" in the repo.
To make this work the sharding function is now always specified as a
string.

Add test to make sure suffix sharding prevents prefix clustering.

This is done by fixing the Rand source to use a seed for deterministic
results and increasing the number of keys used.

Closes #8

Add support for NextToLast shard function.

coverage: switch to codecov

Also fix bug when only the Prefix function was being tested in most cases.

Ignore non-directories in the top level directory.

Ignore non-directories in the top level directory to allow placing a
README or similar file there.

Support using suffix of key for directory in addition to the prefix.

Improve Query Perforamance

Note: Readdirnames does not return entries for "." or ".." but it a
good idea to skip dirs/files starting with '.' anyway.

filepath.Walk() sorts the names and calls lstat() on each entry, both of
which are unnecessary when all we need are the key names in random order.

For large datastores can improve the performance by about 50%.

Limin number of goroutines stuck on Sync syscall

It is rising number of running threads of go-ipfs from 12 to 200+ when
the flatfs datastore works on slowish HDDs.

This is to prevent race that can happen while GCing the repo
Walk will build list of files and one of those files can be removed
by GC, currently it causes a failure which shouldn't be the case.

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

Print formatted string using Errorf

merge ipfs org changes back into jbenet/go-datastore

We have some random crashes there and it would be good
to know what are erros that are causing those crashes.

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>