GitLab

This commit adds a new set of sharness tests for pinning, and addresses
bugs that were pointed out by said tests.

test/sharness: added more pinning tests

Pinning is currently broken. See issue #1051. This commit introduces
a few more pinning tests. These are by no means exhaustive, but
definitely surface the present problems going on. I believe these
tests are correct, but not sure. Pushing them as failing so that
pinning is fixed in this PR.

make pinning and merkledag.Get take contexts

improve 'add' commands usage of pinning

FIXUP: fix 'pin lists look good'

ipfs-pin-stat simple script to help check pinning

This is a simple shell script to help check pinning.

We ought to strive towards making adding commands this easy.
The http api is great and powerful, but our setup right now
gets in the way. Perhaps we can clean up that area.

updated t0081-repo-pinning

- fixed a couple bugs with the tests
- made it a bit clearer (still a lot going on)
- the remaining tests are correct and highlight a problem with
  pinning. Namely, that recursive pinning is buggy. At least:
  towards the end of the test, $HASH_DIR4 and $HASH_FILE4 should
  be pinned indirectly, but they're not. And thus get gc-ed out.
  There may be other problems too.

cc @whyrusleeping

fix grep params for context deadline check

fix bugs in pin and pin tests

check for block local before checking recursive pin

// AddRawLink adds a link to this node
AddRawLink(name string, lnk *Link) error

// Return a copy of the link with given name
GetNodeLink(name string) (*Link, error)

- Modified Godeps/Godeps.json by hand
- [TEST] Updated welcome docs hash to sharness
- [TEST] Updated contact doc
- [TEST] disabled breaking test (t0080-repo refs local)

update context imports to golang.org/x/net

- updated go-ctxgroup and goprocess
ctxgroup: AddChildGroup was changed to AddChild. Used in two files:
- p2p/net/mock/mock_net.go
- routing/dht/dht.go

- updated context from hg repo to git
prev. commit in hg was ad01a6fcc8a19d3a4478c836895ffe883bd2ceab. (context: make parentCancelCtx iterative)
represents commit 84f8955a887232b6308d79c68b8db44f64df455c in git repo

- updated context to master (b6fdb7d8a4ccefede406f8fe0f017fb58265054c)

Aaron Jacobs (2):
net/context: Don't accept a context in the DoSomethingSlow example.
context: Be clear that users must cancel the result of WithCancel.

Andrew Gerrand (1):
go.net: use golang.org/x/... import paths

Bryan C. Mills (1):
net/context: Don't leak goroutines in Done example.

Damien Neil (1):
context: fix removal of cancelled timer contexts from parent

David Symonds (2):
context: Fix WithValue example code.
net: add import comments.

Sameer Ajmani (1):
context: fix TestAllocs to account for ints in interfaces

Let's save log.Error for things the user can take action on.
Moved all our diagnostics to log.Debug. We can ideally reduce them
even further.

All the datastores used by pinners and so on should be mutex
wrapped. One issue with changing all of them from
ds.Datastore -> ds.ThreadSafeDatastore is that we wrap the
incoming ds.ThreadSafeDatastore with other datastores, which
do not implement the interface. Re-wrapping again causes
double locking. (which may be ok..., but...) any ideas?

'ipfs object stat' is a plumbing command to print DAG
node statistics. <key> is a base58 encoded multihash.
It outputs to stdout:

  NumLinks        int number of links in link table
  BlockSize       int size of the raw, encoded data
  LinksSize       int size of the links segment
  DataSize        int size of the data segment
  CumulativeSize  int cumulative size of object and references

May not be necessary to sort when adding each link--
doing so would be unnecessarily expensive O(n^2) when
constructing nodes -- though n wont be big.

improve efficiency of multilayered indirect blocks

clean up tests

panic cleanup

clean up logic, improve readability

add final root node to the dagservice upon creation

importer: simplified dag generation

test: updated hashes using latest code

@whyrusleeping this is why the sharness tests
were failing: the hashes are added manually to
make sure our generation doesn't change.

cleanup after CR

fix merkledag tests

fix small block generation (no subblocks!)

we shouldn't use an arbitrary timeout here. since Get
doesnt take in a context yet, we give a large upper bound.
think of an http request. we want it to go on as long as
the client requests it.

https://travis-ci.org/jbenet/go-ipfs/jobs/45000756

cc @whyrusleeping @jbenet

lol this is starting to happen pretty often

cc @jbenet @whyrusleeping

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>

cc @maybebtc