Skip to content

GitLab

Switch branch/tag
  1. 13 Aug, 2015 1 commit
  3. 12 Aug, 2015 2 commits
    • Juan Benet's avatar
      Merge pull request #1565 from ipfs/add-patch · ace06b47
      Juan Benet authored 
      cmds/add: use dagutils.Editor, like patch
      ace06b47
    • Juan Batiz-Benet's avatar
      cmds/add: use dagutils.Editor, like patch · 978c9fa1
      Juan Batiz-Benet authored 
      This changes the pin behavior. It uses the filenames given through
the api, and allows files to be streamed faltly (not a hierarchy),
which is easier for other things (like vinyl in node-ipfs-api land).
Files can also be entirely out of order, and the garbage intermediate
directories will not be pinned (gc-ed later).

The changes also mean the output of add has changed slightly-- it
no longer shows the local path added, but rather the dag path
relative to the added roots. This is a small difference, but changes
tests.

The dagutils.Editor creates a lot of chaff (intermediate objects)
along the way. Wonder how we might minimize the writes to the
datastore...

This commit also removes the "NilRepo()" part of the --only-hash
mode. We need to store at least in an in-mem repo/datastore because
otherwise the dagutils.Editor breaks.

License: MIT
Signed-off-by: default avatarJuan Batiz-Benet <juan@benet.ai>
      978c9fa1
  5. 11 Aug, 2015 1 commit
  7. 10 Aug, 2015 1 commit
  9. 08 Aug, 2015 1 commit
  11. 05 Aug, 2015 7 commits
  13. 04 Aug, 2015 2 commits
    • Juan Batiz-Benet's avatar
      bitswap/provide: improved rate limiting · 6e705e1e
      Juan Batiz-Benet authored 
      this PR greatly speeds up providing and add.

(1) Instead of idling workers, we move to a ratelimiter-based worker.
We put this max at 512, so that means _up to_ 512 goroutines. This
is very small load on the node, as each worker is providing to the
dht, which means mostly waiting. It DOES put a large load on the DHT.
but i want to try this out for a while and see if it's a problem.
We can decide later if it is a problem for the network (nothing
stops anyone from re-compiling, but the defaults of course matter).

(2) We add a buffer size for provideKeys, which means that we block
the add process much less. this is a very cheap buffer, as it only
stores keys (it may be even cheaper with a lock + ring buffer
instead of a channel...). This makes add blazing fast-- it was being
rate limited by providing. Add should not be ratelimited by providing
(much, if any) as the user wants to just store the stuff in the local
node's repo. This buffer is initially set to 4096, which means:

  4096 * keysize (~258 bytes + go overhead) ~ 1-1.5MB

this buffer only last a few sec to mins, and is an ok thing to do
for the sake of very fast adds. (this could be a configurable
paramter, certainly for low-mem footprint use cases). At the moment
this is not much, compared to block sizes.

(3) We make the providing EventBegin() + Done(), so that we can
track how long a provide takes, and we can remove workers as they
finish in bsdash and similar tools.

License: MIT
Signed-off-by: default avatarJuan Batiz-Benet <juan@benet.ai>
      6e705e1e
    • Juan Benet's avatar
      Merge pull request #1548 from prusnak/remove-p224 · b30d9d4d
      Juan Benet authored 
      remove elliptic.P224 usage
      b30d9d4d
  15. 03 Aug, 2015 1 commit
  17. 02 Aug, 2015 6 commits
  19. 30 Jul, 2015 5 commits
  21. 29 Jul, 2015 13 commits