Skip to content

GitLab

Switch branch/tag
  1. 23 Aug, 2021 1 commit
  3. 27 Jan, 2021 1 commit
  5. 30 Nov, 2020 1 commit
    • Andrew Gillis's avatar
      Datastore based pinner (#4) · 4c920717
      Andrew Gillis authored 
      feat: store pins in datastore instead of a DAG

Adds a new `/pins` namespace to the given datastore and uses that to store pins as cbor binary, keyed by unique pin ID.

The new datastore pinner stores pins in the datastore as individual key-value items. This is faster than the dag pinner, which stored all pins in a single dag that had to be rewritten every time a pin was added or removed.

The new pinner provides a secondary indexing mechanism that can be used to index any data that a pin has. Secondary indexing logic is provided by the `dsindex` package. The new pinner currently includes indexing by CID.

Both the new datastore pinner (`dspinner` package) and the old dag pinner (`ipldpinner` package) implementations are included to support migration between the two.  Migration logic is provided by the `pinconv` package.

Other features in new pinner:
- Benchmarks are provided to compare performance of between the old and new pinners
- New pinner does not keep in-memory set of pinned CIDs, instead it relies on the datastore
- Separate recursive and direct CID indexes allow searching for pins without having to load pin data to check the mode
- New pinner can rebuild indexes on load, if saved pins appear out of sync with the indexes
      4c920717
  7. 19 Nov, 2019 1 commit
  9. 05 Mar, 2019 1 commit
  11. 27 Feb, 2019 2 commits
  13. 21 Feb, 2019 2 commits
  15. 18 Feb, 2019 1 commit
  17. 14 Feb, 2019 1 commit
  19. 08 Feb, 2019 1 commit
  21. 23 Jan, 2019 1 commit
  23. 09 Jan, 2019 1 commit
  25. 07 Dec, 2018 1 commit
  27. 28 Nov, 2018 1 commit
  29. 16 Nov, 2018 1 commit
  31. 03 Nov, 2018 1 commit
  33. 02 Nov, 2018 1 commit
  35. 29 Oct, 2018 1 commit
  37. 24 Oct, 2018 3 commits
  39. 12 Oct, 2018 1 commit
  41. 10 Oct, 2018 1 commit
    • Steven Allen's avatar
      gx: update go-buffer-pool · 843897ca
      Steven Allen authored 
      Turns out that `pool.Put(buf)` had to *allocate* because we needed to turn
`[]byte` into `interface{}`. Apparently, we've never done this correctly we just
never noticed because we never really used buffer pools extensively.

However, since migrating yamux to a buffer-pool backed buffer, this started
showing up in allocation profiles.

License: MIT
Signed-off-by: default avatarSteven Allen <steven@stebalien.com>
      843897ca
  43. 05 Oct, 2018 3 commits
  45. 04 Oct, 2018 1 commit
  47. 24 Sep, 2018 2 commits
  49. 12 Sep, 2018 2 commits
  51. 08 Sep, 2018 1 commit
  53. 31 Aug, 2018 1 commit
  55. 22 Aug, 2018 1 commit
  57. 15 Aug, 2018 2 commits
  59. 13 Aug, 2018 1 commit
  61. 12 Aug, 2018 1 commit