1. 20 Apr, 2021 1 commit
  2. 07 Apr, 2021 1 commit
    • Daniel Martí's avatar
      update go-ipld-prime, use go:generate · 5186aac7
      Daniel Martí authored
      A number of codegen changes have happened recently. Most notably, basic
      types behind a Maybe no longer use pointers, which means far fewer
      allocations in common scenarios.
      
      This results in a nice speed-up for go-merkledag's Roundtrip benchmark:
      
      	name         old time/op    new time/op    delta
      	Roundtrip-8    4.24µs ± 3%    3.78µs ± 0%  -10.87%  (p=0.004 n=6+5)
      
      	name         old alloc/op   new alloc/op   delta
      	Roundtrip-8    6.38kB ± 0%    6.24kB ± 0%   -2.26%  (p=0.002 n=6+6)
      
      	name         old allocs/op  new allocs/op  delta
      	Roundtrip-8       103 ± 0%        61 ± 0%  -40.78%  (p=0.002 n=6+6)
      
      Run gofmt too, as prime's gen does not just yet.
      5186aac7
  3. 05 Apr, 2021 1 commit
    • Daniel Martí's avatar
      allow decoding PBNode fields in any order · d0b86f76
      Daniel Martí authored
      We recently updated the spec to say that decoders "should" allow
      decoding a PBNode in either order of its fields. That is because some
      IPFS data out in the wild is already encoded in the non-canonical
      format, which is what Protobuf uses as a default, too.
      
      This change makes the decoder here comply with the spec change, adding a
      test with the encoded block that caused this entire spec change in the
      first place: go-ipfs/test/sharness/t0110-gateway-data/foofoo.block.
      
      The change to the decoder is slightly subtle, because the decoder used
      to leverage the knowledge that Links must come before Data to start the
      Links list early, and finish it as soon as Data arrives.
      
      Since that order is now unknown, we must have some extra logic to
      support either order. We also need special code to make sure Links is
      always set, just like before.
      
      We also add a test to double check that Data between Links is rejected.
      d0b86f76
  4. 30 Mar, 2021 5 commits
    • Daniel Martí's avatar
      expose APIs without Reader/Writer overhead · 8ec6b0fb
      Daniel Martí authored
      An io.Writer, by definition, will always copy bytes. That's fine in
      general, and can't be worked around without breaking the writer's
      contract.
      
      However, the main use of go-codec-dagpb today is go-merkledag, which
      simply uses the codec to encode a node into a buffer.
      
      So, we had to create a new bytes.Buffer, write to it, and grab its
      bytes. This is one extra allocation (the bytes.Buffer object itself),
      plus copying the encoded bytes an extra time, since we must copy from
      Encode's internal buffer to the bytes.Buffer.
      
      Add a lower-level append-like AppendEncode that cuts the middle man,
      removing both of those extra pieces of work.
      
      For the sake of consistency, we add DecodeBytes to mirror the above on
      the decode side. Decode already had a shortcut for Bytes, but this way
      it's more evident what we're doing, and we also avoid allocating a
      bytes.Buffer just to call Bytes on it.
      
      Using these new APIs in go-merkledag shows nice results:
      
      	name         old time/op    new time/op    delta
      	Roundtrip-8    4.27µs ± 0%    4.07µs ± 0%  -4.50%  (p=0.004 n=5+6)
      
      	name         old alloc/op   new alloc/op   delta
      	Roundtrip-8    6.86kB ± 0%    6.38kB ± 0%  -6.99%  (p=0.002 n=6+6)
      
      	name         old allocs/op  new allocs/op  delta
      	Roundtrip-8       106 ± 0%       103 ± 0%  -2.83%  (p=0.002 n=6+6)
      
      While at it, we formally deprecate Marshal and Unmarshal, since we're
      starting to have lots of redundant API surface.
      8ec6b0fb
    • Daniel Martí's avatar
      preallocate 1KiB on the stack for marshals · a4adba8b
      Daniel Martí authored
      	name         old time/op    new time/op    delta
      	Roundtrip-8    4.35µs ± 1%    4.27µs ± 0%  -1.92%  (p=0.004 n=6+5)
      
      	name         old alloc/op   new alloc/op   delta
      	Roundtrip-8    6.86kB ± 0%    6.86kB ± 0%  +0.01%  (p=0.004 n=5+6)
      
      	name         old allocs/op  new allocs/op  delta
      	Roundtrip-8       112 ± 0%       106 ± 0%  -5.36%  (p=0.002 n=6+6)
      a4adba8b
    • Daniel Martí's avatar
      encode directly with a []byte · fd4638f3
      Daniel Martí authored
      Like the previous commit, this helps reduce allocations as well as
      improve performance thanks to the well-optimized protowire package.
      
      And, as before, we get to remove unnecessary code.
      
      	name         old time/op    new time/op    delta
      	Roundtrip-8    4.81µs ± 0%    4.35µs ± 1%  -9.59%  (p=0.004 n=5+6)
      
      	name         old alloc/op   new alloc/op   delta
      	Roundtrip-8    7.14kB ± 0%    6.86kB ± 0%  -3.83%  (p=0.000 n=6+5)
      
      	name         old allocs/op  new allocs/op  delta
      	Roundtrip-8       119 ± 0%       112 ± 0%  -5.88%  (p=0.002 n=6+6)
      fd4638f3
    • Daniel Martí's avatar
      decode directly with a []byte · b4150aed
      Daniel Martí authored
      IPLD's codec helper reader has a relatively high cost, unfortunately. It
      was the main contributor to a slowdown in go-merkledag when moving from
      the old protobuf gogo-generated decoder to go-codec-dagpb.
      
      Using a []byte also means we can reuse protobuf's well-optimized "wire
      encoding" helpers, which gets us extra speed and allows removing some
      code.
      
      This should not matter in practice for the time being, as the only
      go-codec-dagpb user is go-merkledag and it uses bytes.Buffer everywhere.
      
      In the future it would be nice for go-codec-dagpb to be just as
      efficient with a stream decoder, but right now I don't have the extra
      week to get into that. Plus, if the core protobuf implementation works
      on []byte, one can assume it's reasonable for us to do the same.
      
      Using the new BenchmarkRoundtrip in go-merkledag with go-codec-dagpb, we
      get a significant uplift in performance:
      
      	name         old time/op    new time/op    delta
      	Roundtrip-8    6.49µs ± 1%    5.34µs ± 1%  -17.74%  (p=0.002 n=6+6)
      
      	name         old alloc/op   new alloc/op   delta
      	Roundtrip-8    8.07kB ± 0%    7.50kB ± 0%   -7.04%  (p=0.002 n=6+6)
      
      	name         old allocs/op  new allocs/op  delta
      	Roundtrip-8       171 ± 0%       148 ± 0%  -13.45%  (p=0.002 n=6+6)
      b4150aed
    • Daniel Martí's avatar
      remove unnecessary xerrors dep · d6e141f2
      Daniel Martí authored
      std's errors has had wrapping for years now.
      d6e141f2
  5. 29 Mar, 2021 5 commits
  6. 27 Mar, 2021 1 commit
  7. 17 Mar, 2021 1 commit
    • Daniel Martí's avatar
      re-add the Encoder and Decoder APIs · 27e0a6a4
      Daniel Martí authored
      We're a semver v1, so we can't remove exposed APIs unless we bump to
      v2+. Bumping the major version to just rename two APIs seems a bit
      overkill; keeping the old ones around and deprecating them is much
      easier.
      
      This partially reverts commit b8473079.
      27e0a6a4
  8. 16 Mar, 2021 1 commit
    • Daniel Martí's avatar
      add a prototype chooser API for integrations · b641b0ff
      Daniel Martí authored
      At least go-merkledag and go-graphsync still need a prototype chooser,
      and go-ipld-prime-proto provides one, so this simplifies integration
      with downstreams.
      
      It's worth noting that we just add PBNode here, while
      go-ipld-prime-proto also added RawNode. This is because this dag-pb
      module leaves raw nodes out, since those are now in
      go-ipld-prime/codec/raw.
      
      The logic is just three lines, but it's still easier to not have to
      copy-paste these between different projects.
      b641b0ff
  9. 15 Mar, 2021 4 commits
  10. 08 Mar, 2021 1 commit
  11. 28 Feb, 2021 1 commit
  12. 26 Feb, 2021 2 commits
    • Daniel Martí's avatar
      fix the module path · ca0dd481
      Daniel Martí authored
      The module was defined as github.com/ipld/go-codec-dagpb/dagpb, but the
      repository is at github.com/ipld/go-codec-dagpb. This means that neither
      can be resolved as a valid Go package import path; the former results in
      a GitHub 404, and the latter results in a valid GitHub repository which
      holds a different module.
      
      Rod confirmed that the /dagpb suffix wasn't intentional, so just get rid
      of it.
      
      While at it, mod tidy.
      ca0dd481
    • ipldbot's avatar
      update automerge.yml (#10) · 293edc87
      ipldbot authored
      293edc87
  13. 25 Feb, 2021 6 commits
  14. 10 Jan, 2021 1 commit
  15. 01 Jan, 2021 3 commits
  16. 14 Dec, 2020 2 commits
  17. 09 Dec, 2020 2 commits
  18. 08 Dec, 2020 2 commits