1. 05 Apr, 2021 1 commit
    • Daniel Martí's avatar
      allow decoding PBNode fields in any order · d0b86f76
      Daniel Martí authored
      We recently updated the spec to say that decoders "should" allow
      decoding a PBNode in either order of its fields. That is because some
      IPFS data out in the wild is already encoded in the non-canonical
      format, which is what Protobuf uses as a default, too.
      
      This change makes the decoder here comply with the spec change, adding a
      test with the encoded block that caused this entire spec change in the
      first place: go-ipfs/test/sharness/t0110-gateway-data/foofoo.block.
      
      The change to the decoder is slightly subtle, because the decoder used
      to leverage the knowledge that Links must come before Data to start the
      Links list early, and finish it as soon as Data arrives.
      
      Since that order is now unknown, we must have some extra logic to
      support either order. We also need special code to make sure Links is
      always set, just like before.
      
      We also add a test to double check that Data between Links is rejected.
      d0b86f76
  2. 30 Mar, 2021 3 commits
    • Daniel Martí's avatar
      expose APIs without Reader/Writer overhead · 8ec6b0fb
      Daniel Martí authored
      An io.Writer, by definition, will always copy bytes. That's fine in
      general, and can't be worked around without breaking the writer's
      contract.
      
      However, the main use of go-codec-dagpb today is go-merkledag, which
      simply uses the codec to encode a node into a buffer.
      
      So, we had to create a new bytes.Buffer, write to it, and grab its
      bytes. This is one extra allocation (the bytes.Buffer object itself),
      plus copying the encoded bytes an extra time, since we must copy from
      Encode's internal buffer to the bytes.Buffer.
      
      Add a lower-level append-like AppendEncode that cuts the middle man,
      removing both of those extra pieces of work.
      
      For the sake of consistency, we add DecodeBytes to mirror the above on
      the decode side. Decode already had a shortcut for Bytes, but this way
      it's more evident what we're doing, and we also avoid allocating a
      bytes.Buffer just to call Bytes on it.
      
      Using these new APIs in go-merkledag shows nice results:
      
      	name         old time/op    new time/op    delta
      	Roundtrip-8    4.27µs ± 0%    4.07µs ± 0%  -4.50%  (p=0.004 n=5+6)
      
      	name         old alloc/op   new alloc/op   delta
      	Roundtrip-8    6.86kB ± 0%    6.38kB ± 0%  -6.99%  (p=0.002 n=6+6)
      
      	name         old allocs/op  new allocs/op  delta
      	Roundtrip-8       106 ± 0%       103 ± 0%  -2.83%  (p=0.002 n=6+6)
      
      While at it, we formally deprecate Marshal and Unmarshal, since we're
      starting to have lots of redundant API surface.
      8ec6b0fb
    • Daniel Martí's avatar
      decode directly with a []byte · b4150aed
      Daniel Martí authored
      IPLD's codec helper reader has a relatively high cost, unfortunately. It
      was the main contributor to a slowdown in go-merkledag when moving from
      the old protobuf gogo-generated decoder to go-codec-dagpb.
      
      Using a []byte also means we can reuse protobuf's well-optimized "wire
      encoding" helpers, which gets us extra speed and allows removing some
      code.
      
      This should not matter in practice for the time being, as the only
      go-codec-dagpb user is go-merkledag and it uses bytes.Buffer everywhere.
      
      In the future it would be nice for go-codec-dagpb to be just as
      efficient with a stream decoder, but right now I don't have the extra
      week to get into that. Plus, if the core protobuf implementation works
      on []byte, one can assume it's reasonable for us to do the same.
      
      Using the new BenchmarkRoundtrip in go-merkledag with go-codec-dagpb, we
      get a significant uplift in performance:
      
      	name         old time/op    new time/op    delta
      	Roundtrip-8    6.49µs ± 1%    5.34µs ± 1%  -17.74%  (p=0.002 n=6+6)
      
      	name         old alloc/op   new alloc/op   delta
      	Roundtrip-8    8.07kB ± 0%    7.50kB ± 0%   -7.04%  (p=0.002 n=6+6)
      
      	name         old allocs/op  new allocs/op  delta
      	Roundtrip-8       171 ± 0%       148 ± 0%  -13.45%  (p=0.002 n=6+6)
      b4150aed
    • Daniel Martí's avatar
      remove unnecessary xerrors dep · d6e141f2
      Daniel Martí authored
      std's errors has had wrapping for years now.
      d6e141f2
  3. 01 Jan, 2021 1 commit
  4. 09 Dec, 2020 1 commit
  5. 08 Dec, 2020 2 commits
  6. 07 Dec, 2020 1 commit
  7. 01 Oct, 2020 1 commit
  8. 28 Sep, 2020 2 commits