- 28 Apr, 2021 2 commits
-
-
Will authored
Allow emitting & parsing of bytes per dagjson codec spec
-
Will Scott authored
-
- 26 Apr, 2021 1 commit
-
-
Will Scott authored
fix #165
-
- 25 Apr, 2021 2 commits
-
-
Eric Myhre authored
These are somewhat overdue, and clarify what features are supported, and also note some discrepancies in implementation versus the spec. (As I'm taking this inventory of discrepancies, there's admittedly rather more than I'd like... but step 1: document the current truth. Prioritizing which things to hack on, in the field of infinite possible prioritizations of things that need hacking on, can be a step 2.)
-
Eric Myhre authored
These had drifted a bit, and I just noticed it's due for some freshening up.
-
- 19 Apr, 2021 1 commit
-
-
Eric Myhre authored
schema/gen/go: apply gofmt automatically
-
- 09 Apr, 2021 4 commits
-
-
Daniel Martí authored
Now that we buffer the output, using go/format is trivial. This makes the default behavior better, and means not having to use an extra gofmt go:generate step everywhere.
-
Daniel Martí authored
And re-generate all code in this module. This gets us to a point where go-codec-dagpb has zero vet warnings, for example. schema/dmt still has a few warnings, but those are trickier to fix, so will require another PR.
-
Eric Myhre authored
schema/gen/go: batch file writes via a bytes.Buffer
-
Daniel Martí authored
With this change, running 'go generate ./...' on the entire module while running gopls on one of its files drops gopls's CPU spinning from ~25s to well under a second. They should improve that anyway, but there's no reason for the tens of thousands of tiny FS writes on our end either. The time to run 'go generate ./...' itself is largely unaffected; it goes from ~1.2s to ~1.1s, judging by a handful of runs.
-
- 07 Apr, 2021 1 commit
-
-
Daniel Martí authored
If we know that a schema type can be represented in Go with a small amount of bytes, using a pointer to store its "maybe" is rarely a good idea. For example, an optional string only weighs twice as much as a pointer, so a pointer adds overhead and will barely ever save any memory. Add a function to work out the byte size of a schema.TypeKind, relying on reflection and the basicnode package. Debug prints are also present if one wants to double-check the numbers. As of today, they are: sizeOf(small): 32 (4x pointer size) sizeOf(Bool): 1 sizeOf(Int): 8 sizeOf(Float): 8 sizeOf(String): 16 sizeOf(Bytes): 24 sizeOf(List): 24 sizeOf(Map): 32 sizeOf(Link): 16 Below is the result on go-merkledag's BenchmarkRoundtrip after re-generating go-codec-dagpb with this change. Note that the dag-pb schema contains multiple optional fields, such as strings. name old time/op new time/op delta Roundtrip-8 4.24µs ± 3% 3.78µs ± 0% -10.87% (p=0.004 n=6+5) name old alloc/op new alloc/op delta Roundtrip-8 6.38kB ± 0% 6.24kB ± 0% -2.26% (p=0.002 n=6+6) name old allocs/op new allocs/op delta Roundtrip-8 103 ± 0% 61 ± 0% -40.78% (p=0.002 n=6+6) Schema typekinds which don't directly map to basicnode prototypes, such as structs and unions, are left as a TODO for now. I did not do any measurements to arrive at the magic number of 4x, which is documented in the code. We might well increase it in the future, with more careful benchmarking. For now, it seems like a conservative starting point that should cover all basic types. Finally, re-generate within this repo.
-
- 03 Apr, 2021 1 commit
-
-
Eric Myhre authored
-
- 02 Apr, 2021 1 commit
-
-
Eric Myhre authored
feat(linksystem): add reification to LinkSystem
-
- 01 Apr, 2021 1 commit
-
-
hannahhoward authored
add an optional reifier into the link system process. The reason to do this is to capture the link system itself when reification happens, in case it needs to be put into the node, which is often not accessible by the time you have a node. another alternative would be to make this specific to selector traversal, similar to LinkNodePrototypeChooser
-
- 24 Mar, 2021 2 commits
-
-
Eric Myhre authored
Add option to tell link system storage is trusted and we can skip hash on read
-
hannahhoward authored
Have boolean that specifies whether the storage is trusted. if it is, skip hashing on reads
-
- 23 Mar, 2021 5 commits
-
-
Eric Myhre authored
implement non-dag cbor codec
-
Will Scott authored
-
Eric Myhre authored
add non-dag json codec
-
Will Scott authored
-
Will Scott authored
-
- 22 Mar, 2021 1 commit
-
-
Will Scott authored
-
- 15 Mar, 2021 5 commits
-
-
Eric Myhre authored
:'(
-
Eric Myhre authored
-
Eric Myhre authored
-
Eric Myhre authored
This is just documenting our defacto policies already.
-
Eric Myhre authored
Unless we can keep these up to date with go-mod state automatically (and we don't currently have tools for that), maintaining both these systems is problematic.
-
- 12 Mar, 2021 9 commits
-
-
Eric Myhre authored
introduce LinkSystem
-
Eric Myhre authored
-
Eric Myhre authored
(It can be hard to intuit this just by reading the code, because some of the key relevance is actually in *other* functions that might not be in this repo at all!)
-
Eric Myhre authored
(Some things got better. Several others in the area still have not.)
-
hannahhoward authored
-
Eric Myhre authored
Resolves conflicts in go.sum. (They were conflicts in a textual sense only, but semantically trivial; application of `go mod tidy` was sufficient.)
-
Eric Myhre authored
-
Eric Myhre authored
I'm still not fully convinced this is always a win for performance, but as discussed in https://github.com/ipld/go-ipld-prime/pull/143#discussion_r582335774 , there was past evidence, and reproducing that takes work -- so, we're going to try to be relatively conservative here and keep this logical branch behavior in place for now. (The reason this may be worth re-examining is that the change to hashing interface probably gets rid of one big source of copies. That doens't answer the holistic performance question alone, though; it just reopens it.)
-
Eric Myhre authored
-
- 10 Mar, 2021 1 commit
-
-
Eric Myhre authored
Updated the phrasing on relationship to alterative golang libraries. The state of play there is a bit more advanced than the last time I updated this readme. Was also able to add quite a few more statements on distinguishing features vs alternative libraries. More discussion of version strategy. It's defacto what I've been doing already; now it is documented.
-
- 05 Mar, 2021 1 commit
-
-
Daniel Martí authored
It's small, it's simple, and it's already widely used as part of unixfs. So there's no reason it shouldn't be part of go-ipld-prime. The codec is tiny, but has three noteworthy parts: the Encode and Decode funcs, the cidlink multicodec registration, and the Bytes method shortcut. Each of these has its own dedicated regression test. I'm also using this commit to showcase the use of quicktest instead of go-wish. The result is extremely similar, but with less dot-import magic. For example, if I remove the Bytes shortcut in Decode: --- FAIL: TestDecodeBuffer (0.00s) codec_test.go:115: error: got non-nil error got: e"could not decode raw node: must not call Read" stack: /home/mvdan/src/ipld/codec/raw/codec_test.go:115 qt.Assert(t, err, qt.IsNil)
-
- 25 Feb, 2021 2 commits
-
-
Eric Myhre authored
And, make a package which can be imported to register "all" of the multihashes. (Or at least all of them that you would've expected from go-multihash.) There are also packages that are split roughly per the transitive dependency it brings in, so you can pick and choose. This cascaded into more work than I might've expected. Turns out a handful of the things we have multihash identifiers for actually *do not* implement the standard hash.Hash contract at all. For these, I've made small shims. Test fixtures across the library switch to using sha2-512. Previously I had written a bunch of them to use sha3 variants, but since that is not in the standard library, I'm going to move away from that so as not to re-bloat the transitive dependency tree just for the tests and examples.
-
Eric Myhre authored
This significantly reworks how linking is handled. All of the significant operations involved in storing and loading data are extracted into their own separate features, and the LinkSystem just composes them. The big advantage of this is we can now add as many helper methods to the LinkSystem construct as we want -- whereas previously, adding methods to the Link interface was a difficult thing to do, because that interface shows up in a lot of places. Link is now *just* treated as a data holder -- it doesn't need logic attached to it directly. This is much cleaner. The way we interact with the CID libraries is also different. We're doing multihash registries ourselves, and breaking our direct use of the go-multihash library. The big upside is we're now using the familiar and standard hash.Hash interface from the golang stdlib. (And as a bonus, that actually works streamingly; go-mulithash didn't.) However, this also implies a really big change for downstream users: we're no longer baking as many hashes into the new multihash registry by default.
-