GitLab

Useful for tests that do deep equality tests on structures.

Same caveat about current placement of this method as in the previous
commit: this might be worth detaching and shifting to a 'codectest'
or 'tokentest' package.  But let's see how it shakes out.

This is far too useful in testing to reproduce in each package that
needs something like it.  It's already shown up as desirable again
as soon as I start implementing even a little bit of even one codec
tokenizer, and that's gonna keep happening.

This might be worth moving to some kind of a 'tokentest' or
'codectest' package instead of cluttering up this one, but...
we'll see; I've got a fair amount more code to flush into commits,
and after that we can reshake things and see if packages settle
out differently.

There were already comments about how this would be "probably"
necessary; I don't know why I wavered, it certainly is.

You can write a surprising amount of code where the compiler will shrug
and silently coerce things for you.  Right up until you can't.
(Some test cases that'll be coming down the commit queue shortly
happened to end up checking the type of the constants, and, well.
Suddenly this was noticable.)

We definitely did make a TokenWalker, heh.

The other naming marsh (heh, see what I did there?) is still unresolved
but can stay unresolved a while longer.

The tokenization system may look familiar to refmt's tokens -- and
indeed it surely is inspired by and in the same pattern -- but it
hews a fair bit closer to the IPLD Data Model definitions of kinds,
and it also includes links as a token kind.  Presense of link as
a token kind means if we build codecs around these, the handling
of links will be better and most consistently abstracted (the
current dagjson and dagcbor implementations are instructive for what
an odd mess it is when you have most of the tokenization happen
before you get to the level that figures out links; I think we can
improve on that code greatly by moving the barriers around a bit).

I made both all-at-once and pumpable versions of both the token
producers and the token consumers.  Each are useful in different
scenarios.  The pumpable versions are probably generally a bit slower,
but they're also more composable.  (The all-at-once versions can't
be glued to each other; only to pumpable versions.)

Some new and much reduced contracts for codecs are added,
but not yet implemented by anything in this comment.
The comments on them are lengthy and detail the ways I'm thinking
that codecs should be (re)implemented in the future to maximize
usability and performance and also allow some configurability.
(The current interfaces "work", but irritate me a great deal every
time I use them; to be honest, I just plain guessed badly at what
the API here should be the first time I did it.  Configurability
should be both easy to *not* engage in, but also easier if you do
(and in pariticular, not require reaching to *another* library's
packages to do it!).)  More work will be required to bring this
to fruition.

It may be particularly interesting to notice that the tokenization
systems also allow complex keys -- maps and lists can show up as the
keys to maps!  This is something not allowed by the data model (and
for dare I say obvious reasons)... but it's something that's possible
at the schema layer (e.g. structs with representation strategies that
make them representable as strings can be used as map keys), so,
these functions support it.

Add a demo ADL (rot13adl)

Fixed a symbol to be exported that's needed for this to be possible
when outside the package.  This still probably deserves an interface,
too, though.  Comments on that also updated, but we'll still leave
that for future work (more examples of more ADLs wanted before we try
to solidify on something there).

rot13adl demo: finish documentation; simplify Reify; more recommendations about how to implement Reify; consistent export symbol conventions; some fixes.

Introduce traversal function that selects links out of a tree.

Codegen various improvements

Absence of this is an oversight, and I just happened to catch it while
passing through the vicinity.

Also: dropped a comment for later review on the bytesprefix strategy.
While adding the stringprefix strategy, it's hard not to notice that
variable length strings are allowed; so, it occurs to me we should
probably do the same for byteprefix.  (Also, possibly renaming it to
byte*s*prefix.)  Doing this would also fix the ancient weirdness of
the map being flipped in an awkward way to evade int keys, which is
a very happy coincidence (and in retrospect, I'm not sure why we didn't
think of this solution earlier).

The capitalization on this has varied a bit over time.
It's been tempting to capitalize these things because they're clearly
two english words.  However, I'm taking the line that they're a single
word that just happens to have been derived from two english words,
and such a neologism does not retain mid-word capitalization.

(I'm looking at this right now because I'm attempting to write some new
code around the schema-schema outputs, and so I want any dissonance and
inconsistency gone from the start in this new code.)

No suprise this is needed; just filling in the gaps.

No surprise this is needed; just filling in the gaps.

All of these were fixed by https://github.com/ipld/go-ipld-prime/pulls/85 ,
but it's good to have a test saying so and watching for regression.

Reported via https://github.com/LeastAuthority/go-ipld-prime/issues/7 .

No idea why I didn't do this in the first pass.  Sheesh.
It's like the single most useful thing for banging out fixtures...

codegen: make error info available when tuples process data that is too long.

This requires introducing an error-carrying NodeAssembler,
because the AssembleValue methods don't have the ability to return errors themselves.

AssembleValue methods have not needed to return errors before!
Most lists don't have any reason to error: out of our whole system,
it's only struct-with-tuple-representation that can have errors here,
due to tuples having length limits.
AssembleValue for maps doesn't have a similar challenge either,
because key invalidity can always be indicated by errors returned from
the key assembly process.

I'm not a big fan of this diff -- error carrying thunks like this are
ugly to write and they're also pretty ugly to use -- but I'm not sure
what would be better.  ListAssembler.AssembleValue returning an error?
Turning ListAssembler into a two phase thing, e.g. with an Advance
method that fills some of the same role as AssembleKey does for maps,
and gives us a place to return errors?

I don't think we've resolved the questions already noted in that diff yet,
and I abhor drifting open branches, so I'm going to "merge" this
such that it's in git history for reference, but not an effected diff.

Additionally, I believe we encountered a question about whether this string
should include multibase -- this diff doesn't; the spec suggested it should;
and I think we've agreed the spec should be changed, but I'm not sure if
that's been done yet.

At any rate, someone's welcome to take a look at this again in the future.

I haven't implemented the reader side because I'm not sure it's possible;
the specification is insufficiently clear.
I opened Issue https://github.com/ipld/specs/issues/302 to track this.

Implement resource budgets in dagcbor parsing.

Codegen for links should emit the methods to conform to the schema.TypedLinkNode interface where applicable.

Codegen for links should emit the methods to conform to the schema.TypedLinkNode interface where applicable.

Should fix https://github.com/ipld/go-ipld-prime/issues/90 .

Introduce fluent.Reflect convenience functions.

They do quite a lot of work, including setting up a type system,
generating Go code, and building it.

This package is by far the slowest to test. On a warm cache, 'go test
-count=1 ./...' shows all packages under 0.2s, while this one sits at
~1.5s.

1.5s is not unreasonable with a warm cache, but we can bring that down
to 0.6s on my quad-core laptop by just making all tests parallel. Note
that sub-tests aren't parallel for now, since there are multiple layers
of them and I'm not sure I even follow what's going on.

Mac is also disabled for the time being, since it seems to time out too
often.