GitLab

And re-generate all code in this module.

This gets us to a point where go-codec-dagpb has zero vet warnings, for
example. schema/dmt still has a few warnings, but those are trickier to
fix, so will require another PR.

This reverts commit 6e6625bd.

Discussed at
https://github.com/ipld/go-ipld-prime/pull/126#issuecomment-753003441

Long story short, the motivations of this rename are good,
but the new name also carries some connotations we're really not sure
about, and so we're going to undo this for now, and continue to think
about it in the future.

As discussed on the issue thread, ipld.Kind and schema.TypeKind are more
intuitive, closer to the spec wording, and just generally better in the
long run.

The changes are almost entirely automated via the commands below. Very
minor changes were needed in some of the generators, and then gofmt.

	sed -ri 's/\<Kind\(\)/TypeKind()/g' **/*.go
	git checkout fluent # since it uses reflect.Value.Kind

	sed -ri 's/\<Kind_/TypeKind_/g' **/*.go
	sed -i 's/\<Kind\>/TypeKind/g' **/*.go
	sed -i 's/ReprKind/Kind/g' **/*.go

Plus manually undoing a few renames, as per Eric's review.

Fixes #94.

This should be more intuitive to Go programmers, since assignments are
generally trivial operations, but conversions imply that extra work
might be needed to adapt the value to fit in the recipient.

The entire change is just:

	sed -ri 's/AssignNode/ConvertFrom/g' **/*.go

Downstream users can very likely use the same line to fix their function
declarations and calls.

Fixes #95.

We only supported representing Int nodes as Go's "int" builtin type.
This is fine on 64-bit, but on 32-bit, it limited those node values to
just 32 bits. This is a problem in practice, because it's reasonable to
want more than 32 bits for integers.

Moreover, this meant that IPLD would change behavior if built for a
32-bit platform; it would not be able to decode large integers, for
example, when in fact that was just a software limitation that 64-bit
builds did not have.

To fix this problem, consistently use int64 for AsInt and AssignInt.

A lot more functions are part of this rewrite as well; mainly, those
revolving around collections and iterating. Some might never need more
than 32 bits in practice, but consistency and portability is preferred.
Moreover, many are interfaces, and we want IPLD interfaces to be
flexible, which will be important for ADLs.

Below are some GNU sed lines which can be used to quickly update
function signatures to use int64:

	sed -ri 's/(func.* AsInt.*)\<int\>/\1int64/g' **/*.go
	sed -ri 's/(func.* AssignInt.*)\<int\>/\1int64/g' **/*.go
	sed -ri 's/(func.* Length.*)\<int\>/\1int64/g' **/*.go
	sed -ri 's/(func.* LookupByIndex.*)\<int\>/\1int64/g' **/*.go
	sed -ri 's/(func.* Next.*)\<int\>/\1int64/g' **/*.go
	sed -ri 's/(func.* ValuePrototype.*)\<int\>/\1int64/g' **/*.go

Note that the function bodies, as well as the code that calls said
functions, may need to be manually updated with the integer type change.
That cannot be automated, because it's possible that an automated fix
would silently introduce potential overflows not being handled.

Some TODOs and FIXMEs for overflow checks are removed, since we remove
some now unnecessary int64->int conversions. On the other hand, the
older codecs based on refmt need to gain some overflow check TODOs,
since refmt uses ints. That is okay for now, since we'll phase out refmt
pretty soon.

While at it, update codectools to use int64 for token Length fields, so
that it properly supports full IPLD integers without machine-dependent
behavior and overflow checks. The budget integer is also updated to be
int64, since the lengths it uses are now int64.

Note that this refactor needed changes to the Go code generator as well
as some of the tests, for the purpose of updating all the code.

Finally, note that the code-generated iterator structs do not use int64
fields internally, even though they must return int64 numbers to
implement the interface. This is because they use the numeric fields to
count up to a small finite amount (such as the number of fields in a Go
struct), or up to the length of a map/slice. Neither of them can ever
outgrow "int".

Fixes #124.

(This type isn't really "embedded" in the others directly, so those
docs were somewhat confusing.)

Hopefully this increases clarity and eases comprehension.

Notes and discussion can be found at
https://github.com/ipld/go-ipld-prime/issues/54 (and also
I suppose in some of our weekly video chats, but I'd have
to go on quite a dig to find the relevant links and time).

Many many refernces to 'ns' are also updated to 'np',
making the line count in this diff pretty wild.

See the changelog for discussion; this had already been on the docket
for a while now.

... and it will further become LookupByNode shortly, but that will be
a separate commit.

See the changelog for discussion; this had already been on the docket
for a while now.

It's possible these could be dedup'd... a lot, honestly.  I'm wondering
if they actually need a discrete type per kind at all, anymore; it's
seeming much more tractable to try to turn that into a "no" after all
the other successful work this morning abstracting things.

But I have some other things I want to ship and do demos and prototypes
of that's a lot more interesting than shaving more lines off today.
So just gonna roll with the existing convention and stamp these out.
Might deserve a revisit in the future.  Might not.  Time will tell.

Int is now basically all common stuff.  Now I'm really gonna quit.

Interestingly, the total line count isn't going down very fast.
108 insertions, 165 deletions, according to the git diff stat.
Evidentally the overhead of things opting into this is darn near
the amount of lines saved in many cases.

This one turns out to be different betweens scalars and recursives,
if in a small (and fortunately consistent) way.

Yes, even though in practice in most of codegen we'll just turn around
and title-case it again for symbol export reasons.

The keywords everywhere throughout the schema language are lower-case,
and kinds are such a keyword.  They should therefore be consistently
handled and shown as lowercase.

This has bugged me for a while but it's time to fix it before any more
code starts passing by the area (and it's about to).

These are getting increasingly straightforward as most of the hard
problems have been ironed out already when getting the other recursives
(maps and structs) sorted.

Now, we just need to stamp out the rest of the scalars, and I think
this codegen stuff is at least early-alpha level usable!

Also in this commit: added some new error types and fixed up the
basicnode.List implementation to use it, removing some todos there.

Fix many issues with maybes; adjunct config for if maybe is implemented using a pointer; attempt the finishCallback system for reusable child assemblers.

This... almost appears to be on a dang nice track.

Except one fairly critical thing.  It doesn't work correctly if your
maybe IS implemented as containing a pointer.

Because that pointer starts life as nil in the parent structure.

And that's hard to fix.

We can't have a pointer to a pointer in the assembler;
that'd cause the type bifructation we've been trying to avoid.
(We can give up and do that of course; it would be correct.
Just... more code and larger final assembly size.
And if we did this, a lot of this diff would be rewritten.)

We could have the parent structure allocate a blank of the field,
and put a pointer to it in the maybe and also in the child assembler.
Except... this is a wasted allocation if it turns out to be null.

Rock and hard place detected in a paired configuration.
Might have to back out of this approach entirely.  Not sure.

This cleans up a lot of stuff and reduces the amount of boilerplate
content that's just generating monomorphizing error method stubs.

The nodeGenerator mixins now also use the node mixins.  I don't know
why I failed to do that from the start; I certainly meant to.
It results in shorter generated code, and the compiler turns it into
identical assembly.

In research: I found that there are more low-cost ways to switch which
methods are available to call on a value than I thought.  Also, these
techniques work even for methods of the same name.  This is going to
improve some code in NodeAssemblers significantly -- there are several
situations where this will let us reuse existing pieces of memory
instead of needing to allocate new ones; even the basicnode package
now already needs updates to improve this.  It's also going to make
returning representation nodes from our typed nodes *significantly*
easier and and lower in performance costs.  (Before finding methodsets
are in fact so feasible, I was afraid this was going to require our
typed nodes to embed yet another small struct with a pointer back to
themselves so we can have amortized availability of value that contains
the representation's logic for the Node interface... which while it
certainly would've worked, would've definitely made me sigh deeply.)
Quite exciting for several reasons; only wish I'd noticed this earlier.

Also in research: I found a novel way to make it (I believe) impossible
to create zero values of a type, whilst also making a symbol available
for it in other packages, so that we can do type assertions, etc,
with that symbol.  This is neat.  We're gonna use this to make sure
that types in your schema package can never be created without passing
through any validation logic that the user applies.

In codegen: lots of files disappear.  I'm doing a tabula rasa workflow.
(A bunch of the old files stick around in my working directory, and
are being... "inspirational"... but everything is getting whitelisted
before any of it ports over to the new commits.  This is an effective
way to force myself to do things like naming consistency re-checks
across the board.  And there's *very* little that's getting zero change
since the changes to pointer strategy and assembler interface are so
sweeping, so... there's very little reason *not* to tabula rasa.)

Strings are reimplemented already.  *With* representations.

Most of the codegen interfaces stay roughly the same so far.
I've exported more things this time around.

Lots of "mixins" based on lessons learned in the prior joust.
(Also a bunch of those kind-based rejections look *much* nicer now,
since we also made those standard across the other node packages.)

Some parts of the symbol munging still up in the air a bit.
I think I'm going to go for getting all the infrastructure in place
for allowing symbol-rename adjunct configuration this time.
(I doubt I'll wire it all the way up to real usable configuration yet,
but it'll be nice to get as many of the interventions as possible into
topologically the right places to minimize future effort required.)

There's a HACKME_wip.md file which contains some other notes on
priorities/goals/lessoned-learned-now-being-applied in this rewrite
which may contain some information about what's changing at a higher
level than trying to track the diffs.  (But, caveat: I'm not really
writing it for an audience; more my own tracking.  So, it comes with
no guarantee it will make sense or be useful.)