GitLab

cleanup from #105

An underscore; and less "gen", because reviewers indicated it felt redundant.

I'd still probably prefer to replace this with simply having a stable
order that is carried through consistently, but that remains blocked
behind getting self-hosted types, and while it so happens I also got
about 80% of the way there on those today, the second 80% may take
another day.  Better make this stable rather than wait.

Also, emit some comments around the type definitions.

The old file layout is still available, but renamed to GenerateSplayed.
It will probably be removed in the future.

The new format does not currently have stable output order.
I'd like to preserve the original order given by the schema,
but our current placeholder types for schema data don't have this.
More work needed on this.

So far, the generation itself runs, but the result will not yet compile.
I just want a checkpoint here.  (Most of this was written a few days ago already.)

This is full of mundane plonking away at adding stars and ampersands,
of which approximately none are interesting.

(I'm particularly frustrated by this because these are all placeholder
types, and we're getting *very* close to replacing them, as we get
closer and closer to self-hosting... at which point all of this bonking
about will be made totally irrelevant.  And yet to close the last mile,
these "small" fixes are surprisingly irritating.  Ah well.)

The bits that *are* interesting:

- the Spawn functions for type info now take strings rather than types
(so that they don't provoke a cycle problem for the user when
constructing the information to describe cyclic type info);

- all of the Type info structure hold a pointer to the TypeSystem, and
use that to look up reified Type info for related types, so that their
methods don't force the caller to do that themselves.

(The TypeSystem pointer was already there, amusingly; just never before
initialized, because it hadn't turned out to be load-bearing yet.)

It also would've been possible to just change all the methods on the
Type types to return TypeName rather than full Type info.  That would
avoid the need to use a TypeSystem pointer.  I didn't because:

Overall, this was done in such a way as to minimize the diff that
impacts within the templates.  This was a goal because updating
templates is a fair bit more work than other code due to the weak
compiler support.  And we'll end up reviewing and changing these
methods when we get to our goal of self-hosting generation of the
schema types from the schema-schema, so, it's not worth pushing around
diffs in that same area when they'd be sure to be churned under soon.

Outline for keyed representation, but only enough to get some more
things compilable and start getting all the templates exercised.

Still can't really e2e test the things we've got without getting
the rest of it and its builders finished.  It's coming along pretty
reasonably, though, it seems.

Generates the type structure itself, and a marker interface.
Not much else yet.

Wired test harnesses and plugged out one example.

It's a bummer we don't really have a great way to poke part of it into
running until the whole thing satisfies the generator interface.
But it'll come soon enough.

It's possible these could be dedup'd... a lot, honestly.  I'm wondering
if they actually need a discrete type per kind at all, anymore; it's
seeming much more tractable to try to turn that into a "no" after all
the other successful work this morning abstracting things.

But I have some other things I want to ship and do demos and prototypes
of that's a lot more interesting than shaving more lines off today.
So just gonna roll with the existing convention and stamp these out.
Might deserve a revisit in the future.  Might not.  Time will tell.

These are getting increasingly straightforward as most of the hard
problems have been ironed out already when getting the other recursives
(maps and structs) sorted.

Now, we just need to stamp out the rest of the scalars, and I think
this codegen stuff is at least early-alpha level usable!

Also in this commit: added some new error types and fixed up the
basicnode.List implementation to use it, removing some todos there.

Fixes many issues with maybes.  Some minor typo-scale stuff; some
pretty consequential.  What to do about speciated lookup methods
still has big honking questions on it: returning a maybe there if
you didn't already have data in that format can be *not cheap*.
Figuring out the best DX there is gonna be interesting.

Making progress on how to handle keys, but slow.  Complex keys are...
well, really complex.  So far I've learned several things that *won't*
work, which is fun.  It's looking unlikely that having a KeyAssembler
type at all is actually going to be a good idea.

This diff is pretty fun.  It's our first alternative representation,
and it all works out nicely (no internal syntactical complications
encountered or any other unexpected unpleasantness).

It's also our first representation that involves potentially recursive
destructuring work over a scalar!  So, you can see the new 'construct'
method conventions appearing to handle that.  Works out neatly.
This kind of recursive destructuring happens all within the span of
processing one "token" so to speak, so naturally it can work without
any stateful machinery.

Some utility functions are added to the mixins package.
(It's sort of betraying in the name of the package, but, well,
it's an extremely expedient choice.  See comments about import
management.  tl;dr: consistency makes things easier.)

Tests for a recursive case of this will be coming soon, but requires
addressing some other design flaws with the AdjunctConfig maps.
(StructField is... not a good key.  It's not always hashable... such
as when the StructField.Type field is inhabited by TypeStruct.)
Easy enough to fix, just not going to cram it into this commit.

The way null and the 'fcb' are handled here differs from previous
generators: here, we *always* have an 'fcb', and just make a special
one for the root builder.  This is... well, it works.  It was an
attempt to simplify things and generate fewer method overrides, and
it did.  However, I'm not going to dwell on this too long, because...

The 'fcb' system is not working out well overall for other reasons:
it's causing costly allocations.  (Taking a function pointer from
a method requires at least two words: one for the function pointer
and one for the value to apply it on.  That means an allocation.)
So a serious rewrite of anything involing the 'fcb' strategy is needed.

And that is going to be a little tricky.  The 'fcb' idea was itself
a trying-slightly-too-hard-to-be-clever attempt to avoid an alternative
solution that involves generating additional types per distinct child
value slot (and I'm still unwilling to pursue that one -- it suggests
far too many types).  The next best idea I have now seems to involve a
lot of pointers into other assembler's state machines; this will surely
be a bit touchy.  But no worse than any of the rest of all this, I
suppose: it's *all* "a bit touchy".  Buckle up for fun diffs ahead.)

So, considering these 'fcb' notes, this feels a bit mixed...
"Two steps forward, one step back", so to speak.  Still: progress!
And it's very exciting that we got through several new categories of
behavior without hitting any other new roadbumps.

I'll want this momentarily for doing Generate from other packages.
(I might not actually commit anything relating to this for a while,
but secretly, I'm playing with it.)

Correct a half-dozen laxnesses in path handling while at it.