GitLab

Outline of Traverse* funcs; comment about link loaders in TraversalConfig; confession of working draft of Selector interface; fixed typo in AdvVisitFn return types.

Yes, that's lot of things heaped in one commit.

Yes, (unfortunately,) they're all related.  Almost the entirety of this
has to manifest *at once*, all seamlessly connected, for any part of
it to get off the ground.

Note how very much is handwaved in the comments about TraversalConfig
having linkloader tools of some kind.  There's a lot to unpack there.
Getting it to work well, and let users of the library supply functions
that customize the parts that are interesting to the user, while *not*
getting them hamstrung by a bunch of details about multicodecs and
multihashing (unless they want to!)... requires careful design work.
Signed-off-by: Eric Myhre <hash@exultant.us>

All traversal/transform/selector/path code to date got a rethink.

For the longest time, I was trying to keep all the traversal code in
the main root IPLD package, thinking/hoping that the number of relevant
functions would be minimal enough that this would be fine.

Somewhere around the time where I conceded that yes, we *do* need
separate methods for writable and readonly paths (it's true that
readonly is a degenerate case of writable -- but they have different
performance characteristics!), I gave up: the number of functions in
play is too high to heap alltogether in the root package namespace.

This leads to a number of other surprising results: namely, that
ipld.Path *isn't* -- it's actually traversal.Path!  We *don't use*
paths anywhere else except as guidance to some forms of traversal,
and logs and progress markers during traversal.  Huh!
Despite being initially surprising, I now regard this as a *good*
code smell.

ipld.Traversal as an interface and ipld.Path.Traverse are dropped.
These turned out to be the wrong level of abstraction:
it's both missing things, and not able to return enough things.
By the time we would fix both, it's not an abstraction anymore.

Path.Traverse didn't know how to use a link loader func -- and
that's correct; it shouldn't -- and it also didn't retain and
return a stack of Nodes it traversed -- which again, it shouldn't,
because in general that would be useless overkill and resource waste...
it's just that both these things are essential in some use cases.
So!  Cut that gordian knot.

The Focus/FocusTransform/Traverse/TraverseTransform methods will now
do all recursion work themselves, internally.  The transform family
will keep stacks of nodes encountered along the way, to aid in the
(presumably) coming tree re-build; the focus family won't.
All of these methods will be able to support link loading, though
implementation of that is still upcoming.

There's a few more fields in TraversalProgress, also preparing for when
we have automatic link loading and traversal: we'll want to be able
to inform the user's callbacks about load edges.  (I'm not a fan of the
term 'block' here, but running with it for the moment.)

TraversalConfig is expected to grow.  Namely, to include link
loaders... and subsequently, probably block writers, as well!  This is
a big subject, though.  It'll probably take several commits and
possibly more than one iteration over time to hammer out a good API.

One thing that's interesting to note that neither TraversalConfig nor
TraversalProgress will grow to contain certain things: for example,
a map of "seen" CIDs when doing link traversal.  Why?  Well, if you've
been mentally modelling things as a graph, this is surprising; surely
any graph traversal needs to remember a set of already-visited nodes.
The trick is... we're *not* doing a graph traversal -- not exactly!
Since our visitor gets a (Node,Path) *tuple*... it would be wrong to
memoize on anything other than the complete tuple; and the Path part
of the tuple means we're actually doing a *tree* walk rather than
a graph walk.  (And of course, since we're operating on DAGs,
termination guarantees are already a non-issue.)  Tada; no "seen" set.
Signed-off-by: Eric Myhre <hash@exultant.us>