- 22 Aug, 2021 1 commit
-
-
tavit ohanian authored
-
- 16 Aug, 2021 1 commit
-
-
tavit ohanian authored
-
- 29 Jul, 2021 1 commit
-
-
tavit ohanian authored
-
- 25 May, 2021 1 commit
-
-
Eric Myhre authored
Per https://github.com/ipld/go-ipld-prime/issues/175 and some discussions today. Also touched up the docs for Next vs Done a bit while I was in the area; they're not significantly changed, but there were some dangling antecedents that didn't read very well, and now it's a bit better.
-
- 25 Jan, 2021 1 commit
-
-
Daniel Martí authored
As a method superset of Node, with the Substrate method added on top. This way, we can generally say that any ADL implementation should include this method to allow for easy encoding over the wire. Reify, the opposite of Substrate, is notably missing here. This is because it's generally a top-level function, not a method. We might want to rethink this disparity in the future.
-
- 25 Dec, 2020 1 commit
-
-
Daniel Martí authored
As discussed on the issue thread, ipld.Kind and schema.TypeKind are more intuitive, closer to the spec wording, and just generally better in the long run. The changes are almost entirely automated via the commands below. Very minor changes were needed in some of the generators, and then gofmt. sed -ri 's/\<Kind\(\)/TypeKind()/g' **/*.go git checkout fluent # since it uses reflect.Value.Kind sed -ri 's/\<Kind_/TypeKind_/g' **/*.go sed -i 's/\<Kind\>/TypeKind/g' **/*.go sed -i 's/ReprKind/Kind/g' **/*.go Plus manually undoing a few renames, as per Eric's review. Fixes #94.
-
- 16 Dec, 2020 1 commit
-
-
Daniel Martí authored
We only supported representing Int nodes as Go's "int" builtin type. This is fine on 64-bit, but on 32-bit, it limited those node values to just 32 bits. This is a problem in practice, because it's reasonable to want more than 32 bits for integers. Moreover, this meant that IPLD would change behavior if built for a 32-bit platform; it would not be able to decode large integers, for example, when in fact that was just a software limitation that 64-bit builds did not have. To fix this problem, consistently use int64 for AsInt and AssignInt. A lot more functions are part of this rewrite as well; mainly, those revolving around collections and iterating. Some might never need more than 32 bits in practice, but consistency and portability is preferred. Moreover, many are interfaces, and we want IPLD interfaces to be flexible, which will be important for ADLs. Below are some GNU sed lines which can be used to quickly update function signatures to use int64: sed -ri 's/(func.* AsInt.*)\<int\>/\1int64/g' **/*.go sed -ri 's/(func.* AssignInt.*)\<int\>/\1int64/g' **/*.go sed -ri 's/(func.* Length.*)\<int\>/\1int64/g' **/*.go sed -ri 's/(func.* LookupByIndex.*)\<int\>/\1int64/g' **/*.go sed -ri 's/(func.* Next.*)\<int\>/\1int64/g' **/*.go sed -ri 's/(func.* ValuePrototype.*)\<int\>/\1int64/g' **/*.go Note that the function bodies, as well as the code that calls said functions, may need to be manually updated with the integer type change. That cannot be automated, because it's possible that an automated fix would silently introduce potential overflows not being handled. Some TODOs and FIXMEs for overflow checks are removed, since we remove some now unnecessary int64->int conversions. On the other hand, the older codecs based on refmt need to gain some overflow check TODOs, since refmt uses ints. That is okay for now, since we'll phase out refmt pretty soon. While at it, update codectools to use int64 for token Length fields, so that it properly supports full IPLD integers without machine-dependent behavior and overflow checks. The budget integer is also updated to be int64, since the lengths it uses are now int64. Note that this refactor needed changes to the Go code generator as well as some of the tests, for the purpose of updating all the code. Finally, note that the code-generated iterator structs do not use int64 fields internally, even though they must return int64 numbers to implement the interface. This is because they use the numeric fields to count up to a small finite amount (such as the number of fields in a Go struct), or up to the length of a map/slice. Neither of them can ever outgrow "int". Fixes #124.
-
- 29 Jun, 2020 2 commits
-
-
Eric Myhre authored
-
Eric Myhre authored
Hopefully this increases clarity and eases comprehension. Notes and discussion can be found at https://github.com/ipld/go-ipld-prime/issues/54 (and also I suppose in some of our weekly video chats, but I'd have to go on quite a dig to find the relevant links and time). Many many refernces to 'ns' are also updated to 'np', making the line count in this diff pretty wild.
-
- 26 Jun, 2020 2 commits
-
-
Eric Myhre authored
See the changelog for discussion; this had already been on the docket for a while now.
-
Eric Myhre authored
... and it will further become LookupByNode shortly, but that will be a separate commit. See the changelog for discussion; this had already been on the docket for a while now.
-
- 03 Jun, 2020 1 commit
-
-
Eric Myhre authored
-
- 02 Mar, 2020 1 commit
-
-
Eric Myhre authored
This is a *lot* of changes. It's the most significant change to date, both in semantics and in character count, since the start of this repo. It changes the most central interfaces, and significantly so. But all tests pass. And all benchmarks are *improved*. The Node interface (the reading side) is mostly unchanged -- a lot of consuming code will still compile and work just fine without changes -- but any other Node implementations out there might need some updating. The NodeBuilder interface (the writing side) is *extremely* changed -- any implementations out there will *definitely* need change -- and most consumers will too. It's unavoidable with a semantic fix this big. The performance improvements should make it worth your while, though. If you want more background on how and why we got here, you've got quite a few commits on the "research-admissions" branches to catch up on reading. But here's a rundown of the changes: (Get a glass of water or something calming before reading...) === NodeAssembler introduced! === NodeAssembler is a new interface that describes most of the work of creating and filling data into a new Node. The NodeBuilder interface is still around, but changed in role. A NodeBuilder is now always also a NodeAssembler; additionally, it can return the final Node to you. A NodeAssembler, unlike NodeBuilder, can **not** return a Node to you. In this way, a NodeBuilder represents the ability to allocate memory. A NodeAssembler often *does not*: it's just *filling in* memory. This design overall is much more friendly to efficient operations: in this model, we do allocations in bulk when a NodeBuilder is used, and then NodeAssemblers are used thereafter to fill it in -- this mental model is very friendly to amortizing memory allocations. Previously, the NodeBuilder interface made such a pattern of use somewhere between difficult and outright impossible, because it was modeled around building small values, then creating a bigger value and inserting the smaller ones into it. This is the key change that cascaded into producing the entire other set of changes which land in this commit. The NodeBuilder methods for getting "child builders" are also gone as a result of these changes. The result feels a lot smoother. (You can still ask for the NodeStyle for children of a recursive kind! But you'll find that even though it's possible, it's rarely necessary.) We see some direct improvements from this interface change already. We'll see even more in the future: creating values when using codegen'd implementations of Node was hugely encumbered by the old NodeBuilder model; NodeAssembler *radically* raises the possible ceiling for performance of codegen Node implementations. === NodeStyle introduced === NodeStyle is a new interface type that is used to carry information about concrete node implementations. You can always use a NodeStyle to get a NodeBuilder. NodeStyle may also have additional features on it which can be detected by interface checks. (This isn't heavily used yet, but we imagine it might become handy in the future.) NodeStyle replaces NodeBuilder in many function arguments, because often what we wanted was to communicate a selection of Node implementation strategy, but not actually the start of construction; the NodeStyle interface now allows us to *say that*. NodeStyle typically cost nothing to pass around, whereas a NodeBuilder generally requires an allocation to create and initialize. This means we can use NodeStyle more freely in many contexts. === node package paths changed === Node implementations are now in packages under the "node/*" directory. Previously, they were under an "impl/*" directory. The "impl/free" package is replaced by the the "node/basic" package! The package name was "ipldfree"; it's now "basicnode". === basicnode is an improved runtime/anycontent Node implementation === The `basicnode` package works much the same as the `ipldfree` package used to -- you can store any kind of data in it, and it just does as best it can to represent and handle that, and it works without any kind of type info nor needs of compile-time special support, etc -- while being just quietly *better at it*. The resident memory size of most things has gone down. (We're not using "fat unions" in the implementation anymore.) The cost of iterating maps has gone down *dramatically*. Iteration previously suffered from O(n) allocations due to expensive `runtime.conv*` calls when yielding keys. Iteration is now O(1) (!!) because we redesigned `basicnode` internals to use "internal pointers" more heavily, and this avoids the costs from `runtime.conv*`. (We could've done this separately from the NodeAssembler change, admittedly. But both are the product of research into how impactful clever use of "internal pointers" can be, and lots of code in the neighborhood had to be rewritten for the NodeAssembler interface, so, these diffs arrive as one.) Error messages are more informative. Many small operations should get a few nanoseconds faster. (The implementation uses more concrete types and fewer switch statements. The difference probably isn't the most noticeable part of all these changes, but it's there.) --- basicnode constructor helpers do all return pointers --- All the "New*" helper functions in the basicnode package return interfaces which are filled by a pointer now. This is change from how they worked previously when they were first implemented in the "rsrch" package. The experience of integrating basicnode with the tests in the traversal package made it clear that having a mixture of pointer and non-pointer values flying around will be irritating in practice. And since it is the case that when returning values from inside a larger structure, we *must* end up returning a pointer, pointers are thus what we standardize on. (There was even some writeup in the HACKME file about how we *might* encounter issues on this, and need to change to pointers-everywhere -- the "pointer-vs-value inhabitant consistency" heading. Yep: we did. And since this detail is now resolved, that doc section is dropped.) This doesn't really make any difference to performance. The old way would cause an alloc in those method via 'conv*' methods; the new way just makes it more explicit and go through a different runtime method at the bottom, but it's still the same number of allocations for essentially the same reasons. (I do wonder if at some future point, the golang compiler might get cleverer about eliding 'conv*' calls, and then this change we make here might be unfortunate; but that's certainly not true today, nor in the future at any proximity that I can foresee.) === iterator getters return nil for wrong-kind === The Node.MapIterator and Node.ListIterator methods now return nil if you call them on non-maps or non-lists. Previously, they would return an iterator, but using it would just constantly error. I don't think anyone was honestly really checking those error thunks, and they made a lot of boilerplate white noise in the implementations, and the error is still entirely avoidable by checking the node kind up-front (and this is strictly preferable anyway, since it's faster than getting an error thunk, poking it to get the error, etc)... so, in total, there seem like very few reasons these were useful: the idea is thus dropped. Docs in the Node interface reflect this. === node/mixins makes new Node implementations easier === The mixins package isn't usable directly, but if you're going to make a new Node implementation, it should save you a lot of typing... and also, boost consistency of basic error handling. Codegen will look forward to using this. (Codegen already had much of these semantics internally, and so this package is sort of lifting that back out to be more generally usable. By making it live out here as exported symbols in the core library, we should also reduce the sheer character count of codegen output.) === 'typed.Node' is now 'schema.TypedNode' === A bunch of interfaces that were under the "impl/typed" path moved to be in the "schema" package instead. This probably makes sense to you if you look at them and needs no further explanation. (The reason it comes in this diff, though, is that it was forced: adding better tests to the traversal package highlighted a bunch of cyclic dependency issues that came from 'typed.Node' being in a package that had concrete use of 'basicnode'.) === codecs === The 'encoding' package is now named 'codec'. This name is shorter; it's more in line with vocabulary we use elsewhere in the IPLD project (whereas 'encoding' was more of a nod to the naming found in the golang standard library); and in my personal opinion it does better at describing the both directions of the process (whereas 'encoding' sounds like only the to-linear-bytes direction). I just like it better. === unmarshal functions no longer return node === Unmarshal functions accept an NodeAssembler parameter (rather than a NodeBuilder, as before, nor a NodeStyle, which might also make sense in the new family of interfaces). This means they no longer need to return a Node, either -- the caller can decide where the unmarshalled data lands. If the caller is using a NodeBuilder, it means they can call Build on that to get the value. (If it's a codegen NodeBuilder with More Information, the caller can use any specialized functions to get the more informative pointers without need for casting!) Broadly speaking, this means users of unmarshal functions have more control over how memory allocation comes into play. We may want to add more helper functions to the various codec packages which take a NodeStyle argument and do return a Node. That's not in this diff, though. (Need to decide what pattern of naming these various APIs would deserve, among other things.) === the fluent package === The fluent package changed significantly. The readonly/Node side of it is dropped. It didn't seem to get a ton of exercise in practice; the 'traversal' package (and in the future, perhaps also a 'cursor' package) addresses a lot of the same needs, and what remains is also covered well these days by the 'must' package; and the performance cost of fluent node wrappers as well as the composability obstruction of them... is just too much to be worth it. The few things that used fluent.Node for reading data now mostly use the 'must' package instead (and look better for it, imo). It's possible that some sort of fluent.Node will be rebuilt someday, but it's not entirely clear to me what it should look like, and indeed whether or not it's a good idea to have in the repo at all if the performance of it is counterindicated in a majority of situations... so, it's not part of today's update. The writing/NodeBuilder/NodeAssembler fluent wrappers are continued. It's similar to before (panics promptly on errors, and has a lot of closures)... but also reflects all of the changes made in the migration towards NodeAssembler: it doesn't return intermediate nodes, and there's much less kerfuffle with getting child builders. Overall, the fluent builders are now even more streamlined than before; the closures need even fewer parameters; great success! The fluent.NodeAssembler interface retains the "Create" terminology around maps and lists, even though in the core interfaces, the ipld.NodeAssembler interface now says "Begin" for maps and lists. This is because the fluent.NodeAssembler approach, with its use of closures, really does do the whole operation in one swoop. (It's amusing to note that this change includes finally nuking some fairly old "REVIEW" comment blocks from the old fluent package which regarded the "knb" value and other such sadness around typed recursion. Indeed, we've finally reviewed that: and the answer was indeed to do something drastically different to make those recursions dance well.) === selectors === Selectors essentially didn't change as part of this diff. Neat. (They should get a lot faster when applied, because our node implementations hit a lot less interface boxing in common operations! But the selector code itself didn't need to change to get the gains.) The 'selector/builder' helper package *did* change a bit. The changes are mostly invisible to the user. I do have some questions about the performance of the result; I've got a sneaking suspicion there's now a bunch of improvements that might be easier to get to now than they would've been previously. But, this is not my quest today. Perhaps it will deserve some review in the future. The 'selector/builder' package should be noted as having some interesting error handling strategies. Namely, it doesn't. Any panics raised by the fluent package will just keep rising; there's no place where they're converted to regular error value returns. I'm not sure this is a good interface, but it's the way it was before I started passing through, so that's the way it stays after this patch. ExploreFieldsSpecBuilder.Delete disappears. I hope no one misses it. I don't think anyone will. I suspect it was there only because the ipld.MapBuilder interface had such a method and it seemed like a reasonable conservative choice at the time to proxy it; now that the method proxied is gone, though, so too shall go this. === traversal === Traversal is mostly the same, but a few pieces of config have new names. `traversal.Config.LinkNodeBuilderChooser` is now `traversal.Config.LinkTargetNodeStyleChooser`. Still a mouthful; slightly more accurate; and reflects that it now works in terms of NodeStyle, which gives us a little more finesse in reasoning about where NodeBuilders are actually created, and thus better control and insight into where allocations happen. `traversal.NodeBuilderChooser` is now `traversal.LinkTargetNodeStyleChooser` for the same reasons. The actual type of the `LinkTargetNodeStyleChooser` now requires returning a `NodeStyle`, in case all the naming hasn't made it obvious. === disappearing node packages === A couple of packages under 'impl/*' are just dropped. This is no real loss. The packages dropped were Node implementations that simply weren't done. Deleting them is an increase in honesty. This doesn't mean something with the same intentions as those packages won't come back; it's just not today. --- runtime typed node wrapper disappeared --- This one will come back. It was just too much of a pain to carry along in this diff. Since it was also a fairly unfinished proof-of-concept with no downstream users, it's easier to drop and later reincarnate it than it is to carry it along now. === linking === Link.Load now takes a `NodeAssembler` parameter instead of a `NodeBuilder`, and no longer returns a `Node`! This should result in callers having a little more control over where allocations may occur, letting them potentially reuse builders, etc. This change should also make sense considering how codec.Unmarshal now similarly takes a NodeAssembler argument and does not return a Node value since its understood that the caller has some way to access or gather the effects, and it's none of our business. Something about the Link interface still feels a bit contorted. Having to give the Load method a Loader that takes half the same arguments all over again is definitely odd. And it's tempting to take a peek at fixing this, since the method is getting a signature change. It's unclear what exactly to do about this, though, and probably a consequential design decision space... so it shall not be reopened today during this other large refactor. Maybe soon. Maybe. === the dag-json codec === The dag-json codec got harder to implement. Rrgh. Since we can't tell if something is going to become a Link until *several tokens in*, dag-json is always a bit annoying to deal with. Previously, however, dag-json could still start optimistically building a map node, and then just... quietly drop it if we turn out to be dealing with a link instead. *That's no longer possible*: the process of using NodeAssembler doesn't have a general purpose mechanism for backtracking. So. Now the dag-json codec has to do even more custom work to buffer tokens until it knows what to do with them. Yey. The upside is: of course, the result is actually faster, and does fewer memory allocations, since it gathers enough information to decide what it's doing before it begins to do it. (This is a lovely example of the disciplined design of NodeAssembler's interface forcing other code to be better behaved and disciplined!) === traversal is faster === The `BenchmarkSpec_Walk_MapNStrMap3StrInt/n=32` test has about doubled in speed on the new `basicnode` implementation in comparison to the old `ipldfree.Node` implementation. This is derived primarily from the drop in costs of iteration on `basicnode` compared to the old `ipldfree.Node` implementation. Some back-of-the-envelope math on the allocation still left around suggest it could double in speed again. The next thing to target would be allocations of paths, followed by iterator allocations. Both are a tad trickier, though (see a recently merge-ignore'd commit for notes on iterators; and paths... paths will be a doozy because the path forward almost certainly involves path values becoming invalid if retained beyond a scope, which is... unsafe), so certainly need their own efforts and separate commits. === marshalling is faster === Marshalling is much faster on the new `basicnode` implementation in comparison to the old `ipldfree.Node` implementation. Same reasons as traversal. Some fixes to marshalling which previously caused unnecessary allocations of token objects during recursions have also been made. These improve speed a bit (though it's not nearly as noticeable as the boost provided by the Node implementation improvements to iteration). === size hints showed up all over the place === The appearance of size hint arguments to assembly of maps and lists is of course inevitable from the new NodeAssembler interface. It's particularly interesting to see how many of them showed up in the selector and selectorbuilder packages as constants. And super especially interesting how many of them are very small constants. 44 zeros. 86 ones. 25 twos. 9 threes. 2 fours. (Counted via variations of `grep -r 'Map(.*4, func' | wc -l`.) It's quite a distribution, neh? We should probably consider some more optimizations specifically targeted to small maps. (This is an unscientific sample, and shifted by what we chose to focus on in testing, etc etc, but the general point stands.) `-1` is used to indicate "no idea" for size. There's a small fix to the basicnode implementations to allow this. A zero would work just as well in practice, but using a negative number as a hint to the human seems potentially useful. It's a shame we can't make the argument optional; oh well. === codegen === The codegen packages still all compile... but do nonsensical things, for the moment: they've not been updated to emit NodeAssembler. Since the output of codegen still isn't well rigged to test harnesses, this breakage is silent. The codegen packages will probably undergo a fairly tabula-rasa sweep in the near future. There's been a lot of lessons learned since the start of the code currently there. Updating to emit the NodeAssembler interface will be such a large endeavor it probably represents a good point to just do a fresh pass on the whole thing all at once. -------- ... and that's all! Fun reading, eh? Please do forgive the refactors necessary for all this. Truly, the performance improvements should make it all worth your while.
-
- 27 Feb, 2020 1 commit
-
-
Eric Myhre authored
Previously it was in the 'impl/typed' package, next to the runtime-wrapper implementation of the interface. This was strange. Not only should those two things be separated just on principle, this was also causing more import cycle problems down the road: for example, the traversal package needs to consider the *interface* for a schema-typed node in order to gracefully handle some features... and if this also brings in a *concrete* dependency on the runtime-wrapper implementation of typed nodes, not only is that incorrect bloat, it becomes a show stopper because (currently, at least) that implementation also in turn transitively imports the ipldfree package for some of its scalars. Ouchouch. So. Now the interface lives over in the 'schema' package, with all the other interfaces for that feature set. Where it probably always should have been. ('typed.Maybe' also became known as 'schema.Maybe', which... does not roll off the tongue as nicely. But this is a minor concern and we might reconsider the naming and appearance of that thing later anyway.)
-
- 03 Dec, 2019 1 commit
-
-
Eric Myhre authored
The readme hadn't seen an update for quite a long time, so it was just overdue for it. I think the comments about where to expect changes at this point are practically accurate and now are also stated. The Node interface, despite being so central to the entire system, was missing a doc block on the type as a whole! It now has one. (I'd like to continue to iterate on this and have it do more linking out to ipld docs for some of the more conceptual parts, but we've got a lot of work to do on that content as well, so, for now just doing it here is better than not having it.) Signed-off-by: Eric Myhre <hash@exultant.us>
-
- 30 Aug, 2019 2 commits
-
-
Eric Myhre authored
Fix traversal internals to use it (!) rather than converting segments to strings, which was both wasteful, and in some cases, *wrong* (!) (although by coincidence happened to mostly work at present because of another thing from early-days code that was also technically wrong). Fix ipldfree.Node to reject LookupString if used on a list node! (This is the other "wrong" thing that made the traversal coincidentally work.) LookupSegment method generation also added to codegen. Signed-off-by: Eric Myhre <hash@exultant.us>
-
Eric Myhre authored
ipld.Path is now a slice of ipld.PathSegment instead of strings. This should be an improvement in sanity: there are now several fewer places importing "strconv", and that's just always a good thing. We will also be free in the future to add PathSegment-based accessor methods to ipld.Node, as has already been commented as a desire; and, to use PathSegment in building better typed errors (which is the specific thing that provokes this diff today and now). The implementation of PathSegment is now also based on a struct-style union rather than an interface style union. There are comments about this in the diff. I do not wish to comment on how much time I've spent looking at golang assembler and runtime internals while trying to find a path to a more perfect compromise between ergonomics and performance. tl;dr Selectors will probably get faster and trigger fewer allocations; ipld.Path will probably take slightly more memory (another word per path segment), but not enough to care about for any practical purposes. I did not attempt to hoist the SegmentIterator features from the selector package to anywhere more central. It may be a fine idea to do so someday; I just don't presently have a formed opinion and am not budgeting time to consider it today. Signed-off-by: Eric Myhre <hash@exultant.us>
-
- 12 Aug, 2019 3 commits
-
-
Eric Myhre authored
Signed-off-by: Eric Myhre <hash@exultant.us>
-
Eric Myhre authored
Most important things first! To follow this refactor: ``` sed s/TraverseField/LookupString/g sed s/TraverseIndex/LookupIndex/g ``` It is *literally* a sed-refactor in complexity. --- Now, details. This has been pending for a while, and there is some discussion in https://github.com/ipld/go-ipld-prime/issues/22 . In short, "Traversal" seemed like a mouthful; "Field" was always a misnomer here; and we've discovered several other methods that we *should* have in the area as well, which necessitated a thought about placement. In this commit, only the renames are applied, but you can also see the outlines of two new methods in the Node interface, as comments. These will be added in future commits. Signed-off-by: Eric Myhre <hash@exultant.us>
-
Eric Myhre authored
These have been just slightly out of style for a while now. (In particular, allusions to numeric-keyed maps are removed. While these are still considered a potential "someday" subject rather than fully ruled out, it's also certainly not a *present* feature; until something changes on this front, better not to advertise it.) Signed-off-by: Eric Myhre <hash@exultant.us>
-
- 05 Aug, 2019 2 commits
-
-
Eric Myhre authored
Mind, I don't know if all the details of this have been worked out. In particular, where to get the LinkStorer and other fun stuff passed through this interface might be a fun dance (if, indeed, such a thing ends up needed -- this hasn't been written yet!). We'll see. One thing is clear: we want this contract here for all the same already-known reasons: a traversal.Transform over a structure using an ADL should "just work". Signed-off-by: Eric Myhre <hash@exultant.us>
-
Eric Myhre authored
Prompted in part by https://github.com/ipld/go-ipld-prime/pull/21#issuecomment-518115245 . Turns out some of the stuff our earliest Node implementations do are actually pretty unusual. Many other features have more strictures. Therefore, it's important to specify the behavior such that the features that have to operate within more restrictions can still comply, and it's clear that anything else shouldn't be expected! (Perhaps we should just... make the ipldfree.Node implementations not do the flexible things they do right now. That package is starting to look in need of some renovation anyway. We'll see.) Signed-off-by: Eric Myhre <hash@exultant.us>
-
- 20 Jul, 2019 1 commit
-
-
Eric Myhre authored
The comment in the ipld.Node type about IsUndefined has been there quite a while; today we're finally using it. With this set of changes, the generated code for structs now compiles successfully again: the Undef and Null thunks are both used there.
-
- 25 Jun, 2019 1 commit
-
-
Eric Myhre authored
Noticed while reviewing the parent commit, so double thanks for that one...!
-
- 17 Jun, 2019 1 commit
-
-
Matthias Beyer authored
The iteration can stop if the `Done()` method returns true, not false.
-
- 21 Mar, 2019 1 commit
-
-
Eric Myhre authored
We now have both MapIterator and ListIterator interfaces. Both return key-value (or index-value) pairs, rather than just keys. List iterators may seem a tad redundant: you just loop over the length, right? Well, sure. But there's one place a list iterator shines: selecting only a subset of elements. And indeed, we'll be doing exactly that in the traversal/selector package; therefore, we definitely need list iterators. We might want keys-only iterators again in the future, but at present, I'm deferring that. It's definitely true that we should have iterators returning values as a core feature, since they're likely to be more efficiently supportable than "random" access (especially when we get to some Advanced Layout data systems), so we'll implement those first. Additionally, note that MapIterator now returns a Node for the key. This is to account for that fact that when using the schema system and typed nodes, map keys can be more *specific* types. Such nodes are still required to be kind==ReprKind_String, but string might not be their *preferred* native format (think: tuples with serialized to be delimiter-separated strings); we need to account for that. (MapBuilder.Insert method already takes a Node parameter for similar reasons: so it can take *typed* nodes. Node.TraverseField accepting a plain string is the oddball out here, and should be rectified.) Signed-off-by: Eric Myhre <hash@exultant.us>
-
- 19 Mar, 2019 1 commit
-
-
Eric Myhre authored
Having a function called "Kind" return a "ReprKind" was inconsistent. Also, we want to introduce a "Kind" method on `typed.Node` in the future. No logical content to this change: you can safely refactor with sed. Signed-off-by: Eric Myhre <hash@exultant.us>
-
- 16 Mar, 2019 3 commits
-
-
Eric Myhre authored
(... offically. Lots of docs have probably already been stating that this is there. Now it actually... is.) Signed-off-by: Eric Myhre <hash@exultant.us>
-
Eric Myhre authored
This has been deprecated and replaced by the NodeBuilder system for a good while now; time to scrape it into the dustbin completely. Tests that were primarily on the mutable node system itself also drop, so, this is a *very* large delete diff. A few other tests used MutableNode just incidentally, and those are quick fixed to use NodeBuilder. Signed-off-by: Eric Myhre <hash@exultant.us>
-
Eric Myhre authored
As detailed in comments a few commits ago, this is part of a big, big roll towards keeping linking details far enough off to one side that one can actually use most of the IPLD system without forming an explicit compile-time dependency on any linking features (until, of course, one uses the linking features). This is a surprisingly small diff, because... well, because most of the *interesting* features around linking simply weren't implemented yet, and at this point everything that is has already been isolated in the new cidlink and related encoding packages. "CID" was *already* just a semantic placeholder that meant "eh, link". Signed-off-by: Eric Myhre <hash@exultant.us>
-
- 05 Feb, 2019 2 commits
-
-
Eric Myhre authored
An "immediate" rather than generative function is also available; the docs contain caveats. At the moment, these distinctions are a bit forced, but we want to be ready for when we start getting truly huge maps where the generative usage actually *matters* for either memory reasons, latency reasons, or both. Separated Length rather than trying to pull double-duty the Keys method; the previous combination was just utterly silly. The implementations in the bind package are stubs; that package is going to take a lot of work, and so the focus is going to be entirely on keeping the 'free' package viable; and we'll revisit bind and such when there's more dev time budget. Signed-off-by: Eric Myhre <hash@exultant.us>
-
Eric Myhre authored
Signed-off-by: Eric Myhre <hash@exultant.us>
-
- 15 Jan, 2019 1 commit
-
-
Eric Myhre authored
It should return an error explicitly. A pair of "(nil, nil)" reponses would also be an unambiguous option for representing an absent member, but seems more likely to create pitfalls than save a significant amount of code for callers. Signed-off-by: Eric Myhre <hash@exultant.us>
-
- 10 Jan, 2019 1 commit
-
-
Eric Myhre authored
You could now readily rig up the ipldfree.Node implementation to a refmt cbor.TokenSink, for example, and go to town. (At the moment, doing so is left as an exercise to the reader. We'll make that a smooth built-in/one-word function at some point, but I'm not yet sure exactly how that should look, so it's deferred for now.) While working on this, a lot of other things are cooking on simmer: I'm churning repeatedly over a lot of thoughts about A) API semantics in general, B) how to cache CIDs of nodes, and C) how to memoize serializations / reduce memcopies during partial tree updates... And (unsurprisingly) I keep coming back to the conclusion that the API for dang near everything should be immutable at heart in order to keep things sane. The problem is figuring out how to pursue this A) efficiently, B) in tandem with reasonably low-friction nativeness (i.e. I want autocompletion in a standard golang editor to be as useful as possible!), and C) given an (as yet) lack of good builder or mutation-applier patterns. ipldbind was meant to be a solution to the majority of the B and C issues there, but that rubs smack against the grain of "let's be immutable" in golang >:/ So... a rock and a hard place, in short. Signed-off-by: Eric Myhre <hash@exultant.us>
-
- 08 Jan, 2019 2 commits
-
-
Eric Myhre authored
ipldfree and ipldbound and fluent and all the interfaces now have all the isNull methods, asBytes, setBytes, etc. (Previously this was a little swiss-cheesy.) SMOP. Signed-off-by: Eric Myhre <hash@exultant.us>
-
Eric Myhre authored
Fairly sure this is going to come up in our handling of typed nodes (and may also be relevant to some of the REVIEW-todo comments about sparseness of arrays; haven't yet fully considered). Signed-off-by: Eric Myhre <hash@exultant.us>
-
- 08 Dec, 2018 1 commit
-
-
Eric Myhre authored
This will be for the active path -- if we also follow through on the idea of having a just-in-time checked Node wrapper, a lot of these checks might end up duplicated. We'll DRY that up when we get there. Doing accumulation of errors. Might get loud. We can come back and and early-halt parameters or other accumulation strategies later. Added IsNull predicate to core Node interface. Going to want enumerated error categories here for sure, but punting on that until we get more examples of what all the errors *are*; then I'll come back and tear all this 'fmt.Errorf' nonsense back out. Signed-off-by: Eric Myhre <hash@exultant.us>
-
- 06 Dec, 2018 1 commit
-
-
Eric Myhre authored
And ReprKind moves from the typed package to the ipld main package. It's hard to get too much done without the standardization of ReprKind. Between the Kind() and Keys() methods, it should now be possible to perform traversals of unknown nodes. This diff just worries about implementing all the Kind() methods. Keys() has some additional questions to handle (namely, map ordering?). Signed-off-by: Eric Myhre <hash@exultant.us>
-
- 10 Nov, 2018 2 commits
-
-
Eric Myhre authored
(Mostly; all the scalars work; the composites are todo's.) Heading in the direction of being able to construct stuff for testing. Almost there. Signed-off-by: Eric Myhre <hash@exultant.us>
-
Eric Myhre authored
That subject has more to do with SerializableNode. Which also might not be the name we end up with for that. ID'able Node? LinkableNode? Signed-off-by: Eric Myhre <hash@exultant.us>
-