GitLab

importer: remove `UnixfsNode` from the balanced builder

The `UnixfsNode` structure has multiple pointers to many (non-complementary)
mutually exclusive node types, only some of them are active (not-`nil`) at a
given time in the code path which made the code too convoluted. Specifically,
the most important distinction between node types was being hidden: leaf nodes
vs internal (non-leaf) nodes.

Remove entirely the use of `UnixfsNode` from the `balanced` package replacing it
in turn with the newly created `FSNodeOverDag` structure that represents the
UnixFS node encoded inside the DAG node, primarily used for internal node
representations. Leaf nodes are handled exclusively in the `NewLeafDataNode`
encapsulating its multiple representations (that we're previously exposed in
`UnixfsNode` as conflicting pointers).

The `builder.go` file has been completely rewritten, although the basic DAG
creation algorithm has been preserved (extending a full DAG by creating a new
root and linking the old one as its child), the most significant modification
has been in the loop of `Layout` that now only handles internal nodes (i.e.,
nodes with `depth` bigger than zero) to be able to adapt `fillNodeRec` to only
that scenario (avoiding the replace logic of the zero `depth` case with the
defective `Set` function, now removed). The `fillNodeRec` now explicitly returns
the `ipld.Node` and the size of the file data it's storing to propagate it
upwards into the DAG.

The `DagBuilderHelper` was heavily extended to incorporate `ipld.Node` functions
that would replace the `UnixfsNode` ones used by the balanced builder:
`NewLeafNode()`, `NewLeafDataNode()` and `AddNodeAndClose()`. Also, the
`ProcessFileStore` function was incorporated to encapsulate all the logic
related to the Filestore support which was scattered throughout the builder
logic, the `offset` that was being passed through most functions is now a part
of the `DagBuilderHelper`.

This has turned out to be a rather big commit, it should have been split into
more smaller and logically cohesive commits, but the `UnixfsNode` was too
entangled inside the logic and that would have required a progressive
modification of the `UnixfsNode` structure as well, which wasn't possible as it
is still being used by the balanced builder (the same reason why most of the
`UnixfsNode`-related functions cannot yet be removed, leaving the `helpers.go`
file mostly untouched).

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>

unixfs: fix `FSNode` accessor in `NewDirectoryFromNode`

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>

Add information about bloom filter to config.md

unixfs: add a directory interface

Refactor code

cmd: fix `files ls` to report hash and size for files

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

Efficient "repo stat" (DiskUsage) and "--size-only" flag

feat: Allow specifing how object data is encoded

mfs: make `Root` value a `Directory`

pbdagreader: use FSNode instead of protobuf structure

dagreader: remove `Offset()` method

URL store

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

Ipfs 0.4.16 release

License: MIT
Signed-off-by: Whyrusleeping <why@ipfs.com>

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>

License: MIT
Signed-off-by: Rob Brackett <rob@robbrackett.com>

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>

publish the gx 0.4.16-rc3 release

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>

Ipfs v0.4.16-rc3

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>

Add a UnixFS `Directory` that hides implementation details and helps to
distinguish *what* is a UnixFS directory.

Replace the `unixfs.io.Directory` structure that contained the HAMT and basic
directory implementations (through inner pointers) with an interface containing
the same methods. Implement those methods in two clearly distinct structures for
each implementation (`BasicDirectory` and `HAMTDirectory`) avoiding pointer
logic and clearly differentiating which implementation does what.

The potential basic to HAMT transition was being hidden behind the `AddChild`
call at the UnixFS layer (changing one implementation pointer  with the other
one), it is now being explicitly done at the MFS layer.

Rename the `dirbuilder.go` file to `directory.go` and change the `Directory` MFS
attribute `dirbuilder` to `unixfsDir` to be consistent.

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>