GitLab

unixfile: precalc dir size

Archive refactor

decouple the DAG traversal logic from the DAG reader (local branch)

This version writes one node at a time instead of first buffering the entire
file contents in a single array (taking up too much memory) before actually
writing it.

Decouple the DAG traversal logic from the `PBDagReader` encapsulating it in the
(new) `Walker` structure.

Collapse PB and Buffer DAG readers into one (`dagReader`) removing the
`bufdagreader.go` and `pbdagreader.go` files, moving all the code to
`dagreader.go`.

Remove `TestSeekAndReadLarge` and `TestReadAndCancel` which operated directly on
the `NodePromise` structure that is now abstracted away in `NavigableIPLDNode`,
in the `go-ipld-format` repo, where they should be recreated.

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>

Unixfs: enforce refs on files when using nocopy

Notes of removing UnixfsNode:
- `NewLeafNode` will return the `FSNodeOverDag` with the given
  `fsNodeType`. While the old `NewLeaf`: `if data is nil the type
  field will be TRaw (for backwards compatibility), if data is
  defined (but possibly empty) the type field will be TRaw.`. Not
  sure if I should follow this. And because of this, I keep the
  `NewLeafNode` and `NewLeafDataNode`, not rename them to
  `NewLeaf` and `GetNextDataNode`.
- There is no functions in importer/helpers/helpers.go. I am
  thinking if I should move the `FSNodeOverDag` part of
  importer/helpers/dagbuilder.go into importer/helpers/helpers.go.
- `GetDagNode` return FilestoreNode for RawNode. But I do not
  understand how it is used in the `DagBuilderHelper.AddChild`. And
  `FileSize` do not calculate the size of RawNode because there is
  no flag of Raw in `FSNodeOverDag`.

License: MIT
Signed-off-by: Bamvor Zhang <jian.zhang@ipfsbit.com>

License: MIT
Signed-off-by: Bamvor Zhang <jian.zhang@ipfsbit.com>

License: MIT
Signed-off-by: Bamvor Zhang <jian.zhang@ipfsbit.com>

License: MIT
Signed-off-by: Bamvor Zhang <jian.zhang@ipfsbit.com>

License: MIT
Signed-off-by: Bamvor Zhang <jian.zhang@ipfsbit.com>

Signed-off-by: Bamvor Zhang <jian.zhang@ipfsbit.com>