GitLab

Including the offset.

We had a hack that "pretended" seeking worked when it didn't as go's HTTP
library uses seeks to determine the file size. However,

1. I'm changing the gateway to actually rely on seeking working as specified.
2. We don't even need this hack. The gateway performs two types of seeks (unless
   a range query is passed):
    1. It seeks to the beginning. We can always shortcut this.
    2. It seeks to the end. The gateway now has a special "lazy" seeker to avoid
      seeking until we actually try to _read_. Therefore, we don't need a hack
      for that either.

This version writes one node at a time instead of first buffering the entire
file contents in a single array (taking up too much memory) before actually
writing it.

Decouple the DAG traversal logic from the `PBDagReader` encapsulating it in the
(new) `Walker` structure.

Collapse PB and Buffer DAG readers into one (`dagReader`) removing the
`bufdagreader.go` and `pbdagreader.go` files, moving all the code to
`dagreader.go`.

Remove `TestSeekAndReadLarge` and `TestReadAndCancel` which operated directly on
the `NodePromise` structure that is now abstracted away in `NavigableIPLDNode`,
in the `go-ipld-format` repo, where they should be recreated.

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>

- Add LinkResult type to unix-fs - is an ipld Link or an error
- Add EnumLinksAsync method to Directory interface, returns channel of directory links or error
- Add EnumLinksAsync method to Shard interface in HAMT, returns channel
of directory links or error
- EnumLinks method in Shard interface in HAMT uses EnumLinksAsync now
- modify makeAsyncTrieGetLinks to use channel

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

alternative to #5243 that updates go-cid and all packages that depend on it

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>

Create new `loadBufNode` function to handle the `buf` logic which is unrelated
to the main `precalcNextBuf` logic of processing promises to fetch nodes.

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>

Do not use `NewDagReader` just for the `RawNode` case.
Treat invalid UnixFS types in the same case.

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>

Add a UnixFS `Directory` that hides implementation details and helps to
distinguish *what* is a UnixFS directory.

Replace the `unixfs.io.Directory` structure that contained the HAMT and basic
directory implementations (through inner pointers) with an interface containing
the same methods. Implement those methods in two clearly distinct structures for
each implementation (`BasicDirectory` and `HAMTDirectory`) avoiding pointer
logic and clearly differentiating which implementation does what.

The potential basic to HAMT transition was being hidden behind the `AddChild`
call at the UnixFS layer (changing one implementation pointer  with the other
one), it is now being explicitly done at the MFS layer.

Rename the `dirbuilder.go` file to `directory.go` and change the `Directory` MFS
attribute `dirbuilder` to `unixfsDir` to be consistent.

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>

See https://golang.org/doc/effective_go.html#Getters.

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>

Focus on the UnixFS layer and avoid explicit references to protocol buffers
format (used to serialize objects of that layer). Use the `unixfs.FSNode`
structure which it abstracts from the `unixfs.pb.Data` format.

Replace `PBDagReader` field `ftpb.Data` with `ft.FSNode`, renaming it to `file`
(which is the type of UnixFS object represented in the reader) and changing its
comment removing the "cached" reference, as this structure is not used here as a
cache (`PBDagReader` doesn't modify the DAG, it's read-only). Also, removed
unused `ProtoNode` field to avoid confusions, as it would normally be present if
the `FSNode` was in fact used as a cache of the contents of the `ProtoNode`.

An example of the advantage of shifting the focus from the format to the UnixFS
layer is dropping the of use `len(pb.Blocksizes)` in favor of the more clear
`NumChildren()` abstraction.

Added `BlockSize()` accessor.

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>

Remove `Offset()` from the `DagReader` interface. It's not part of the Unix API
and it wasn't used anywhere except for the tests (a helper function was added to
replace it).

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>