GitLab

This allows closing a read-write blockstore without doing the extra work
to finalize its header and index. Can be useful if an entire piece of
work is cancelled, and also simplifies the tests.

Also make ReadOnly error in a straightforward way if it is used after
being closed. Before, this could lead to panics, as we'd attempt to read
the CARv1 file when it's closed.

Both mechanisms now use a "closed" boolean, which is consistent and
simpler than checking a header field.

Finally, add tests that ensure both ReadOnly and ReadWrite behave as
intended once they have been closed.

The tests also uncovered that AllKeysChan would not release the mutex if
it encountered an error early on. Fix that, too.

While at it, fix some now-obsolete references to panics on unsupported
or after-close method calls.

Fixes #205.

While at it, make v2's license a symlink, so the two stay in sync.
The reason we want the v2 module to have its own license is so that its
module zip includes the file as well.

Besides mentioning v0 and v2, the root README now also links to pkgsite
and lists Masih and myself as default maintainers, given that we're the
ones that did major work on this library last.

Fixes #124.
Fixes #179.

Rename terminology to match what the spec uses to describe the inner
CARv1 payload, i.e. Data payload.

Update docs to use CARv1 and CARv2 consistently.

Fix typo in Options API.

See:
- https://ipld.io/specs/transport/car/carv2/

All the "New" APIs take IO interfaces, so they don't open any file by
themselves. However, the APIs that take a path to disk and return a
blockstore or a reader need closing, so the prefix "Open" helps clarify
that.

Plus, it makes names more consistent.

When file exists, attempt resumption by default. This seems like a more
user-friendly API and less requirements to explicitly tune knobs.

Explicitly fail of the file is finalized. We technically can append to a
finalized file but that involves changes in multiple places in reader
abstractions. Left a TODO as a feature for future when asked for.

Close off file if construction of ReadWrite blockstore fails after file
is opened. If left open, it causes test failures when t cleans up temp
files.

* Implement an option for read-write blockstore, that if enabled the
write can resume from where the writer left off. For resumption to work
the `WithResumption` option needs to be set explicitly. Otherwise, if
path to an existing file is passed, the blockstore construction will
return an error. The resumption requires the roots passed to constructor
as well as padding options to be identical with roots in file.
Resumption only works on paths where at least V2 pragma and CAR v1
header was successfully written onto the file. Otherwise an error is
returned.

* Implement resumption test that verifies files resumed from match
expected  header, data and index.

* Implement a CAR v1 equals function to check if two given headers are
identical. This implementation requires exact ordering of root elements.
A TODO is left to relax the exact ordering requirement.

* Implement Seeker in internal offset writer in order to forward offset
of CAR v1 writer within a resumed read-write blockstore after
resumption. The offset of the writer needs to be set to the latest
written frame in order for consecutive writes to be at the right offset.

* Fix bug in offset read seeker in internal IO, where seek and returned
position was not normalized by the base. Reflect the fix in read-only
blockstore AllKeysChan where reader was twisted to work. We now read the
 header to get its size, then seek past it to then iterate over blocks
 to populate the channel.

* Add TODOs in places to make treating zero-length frames as EOF
optional; See #140 for context.

* Run `gofumpt -l -w .` on everything to maintain consistent formatting.

Address review comments

* Implement equality check for CAR v1 headers where roots in different
  order are considered to be equal.

* Improve resumption docs to clarify what matching roots mean.

Since `blockstore.NewReadOnly` accepts both versions, and the `attach`
flag is never used in `OpenReadOnly` make things consistent and arguably
more useful by accepting both v1 and v2 CAR files. This also avoids
modifying files passed to a read-only API which is always sweet as
sugar.

Rename the `blockstore.ReadOnlyOf` to `blockstore.NewReadOnly` for
better readability and consistent naming.

Allow the function to accept both v1 and v2 payloads and "do the right
thing" depending on the input.

Optionally, allow the caller to supply an index that, if present, will
overrider any existing index. Otherwise an in-memory index is generated.

Note, `blockstore.OpenReadOnly` only accepts v2 payload. Future PRs will
change this function to also accept both versions for consistency, and
drop modification of given path, delegating any writing to the
write-specific APIs like `index.Attach` and `index.Generate`.

Fixes #128

Add documentation where it is sparse to help folks learn how to use the
library easier.

Unexport the ability to select codec, until we decide what codec we want
to expose. Fix all encodings of index to `IndexSorted`.

Fix a bug in writer where index is written by direct marshalling instead
of index.WriteTo which includes index codec.

Refactor in places for better readability.