Commit e8ba18e3 authored by hannahhoward's avatar hannahhoward

docs(README): update interface descriptions

Add storer to Graphsync Initialization
parent 3047e890
...@@ -13,6 +13,7 @@ ...@@ -13,6 +13,7 @@
- [Background](#background) - [Background](#background)
- [Install](#install) - [Install](#install)
- [Usage](#usage) - [Usage](#usage)
- [Architecture](#architecture)
- [Contribute](#contribute) - [Contribute](#contribute)
- [License](#license) - [License](#license)
...@@ -47,10 +48,11 @@ import ( ...@@ -47,10 +48,11 @@ import (
var ctx context.Context var ctx context.Context
var host libp2p.Host var host libp2p.Host
var loader ipld.Loader var loader ipld.Loader
var storer ipld.Storer
network := gsnet.NewFromLibp2pHost(host) network := gsnet.NewFromLibp2pHost(host)
ipldBridge := gsbridge.NewIPLDBridge() ipldBridge := gsbridge.NewIPLDBridge()
exchange := graphsync.New(ctx, network, ipldBridge, loader) exchange := graphsync.New(ctx, network, ipldBridge, loader, storer)
``` ```
Parameter Notes: Parameter Notes:
...@@ -60,10 +62,19 @@ Parameter Notes: ...@@ -60,10 +62,19 @@ Parameter Notes:
of libp2p. This allows graphsync to be tested without the actual network of libp2p. This allows graphsync to be tested without the actual network
3. `ipldBridge` is an IPLD abstraction provided to Graphsync on top of go-ipld-prime. This makes the graphsync library testable in isolation 3. `ipldBridge` is an IPLD abstraction provided to Graphsync on top of go-ipld-prime. This makes the graphsync library testable in isolation
4. `loader` is used to load blocks from content ids from the local block store. It's used when RESPONDING to requests from other clients. It should conform to the IPLD loader interface: https://github.com/ipld/go-ipld-prime/blob/master/linking.go 4. `loader` is used to load blocks from content ids from the local block store. It's used when RESPONDING to requests from other clients. It should conform to the IPLD loader interface: https://github.com/ipld/go-ipld-prime/blob/master/linking.go
5. `storer` is used to store incoming blocks to the local block store. It's used when REQUESTING a graphsync query, to store blocks locally once they are validated as part of the correct response. It should conform to the IPLD storer interface: https://github.com/ipld/go-ipld-prime/blob/master/linking.go
### Write A Loader From The Stuff You Know ### Write A Loader An IPFS BlockStore
Coming from a pre-`go-ipld-prime` world, you probably expect a link loading function signature to look like this: If you are using a traditional go-ipfs-blockstore, your link loading function looks like this:
```golang
type BlockStore interface {
Get(lnk cid.Cid) (blocks.Block, error)
}
```
or, more generally:
```golang ```golang
type Cid2BlockFn func (lnk cid.Cid) (blocks.Block, error) type Cid2BlockFn func (lnk cid.Cid) (blocks.Block, error)
...@@ -75,7 +86,7 @@ in `go-ipld-prime`, the signature for a link loader is as follows: ...@@ -75,7 +86,7 @@ in `go-ipld-prime`, the signature for a link loader is as follows:
type Loader func(lnk Link, lnkCtx LinkContext) (io.Reader, error) type Loader func(lnk Link, lnkCtx LinkContext) (io.Reader, error)
``` ```
`go-ipld-prime` intentionally keeps its interfaces as abstract as possible to limit dependencies on other ipfs/filecoin specific packages. An IPLD Link is an abstraction for a CID, and IPLD expects io.Reader's rather than an actual block. IPLD provides a `cidLink` package for working with Links that use CIDs as the underlying data, and it's safe to assume that's the type in use if your code deals only with CIDs. Anyway, a conversion would look something like this: `go-ipld-prime` intentionally keeps its interfaces as abstract as possible to limit dependencies on other ipfs/filecoin specific packages. An IPLD Link is an abstraction for a CID, and IPLD expects io.Reader's rather than an actual block. IPLD provides a `cidLink` package for working with Links that use CIDs as the underlying data, and it's safe to assume that's the type in use if your code deals only with CIDs. A conversion would look something like this:
```golang ```golang
import ( import (
...@@ -98,76 +109,75 @@ func LoaderFromCid2BlockFn(cid2BlockFn Cid2BlockFn) ipld.Loader { ...@@ -98,76 +109,75 @@ func LoaderFromCid2BlockFn(cid2BlockFn Cid2BlockFn) ipld.Loader {
} }
``` ```
Alternatively, you can just call: ### Write A Storer From An IPFS BlockStore
If you are using a traditional go-ipfs-blockstore, your storage function looks like this:
```golang ```golang
loader := graphsync.LoaderFromCid2BlockFn(cid2BlockFn) type BlockStore interface {
Put(blocks.Block) error
}
``` ```
### Calling Graphsync or, more generally:
```golang ```golang
var exchange graphsync.GraphSync type BlockStoreFn func (blocks.Block) (error)
var ctx context.Context
var p peer.ID
var cidRootedSelector ipld.Node
var responseProgress <-chan graphsync.ResponseProgress
var errors <-chan error
responseProgress, errors = exchange.Request(ctx context.Context, p peer.ID, rootedSelector Node)
``` ```
Paramater Notes: in `go-ipld-prime`, the signature for a link storer is a bit different:
1. `ctx` is the context for this request. To cancel an in progress request, cancel the context.
2. `p` is the peer you will send this request to
3. `rootedSelector` is the a go-ipld-prime node the specifies a rooted selector
### Building a path selector
A rooted selector is a `go-ipld-prime` node that follows the spec outlined here: https://github.com/ipld/specs/blob/master/block-layer/selectors/selectors.md ```golang
type StoreCommitter func(Link) error
type Storer func(lnkCtx LinkContext) (io.Writer, StoreCommitter, error)
```
`go-ipld-prime` provides a series of builder interfaces for building this kind of structured data into a node. If your library simply wants to make a selector from CID and a path, represented by an array of strings, you could construct the node as follows: `go-ipld-prime` stores in two parts to support streaming -- the storer is called and returns an IO.Writer and a function to commit changes when finished. Here's how you can write a storer from a traditional block storing signature.
```golang ```golang
import ( import (
ipld "github.com/ipld/go-ipld-prime" blocks "github.com/ipfs/go-block-format"
free "github.com/ipld/go-ipld-prime/impl/free" ipld "github.com/ipld/go-ipld-prime"
fluent "github.com/ipld/go-ipld-prime/fluent" cidLink "github.com/ipld/go-ipld-prime/linking/cid"
cidLink "github.com/ipld/go-ipld-prime/linking/cid"
) )
func SelectorSpecFromCidAndPath(lnk cid.Cid, pathSegments []string) (ipld.Node, error) { func StorerFromBlockStoreFn(blockStoreFn BlockStoreFn) ipld.Storer {
var node ipld.Node return func(lnkCtx ipld.LinkContext) (io.Writer, ipld.StoreCommitter, error) {
err := fluent.Recover(func() { var buffer bytes.Buffer
builder := fluent.WrapNodeBuilder(free.NodeBuilder()) committer := func(lnk ipld.Link) error {
node = builder.CreateMap(func (mb fluent.MapBuilder, knb fluent.NodeBuilder, vnb fluent.NodeBuilder) { asCidLink, ok := lnk.(cidlink.Link)
mb.Insert(knb.CreateString("root"), vnb.CreateLink(cidLink.Link{lnk})) if !ok {
mb.Insert(knb.CreateString("selectors"), return fmt.Errorf("Unsupported Link Type")
vnb.CreateList(func (lb fluent.ListBuilder, vnb fluent.NodeBuilder) { }
for _, pathSegment := range pathSegments { block := blocks.NewBlockWithCid(buffer.Bytes(), asCidLink.Cid)
lb.Append(CreateMap( return blockStoreFn(block)
func (mb fluent.MapBuilder, knb fluent.NodeBuilder, vnb fluent.NodeBuilder) { }
mb.Insert(knb.CreateString("selectPath"), vnb.CreateString(pathSegment)) return &buffer, committer, nil
},
))
}
}))
});
})
if err != nil {
return nil, err
} }
return node, nil
} }
``` ```
Alternatively, just call: ### Calling Graphsync
```golang ```golang
rootedSelector := graphsync.SelectorSpecFromCidAndPath(lnk, pathSegments) var exchange graphsync.GraphSync
var ctx context.Context
var p peer.ID
var selector ipld.Node
var rootLink ipld.Link
var responseProgress <-chan graphsync.ResponseProgress
var errors <-chan error
responseProgress, errors = exchange.Request(ctx context.Context, p peer.ID, root ipld.Link, selector ipld.Node)
``` ```
Paramater Notes:
1. `ctx` is the context for this request. To cancel an in progress request, cancel the context.
2. `p` is the peer you will send this request to
3. `link` is an IPLD Link, i.e. a CID (cidLink.Link{Cid})
4. `selector` is an IPLD selector node. Recommend using selector builders from go-ipld-prime to construct these
### Response Type ### Response Type
```golang ```golang
...@@ -185,25 +195,12 @@ type ResponseProgress struct { ...@@ -185,25 +195,12 @@ type ResponseProgress struct {
The above provides both immediate and relevant metadata for matching nodes in a traversal, and is very similar to the information provided by a local IPLD selector traversal in `go-ipld-prime` The above provides both immediate and relevant metadata for matching nodes in a traversal, and is very similar to the information provided by a local IPLD selector traversal in `go-ipld-prime`
## Compatibility: Block Requests
While the above is extremely useful if your library already uses `go-ipld-prime`, if your library uses an older version of go ipld libraries, working with these types of `go-ipld-prime` data may prove challenging.
To support these clients, Graphsync provides a compatibility version of the above function that returns just blocks that were traversed:
```golang
var blocksChan <-chan blocks.Block
var errors <-chan error
blocksChan, errors = exchange.GetBlocks(ctx context.Context, p peer.ID, rootedSelector Node)
```
This is provided as a transitional layer and `go-graphsync` may drop support for this format in the future.
## Contribute ## Contribute
PRs are welcome! PRs are welcome!
Before doing anything heavy, checkout the [Graphsync Architecture](docs/architecture.md)
Small note: If editing the Readme, please conform to the [standard-readme](https://github.com/RichardLitt/standard-readme) specification. Small note: If editing the Readme, please conform to the [standard-readme](https://github.com/RichardLitt/standard-readme) specification.
## License ## License
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment