Commit d168547a authored by Eric Myhre's avatar Eric Myhre

Beginye another pass on docs.

Starting with a new index, which enumerates lots of things which shall
deserve at least one page or section.

Big picture also includes links out to things which shall require their
own (as yet unwritten) pages.

Old ball-of-mud "dev.md" already being eroded into the new files.
Signed-off-by: default avatarEric Myhre <hash@exultant.us>
parent aaea73ad
- [IPLD Big Picture](./big-picture.md)
- Nodes
- Node interface overview
- Node implementation diversity
- how to use NodeBuilder
- intro to typed nodes
- using the fluent API
- operationals
- focus & paths
- traverse & selectors
- transform
- constructing other operations
- storing, loading, and linking
- encoding
- (this covers only cbor and json builtins)
- (brief description of how to build-your-own (either with refmt tokens or nodewalking))
- cids are links
- linkloader and linkbuilder
- automatical link loading in operationals
IPLD Big Picture
================
IPLD is a system for describing, storing, sharing, and hashing data.
Layers
------
IPLD comes in several layers:
1. The IPLD Data Model
2. The IPLD Schema system
3. "IPFN"
The IPLD Data Model is a specification of the core serializable types of data.
You can think of it as roughly isomorphic to the familiar JSON and CBOR
serialization systems (and yes, those parsers usually work "out of the box").
You can read more about
[the Data Model specification in the Specs repo](https://github.com/ipld/specs/blob/master/IPLD-Data-Model-v1.md).
The IPLD Schema system builds atop the IPLD Data Model to add a system of
user-defined types. These type schema are meant to be easy to write, easy to
read, and easy for the machine to parse and check efficiently.
IPLD Schema bring common concepts like "structs" with named "fields", as well
as "enums" and even "unions" -- and make clear definitions of how to map all of
these concepts onto the basic serializable types of the Data Model.
You can use IPLD at any of these levels you choose. We recommend having a
schema for your data, because it makes validity checking, sanitization,
migration, and documentation all easier in the long run -- but the Data Model
is also absolutely usable stand-alone.
IPFN is mentioned here as an intent to design something between "dependent types"
and a full network and host agnostic computation platform. It is more of an
outline of a research direction than a particular project; the mention here is
mostly to make it clear what *not* to expect from this repo.
Nodes, operations, encoding, & linking
--------------------------------------
Three (or four) big picture groups of concepts you'll want to keep in mind when
reading the rest of the docs are nodes, operations, encoding, and linking.
Nodes: every piece of data in IPLD can be handled as a "node".
Strings are nodes; ints are nodes; maps are nodes; etc.
[Nodes have their own full chapter of the documentation](./nodes.md).
Operations: some basic computation we can do *on* a node is an operation.
Traversals and simply updates have built-in functions; more advanced operations
can be built with some use of callbacks.
[Operations have their own full chapter of the documentation](./operations.md).
Encoding: every piece of data in IPLD is serializable. Any node can be
converted to a tokenized representation, and several built-in serialization
systems are supported, plus a pluggable mechanism for implementing more.
[Encoding has its own full chapter of the documentation](./encoding.md)
Linking: every piece of data in IPLD is serializable ***and hashable***...
and can be linked by those hashes. These links can be manipulated explicitly,
and also used transparently by some of the built-in operation support.
[Linking has its own full chapter of the documentation](./linking.md)
All of these features work on both the Data Model (unityped) layer
and also with Schema layer (typed) nodes.
Developer Docs
==============
Core Concepts
-------------
### IPLD Big Picture
IPLD is a system for describing, storing, sharing, and hashing data.
IPLD comes in several layers:
1. The IPLD Data Model
2. The IPLD Schema system
3. "IPFN"
The IPLD Data Model is a specification of the core serializable types of data.
You can think of it as roughly isomorphic to the familiar JSON and CBOR
serialization systems (and yes, those parsers usually work "out of the box").
You can read more about
[the Data Model specification in the Specs repo](https://github.com/ipld/specs/blob/master/IPLD-Data-Model-v1.md).
The IPLD Schema system builds atop the IPLD Data Model to add a system of
user-defined types. These type schema are meant to be easy to write, easy to
read, and easy for the machine to parse and check efficiently.
IPLD Schema bring common concepts like "structs" with named "fields", as well
as "enums" and even "unions" -- and make clear definitions of how to map all of
these concepts onto the basic serializable types of the Data Model.
You can use IPLD at any of these levels you choose. We recommend having a
schema for your data, because it makes validity checking, sanitization,
migration, and documentation all easier in the long run -- but the Data Model
is also absolutely usable stand-alone.
IPFN is mentioned here as an intent to design something between "dependent types"
and a full network and host agnostic computation platform. It is not part of
this repo.
### IPLD Nodes
Everything is a Node. Maps are a node. Arrays are node. Ints are a node.
Everything in the IPLD Data Model is a Node.
Nodes are traversable. Given a node which is one of the recursive kinds
(e.g. map or array), you can list child indexes, and you can traverse that
index to get another Node.
Nodes are trees. It is not permissible to make a cycle of Node references.
Everything in the IPLD *Schema* layer is *also* a node.
This sometimes means you'll jump over *several* nodes in the raw Data Model
representation of the data when traversing one link in the Schema layer.
(Go programmers already familiar with the standard library `reflect` package
may find it useful to think of `ipld.Node` as similar to `reflect.Value`.)
Code Layout
-----------
......
Nodes
=====
Everything is a Node. Maps are a node. Arrays are node. Ints are a node.
Everything in the IPLD Data Model is a Node.
Nodes are traversable. Given a node which is one of the recursive kinds
(e.g. map or array), you can list child indexes, and you can traverse that
index to get another Node.
Nodes are trees. It is not permissible to make a cycle of Node references.
(Go programmers already familiar with the standard library `reflect` package
may find it useful to think of `ipld.Node` as similar to `reflect.Value`.)
the Node interface
------------------
TODO
Node implementations
--------------------
TODO
using NodeBuilder
-----------------
TODO
Typed Nodes
-----------
TODO
Everything in the IPLD *Schema* layer is *also* a node.
This sometimes means you'll jump over *several* nodes in the raw Data Model
representation of the data when traversing one link in the Schema layer.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment