walker.go

package format

import (
	"context"
	"errors"
)

// Walker provides methods to move through a DAG of nodes that implement
// the `NavigableNode` interface. It uses iterative algorithms (instead
// of recursive ones) that expose the `path` of nodes from the root to
// the `ActiveNode` it currently points to.
//
// It provides multiple ways to walk through the DAG (e.g. `Iterate`
// and `Seek`). When using them, you provide a Visitor function that
// will be called for each node the Walker traverses. The Visitor can
// read data from those nodes and, optionally, direct the movement of
// the Walker by calling `Pause` (to stop traversing and return) or
// `NextChild` (to skip a child and its descendants). See the DAG reader
// in `gitlab.dms3.io/dms3/public/go-unixfs/io/dagreader.go` for a usage example.
// TODO: This example isn't merged yet.
type Walker struct {

	// Sequence of nodes in the DAG from the root to the `ActiveNode`, each
	// position in the slice being the parent of the next one. The `ActiveNode`
	// resides in the position indexed by `currentDepth` (the slice may contain
	// more elements past that point but they should be ignored since the slice
	// is not truncated to leverage the already allocated space).
	//
	// Every time `down` is called the `currentDepth` increases and the child
	// of the `ActiveNode` is inserted after it (effectively becoming the new
	// `ActiveNode`).
	//
	// The slice must *always* have a length bigger than zero with the root
	// of the DAG at the first position (empty DAGs are not valid).
	path []NavigableNode

	// Depth of the `ActiveNode`. It grows downwards, root being 0, its child 1,
	// and so on. It controls the effective length of `path` and `childIndex`.
	//
	// A currentDepth of -1 signals the start case of a new `Walker` that hasn't
	// moved yet. Although this state is an invalid index to the slices, it
	// allows to centralize all the visit calls in the `down` move (starting at
	// zero would require a special visit case inside every walk operation like
	// `Iterate()` and `Seek`). This value should never be returned to after
	// the first `down` movement, moving up from the root should always return
	// `errUpOnRoot`.
	currentDepth int

	// This slice has the index of the child each node in `path` is pointing
	// to. The child index in the node can be set past all of its child nodes
	// (having a value equal to `ChildTotal`) to signal it has visited (or
	// skipped) all of them. A leaf node with no children that has its index
	// in zero would also comply with this format.
	//
	// Complement to `path`, not only do we need to know which nodes have been
	// traversed to reach the `ActiveNode` but also which child nodes they are
	// to correctly have the active path of the DAG. (Reword this paragraph.)
	childIndex []uint

	// Flag to signal that a pause in the current walk operation has been
	// requested by the user inside `Visitor`.
	pauseRequested bool

	// Used to pass information from the central `Walker` structure to the
	// distributed `NavigableNode`s (to have a centralized configuration
	// structure to control the behavior of all of them), e.g., to tell
	// the `NavigableDMS3LDNode` which context should be used to load node
	// promises (but this could later be used in more elaborate ways).
	ctx context.Context
}

// `Walker` implementation details:
//
// The `Iterate` and `Seek` walk operations are implemented through two
// basic move methods `up` and `down`, that change which node is the
// `ActiveNode` (modifying the `path` that leads to it). The `NextChild`
// method allows to change which child the `ActiveNode` is pointing to
// in order to change the direction of the descent.
//
// The `down` method is the analogous of a recursive call and the one in
// charge of visiting (possible new) nodes (through `Visitor`) and performing
// some user-defined logic. A `Pause` method is available to interrupt the
// current walk operation after visiting a node.
//
// Key terms and concepts:
// * Walk operation (e.g., `Iterate`).
// * Move methods: `up` and `down`.
// * Active node.
// * Path to the active node.

// Function called each time a node is arrived upon in a walk operation
// through the `down` method (not when going back `up`). It is the main
// API to implement DAG functionality (e.g., read and seek a file DAG)
// on top of the `Walker` structure.
//
// Its argument is the current `node` being visited (the `ActiveNode`).
// Any error it returns (apart from the internal `errPauseWalkOperation`)
// will be forwarded to the caller of the walk operation (pausing it).
//
// Any of the exported methods of this API should be allowed to be called
// from within this method, e.g., `NextChild`.
// TODO: Check that. Can `ResetPosition` be called without breaking
// the `Walker` integrity?
type Visitor func(node NavigableNode) error

// NavigableNode is the interface the nodes of a DAG need to implement in
// order to be traversed by the `Walker`.
type NavigableNode interface {

	// FetchChild returns the child of this node pointed to by `childIndex`.
	// A `Context` stored in the `Walker` is passed (`ctx`) that may contain
	// configuration attributes stored by the user before initiating the
	// walk operation.
	FetchChild(ctx context.Context, childIndex uint) (NavigableNode, error)

	// ChildTotal returns the number of children of the `ActiveNode`.
	ChildTotal() uint

	// GetDMS3LDNode returns actual DMS3LD Node
	GetDMS3LDNode() Node

	// TODO: Evaluate providing the `Cleanup` and `Reset` methods.

	// Cleanup is an optional method that is called by the `Walker` when
	// this node leaves the active `path`, i.e., when this node is the
	// `ActiveNode` and the `up` movement is called.
	//Cleanup()
	// Allow this method to return an error? That would imply
	// modifying the `Walker` API, `up()` would now return an error
	// different than `errUpOnRoot`.

	// Reset is an optional function that is called by the `Walker` when
	// `ResetPosition` is called, it is only applied to the root node
	// of the DAG.
	//Reset()
}

// NewWalker creates a new `Walker` structure from a `root`
// NavigableNode.
func NewWalker(ctx context.Context, root NavigableNode) *Walker {
	return &Walker{
		ctx: ctx,

		path:       []NavigableNode{root},
		childIndex: []uint{0},

		currentDepth: -1,
		// Starting position, "on top" of the root node, see `currentDepth`.
	}
}

// ActiveNode returns the `NavigableNode` that `Walker` is pointing
// to at the moment. It changes when `up` or `down` is called.
func (w *Walker) ActiveNode() NavigableNode {
	return w.path[w.currentDepth]
	// TODO: Add a check for the initial state of `currentDepth` -1?
}

// ErrDownNoChild signals there is no child at `ActiveChildIndex` in the
// `ActiveNode` to go down to.
var ErrDownNoChild = errors.New("can't go down, the child does not exist")

// errUpOnRoot signals the end of the DAG after returning to the root.
var errUpOnRoot = errors.New("can't go up, already on root")

// EndOfDag wraps the `errUpOnRoot` and signals to the user that the
// entire DAG has been iterated.
var EndOfDag = errors.New("end of DAG")

// ErrNextNoChild signals the end of this parent child nodes.
var ErrNextNoChild = errors.New("can't go to the next child, no more child nodes in this parent")

// errPauseWalkOperation signals the pause of the walk operation.
var errPauseWalkOperation = errors.New("pause in the current walk operation")

// ErrNilVisitor signals the lack of a `Visitor` function.
var ErrNilVisitor = errors.New("no Visitor function specified")

// Iterate the DAG through the DFS pre-order walk algorithm, going down
// as much as possible, then `NextChild` to the other siblings, and then up
// (to go down again). The position is saved throughout iterations (and
// can be previously set in `Seek`) allowing `Iterate` to be called
// repeatedly (after a `Pause`) to continue the iteration.
//
// This function returns the errors received from `down` (generated either
// inside the `Visitor` call or any other errors while fetching the child
// nodes), the rest of the move errors are handled within the function and
// are not returned.
func (w *Walker) Iterate(visitor Visitor) error {

	// Iterate until either: the end of the DAG (`errUpOnRoot`), a `Pause`
	// is requested (`errPauseWalkOperation`) or an error happens (while
	// going down).
	for {

		// First, go down as much as possible.
		for {
			err := w.down(visitor)

			if err == ErrDownNoChild {
				break
				// Can't keep going down from this node, try to move Next.
			}

			if err == errPauseWalkOperation {
				return nil
				// Pause requested, `errPauseWalkOperation` is just an internal
				// error to signal to pause, don't pass it along.
			}

			if err != nil {
				return err
				// `down` is the only movement that can return *any* error.
			}
		}

		// Can't move down anymore, turn to the next child in the `ActiveNode`
		// to go down a different path. If there are no more child nodes
		// available, go back up.
		for {
			err := w.NextChild()
			if err == nil {
				break
				// No error, it turned to the next child. Try to go down again.
			}

			// It can't go Next (`ErrNextNoChild`), try to move up.
			err = w.up()
			if err != nil {
				// Can't move up, on the root again (`errUpOnRoot`).
				return EndOfDag
			}

			// Moved up, try `NextChild` again.
		}

		// Turned to the next child (after potentially many up moves),
		// try going down again.
	}
}

// Seek a specific node in a downwards manner. The `Visitor` should be
// used to steer the seek selecting at each node which child will the
// seek continue to (extending the `path` in that direction) or pause it
// (if the desired node has been found). The seek always starts from
// the root. It modifies the position so it shouldn't be used in-between
// `Iterate` calls (it can be used to set the position *before* iterating).
// If the visitor returns any non-`nil` errors the seek will stop.
//
// TODO: The seek could be extended to seek from the current position.
// (Is there something in the logic that would prevent it at the moment?)
func (w *Walker) Seek(visitor Visitor) error {

	if visitor == nil {
		return ErrNilVisitor
		// Although valid, there is no point in calling `Seek` without
		// any extra logic, it would just go down to the leftmost leaf,
		// so this would probably be a user error.
	}

	// Go down until it the desired node is found (that will be signaled
	// pausing the seek with `errPauseWalkOperation`) or a leaf node is
	// reached (end of the DAG).
	for {
		err := w.down(visitor)

		if err == errPauseWalkOperation {
			return nil
			// Found the node, `errPauseWalkOperation` is just an internal
			// error to signal to pause, don't pass it along.
		}

		if err == ErrDownNoChild {
			return nil
			// Can't keep going down from this node, either at a leaf node
			// or the `Visitor` has moved the child index past the
			// available index (probably because none indicated that the
			// target node could be down from there).
		}

		if err != nil {
			return err
			// `down()` is the only movement that can return *any* error.
		}
	}
	// TODO: Copied from the first part of `Iterate()` (although conceptually
	// different from it). Could this be encapsulated in a function to avoid
	// repeating code? The way the pause signal is handled it wouldn't seem
	// very useful: the `errPauseWalkOperation` needs to be processed at this
	// depth to return from the function (and pause the seek, returning
	// from another function here wouldn't cause it to stop).
}

// Go down one level in the DAG to the child of the `ActiveNode`
// pointed to by `ActiveChildIndex` and perform some logic on it by
// through the user-specified `visitor`.
//
// This should always be the first move in any walk operation
// (to visit the root node and move the `currentDepth` away
// from the negative value).
func (w *Walker) down(visitor Visitor) error {
	child, err := w.fetchChild()
	if err != nil {
		return err
	}

	w.extendPath(child)

	return w.visitActiveNode(visitor)
}

// Fetch the child from the `ActiveNode` through the `FetchChild`
// method of the `NavigableNode` interface.
func (w *Walker) fetchChild() (NavigableNode, error) {
	if w.currentDepth == -1 {
		// First time `down()` is called, `currentDepth` is -1,
		// return the root node. Don't check available child nodes
		// (as the `Walker` is not actually on any node just yet
		// and `ActiveChildIndex` is of no use yet).
		return w.path[0], nil
	}

	// Check if the child to fetch exists.
	if w.ActiveChildIndex() >= w.ActiveNode().ChildTotal() {
		return nil, ErrDownNoChild
	}

	return w.ActiveNode().FetchChild(w.ctx, w.ActiveChildIndex())

	// TODO: Maybe call `extendPath` here and hide it away
	// from `down`.
}

// Increase the `currentDepth` and extend the `path` to the fetched
// `child` node (which now becomes the new `ActiveNode`)
func (w *Walker) extendPath(child NavigableNode) {
	w.currentDepth++

	// Extend the slices if needed (doubling its capacity).
	if w.currentDepth >= len(w.path) {
		w.path = append(w.path, make([]NavigableNode, len(w.path))...)
		w.childIndex = append(w.childIndex, make([]uint, len(w.childIndex))...)
		// TODO: Check the performance of this grow mechanism.
	}

	// `child` now becomes the `ActiveNode()`.
	w.path[w.currentDepth] = child
	w.childIndex[w.currentDepth] = 0
}

// Call the `Visitor` on the `ActiveNode`. This function should only be
// called from `down`. This is a wrapper function to `Visitor` to process
// the `Pause` signal and do other minor checks (taking this logic away
// from `down`).
func (w *Walker) visitActiveNode(visitor Visitor) error {
	if visitor == nil {
		return nil
		// No need to check `pauseRequested` as `Pause` should
		// only be called from within the `Visitor`.
	}

	err := visitor(w.ActiveNode())

	if w.pauseRequested {
		// If a pause was requested make sure an error is returned
		// that will cause the current walk operation to return. If
		// `Visitor` didn't return an error set an artificial one
		// generated by the `Walker`.
		if err == nil {
			err = errPauseWalkOperation
		}

		w.pauseRequested = false
	}

	return err
}

// Go up from the `ActiveNode`. The only possible error this method
// can return is to signal it's already at the root and can't go up.
func (w *Walker) up() error {
	if w.currentDepth < 1 {
		return errUpOnRoot
	}

	w.currentDepth--

	// w.ActiveNode().Cleanup()
	// If `Cleanup` is supported this would be the place to call it.

	return nil
}

// NextChild increases the child index of the `ActiveNode` to point
// to the next child (which may exist or may be the end of the available
// child nodes).
//
// This method doesn't change the `ActiveNode`, it just changes where
// is it pointing to next, it could be interpreted as "turn to the next
// child".
func (w *Walker) NextChild() error {
	w.incrementActiveChildIndex()

	if w.ActiveChildIndex() == w.ActiveNode().ChildTotal() {
		return ErrNextNoChild
		// At the end of the available children, signal it.
	}

	return nil
}

// incrementActiveChildIndex increments the child index of the `ActiveNode` to
// point to the next child (if it exists) or to the position past all of
// the child nodes (`ChildTotal`) to signal that all of its children have
// been visited/skipped (if already at that last position, do nothing).
func (w *Walker) incrementActiveChildIndex() {
	if w.ActiveChildIndex()+1 <= w.ActiveNode().ChildTotal() {
		w.childIndex[w.currentDepth]++
	}
}

// ActiveChildIndex returns the index of the child the `ActiveNode()`
// is pointing to.
func (w *Walker) ActiveChildIndex() uint {
	return w.childIndex[w.currentDepth]
}

// SetContext changes the internal `Walker` (that is provided to the
// `NavigableNode`s when calling `FetchChild`) with the one passed
// as argument.
func (w *Walker) SetContext(ctx context.Context) {
	w.ctx = ctx
}

// Pause the current walk operation. This function must be called from
// within the `Visitor` function.
func (w *Walker) Pause() {
	w.pauseRequested = true
}