Add position tracking fields to Token.

These aren't excersied yet -- and this is accordingly still highly
subject to change -- but so far in developing this package, the pattern
has been "if I say maybe this should have X", it's always turned out
it indeed should have X.  So let's just do that and then try it out,
and have the experimental code instead of the comments.

codec/codectools/token.go

@@ -19,9 +19,15 @@ type Token struct {
 
 	Node ipld.Node // Direct pointer to the original data, if this token is used to communicate data during a walk of existing in-memory data.  Absent when token is being used during deserialization.
 
-	// TODO: position info?  We seem to want this basically everywhere the token goes, so it might as well just live here.
-	//  Putting this position info into the token would require writing those fields many times, though;
-	//   hopefully we can also use them as the primary accounting position then, or else this might be problematic for speed.
+	// The following fields all track position and progress:
+	// (These may be useful to copy into any error messages if errors arise.)
+	// (Implementations may assume token reuse and treat these as state keeping;
+	// you may experience position accounting accuracy problems if *not* reusing tokens or if zeroing these fields.)
+
+	pth          []ipld.PathSegment // Set by token producers (whether marshallers or deserializers) to track logical position.
+	offset       int64              // Set by deserializers (for both textual or binary formats alike) to track progress.
+	lineOffset   int64              // Set by deserializers that work with textual data.  May be ignored by binary deserializers.
+	columnOffset int64              // Set by deserializers that work with textual data.  May be ignored by binary deserializers.
 }
 
 func (tk Token) String() string {

codec/codectools/token_producers.go

@@ -129,12 +129,6 @@ var TokenWalkSkip = errors.New("token walk: skip")
 // TokenReader implementations are encouraged to yield the same token pointer repeatedly,
 // just varying the contents of the value, in order to avoid unnecessary allocations.
 //
-// TODO: as elegant as this is, it's not able to provide much help for error reporting:
-//  if I hit an error while handling the token, I'd like to be able to ask this thing where it thinks it is,
-//   and include that position info in my error report.
-//  Maybe putting position info directly into the Token struct would solve this satisfactorily?
-//   More comments can be found in the Token definition.
-//
 // A 'budget' parameter must be provided to a TokenReader as a pointer to an integer.
 // The TokenReader should limit how much memory it uses according to the budget remaining.
 // (The budget is considered to be roughly in units of bytes, but can be treated as an approximation.)