schema: working to unify interfaces and dmt. Intermediate checkpoint commit.

This commit does not pass CI or even fully compile, and while I usually
try to avoid those, A) I need a checkpoint!, and B) I think this one is
interestingly illustrative, and I'll probably want to refer to this
diff and the one that will follow it in the future as part of
architecture design records (or even possibly experience reports about
golang syntax).

In this commit: we have three packages:

- schema: full of interfaces (and only interfaces)
- schema/compiler: creates values matching schema interfaces
- schema/dmt: contains codegen'd types that parse schema documents.

The dmt package feeds data to the compiler package, and the compiler
package emits values matching the schema interface.
This all works very nicely and avoids import cycles.

(Avoiding import cycles has been nontrivial, here, unfortunately.
The schema/schema2 package (which is still present in this commit,
but will be removed shortly -- I've scraped most of it over into
this new 'compiler' package already, just not a bunch of the validation
rules stuff, yet) was a dream of making this all work by just having
thin wrapper types around the dmt types.  This didn't fly...
because codegen'd nodes comply with `schema.TypedNode`, and complying
with `schema.TypedNode` means they have a function which references
`schema.Type`... and that means we really must depend on that
interface and the package it's in.  Ooof.)

The big downer with this state, and why things are currently
non-compiling at this checkpoint I've made here, is that we have to
replicate a *lot* of methods into single-use interfaces in the schema
package for this to work.  This belies the meaning of "interface".
The reason we'd do this -- the reason to split 'compiler' into its own
package -- is most because I wanted to keep all the constructor
mechanisms for schema values out of the direct path of the user's eye,
because most users shouldn't be using the compiler directly at all.

But... I'm shifting to thinking this attempt to segregate the compiler
details isn't worth it.  A whole separate package costs too much.
Most concretely, it would make it impossible to make the `schema.Type`
interface "closed" (e.g. by having an unexported method), and I think
at that point we would be straying quite far from desired semantics.

schema/compiler/compiler.go

+/*
+	The schema/compiler package contains concrete implementations of the
+	interfaces in the schema package which are used to describe IPLD Schemas,
+	and it also provides a Compiler type which is used to construct them.
+*/
+package compiler
+
+import (
+	"fmt"
+
+	"github.com/ipld/go-ipld-prime/schema"
+)
+
+// Compiler creates new TypeSystem instances.
+// Methods are called on a Compiler instance to add types to the set,
+// and when done, the Compile method is called, which can return
+// either a list of error values or a new TypeSystem.
+//
+// Users don't usually use Compiler themselves,
+// and this API isn't meant to be especially user-friendly.
+// It's better to write IPLD Schemas using the DSL,
+// parse and transpile that into the standard DMT format,
+// and then read that with `schema/dmt` package and use the `dmt.Compile` feature.
+// This lets you spend more time with the human-readable syntax and DMT format,
+// which in addition to being better suited for documentation and review,
+// is also usable with other IPLD tools and IPLD implementations in other languages.
+// (Inside, the `dmt.Compile` feature uses this Compiler for you.)
+//
+// On error handling:
+// Since several sorts of error can't be checked until the whole group of types has been stated
+// (for example, referential completeness checks),
+// almost none of the methods on Compiler return errors as they go.
+// All errors will just be reported altogether at once at the end, when Compile is called.
+// Some extremely obvious errors, like trying to use the same TypeName twice, will cause a panic immediately.
+// The rule for errors that are raised as panics is that they must have been already avoided if the data were coming from the schemadmt package.
+// (E.g., if something could be invalidly sent to the Compiler twice, but was a map key in the schemadmt and so already checked as unique, that's panic-worthy here.
+// But if repeats of some identifier are invalid but would a list when expressed in the schemadmt, that's *not* allowed to panic here.)
+//
+// On immutability:
+// The TypeSystem returned by a successful Compile call will be immutable.
+// Many methods on the Compiler type are structured to accept data in a way that works towards this immutability.
+// In particular, many methods on Compiler take arguments which are "carrier types" for segments of immutable data,
+// which must be produced by constructor functions; for one example of this pattern, see the interplay of Compiler.TypeStruct() and MakeStructFieldList().
+type Compiler struct {
+	// ... and if you're wondering why this type is exported at all?
+	//  Well, arguably, it's useful to be able to construct these values without going through the dmt.
+	//  At the end of the day, though?  Honestly, import cycle breaking.  This was not the first choice.
+	// An implementation which wraps the schemadmt package to make it fit the schema interfaces was the first choice
+	//  because it would've saved a *lot* of work (it would've removed the need for this compiler system entirely, among other things);
+	//  but that doesn't fly, because the dmt types have to implement schema.Type, and that interface refers to yet more schema.* types.
+	//  And that would make an import cycle if we tried to put types wrapping the dmt types into the schema package.  Whoops.
+	//  So, here we are.
+	//
+	// The decision to split out this Compiler type and all its other related Make* functions
+	//  from the schema package is largely cosmetic; it technically could've been placed in the schema package.
+	//  However, the result of splitting the readable and writable types into packages seemed more readable,
+	//   and gives us more elbow room in the godocs to suggest "you probably shouldn't use these directly".
+	//  Compared to the already-forced dmt package split, having the creation stuff in one package
+	//   and the read-only interfaces in another package just isn't much additional burden.
+
+	// ts gathers all the in-progress types (including anonymous ones),
+	// and is eventually the value we return (if Compile is ultimately successful).
+	// We insert into this blindly as we go, and check everything for consistency at the end;
+	// if those logical checks flunk, we don't allow any reference to it to escape.
+	// This is nil'd after any Compile, so when we give a reference to it away,
+	// it's immutable from there on out.
+	ts *TypeSystem
+}
+
+func (c *Compiler) Init() {
+	c.ts = &TypeSystem{
+		map[schema.TypeReference]schema.Type{},
+		nil,
+	}
+}
+
+func (c *Compiler) Compile() (schema.TypeSystem, error) {
+	panic("TODO")
+}
+
+func (c *Compiler) addType(t schema.Type) {
+	c.mustHaveNameFree(t.Name())
+	c.ts.types[schema.TypeReference(t.Name())] = t
+	c.ts.list = append(c.ts.list, t)
+}
+func (c *Compiler) addAnonType(t schema.Type) {
+	c.ts.types[schema.TypeReference(t.Name())] = t // FIXME it's... probably a bug that the schema.Type.Name() method doesn't return a TypeReference.  Yeah, it definitely is.  TypeMap and TypeList should have their own name field internally be TypeReference, too, because it's true.  wonder if we should have separate methods on the schema.Type interface for this.  would probably be a usability trap to do so, though (too many user printfs would use the Name function and get blanks and be surprised).
+}
+
+func (c *Compiler) mustHaveNameFree(name schema.TypeName) {
+	if _, exists := c.ts.types[schema.TypeReference(name)]; exists {
+		panic(fmt.Errorf("type name %q already used", name))
+	}
+}
+
+func (c *Compiler) TypeBool(name schema.TypeName) {
+	c.addType(&TypeBool{c.ts, name})
+}
+
+func (c *Compiler) TypeString(name schema.TypeName) {
+	c.addType(&TypeString{c.ts, name})
+}
+
+func (c *Compiler) TypeBytes(name schema.TypeName) {
+	c.addType(&TypeBytes{c.ts, name})
+}
+
+func (c *Compiler) TypeInt(name schema.TypeName) {
+	c.addType(&TypeInt{c.ts, name})
+}
+
+func (c *Compiler) TypeFloat(name schema.TypeName) {
+	c.addType(&TypeFloat{c.ts, name})
+}
+
+func (c *Compiler) TypeLink(name schema.TypeName, expectedTypeRef schema.TypeName) {
+	c.addType(&TypeLink{c.ts, name, expectedTypeRef})
+}
+
+func (c *Compiler) TypeStruct(name schema.TypeName, fields structFieldList, rstrat StructRepresentation) {
+	t := TypeStruct{
+		ts:        c.ts,
+		name:      name,
+		fields:    fields.x, // it's safe to take this directly because the carrier type means a reference to this slice has never been exported.
+		fieldsMap: make(map[StructFieldName]*StructField, len(fields.x)),
+		rstrat:    rstrat,
+	}
+	c.addType(&t)
+	for i, f := range fields.x {
+		// duplicate names are rejected with a *panic* here because we expect these to already be unique (if this data is coming from the dmt, these were map keys there).
+		if _, exists := t.fieldsMap[f.name]; exists {
+			panic(fmt.Errorf("type %q already has field named %q", t.name, f.name))
+		}
+		t.fieldsMap[f.name] = &fields.x[i]
+		fields.x[i].parent = &t
+	}
+}
+
+// structFieldList is a carrier type that just wraps a slice reference.
+// It is used so we can let code outside this package hold a value of this type without letting the slice become mutable.
+type structFieldList struct {
+	x []StructField
+}
+
+func MakeStructFieldList(fields ...StructField) structFieldList {
+	return structFieldList{fields}
+}
+func MakeStructField(name StructFieldName, typ schema.TypeReference, optional, nullable bool) StructField {
+	return StructField{nil, name, typ, optional, nullable}
+}
+
+func MakeStructRepresentation_Map(fieldDetails ...StructRepresentation_Map_FieldDetailsEntry) StructRepresentation {
+	rstrat := StructRepresentation_Map{nil, make(map[StructFieldName]StructRepresentation_Map_FieldDetails, len(fieldDetails))}
+	for _, fd := range fieldDetails {
+		if _, exists := rstrat.fieldDetails[fd.FieldName]; exists {
+			panic(fmt.Errorf("field name %q duplicated", fd.FieldName))
+		}
+		rstrat.fieldDetails[fd.FieldName] = fd.Details
+	}
+	return rstrat
+}
+
+// StructRepresentation_Map_FieldDetailsEntry is a carrier type that associates a field name
+// with field detail information that's appropriate to a map representation strategy for a struct.
+// It is used to feed data to MakeStructRepresentation_Map in so that that method can build a map
+// without exposing a reference to it in a way that would make that map mutable.
+type StructRepresentation_Map_FieldDetailsEntry struct {
+	FieldName StructFieldName
+	Details   StructRepresentation_Map_FieldDetails
+}
+
+func (c *Compiler) TypeMap(name schema.TypeName, keyTypeRef schema.TypeName, valueTypeRef schema.TypeReference, valueNullable bool) {
+	c.addType(&TypeMap{c.ts, name, keyTypeRef, valueTypeRef, valueNullable})
+}
+
+func (c *Compiler) TypeList(name schema.TypeName, valueTypeRef schema.TypeReference, valueNullable bool) {
+	c.addType(&TypeList{c.ts, name, valueTypeRef, valueNullable})
+}
+
+func (c *Compiler) TypeUnion(name schema.TypeName, members unionMemberList, rstrat UnionRepresentation) {
+	t := TypeUnion{
+		ts:      c.ts,
+		name:    name,
+		members: members.x, // it's safe to take this directly because the carrier type means a reference to this slice has never been exported.
+		rstrat:  rstrat,
+	}
+	c.addType(&t)
+	// note! duplicate member names *not* rejected at this moment -- that's a job for the validation phase.
+	//  this is an interesting contrast to how when buildings struct, dupe field names may be rejected proactively:
+	//   the difference is, member names were a list in the dmt form too, so it's important we format a nice error rather than panic if there was invalid data there.
+}
+
+// unionMemberList is a carrier type that just wraps a slice reference.
+// It is used so we can let code outside this package hold a value of this type without letting the slice become mutable.
+type unionMemberList struct {
+	x []schema.TypeName
+}
+
+func MakeUnionMemberList(members ...schema.TypeName) unionMemberList {
+	return unionMemberList{members}
+}
+
+func MakeUnionRepresentation_Keyed(discriminantTable unionDiscriminantStringTable) UnionRepresentation {
+	return &UnionRepresentation_Keyed{nil, discriminantTable.x}
+}
+
+// unionMemberList is a carrier type that just wraps a map reference.
+// It is used so we can let code outside this package hold a value of this type without letting the map become mutable.
+type unionDiscriminantStringTable struct {
+	x map[string]schema.TypeName
+}
+
+func MakeUnionDiscriminantStringTable(entries ...UnionDiscriminantStringEntry) unionDiscriminantStringTable {
+	x := make(map[string]schema.TypeName, len(entries))
+	for _, y := range entries {
+		if _, exists := x[y.Discriminant]; exists {
+			panic(fmt.Errorf("discriminant string %q duplicated", y.Discriminant))
+		}
+		x[y.Discriminant] = y.Member
+	}
+	return unionDiscriminantStringTable{x}
+}
+
+// UnionRepresentation_DiscriminantStringEntry is a carrier type that associates a string with a TypeName.
+// It is used to feed data to several of the union representation constructors so that those functions
+// can build their results without exposing a reference to a map in a way that would make that map mutable.
+type UnionDiscriminantStringEntry struct {
+	Discriminant string
+	Member       schema.TypeName
+}

schema/compiler/rules.go

+package compiler

schema/compiler/type_bool.go

+package compiler
+
+import (
+	"github.com/ipld/go-ipld-prime"
+	"github.com/ipld/go-ipld-prime/schema"
+)
+
+type TypeBool struct {
+	ts   *TypeSystem
+	name schema.TypeName
+}
+
+// -- schema.Type interface satisfaction -->
+
+var _ schema.Type = (*TypeBool)(nil)
+
+func (t *TypeBool) TypeSystem() schema.TypeSystem {
+	return t.ts
+}
+
+func (TypeBool) TypeKind() schema.TypeKind {
+	return schema.TypeKind_Bool
+}
+
+func (t *TypeBool) Name() schema.TypeName {
+	return t.name
+}
+
+func (t TypeBool) RepresentationBehavior() ipld.Kind {
+	return ipld.Kind_Bool
+}

schema/compiler/type_bytes.go

+package compiler
+
+import (
+	"github.com/ipld/go-ipld-prime"
+	"github.com/ipld/go-ipld-prime/schema"
+)
+
+type TypeBytes struct {
+	ts   *TypeSystem
+	name schema.TypeName
+}
+
+// -- schema.Type interface satisfaction -->
+
+var _ schema.Type = (*TypeBytes)(nil)
+
+func (t *TypeBytes) TypeSystem() schema.TypeSystem {
+	return t.ts
+}
+
+func (TypeBytes) TypeKind() schema.TypeKind {
+	return schema.TypeKind_Bytes
+}
+
+func (t *TypeBytes) Name() schema.TypeName {
+	return t.name
+}
+
+func (t TypeBytes) RepresentationBehavior() ipld.Kind {
+	return ipld.Kind_Bytes
+}

schema/compiler/type_float.go

+package compiler
+
+import (
+	"github.com/ipld/go-ipld-prime"
+	"github.com/ipld/go-ipld-prime/schema"
+)
+
+type TypeFloat struct {
+	ts   *TypeSystem
+	name schema.TypeName
+}
+
+// -- schema.Type interface satisfaction -->
+
+var _ schema.Type = (*TypeFloat)(nil)
+
+func (t *TypeFloat) TypeSystem() schema.TypeSystem {
+	return t.ts
+}
+
+func (TypeFloat) TypeKind() schema.TypeKind {
+	return schema.TypeKind_Float
+}
+
+func (t *TypeFloat) Name() schema.TypeName {
+	return t.name
+}
+
+func (t TypeFloat) RepresentationBehavior() ipld.Kind {
+	return ipld.Kind_Float
+}

schema/compiler/type_int.go

+package compiler
+
+import (
+	"github.com/ipld/go-ipld-prime"
+	"github.com/ipld/go-ipld-prime/schema"
+)
+
+type TypeInt struct {
+	ts   *TypeSystem
+	name schema.TypeName
+}
+
+// -- schema.Type interface satisfaction -->
+
+var _ schema.Type = (*TypeInt)(nil)
+
+func (t *TypeInt) TypeSystem() schema.TypeSystem {
+	return t.ts
+}
+
+func (TypeInt) TypeKind() schema.TypeKind {
+	return schema.TypeKind_Int
+}
+
+func (t *TypeInt) Name() schema.TypeName {
+	return t.name
+}
+
+func (t TypeInt) RepresentationBehavior() ipld.Kind {
+	return ipld.Kind_Int
+}

schema/compiler/type_link.go

+package compiler
+
+import (
+	"github.com/ipld/go-ipld-prime"
+	"github.com/ipld/go-ipld-prime/schema"
+)
+
+type TypeLink struct {
+	ts              *TypeSystem
+	name            schema.TypeName
+	expectedTypeRef schema.TypeName // can be empty
+}
+
+// -- schema.Type interface satisfaction -->
+
+var _ schema.Type = (*TypeLink)(nil)
+
+func (t *TypeLink) TypeSystem() schema.TypeSystem {
+	return t.ts
+}
+
+func (TypeLink) TypeKind() schema.TypeKind {
+	return schema.TypeKind_Link
+}
+
+func (t *TypeLink) Name() schema.TypeName {
+	return t.name
+}
+
+func (t TypeLink) RepresentationBehavior() ipld.Kind {
+	return ipld.Kind_Link
+}
+
+// -- specific to TypeLink -->
+
+// HasExpectedType returns true if the link has a hint about the type it references.
+func (t *TypeLink) HasExpectedType() bool {
+	return t.expectedTypeRef != ""
+}
+
+// ExpectedType returns the type which is expected for the node on the other side of the link.
+// Nil is returned if there is no information about the expected type
+// (which may be interpreted as "any").
+func (t *TypeLink) ExpectedType() schema.Type {
+	if !t.HasExpectedType() {
+		return nil
+	}
+	return t.ts.types[schema.TypeReference(t.expectedTypeRef)]
+}

schema/compiler/type_list.go

+package compiler
+
+import (
+	"github.com/ipld/go-ipld-prime"
+	"github.com/ipld/go-ipld-prime/schema"
+)
+
+type TypeList struct {
+	ts            *TypeSystem
+	name          schema.TypeName
+	valueTypeRef  schema.TypeReference
+	valueNullable bool
+}
+
+// -- schema.Type interface satisfaction -->
+
+var _ schema.Type = (*TypeList)(nil)
+
+func (t *TypeList) TypeSystem() schema.TypeSystem {
+	return t.ts
+}
+
+func (TypeList) TypeKind() schema.TypeKind {
+	return schema.TypeKind_List
+}
+
+func (t *TypeList) Name() schema.TypeName {
+	return t.name
+}
+
+func (t TypeList) RepresentationBehavior() ipld.Kind {
+	return ipld.Kind_List
+}
+
+// -- specific to TypeList -->
+
+// ValueType returns the Type of the list values.
+func (t *TypeList) ValueType() schema.Type {
+	return t.ts.types[schema.TypeReference(t.valueTypeRef)]
+}
+
+// ValueIsNullable returns a bool describing if the list values are permitted to be null.
+func (t *TypeList) ValueIsNullable() bool {
+	return t.valueNullable
+}

schema/compiler/type_map.go

+package compiler
+
+import (
+	"github.com/ipld/go-ipld-prime"
+	"github.com/ipld/go-ipld-prime/schema"
+)
+
+type TypeMap struct {
+	ts            *TypeSystem
+	name          schema.TypeName
+	keyTypeRef    schema.TypeName // is a TypeName and not a TypeReference because it can't be an anon.
+	valueTypeRef  schema.TypeReference
+	valueNullable bool
+}
+
+// -- schema.Type interface satisfaction -->
+
+var _ schema.Type = (*TypeMap)(nil)
+
+func (t *TypeMap) TypeSystem() schema.TypeSystem {
+	return t.ts
+}
+
+func (TypeMap) TypeKind() schema.TypeKind {
+	return schema.TypeKind_Map
+}
+
+func (t *TypeMap) Name() schema.TypeName {
+	return t.name
+}
+
+func (t TypeMap) RepresentationBehavior() ipld.Kind {
+	return ipld.Kind_Map
+}
+
+// -- specific to TypeMap -->
+
+// KeyType returns the Type of the map keys.
+//
+// Note that map keys must always be some type which is representable as a
+// string in the IPLD Data Model (e.g. any string type is valid,
+// but something with enum typekind and a string representation is also valid,
+// and a struct typekind with a representation that has a string kind is also valid, etc).
+func (t *TypeMap) KeyType() schema.Type {
+	return t.ts.types[schema.TypeReference(t.keyTypeRef)]
+}
+
+// ValueType returns the Type of the map values.
+func (t *TypeMap) ValueType() schema.Type {
+	return t.ts.types[schema.TypeReference(t.valueTypeRef)]
+}
+
+// ValueIsNullable returns a bool describing if the map values are permitted to be null.
+func (t *TypeMap) ValueIsNullable() bool {
+	return t.valueNullable
+}

schema/compiler/type_string.go

+package compiler
+
+import (
+	"github.com/ipld/go-ipld-prime"
+	"github.com/ipld/go-ipld-prime/schema"
+)
+
+type TypeString struct {
+	ts   *TypeSystem
+	name schema.TypeName
+}
+
+// -- schema.Type interface satisfaction -->
+
+var _ schema.Type = (*TypeString)(nil)
+
+func (t *TypeString) TypeSystem() schema.TypeSystem {
+	return t.ts
+}
+
+func (TypeString) TypeKind() schema.TypeKind {
+	return schema.TypeKind_String
+}
+
+func (t *TypeString) Name() schema.TypeName {
+	return t.name
+}
+
+func (t TypeString) RepresentationBehavior() ipld.Kind {
+	return ipld.Kind_String
+}

schema/compiler/type_struct.go

+package compiler
+
+import (
+	"github.com/ipld/go-ipld-prime"
+	"github.com/ipld/go-ipld-prime/schema"
+)
+
+type TypeStruct struct {
+	ts        *TypeSystem
+	name      schema.TypeName
+	fields    []StructField
+	fieldsMap map[StructFieldName]*StructField // same content, indexed for lookup.
+	rstrat    StructRepresentation
+}
+
+type StructField struct {
+	parent   *TypeStruct // a pointer back up is used so we can provide the method that gives a reified type instead of just the TypeReference.
+	name     StructFieldName
+	typeRef  schema.TypeReference
+	optional bool
+	nullable bool
+}
+
+type StructFieldName string
+
+type StructRepresentation interface{ _StructRepresentation() }
+
+func (StructRepresentation_Map) _StructRepresentation()         {}
+func (StructRepresentation_Tuple) _StructRepresentation()       {}
+func (StructRepresentation_Stringpairs) _StructRepresentation() {}
+func (StructRepresentation_Stringjoin) _StructRepresentation()  {}
+func (StructRepresentation_Listpairs) _StructRepresentation()   {}
+
+type StructRepresentation_Map struct {
+	parent       *TypeStruct // this one needs a pointer back up to figure out its defaults.
+	fieldDetails map[StructFieldName]StructRepresentation_Map_FieldDetails
+}
+type StructRepresentation_Map_FieldDetails struct {
+	Rename   string
+	Implicit interface{}
+}
+
+type StructRepresentation_Tuple struct {
+	fieldOrder []StructFieldName
+}
+
+type StructRepresentation_Stringpairs struct {
+	innerDelim string
+	entryDelim string
+}
+
+type StructRepresentation_Stringjoin struct {
+	delim      string
+	fieldOrder []StructFieldName
+}
+
+type StructRepresentation_Listpairs struct {
+}
+
+// -- schema.Type interface satisfaction -->
+
+var _ schema.Type = (*TypeStruct)(nil)
+
+func (t *TypeStruct) TypeSystem() schema.TypeSystem {
+	return t.ts
+}
+
+func (TypeStruct) TypeKind() schema.TypeKind {
+	return schema.TypeKind_Struct
+}
+
+func (t *TypeStruct) Name() schema.TypeName {
+	return t.name
+}
+
+func (t TypeStruct) RepresentationBehavior() ipld.Kind {
+	switch t.rstrat.(type) {
+	case StructRepresentation_Map:
+		return ipld.Kind_Map
+	case StructRepresentation_Tuple:
+		return ipld.Kind_List
+	case StructRepresentation_Stringpairs:
+		return ipld.Kind_String
+	case StructRepresentation_Stringjoin:
+		return ipld.Kind_String
+	case StructRepresentation_Listpairs:
+		return ipld.Kind_List
+	default:
+		panic("unreachable")
+	}
+}
+
+// -- specific to TypeStruct -->
+
+// Fields returns a slice of descriptions of the object's fields.
+func (t *TypeStruct) Fields() []StructField {
+	// A defensive copy to preserve immutability is performed.
+	a := make([]StructField, len(t.fields))
+	copy(a, t.fields)
+	return a
+}
+
+// Field looks up a StructField by name, or returns nil if no such field.
+func (t *TypeStruct) Field(name string) *StructField {
+	return t.fieldsMap[StructFieldName(name)]
+}
+
+// Parent returns the type information that this field describes a part of.
+//
+// While in many cases, you may know the parent already from context,
+// there may still be situations where want to pass around a field and
+// not need to continue passing down the parent type with it; this method
+// helps your code be less redundant in such a situation.
+// (You'll find this useful for looking up any rename directives, for example,
+// when holding onto a field, since that requires looking up information from
+// the representation strategy, which is a property of the type as a whole.)
+func (f *StructField) Parent() *TypeStruct { return f.parent }
+
+// Name returns the string name of this field.  The name is the string that
+// will be used as a map key if the structure this field is a member of is
+// serialized as a map representation.
+func (f *StructField) Name() StructFieldName { return f.name }
+
+// Type returns the Type of this field's value.  Note the field may
+// also be unset if it is either Optional or Nullable.
+func (f *StructField) Type() schema.Type { return f.parent.ts.types[f.typeRef] }
+
+// IsOptional returns true if the field is allowed to be absent from the object.
+// If IsOptional is false, the field may be absent from the serial representation
+// of the object entirely.
+//
+// Note being optional is different than saying the value is permitted to be null!
+// A field may be both nullable and optional simultaneously, or either, or neither.
+func (f *StructField) IsOptional() bool { return f.optional }
+
+// IsNullable returns true if the field value is allowed to be null.
+//
+// If is Nullable is false, note that it's still possible that the field value
+// will be absent if the field is Optional!  Being nullable is unrelated to
+// whether the field's presence is optional as a whole.
+//
+// Note that a field may be both nullable and optional simultaneously,
+// or either, or neither.
+func (f *StructField) IsNullable() bool { return f.nullable }
+
+// IsMaybe returns true if the field value is allowed to be either null or absent.
+//
+// This is a simple "or" of the two properties,
+// but this method is a shorthand that turns out useful often.
+func (f *StructField) IsMaybe() bool { return f.IsNullable() || f.IsOptional() }
+
+func (t *TypeStruct) RepresentationStrategy() StructRepresentation {
+	return t.rstrat
+}
+
+// GetFieldKey returns the string that should be the key when serializing this field.
+// For some fields, it's the same as the field name; for others, a rename directive may provide a different value.
+func (r StructRepresentation_Map) GetFieldKey(field StructField) string {
+	details, exists := r.fieldDetails[field.name]
+	if !exists {
+		return string(field.Name())
+	}
+	if details.Rename == "" {
+		return string(field.Name())
+	}
+	return details.Rename
+}
+
+func (r StructRepresentation_Stringjoin) GetJoinDelim() string {
+	return r.delim
+}

schema/compiler/type_union.go

+package compiler
+
+import (
+	"github.com/ipld/go-ipld-prime"
+	"github.com/ipld/go-ipld-prime/schema"
+)
+
+type TypeUnion struct {
+	ts      *TypeSystem
+	name    schema.TypeName
+	members []schema.TypeName // all of these are TypeName because we ruled by fiat that unions are not allowed to use anon types (for sheer syntactic complexity boundary reasons).
+	rstrat  UnionRepresentation
+}
+
+type UnionRepresentation interface{ _UnionRepresentation() }
+
+func (UnionRepresentation_Keyed) _UnionRepresentation()        {}
+func (UnionRepresentation_Kinded) _UnionRepresentation()       {}
+func (UnionRepresentation_Envelope) _UnionRepresentation()     {}
+func (UnionRepresentation_Inline) _UnionRepresentation()       {}
+func (UnionRepresentation_StringPrefix) _UnionRepresentation() {}
+func (UnionRepresentation_BytePrefix) _UnionRepresentation()   {}
+
+type UnionRepresentation_Keyed struct {
+	ts                *TypeSystem
+	discriminantTable map[string]schema.TypeName
+}
+type UnionRepresentation_Kinded struct {
+	ts                *TypeSystem
+	discriminantTable map[ipld.Kind]schema.TypeName
+}
+type UnionRepresentation_Envelope struct {
+	ts                *TypeSystem
+	discriminantKey   string
+	contentKey        string
+	discriminantTable map[string]schema.TypeName
+}
+type UnionRepresentation_Inline struct {
+	ts                *TypeSystem
+	discriminantKey   string
+	discriminantTable map[string]schema.TypeName
+}
+type UnionRepresentation_StringPrefix struct {
+	ts                *TypeSystem
+	discriminantTable map[string]schema.TypeName
+}
+type UnionRepresentation_BytePrefix struct {
+	ts                *TypeSystem
+	discriminantTable map[string]schema.TypeName
+}
+
+// -- schema.Type interface satisfaction -->
+
+var _ schema.Type = (*TypeUnion)(nil)
+
+func (t *TypeUnion) TypeSystem() schema.TypeSystem {
+	return t.ts
+}
+
+func (TypeUnion) TypeKind() schema.TypeKind {
+	return schema.TypeKind_Union
+}
+
+func (t *TypeUnion) Name() schema.TypeName {
+	return t.name
+}
+
+func (t *TypeUnion) RepresentationBehavior() ipld.Kind {
+	switch t.rstrat.(type) {
+	case UnionRepresentation_Keyed:
+		return ipld.Kind_Map
+	case UnionRepresentation_Kinded:
+		return ipld.Kind_Invalid // you can't know with this one, until you see the value (and thus can see its inhabitant's behavior)!
+	case UnionRepresentation_Envelope:
+		return ipld.Kind_Map
+	case UnionRepresentation_Inline:
+		return ipld.Kind_Map
+	case UnionRepresentation_StringPrefix:
+		return ipld.Kind_String
+	case UnionRepresentation_BytePrefix:
+		return ipld.Kind_Bytes
+	default:
+		panic("unreachable")
+	}
+}
+
+// -- specific to TypeUnion -->
+
+func (t *TypeUnion) RepresentationStrategy() UnionRepresentation {
+	return t.rstrat
+}
+
+// GetDiscriminantForType looks up the discriminant key for the given type.
+// It panics if the given type is not a member of this union.
+func (r UnionRepresentation_Keyed) GetDiscriminantForType(t schema.Type) string {
+	if t.TypeSystem() != r.ts {
+		panic("that type isn't even from the same universe!")
+	}
+	for k, v := range r.discriminantTable {
+		if v == t.Name() {
+			return k
+		}
+	}
+	panic("that type isn't a member of this union")
+}
+
+// GetMember returns the type info for the member that would be indicated by the given kind,
+// or may return nil if that kind is not mapped to a member of this union.
+func (r UnionRepresentation_Kinded) GetMember(k ipld.Kind) schema.Type {
+	if tn, exists := r.discriminantTable[k]; exists {
+		return r.ts.types[schema.TypeReference(tn)]
+	}
+	return nil
+}

schema/compiler/typesystem.go

+package compiler
+
+import (
+	"github.com/ipld/go-ipld-prime/schema"
+)
+
+type TypeSystem struct {
+	// Mind the key type here: TypeReference, not TypeName.
+	// The key might be a computed anon "name" which is not actually a valid type name itself.
+	types map[schema.TypeReference]schema.Type
+
+	// List of types, retained in the original order they were specified,
+	// including only those which are named (not any computed anonymous types).
+	// This is kept so we can do any listing in the order the user expects,
+	// report any errors during rule validation in the same order as the input, etc.
+	list []schema.Type
+}

schema/dmt/compile.go

+package schemadmt
+
+import (
+	"github.com/ipld/go-ipld-prime/schema"
+	"github.com/ipld/go-ipld-prime/schema/compiler"
+)
+
+// This code is broken up into a bunch of individual 'compile' methods,
+//  each attached to the type that's their input information.
+// However, many of them return distinct concrete types,
+//  and so we've just chained it all together with switch statements;
+//   creating a separate interface per result type seems just not super relevant.
+
+func (schdmt Schema) Compile() (schema.TypeSystem, error) {
+	c := &compiler.Compiler{}
+	typesdmt := schdmt.FieldTypes()
+	for itr := typesdmt.Iterator(); !itr.Done(); {
+		tn, t := itr.Next()
+		switch t2 := t.AsInterface().(type) {
+		case TypeBool:
+			c.TypeBool(schema.TypeName(tn.String()))
+		case TypeString:
+			c.TypeString(schema.TypeName(tn.String()))
+		case TypeBytes:
+			c.TypeBytes(schema.TypeName(tn.String()))
+		case TypeInt:
+			c.TypeInt(schema.TypeName(tn.String()))
+		case TypeFloat:
+			c.TypeFloat(schema.TypeName(tn.String()))
+		case TypeLink:
+			if t2.FieldExpectedType().Exists() {
+				c.TypeLink(schema.TypeName(tn.String()), schema.TypeName(t2.FieldExpectedType().Must().String()))
+			} else {
+				c.TypeLink(schema.TypeName(tn.String()), "")
+			}
+		case TypeMap:
+			c.TypeMap(
+				schema.TypeName(tn.String()),
+				schema.TypeName(t2.FieldKeyType().String()),
+				t2.FieldValueType().TypeReference(),
+				t2.FieldValueNullable().Bool(),
+			)
+			// If the field typeReference is TypeDefnInline, that needs a chance to take additional action.
+			t2.FieldValueType().compile(c)
+		case TypeList:
+			c.TypeList(
+				schema.TypeName(tn.String()),
+				t2.FieldValueType().TypeReference(),
+				t2.FieldValueNullable().Bool(),
+			)
+			// If the field typeReference is TypeDefnInline, that needs a chance to take additional action.
+			t2.FieldValueType().compile(c)
+		case TypeStruct:
+			// Flip fields info from DMT to compiler argument format.
+			fields := make([]compiler.StructField, t2.FieldFields().Length())
+			for itr := t2.FieldFields().Iterator(); !itr.Done(); {
+				fname, fdmt := itr.Next()
+				fields = append(fields, compiler.MakeStructField(
+					compiler.StructFieldName(fname.String()),
+					fdmt.FieldType().TypeReference(),
+					fdmt.FieldOptional().Bool(),
+					fdmt.FieldNullable().Bool(),
+				))
+				// If the field typeReference is TypeDefnInline, that needs a chance to take additional action.
+				fdmt.FieldType().compile(c)
+			}
+			// Flip the representaton strategy DMT to compiler argument format.
+			rstrat := func() compiler.StructRepresentation {
+				switch r := t2.FieldRepresentation().AsInterface().(type) {
+				case StructRepresentation_Map:
+					return r.compile()
+				case StructRepresentation_Tuple:
+					return r.compile()
+				case StructRepresentation_Stringpairs:
+					return r.compile()
+				case StructRepresentation_Stringjoin:
+					return r.compile()
+				case StructRepresentation_Listpairs:
+					return r.compile()
+				default:
+					panic("unreachable")
+				}
+			}()
+			// Feed it all into the compiler.
+			c.TypeStruct(
+				schema.TypeName(tn.String()),
+				compiler.MakeStructFieldList(fields...),
+				rstrat,
+			)
+		case TypeUnion:
+			// Flip members info from DMT to compiler argument format.
+			members := make([]schema.TypeName, t2.FieldMembers().Length())
+			for itr := t2.FieldMembers().Iterator(); !itr.Done(); {
+				_, memberName := itr.Next()
+				members = append(members, schema.TypeName(memberName.String()))
+				// n.b. no need to check for TypeDefnInline here, because schemas don't allow those in union defns.
+			}
+			// Flip the representaton strategy DMT to compiler argument format.
+			rstrat := func() compiler.UnionRepresentation {
+				switch r := t2.FieldRepresentation().AsInterface().(type) {
+				case UnionRepresentation_Keyed:
+					return r.compile()
+				case UnionRepresentation_Kinded:
+					return r.compile()
+				case UnionRepresentation_Envelope:
+					return r.compile()
+				case UnionRepresentation_Inline:
+					return r.compile()
+				case UnionRepresentation_StringPrefix:
+					return r.compile()
+				case UnionRepresentation_BytePrefix:
+					return r.compile()
+				default:
+					panic("unreachable")
+				}
+			}()
+			// Feed it all into the compiler.
+			c.TypeUnion(
+				schema.TypeName(tn.String()),
+				compiler.MakeUnionMemberList(members...),
+				rstrat,
+			)
+		case TypeEnum:
+			panic("TODO")
+		case TypeCopy:
+			panic("no support for 'copy' types.  I might want to reneg on whether these are even part of the schema dmt.")
+		default:
+			panic("unreachable")
+		}
+	}
+	return c.Compile()
+}
+
+// If the typeReference is TypeDefnInline, create the anonymous type and feed it to the compiler.
+// It's fine if anonymous type has been seen before; we let dedup of that be handled by the compiler.
+func (dmt TypeNameOrInlineDefn) compile(c *compiler.Compiler) {
+	switch dmt.AsInterface().(type) {
+	case TypeDefnInline:
+		panic("nyi") // TODO this needs to engage in anonymous type spawning.
+	}
+}
+
+func (dmt StructRepresentation_Map) compile() compiler.StructRepresentation {
+	if !dmt.FieldFields().Exists() {
+		return compiler.MakeStructRepresentation_Map()
+	}
+	fields := make([]compiler.StructRepresentation_Map_FieldDetailsEntry, dmt.FieldFields().Must().Length())
+	for itr := dmt.FieldFields().Must().Iterator(); !itr.Done(); {
+		fn, det := itr.Next()
+		fields = append(fields, compiler.StructRepresentation_Map_FieldDetailsEntry{
+			FieldName: compiler.StructFieldName(fn.String()),
+			Details: compiler.StructRepresentation_Map_FieldDetails{
+				Rename: func() string {
+					if det.FieldRename().Exists() {
+						return det.FieldRename().Must().String()
+					}
+					return ""
+				}(),
+				Implicit: nil, // TODO
+			},
+		})
+	}
+	return compiler.MakeStructRepresentation_Map(fields...)
+}
+
+func (dmt StructRepresentation_Tuple) compile() compiler.StructRepresentation {
+	panic("TODO")
+}
+
+func (dmt StructRepresentation_Stringpairs) compile() compiler.StructRepresentation {
+	panic("TODO")
+}
+
+func (dmt StructRepresentation_Stringjoin) compile() compiler.StructRepresentation {
+	panic("TODO")
+}
+
+func (dmt StructRepresentation_Listpairs) compile() compiler.StructRepresentation {
+	panic("TODO")
+}
+
+func (dmt UnionRepresentation_Keyed) compile() compiler.UnionRepresentation {
+	ents := make([]compiler.UnionDiscriminantStringEntry, 0, dmt.Length())
+	for itr := dmt.Iterator(); !itr.Done(); {
+		k, v := itr.Next()
+		ents = append(ents, compiler.UnionDiscriminantStringEntry{k.String(), schema.TypeName(v.String())})
+	}
+	return compiler.MakeUnionRepresentation_Keyed(compiler.MakeUnionDiscriminantStringTable(ents...))
+}
+
+func (dmt UnionRepresentation_Kinded) compile() compiler.UnionRepresentation {
+	panic("TODO")
+}
+
+func (dmt UnionRepresentation_Envelope) compile() compiler.UnionRepresentation {
+	panic("TODO")
+}
+
+func (dmt UnionRepresentation_Inline) compile() compiler.UnionRepresentation {
+	panic("TODO")
+}
+
+func (dmt UnionRepresentation_StringPrefix) compile() compiler.UnionRepresentation {
+	panic("TODO")
+}
+
+func (dmt UnionRepresentation_BytePrefix) compile() compiler.UnionRepresentation {
+	panic("TODO")
+}

schema/dmt/typereference.go

 package schemadmt
 
-import "fmt"
+import (
+	"fmt"
 
-// TypeReference is a string that's either a TypeName or a computed string from an InlineDefn.
-// This string is often useful as a map key.
-//
-// The computed string for an InlineDefn happens to match the IPLD Schema DSL syntax,
-// but it would be very odd for any code to depend on that detail.
-type TypeReference string
+	"github.com/ipld/go-ipld-prime/schema"
+)
 
-func (x TypeNameOrInlineDefn) TypeReference() TypeReference {
+func (x TypeNameOrInlineDefn) TypeReference() schema.TypeReference {
 	switch y := x.AsInterface().(type) {
 	case TypeName:
-		return TypeReference(y.String())
+		return schema.TypeReference(y.String())
 	case TypeDefnInline:
 		return y.TypeReference()
 	default:
@@ -20,23 +17,23 @@ func (x TypeNameOrInlineDefn) TypeReference() TypeReference {
 	}
 }
 
-func (x TypeDefnInline) TypeReference() TypeReference {
+func (x TypeDefnInline) TypeReference() schema.TypeReference {
 	switch y := x.AsInterface().(type) {
 	case TypeMap:
 		if y.FieldValueNullable().Bool() {
-			return TypeReference(fmt.Sprintf("{%s : nullable %s}", y.FieldKeyType(), y.FieldValueType().TypeReference()))
+			return schema.TypeReference(fmt.Sprintf("{%s : nullable %s}", y.FieldKeyType(), y.FieldValueType().TypeReference()))
 		}
-		return TypeReference(fmt.Sprintf("{%s:%s}", y.FieldKeyType(), y.FieldValueType().TypeReference()))
+		return schema.TypeReference(fmt.Sprintf("{%s:%s}", y.FieldKeyType(), y.FieldValueType().TypeReference()))
 	case TypeList:
 		if y.FieldValueNullable().Bool() {
-			return TypeReference(fmt.Sprintf("[nullable %s]", y.FieldValueType().TypeReference()))
+			return schema.TypeReference(fmt.Sprintf("[nullable %s]", y.FieldValueType().TypeReference()))
 		}
-		return TypeReference(fmt.Sprintf("[%s]", y.FieldValueType().TypeReference()))
+		return schema.TypeReference(fmt.Sprintf("[%s]", y.FieldValueType().TypeReference()))
 	default:
 		panic("unreachable")
 	}
 }
 
-func (x TypeName) TypeReference() TypeReference {
-	return TypeReference(x.String())
+func (x TypeName) TypeReference() schema.TypeReference {
+	return schema.TypeReference(x.String())
 }

schema/tmpBuilders.go

-package schema
-
-import (
-	"fmt"
-
-	"github.com/ipld/go-ipld-prime"
-)
-
-// Everything in this file is __a temporary hack__ and will be __removed__.
-//
-// These methods will only hang around until more of the "ast" packages are finished;
-// thereafter, building schema.Type and schema.TypeSystem values will only be
-// possible through first constructing a schema AST, and *then* using Reify(),
-// which will validate things correctly, cycle-check, cross-link, etc.
-//
-// (Meanwhile, we're using these methods in the codegen prototypes.)
-
-// These methods use Type objects as parameters when pointing to other things,
-//  but this is... turning out consistently problematic.
-//   Even when we're doing this hacky direct-call doesn't-need-to-be-serializable temp stuff,
-//    as written, this doesn't actually let us express cyclic things viably!
-//   The same initialization questions are also going to come up again when we try to make
-//    concrete values in the output of codegen.
-// Maybe it's actually just a bad idea to have our reified Type types use Type pointers at all.
-//  (I will never get tired of the tongue twisters, evidently.)
-//  I'm not actually using that much, and it's always avoidable (it's trivial to replace with a map lookup bouncing through a 'ts' variable somewhere).
-//  And having the AST gen'd types be... just... the thing... sounds nice.  It could save a lot of work.
-//   (It would mean the golang types don't tell you whether the values have been checked for global properties or not, but, eh.)
-//   (It's not really compatible with "Prototype and Type are the same thing for codegen'd stuff", either (or, we need more interfaces, and to *really* lean into them), but maybe that's okay.)
-
-func SpawnString(name TypeName) *TypeString {
-	return &TypeString{typeBase{name, nil}}
-}
-
-func SpawnBool(name TypeName) *TypeBool {
-	return &TypeBool{typeBase{name, nil}}
-}
-
-func SpawnInt(name TypeName) *TypeInt {
-	return &TypeInt{typeBase{name, nil}}
-}
-
-func SpawnFloat(name TypeName) *TypeFloat {
-	return &TypeFloat{typeBase{name, nil}}
-}
-
-func SpawnBytes(name TypeName) *TypeBytes {
-	return &TypeBytes{typeBase{name, nil}}
-}
-
-func SpawnLink(name TypeName) *TypeLink {
-	return &TypeLink{typeBase{name, nil}, "", false}
-}
-
-func SpawnLinkReference(name TypeName, pointsTo TypeName) *TypeLink {
-	return &TypeLink{typeBase{name, nil}, pointsTo, true}
-}
-
-func SpawnList(name TypeName, valueType TypeName, nullable bool) *TypeList {
-	return &TypeList{typeBase{name, nil}, false, valueType, nullable}
-}
-
-func SpawnMap(name TypeName, keyType TypeName, valueType TypeName, nullable bool) *TypeMap {
-	return &TypeMap{typeBase{name, nil}, false, keyType, valueType, nullable}
-}
-
-func SpawnStruct(name TypeName, fields []StructField, repr StructRepresentation) *TypeStruct {
-	v := &TypeStruct{
-		typeBase{name, nil},
-		fields,
-		make(map[string]StructField, len(fields)),
-		repr,
-	}
-	for i := range fields {
-		fields[i].parent = v
-		v.fieldsMap[fields[i].name] = fields[i]
-	}
-	switch repr.(type) {
-	case StructRepresentation_Stringjoin:
-		for _, f := range fields {
-			if f.IsMaybe() {
-				panic("neither nullable nor optional is supported on struct stringjoin representation")
-			}
-		}
-	}
-	return v
-}
-func SpawnStructField(name string, typ TypeName, optional bool, nullable bool) StructField {
-	return StructField{nil /*populated later*/, name, typ, optional, nullable}
-}
-func SpawnStructRepresentationMap(renames map[string]string) StructRepresentation_Map {
-	return StructRepresentation_Map{renames, nil}
-}
-func SpawnStructRepresentationTuple() StructRepresentation_Tuple {
-	return StructRepresentation_Tuple{}
-}
-func SpawnStructRepresentationStringjoin(delim string) StructRepresentation_Stringjoin {
-	return StructRepresentation_Stringjoin{delim}
-}
-
-func SpawnUnion(name TypeName, members []TypeName, repr UnionRepresentation) *TypeUnion {
-	return &TypeUnion{typeBase{name, nil}, members, repr}
-}
-func SpawnUnionRepresentationKeyed(table map[string]TypeName) UnionRepresentation_Keyed {
-	return UnionRepresentation_Keyed{table}
-}
-func SpawnUnionRepresentationKinded(table map[ipld.Kind]TypeName) UnionRepresentation_Kinded {
-	return UnionRepresentation_Kinded{table}
-}
-func SpawnUnionRepresentationStringprefix(delim string, table map[string]TypeName) UnionRepresentation_Stringprefix {
-	return UnionRepresentation_Stringprefix{delim, table}
-}
-
-// The methods relating to TypeSystem are also mutation-heavy and placeholdery.
-
-func (ts *TypeSystem) Init() {
-	ts.namedTypes = make(map[TypeName]Type)
-}
-func (ts *TypeSystem) Accumulate(typ Type) {
-	typ._Type(ts)
-	ts.namedTypes[typ.Name()] = typ
-}
-func (ts TypeSystem) GetTypes() map[TypeName]Type {
-	return ts.namedTypes
-}
-func (ts TypeSystem) TypeByName(n string) Type {
-	return ts.namedTypes[TypeName(n)]
-}
-
-// ValidateGraph checks that all type names referenced are defined.
-//
-// It does not do any other validations of individual type's sensibleness
-// (that should've happened when they were created
-// (although also note many of those validates are NYI,
-// and are roadmapped for after we research self-hosting)).
-func (ts TypeSystem) ValidateGraph() []error {
-	var ee []error
-	for tn, t := range ts.namedTypes {
-		switch t2 := t.(type) {
-		case *TypeBool,
-			*TypeInt,
-			*TypeFloat,
-			*TypeString,
-			*TypeBytes,
-			*TypeEnum:
-			continue // nothing to check: these are leaf nodes and refer to no other types.
-		case *TypeLink:
-			if !t2.hasReferencedType {
-				continue
-			}
-			if _, ok := ts.namedTypes[t2.referencedType]; !ok {
-				ee = append(ee, fmt.Errorf("type %s refers to missing type %s (as link reference type)", tn, t2.referencedType))
-			}
-		case *TypeStruct:
-			for _, f := range t2.fields {
-				if _, ok := ts.namedTypes[f.typ]; !ok {
-					ee = append(ee, fmt.Errorf("type %s refers to missing type %s (in field %s)", tn, f.typ, f.name))
-				}
-			}
-		case *TypeMap:
-			if _, ok := ts.namedTypes[t2.keyType]; !ok {
-				ee = append(ee, fmt.Errorf("type %s refers to missing type %s (as key type)", tn, t2.keyType))
-			}
-			if _, ok := ts.namedTypes[t2.valueType]; !ok {
-				ee = append(ee, fmt.Errorf("type %s refers to missing type %s (as key type)", tn, t2.valueType))
-			}
-		case *TypeList:
-			if _, ok := ts.namedTypes[t2.valueType]; !ok {
-				ee = append(ee, fmt.Errorf("type %s refers to missing type %s (as key type)", tn, t2.valueType))
-			}
-		case *TypeUnion:
-			for _, mn := range t2.members {
-				if _, ok := ts.namedTypes[mn]; !ok {
-					ee = append(ee, fmt.Errorf("type %s refers to missing type %s (as a member)", tn, mn))
-				}
-			}
-		}
-	}
-	return ee
-}

schema/type.go

@@ -4,10 +4,25 @@ import (
 	ipld "github.com/ipld/go-ipld-prime"
 )
 
-type TypeName string // = ast.TypeName
+// TypeName is a string that names a type.
+// TypeName is restricted to UTF-8 numbers and letter, must start with a letter,
+// excludes whitespace, and excludes multiple consecutive underscores.
+//
+// More specifically, the definitions used in https://golang.org/ref/spec#Identifiers
+// apply for defining numbers and letters.  We don't recommend pushing the limits
+// and corner cases in schemas you author, either, however; tooling in other
+// langauges may be made more difficult to use if you do so.
+type TypeName string
 
 func (tn TypeName) String() string { return string(tn) }
 
+// TypeReference is a string that's either a TypeName or a computed string from an InlineDefn.
+// This string is often useful as a map key.
+//
+// The computed string for an InlineDefn happens to match the IPLD Schema DSL syntax,
+// but it would be very odd for any code to depend on that detail.
+type TypeReference string
+
 // typesystem.Type is an union interface; each of the `Type*` concrete types
 // in this package are one of its members.
 //
@@ -27,27 +42,20 @@ func (tn TypeName) String() string { return string(tn) }
 //
 // are all of the kinds of Type.
 //
-// This is a closed union; you can switch upon the above members without
-// including a default case.  The membership is closed by the unexported
-// '_Type' method; you may use the BurntSushi/go-sumtype tool to check
-// your switches for completeness.
-//
-// Many interesting properties of each Type are only defined for that specific
-// type, so it's typical to use a type switch to handle each type of Type.
-// (Your humble author is truly sorry for the word-mash that results from
-// attempting to describe the types that describe the typesystem.Type.)
+// Many typekinds have interesting properties which only defined for that specific typekind,
+// For example, map typekinds have additional type info about their keys and values,
+// while struct types have additional information about their fields.
+// Since none of these are uniformly true of all types, they aren't in this interface,
+// and it's typical to use a type switch to refine to one of the more specific "Type*"
+// interfaces to get those more specific pieces of information
 //
 // For example, to inspect the kind of fields in a struct: you might
 // cast a `Type` interface into `TypeStruct`, and then the `Fields()` on
 // that `TypeStruct` can be inspected.  (`Fields()` isn't defined for any
 // other kind of Type.)
 type Type interface {
-	// Unexported marker method to force the union closed.
-	// Also used to set the internal pointer back to the universe its part of.
-	_Type(*TypeSystem)
-
 	// Returns a pointer to the TypeSystem this Type is a member of.
-	TypeSystem() *TypeSystem
+	TypeSystem() TypeSystem
 
 	// Returns the string name of the Type.  This name is unique within the
 	// universe this type is a member of, *unless* this type is Anonymous,
@@ -90,156 +98,3 @@ type Type interface {
 	// You can use `TypeKind.ActsLike()` to get type-level behavioral information.
 	RepresentationBehavior() ipld.Kind
 }
-
-var (
-	_ Type = &TypeBool{}
-	_ Type = &TypeString{}
-	_ Type = &TypeBytes{}
-	_ Type = &TypeInt{}
-	_ Type = &TypeFloat{}
-	_ Type = &TypeMap{}
-	_ Type = &TypeList{}
-	_ Type = &TypeLink{}
-	_ Type = &TypeUnion{}
-	_ Type = &TypeStruct{}
-	_ Type = &TypeEnum{}
-)
-
-type typeBase struct {
-	name     TypeName
-	universe *TypeSystem
-}
-
-type TypeBool struct {
-	typeBase
-}
-
-type TypeString struct {
-	typeBase
-}
-
-type TypeBytes struct {
-	typeBase
-}
-
-type TypeInt struct {
-	typeBase
-}
-
-type TypeFloat struct {
-	typeBase
-}
-
-type TypeMap struct {
-	typeBase
-	anonymous     bool
-	keyType       TypeName // must be Kind==string (e.g. Type==String|Enum).
-	valueType     TypeName
-	valueNullable bool
-}
-
-type TypeList struct {
-	typeBase
-	anonymous     bool
-	valueType     TypeName
-	valueNullable bool
-}
-
-type TypeLink struct {
-	typeBase
-	referencedType    TypeName
-	hasReferencedType bool
-	// ...?
-}
-
-type TypeUnion struct {
-	typeBase
-	// Members are listed in the order they appear in the schema.
-	// To find the discriminant info, you must look inside the representation; they all contain a 'table' of some kind in which the member types are the values.
-	// Note that multiple appearances of the same type as distinct members of the union is not possible.
-	//  While we could do this... A: that's... odd, and nearly never called for; B: not possible with kinded mode; C: imagine the golang-native type switch!  it's impossible.
-	//  We rely on this clarity in many ways: most visibly, the type-level Node implementation for a union always uses the type names as if they were map keys!  This behavior is consistent for all union representations.
-	members        []TypeName
-	representation UnionRepresentation
-}
-
-type UnionRepresentation interface{ _UnionRepresentation() }
-
-func (UnionRepresentation_Keyed) _UnionRepresentation()        {}
-func (UnionRepresentation_Kinded) _UnionRepresentation()       {}
-func (UnionRepresentation_Envelope) _UnionRepresentation()     {}
-func (UnionRepresentation_Inline) _UnionRepresentation()       {}
-func (UnionRepresentation_Stringprefix) _UnionRepresentation() {}
-
-// A bunch of these tables in union representation might be easier to use if flipped;
-//  we almost always index into them by type (since that's what we have an ordered list of);
-//  and they're unique in both directions, so it's equally valid either way.
-//  The order they're currently written in matches the serial form in the schema AST.
-
-type UnionRepresentation_Keyed struct {
-	table map[string]TypeName // key is user-defined freetext
-}
-type UnionRepresentation_Kinded struct {
-	table map[ipld.Kind]TypeName
-}
-type UnionRepresentation_Envelope struct {
-	discriminantKey string
-	contentKey      string
-	table           map[string]TypeName // key is user-defined freetext
-}
-type UnionRepresentation_Inline struct {
-	discriminantKey string
-	table           map[string]TypeName // key is user-defined freetext
-}
-type UnionRepresentation_Stringprefix struct {
-	delim string
-	table map[string]TypeName // key is user-defined freetext
-}
-
-type TypeStruct struct {
-	typeBase
-	// n.b. `Fields` is an (order-preserving!) map in the schema-schema;
-	//  but it's a list here, with the keys denormalized into the value,
-	//   because that's typically how we use it.
-	fields         []StructField
-	fieldsMap      map[string]StructField // same content, indexed for lookup.
-	representation StructRepresentation
-}
-type StructField struct {
-	parent   *TypeStruct
-	name     string
-	typ      TypeName
-	optional bool
-	nullable bool
-}
-
-type StructRepresentation interface{ _StructRepresentation() }
-
-func (StructRepresentation_Map) _StructRepresentation()         {}
-func (StructRepresentation_Tuple) _StructRepresentation()       {}
-func (StructRepresentation_StringPairs) _StructRepresentation() {}
-func (StructRepresentation_Stringjoin) _StructRepresentation()  {}
-
-type StructRepresentation_Map struct {
-	renames   map[string]string
-	implicits map[string]ImplicitValue
-}
-type StructRepresentation_Tuple struct{}
-type StructRepresentation_StringPairs struct{ sep1, sep2 string }
-type StructRepresentation_Stringjoin struct{ sep string }
-
-type TypeEnum struct {
-	typeBase
-	members []string
-}
-
-// ImplicitValue is an sum type holding values that are implicits.
-// It's not an 'Any' value because it can't be recursive
-// (or to be slightly more specific, it can be one of the recursive kinds,
-// but if so, only its empty value is valid here).
-type ImplicitValue interface{ _ImplicitValue() }
-
-type ImplicitValue_EmptyList struct{}
-type ImplicitValue_EmptyMap struct{}
-type ImplicitValue_String struct{ x string }
-type ImplicitValue_Int struct{ x int }

schema/typeMethods.go

-package schema
-
-import (
-	ipld "github.com/ipld/go-ipld-prime"
-)
-
-/* cookie-cutter standard interface stuff */
-
-func (t *typeBase) _Type(ts *TypeSystem) {
-	t.universe = ts
-}
-func (t typeBase) TypeSystem() *TypeSystem { return t.universe }
-func (t typeBase) Name() TypeName          { return t.name }
-
-func (TypeBool) TypeKind() TypeKind   { return TypeKind_Bool }
-func (TypeString) TypeKind() TypeKind { return TypeKind_String }
-func (TypeBytes) TypeKind() TypeKind  { return TypeKind_Bytes }
-func (TypeInt) TypeKind() TypeKind    { return TypeKind_Int }
-func (TypeFloat) TypeKind() TypeKind  { return TypeKind_Float }
-func (TypeMap) TypeKind() TypeKind    { return TypeKind_Map }
-func (TypeList) TypeKind() TypeKind   { return TypeKind_List }
-func (TypeLink) TypeKind() TypeKind   { return TypeKind_Link }
-func (TypeUnion) TypeKind() TypeKind  { return TypeKind_Union }
-func (TypeStruct) TypeKind() TypeKind { return TypeKind_Struct }
-func (TypeEnum) TypeKind() TypeKind   { return TypeKind_Enum }
-
-func (TypeBool) RepresentationBehavior() ipld.Kind   { return ipld.Kind_Bool }
-func (TypeString) RepresentationBehavior() ipld.Kind { return ipld.Kind_String }
-func (TypeBytes) RepresentationBehavior() ipld.Kind  { return ipld.Kind_Bytes }
-func (TypeInt) RepresentationBehavior() ipld.Kind    { return ipld.Kind_Int }
-func (TypeFloat) RepresentationBehavior() ipld.Kind  { return ipld.Kind_Float }
-func (TypeMap) RepresentationBehavior() ipld.Kind    { return ipld.Kind_Map }
-func (TypeList) RepresentationBehavior() ipld.Kind   { return ipld.Kind_List }
-func (TypeLink) RepresentationBehavior() ipld.Kind   { return ipld.Kind_Link }
-func (t TypeUnion) RepresentationBehavior() ipld.Kind {
-	switch t.representation.(type) {
-	case UnionRepresentation_Keyed:
-		return ipld.Kind_Map
-	case UnionRepresentation_Kinded:
-		return ipld.Kind_Invalid // you can't know with this one, until you see the value (and thus can its inhabitant's behavior)!
-	case UnionRepresentation_Envelope:
-		return ipld.Kind_Map
-	case UnionRepresentation_Inline:
-		return ipld.Kind_Map
-	default:
-		panic("unreachable")
-	}
-}
-func (t TypeStruct) RepresentationBehavior() ipld.Kind {
-	switch t.representation.(type) {
-	case StructRepresentation_Map:
-		return ipld.Kind_Map
-	case StructRepresentation_Tuple:
-		return ipld.Kind_List
-	case StructRepresentation_StringPairs:
-		return ipld.Kind_String
-	case StructRepresentation_Stringjoin:
-		return ipld.Kind_String
-	default:
-		panic("unreachable")
-	}
-}
-func (t TypeEnum) RepresentationBehavior() ipld.Kind {
-	// TODO: this should have a representation strategy switch too; sometimes that will indicate int representation behavior.
-	return ipld.Kind_String
-}
-
-/* interesting methods per Type type */
-
-// beware: many of these methods will change when we successfully bootstrap self-hosting.
-//
-// The current methods return reified Type objects; in the future, there might be less of that.
-// Returning reified Type objects requires bouncing lookups through the typesystem map;
-// this is unavoidable because we need to handle cycles in definitions.
-// However, the extra (and cyclic) pointers that requires won't necessarily jive well if
-// we remake the Type types to have close resemblances to the Data Model tree data.)
-//
-// It's also unfortunate that some of the current methods collide in name with
-// the names of the Data Model fields.  We might reshuffling things to reduce this.
-//
-// At any rate, all of these changes will come as a sweep once we
-// get a self-hosting gen of the schema-schema, not before
-// (the effort of updating template references is substantial).
-
-// IsAnonymous is returns true if the type was unnamed.  Unnamed types will
-// claim to have a Name property like `{Foo:Bar}`, and this is not guaranteed
-// to be a unique string for all types in the universe.
-func (t TypeMap) IsAnonymous() bool {
-	return t.anonymous
-}
-
-// KeyType returns the Type of the map keys.
-//
-// Note that map keys will must always be some type which is representable as a
-// string in the IPLD Data Model (e.g. either TypeString or TypeEnum).
-func (t TypeMap) KeyType() Type {
-	return t.universe.namedTypes[t.keyType]
-}
-
-// ValueType returns the Type of the map values.
-func (t TypeMap) ValueType() Type {
-	return t.universe.namedTypes[t.valueType]
-}
-
-// ValueIsNullable returns a bool describing if the map values are permitted
-// to be null.
-func (t TypeMap) ValueIsNullable() bool {
-	return t.valueNullable
-}
-
-// IsAnonymous is returns true if the type was unnamed.  Unnamed types will
-// claim to have a Name property like `[Foo]`, and this is not guaranteed
-// to be a unique string for all types in the universe.
-func (t TypeList) IsAnonymous() bool {
-	return t.anonymous
-}
-
-// ValueType returns to the Type of the list values.
-func (t TypeList) ValueType() Type {
-	return t.universe.namedTypes[t.valueType]
-}
-
-// ValueIsNullable returns a bool describing if the list values are permitted
-// to be null.
-func (t TypeList) ValueIsNullable() bool {
-	return t.valueNullable
-}
-
-// Members returns the list of all types that are possible inhabitants of this union.
-func (t TypeUnion) Members() []Type {
-	a := make([]Type, len(t.members))
-	for i := range t.members {
-		a[i] = t.universe.namedTypes[t.members[i]]
-	}
-	return a
-}
-
-func (t TypeUnion) RepresentationStrategy() UnionRepresentation {
-	return t.representation
-}
-
-func (r UnionRepresentation_Keyed) GetDiscriminant(t Type) string {
-	for d, t2 := range r.table {
-		if t2 == t.Name() {
-			return d
-		}
-	}
-	panic("that type isn't a member of this union")
-}
-
-func (r UnionRepresentation_Stringprefix) GetDelim() string {
-	return r.delim
-}
-
-func (r UnionRepresentation_Stringprefix) GetDiscriminant(t Type) string {
-	for d, t2 := range r.table {
-		if t2 == t.Name() {
-			return d
-		}
-	}
-	panic("that type isn't a member of this union")
-}
-
-// GetMember returns type info for the member matching the kind argument,
-// or may return nil if that kind is not mapped to a member of this union.
-func (r UnionRepresentation_Kinded) GetMember(k ipld.Kind) TypeName {
-	return r.table[k]
-}
-
-// Fields returns a slice of descriptions of the object's fields.
-func (t TypeStruct) Fields() []StructField {
-	a := make([]StructField, len(t.fields))
-	for i := range t.fields {
-		a[i] = t.fields[i]
-	}
-	return a
-}
-
-// Field looks up a StructField by name, or returns nil if no such field.
-func (t TypeStruct) Field(name string) *StructField {
-	if v, ok := t.fieldsMap[name]; ok {
-		return &v
-	}
-	return nil
-}
-
-// Parent returns the type information that this field describes a part of.
-//
-// While in many cases, you may know the parent already from context,
-// there may still be situations where want to pass around a field and
-// not need to continue passing down the parent type with it; this method
-// helps your code be less redundant in such a situation.
-// (You'll find this useful for looking up any rename directives, for example,
-// when holding onto a field, since that requires looking up information from
-// the representation strategy, which is a property of the type as a whole.)
-func (f StructField) Parent() *TypeStruct { return f.parent }
-
-// Name returns the string name of this field.  The name is the string that
-// will be used as a map key if the structure this field is a member of is
-// serialized as a map representation.
-func (f StructField) Name() string { return f.name }
-
-// Type returns the Type of this field's value.  Note the field may
-// also be unset if it is either Optional or Nullable.
-func (f StructField) Type() Type { return f.parent.universe.namedTypes[f.typ] }
-
-// IsOptional returns true if the field is allowed to be absent from the object.
-// If IsOptional is false, the field may be absent from the serial representation
-// of the object entirely.
-//
-// Note being optional is different than saying the value is permitted to be null!
-// A field may be both nullable and optional simultaneously, or either, or neither.
-func (f StructField) IsOptional() bool { return f.optional }
-
-// IsNullable returns true if the field value is allowed to be null.
-//
-// If is Nullable is false, note that it's still possible that the field value
-// will be absent if the field is Optional!  Being nullable is unrelated to
-// whether the field's presence is optional as a whole.
-//
-// Note that a field may be both nullable and optional simultaneously,
-// or either, or neither.
-func (f StructField) IsNullable() bool { return f.nullable }
-
-// IsMaybe returns true if the field value is allowed to be either null or absent.
-//
-// This is a simple "or" of the two properties,
-// but this method is a shorthand that turns out useful often.
-func (f StructField) IsMaybe() bool { return f.nullable || f.optional }
-
-func (t TypeStruct) RepresentationStrategy() StructRepresentation {
-	return t.representation
-}
-
-func (r StructRepresentation_Map) GetFieldKey(field StructField) string {
-	if n, ok := r.renames[field.name]; ok {
-		return n
-	}
-	return field.name
-}
-
-func (r StructRepresentation_Map) FieldHasRename(field StructField) bool {
-	_, ok := r.renames[field.name]
-	return ok
-}
-
-func (r StructRepresentation_Stringjoin) GetDelim() string {
-	return r.sep
-}
-
-// Members returns a slice the strings which are valid inhabitants of this enum.
-func (t TypeEnum) Members() []string {
-	a := make([]string, len(t.members))
-	for i := range t.members {
-		a[i] = t.members[i]
-	}
-	return a
-}
-
-// Links can keep a referenced type, which is a hint only about the data on the
-// other side of the link, no something that can be explicitly validated without
-// loading the link
-
-// HasReferencedType returns true if the link has a hint about the type it references
-// false if it's generic
-func (t TypeLink) HasReferencedType() bool {
-	return t.hasReferencedType
-}
-
-// ReferencedType returns the type hint for the node on the other side of the link
-func (t TypeLink) ReferencedType() Type {
-	return t.universe.namedTypes[t.referencedType]
-}

schema/typesystem.go

 package schema
 
-type TypeSystem struct {
-	// namedTypes is the set of all named types in this universe.
-	// The map's key is the value's Name() property and must be unique.
-	//
-	// The IsAnonymous property is false for all values in this map that
-	// support the IsAnonymous property.
-	//
-	// Each Type in the universe may only refer to other types in their
-	// definition if those type are either A) in this namedTypes map,
-	// or B) are IsAnonymous==true.
-	namedTypes map[TypeName]Type
+type TypeSystem interface {
+	// FIXME accessor methods will reappear here (just hasn't been forced by the progress of the refactor yet)
 }