Strongly-Typed Object Schemas for `parameters.yaml`

[brianaydemir]

Scope: docs/parameters.yaml, generate/param_generator.go, .github/scripts/validate-parameters/, TypeScript and web-form codegen

Summary

Pelican describes its ~424 configuration parameters in docs/parameters.yaml. For scalar types this single file drives everything downstream. For type: object, it does not: the shape of an object parameter lives only in prose, and every consumer of that shape — Go structs, TypeScript interfaces, web forms, CI checks, rendered documentation — is reimplemented and maintained by hand.

This proposal makes object shapes machine-readable. A schema: block describes an object parameter's fields once; the generator emits the Go types, TypeScript types, the JSON Schema the web UI uses to GET and PATCH the configuration, the web forms, and the documentation from it. When two parameters share a shape, a schema_ref: reuses the canonical definition by reference instead of restating it. The schema also carries lifecycle metadata (introduced, deprecated, removed), so a parameter's history becomes data rather than tribal knowledge.

The Problem

type: object is the only opaque type in the vocabulary. The other eight (bool, byterate, duration, filename, int, string, stringSlice, url) fully describe their values; object describes nothing beyond the word "object." Of the 424 parameters, 9 are objects, and each one is a place where the documented contract and the implemented contract can disagree with nothing to catch it.

Today an object parameter fans out into hand-maintained, independently-edited artifacts:

• Go. param/parameters_struct.go types every object field as bare any (e.g. Exports any, AuthorizationTemplates any). The real shape is recovered ad hoc at each call site via Unmarshal into a hand-written struct.
• TypeScript. web_ui/frontend/.../Fields/index.ts declares interfaces like Export by hand.
• Web forms. Each object gets a bespoke form component (ExportForm.tsx, …).
• CI. validate-parameters/main.py gates objects behind a VERIFIED_OBJECT_STRUCTURES allowlist — a promise that a human checked the web UI, not a check that anything matches.
• Documentation. Field listings are written into the description: prose by hand.

Nothing ties these together. They drift independently, and the drift is silent.

Concrete symptom. Origin.Exports[].AuthorizationTemplates is fully wired in the Go backend (server_utils/origin.go:69) and consumed by the issuer (oa4mp/serve.go). It is absent from the Export TypeScript interface (Fields/index.ts:143) and from ExportForm.tsx. A working backend feature is invisible and unconfigurable in the web UI, and no test, type checker, or CI step reports the gap. This is not a bug to be fixed once; it is the predictable output of a process where one source of truth is copied by hand into five places.

Why Strong Typing

This is the core argument, and it is deliberately a philosophical one.

With 424 parameters, correctness cannot rest on reviewers noticing that five hand-edited files still agree. The number is too large and the edits too frequent. Strong typing — deriving every consumer from one declared shape — converts a class of errors from "caught if a human happens to notice" into "impossible to express." A field that exists in the schema exists everywhere; a field that does not exists nowhere. Drift stops being a thing that can happen.

The leverage is highest precisely where it is missing today. Scalars are already strongly typed end to end, and they do not drift. Objects are the untyped remainder, and they are exactly where the AuthorizationTemplates gap appeared. Extending the same discipline to objects closes the last hand-copied seam.

Two consequences follow, and both are first-class goals:

• Operational code is correct by construction. Go and TypeScript types, decode hooks, and validation come from the schema, so the code that runs the software cannot disagree with the documented contract.
• Documentation is correct by construction. Field listings, allowed enum values, and lifecycle notes are generated from the same schema, so the docs a human reads cannot disagree with the code that runs.

Generation is not about typing less today. It is about making the documented contract and the executed contract provably the same artifact, every day, without anyone having to remember to keep them aligned.

Design

Three additions to parameters.yaml. Scalar parameters are untouched. Every object/objectList parameter declares its shape exactly one way: an inline schema: block, a schema_ref: to another parameter's schema, or an explicit schema_manual: true. None is left unannotated.

1. enum and enumSlice as first-class types

Replaces the current pattern of type: string with allowed values buried in prose. An enum/enumSlice parameter lists its values:; each entry's value: is the canonical wire/config string and its description: is documentation. Unknown values are rejected at config-load time, so a typo fails fast instead of decoding to a silent zero value. For enumSlice, an empty list is distinct from unset.

name: Origin.StorageType
type: enum
description: Backend storage driver for this origin.
values:
  - { value: posix,  description: "POSIX filesystem." }
  - { value: s3,     description: "S3-compatible object storage." }
  - { value: xroot,  description: "XRootD storage backend." }
  - { value: globus, description: "Globus collection." }
  - { value: http,   description: "Plain HTTP backend." }

Existing call sites that read these as strings (param.Origin_StorageType.GetString()) keep compiling — the generated enum type is a string alias.

2. object splits into object and objectList

The split is semantic, not a codegen convenience. A single keyword conflating "one value" with "many values" is the root cause of several current UI and codegen inconsistencies; separating them lets each carry the right generated type and the right UI control.

• object — exactly one structured value. Go type T; UI a single form. No current parameter is a non-manual singleton; the keyword is reserved for future use and is exercised by generator fixtures so the path cannot rot.
• objectList — an ordered, homogeneous list of structured values. Order is significant and preserved on round-trip. Go type []T; UI an add/remove list. Today's non-manual objects are all lists: Origin.Exports, Issuer.AuthorizationTemplates, Issuer.OIDCAuthenticationRequirements, Registry.Institutions, GeoIPOverrides.

Parameters that opt out (below) keep plain type: object: the container distinction is meaningless when the element shape is opaque.

3. The schema: block

A parameter declares its shape in one of two ways: an inline schema: block, or a schema_ref: that reuses the schema (and generated type) of another parameter. The two are mutually exclusive on a given parameter or field.

schema:
  type_name: AuthorizationTemplate   # optional; default derived from param name
  fields:
    - name: <PascalCase>             # drives Go field and YAML key
      type: <string|int|bool|stringSlice|enum|enumSlice|object|objectList>
      description: <prose>
      required: <bool>
      # type- and feature-specific keys below

# Reuse another parameter's schema instead of restating it.
schema_ref: Issuer.AuthorizationTemplates   # dotted name of the owning parameter

Key	When	Purpose
name	always	PascalCase. Drives the Go struct field and the YAML key.
type	always	One of the eight listed types. (duration, url, filename, byterate are valid at top level but not yet for nested fields — a bounded follow-on.)
description	always	Field doc; flows into tooltips and the parent's generated description.
required	always	Drives web-form validation and CI checks.
default	optional	"New item" initializer in the web form. No effect on decoding.
values	enum types	Ordered { value, description, hidden } entries. hidden defaults false.
enum_name	enum types	Override the generated enum type name.
type_name	object types	Override the generated struct type name.
mapstructure_key	exceptions	Override the default lowercased key (e.g. group_regexes).
schema	object types	Inline nested schema (recursive). Mutually exclusive with schema_ref.
schema_ref	object types	Dotted name of another parameter whose schema (and generated type) this reuses. Mutually exclusive with schema. See Reusing a schema below.
hidden	optional	Field/value is generated but omitted from docs and UI. For expert or not-yet-public settings.
introduced	optional	Semver in which the field/param appeared.
deprecated	optional	Semver in which it was deprecated.
removed	optional	Semver in which it was removed.
replaced_by	optional	Name of the successor field/param. Requires deprecated; surfaced in docs and deprecation warnings.

Name forms

One name: produces several identifiers deterministically. For StoragePrefix with no override:

Form	Result
Go struct field	StoragePrefix (verbatim)
YAML key in user config	StoragePrefix (verbatim, yaml: tag)
JSON key in config GET/PATCH	StoragePrefix (verbatim, json: tag)
JSON Schema property	StoragePrefix (matches the JSON key)
Mapstructure key (Viper internal)	storageprefix (lowercased, no separators)
TypeScript property / form state key	StoragePrefix (matches the JSON key)

The mapstructure key is the field name lowercased with separators stripped, which collapses multi-word names into a run of letters (GroupRegexes → groupregexes). When that is hard to read or needs an explicit separator, mapstructure_key: overrides it (e.g. group_regexes); the yaml: and json: keys are unaffected and stay PascalCase.

Generated config structs carry json: tags equal to the PascalCase YAML key. This matters because param.Config is the web UI's wire type: web_ui/ui.go serves the live configuration as JSON over a *param.Config and binds the PATCH body straight back into one. Pinning the JSON keys to PascalCase keeps the JSON key, the YAML key, and the Go field identical.

Other camelCase-tagged structs such as server_utils.OriginExport are a separate wire type used by different APIs (e.g. director registration), not the config struct repurposed; they keep their own json: tags and stay hand-owned.

Generated type names

• Struct, top-level: drop the first dotted component and singularize (Origin.Exports → Export). Override with schema.type_name:.
• Struct, inline field: singularize the field name (objectList) or use it verbatim (object). Override with field type_name:.
• Enum, top-level: last dotted component (Origin.StorageType → StorageType).
• Enum, field: {Type}{Field} singularized (AuthorizationTemplate.Actions → AuthorizationTemplateAction).

Pluralization uses a standard inflector; edge cases override with an explicit name.

Shared shapes are referenced, not copied. When two parameters need the same type — Origin.Exports[].AuthorizationTemplates and Issuer.AuthorizationTemplates are one example — the reuser declares schema_ref: (see Reusing a schema below) rather than transcribing the field list into both definitions and keeping the copies in lockstep by hand. That hand-copying is exactly what this proposal exists to eliminate, so it is the recommended way to share a type.

As a backstop against accidental collisions, two inline declarations that share a type_name:/enum_name: must be byte-for-byte identical in every attribute and order, or the generator errors; identical declarations emit the type once. An unintended name collision without an explicit override is also an error.

Reusing a schema

schema_ref: <Param.Name> binds a parameter or field to the schema (and the generated Go/TypeScript type) of the named parameter, which becomes the canonical owner:

• The reuser inherits the owner's fields, generated struct name, and JSON Schema definition. It supplies its own name:, description:, required:, and lifecycle metadata — the things that are local to where the type is used — but not fields: or type_name:.
• The target must be an object/objectList parameter with an inline schema: (a schema_ref: may not point at another schema_ref: — references are one hop, so the canonical owner is always unambiguous).
• The reuser's container kind must match the target's (objectList reuses objectList). The generated JSON Schema emits the shared shape once under $defs and points both $refs at it.
• CI rejects a dangling schema_ref:, a reference to a non-object parameter, and a reference chain. Removing or restructuring a referenced parameter therefore fails fast at generation time rather than silently desyncing a copy.

Composed documentation

The parent's rendered docs (HTML, Go docstrings, TS JSDoc, --help) are built mechanically:

• description: supplies the high-level paragraph only.
• The generator appends a Fields block — each field's name, Go type, requiredness, lifecycle, and description:, in declaration order. hidden: fields and values are excluded.

Because the field list is generated, description: must not hand-list fields; hand-listed fields drift, generated ones cannot.

Backwards compatibility

The change is additive and incremental:

• Scalar parameters are untouched. No edit, no regeneration churn.
• Generated constant names are stable. param.Origin_Exports and friends keep their names; only their underlying type tightens from any to []Export.
• Existing accessors keep compiling. New EnumParam/EnumSliceParam and ObjectListParam mirror the accessor surface of the types they replace; current call sites such as param.Origin_StorageType.GetString() are unchanged.
• Migration is one parameter at a time. Each object can be converted, reviewed, and merged independently. An object that has not yet been converted simply keeps schema_manual: true and its existing hand-written code.

Human review stays central

Generation removes hand-copying, not human judgment. Every change a human makes is a parameters.yaml edit, and every effect of that edit lands as a reviewable diff:

• One editable source. Humans edit parameters.yaml. Generated Go, TypeScript, forms, and docs are never hand-edited.
• Deterministic, byte-stable output, so regeneration yields clean, reviewable diffs rather than noise.
• CI freshness check. CI regenerates and fails if committed output differs from fresh output — generated code can never silently lag the schema.
• Golden tests exercise the hard cases (singleton object, nested objects, field enums, schema_ref: reuse), so paths with no current real-world user cannot rot.

Escape hatches are explicit and bounded

Some objects are genuinely not representable: polymorphic, union-typed, or DSL-like. Likely candidates: Shoveler.IPMapping (union), LocalCache.StorageDirs (string-or-object), Registry.CustomRegistrationFields (dynamic), Lotman.PolicyDefinitions (nested DSL).

These declare schema_manual: true (a sibling of type:, never nested) and must appear in a MANUAL_OBJECT_STRUCTURES allowlist naming the hand-maintained code. A form that the renderer cannot express declares form_manual: true and joins its own allowlist while the type still generates. CI fails if an object/objectList is neither schema-backed nor allowlisted. Every exception is therefore visible, named, and counted — never silent drift.

Generation Targets

generate/param_generator.go (run via go generate) currently emits param/parameters.go and param/parameters_struct.go. This extends both and adds two new outputs: the typed Go object types and a JSON Schema.

• param/parameters.go — add EnumParam, EnumSliceParam, and ObjectListParam, mirroring the accessor surface of StringParam/ObjectParam.
• param/object_types_gen.go (new) — walk every schema: and top-level enum; emit typed Go structs and enum constants, plus a Validate() method and decode hook per enum so unknown values are rejected at load time.
• param/parameters_struct.go — each list-shaped field resolves from any to its generated slice type (Exports any → Exports []Export), and every field gains a PascalCase json: tag so the struct's config GET/PATCH wire format is generated rather than incidental.
• JSON Schema (new) — emit a JSON Schema (Draft 2020-12) for the full configuration, with PascalCase property names matching the json:/yaml: keys. Shared shapes appear once under $defs, so a schema_ref: reuser and its owner point at the same $ref. The web UI consumes this schema to validate and render the config editor that GETs and PATCHes param.Config; CI can additionally assert that a sample config round-trips against it. Enum values:, required:, and lifecycle metadata become enum, required, and annotations.
• TypeScript — the target that emits parameters.json also emits typed interfaces keyed by the PascalCase JSON key (required: false → optional ?:); the hand-written interfaces are deleted. This also corrects them: existing interfaces key nested object fields by the lowercased mapstructure form (e.g. interface Export { storageprefix … }), which does not match the PascalCase the backend actually serves.
• Web forms — Option A (target): a generic SchemaForm component reads the generated JSON Schema (and the per-field metadata in parameters.json) and dispatches per field type, so the editor's fields are derived from the schema rather than hand-built. Option B (fallback): generated *.generated.tsx for cases A cannot express.
• CI — validate-parameters/main.py replaces VERIFIED_OBJECT_STRUCTURES with structural checks: every object has schema:, a one-hop schema_ref: to an object parameter that itself has a schema:, or an allowlisted schema_manual:; every enum has well-formed values:; every field has the required keys; names and effective mapstructure keys are unique within a schema; all generated identifiers are valid and unique; lifecycle fields are well-formed.

type Action string
const (
    ActionRead   Action = "read"
    ActionCreate Action = "create"
    ActionModify Action = "modify"
)

type AuthorizationTemplate struct {
    Actions      []Action `mapstructure:"actions"       yaml:"Actions"                json:"Actions"`
    Prefix       string   `mapstructure:"prefix"        yaml:"Prefix"                 json:"Prefix"`
    Users        []string `mapstructure:"users"         yaml:"Users,omitempty"        json:"Users,omitempty"`
    Groups       []string `mapstructure:"groups"        yaml:"Groups,omitempty"       json:"Groups,omitempty"`
    GroupRegexes []string `mapstructure:"group_regexes" yaml:"GroupRegexes,omitempty" json:"GroupRegexes,omitempty"`
}

Hand-written equivalents — authzTemplate (oa4mp/serve.go), AuthorizationTemplates []interface{} (server_utils/origin.go) — are retired in favor of the generated type.

Worked Example

Issuer.AuthorizationTemplates:

name: Issuer.AuthorizationTemplates
type: objectList
description: Per-issuer authorization rules applied when minting tokens.
schema:
  fields:
    - name: Actions
      type: enumSlice
      enum_name: Action
      required: true
      description: "Actions the template authorizes."
      values:
        - { value: read,   description: "Read access to objects under Prefix." }
        - { value: create, description: "Create new objects under Prefix." }
        - { value: modify, description: "Modify existing objects under Prefix." }
    - { name: Prefix, type: string, required: true,
        description: "Path prefix; $USER and $GROUP are substituted at runtime." }
    - { name: Users, type: stringSlice, required: false,
        description: "If non-empty, template activates only for these usernames." }
    - { name: Groups, type: stringSlice, required: false,
        description: "If non-empty, at least one authenticated group must match." }
    - { name: GroupRegexes, type: stringSlice, mapstructure_key: group_regexes,
        required: false,
        description: "Regex patterns; at least one authenticated group must match." }

Origin.Exports then reuses that exact type for its nested field by reference — no transcribed field list, nothing to keep in lockstep:

    - name: AuthorizationTemplates
      type: objectList
      schema_ref: Issuer.AuthorizationTemplates   # reuse the canonical definition
      required: false
      description: "Per-export auth templates; overrides Issuer.AuthorizationTemplates."

The shared AuthorizationTemplate shape is defined once, in Issuer.AuthorizationTemplates. Both the Go []AuthorizationTemplate field and the $defs/AuthorizationTemplate entry in the generated JSON Schema come from that single definition; the two parameters cannot disagree because there is only one shape.

Forward-Looking: Lifecycle Metadata

The schema is built to absorb a parameter's history as data. Three optional semver fields plus a successor pointer — at the parameter level or the field level — turn version lore into something the toolchain enforces and renders:

• introduced — version added. Emitted into Go docstrings and docs (// Introduced in v7.3.0); CI can warn when a config references a param or field newer than the running version.
• deprecated — version deprecated. The param/field stays live and generated; CI warns on configs that use it.
• removed — version removed. Excluded from generated types; CI rejects configs that reference it with a clear error. Setting removed without deprecated is an error; deprecated alone is fine.
• replaced_by — name of the successor field/param. Requires deprecated (replacing something not deprecated is a schema error); the generated docs and the deprecation warning name the successor, so a reader is told what to migrate to, not just that the old name is going away.

Set at the parameter level, lifecycle gates the whole parameter; at the field level, only that field — so an old parameter can deprecate individual fields while the rest stay live. CI compares versions with standard semver, with pre-release segments sorting before their releases and a special "dev" that sorts before all releases for in-development fields.

Because this metadata rides the same generated path as everything else, recording that a field arrived in 8.0 or leaves in 9.0 is one YAML edit that simultaneously updates the types, the docs, and the validation — never three places to remember.

Semantic Guarantees

• Enum values validated at load. A Go enum alias does not stop mapstructure from decoding an unknown string. The generated Validate() method and decode hook reject out-of-range values across every input form — YAML, env-var comma-strings, slices — so an unsupported value fails fast.
• Domain invariants stay single-sourced. Capability-style enums carry meaning beyond shape (e.g. PublicReads implies Reads). One canonical conversion from the generated []Capability to the server's internal form defines the implication and rejection rules once, not per call site.

Out of Scope (Follow-ons)

• Runtime required-field validation. required: today drives web-form and CI checks, not Go validation; a missing required field decodes to a silent zero. A follow-on emits a Validate() error per generated struct, run once at config load before any server starts. required: true together with default: is then a schema error — runtime defaults belong in defaults.yaml.
• Typed runtime defaults. default: in parameters.yaml stays documentation; real defaults remain in config/resources/defaults.yaml. Deriving typed defaults from the schema is a separate change to defaults loading.
• Remaining scalar field types in nested schemas (duration, url, filename, byterate) — straightforward to add when a real field needs them.