Skip to content

feat(algorithm): Add CONSTRAINTS-BEFORE-WRITE as Type 5 probe in PROBE-BEFORE-BUILD gate #1107

@schmetti-dev

Description

@schmetti-dev

Summary

Algorithm v3.7.0 has no mandate to read the constraint model of a target system before writing to it. When a PLAN targets a config file, API, CLI tool, or structured file format, the agent proceeds directly to writing — guessing property names, nesting levels, and field semantics from context instead of reading the schema or spec first. Silent validation failures (strict-schema rejection of unknown keys, wrong nesting depth, misnamed fields) are only discovered after a restart or test run, triggering 2-5 wasted debug iterations per incident.

This is distinct from the prerequisite existence checks in #1001 (PROBE-BEFORE-BUILD). That gate asks "Does the tool/data/endpoint exist?" This proposal asks "Do I understand the rules I must follow when writing to it?" Both are needed; neither subsumes the other.

Evidence

Real incident (gateway-config.json, April 2026):
A config property allowInternalRouting was placed at the top level of the JSON config instead of nested under integrations.chatService.network where the Zod schema requires it. The config uses .strict() — unknown top-level keys are silently discarded, not rejected with an error. The gateway restarted cleanly, the chat plugin appeared connected, but the internal security filter remained active. Three additional debug iterations followed before the schema was actually read.

Root cause: the property name was inferred from documentation prose, not from the schema definition. The schema would have made the required nesting unambiguous in seconds.

Recurring constraint-mismatch failure categories:

Pattern Root cause
Config key at wrong nesting level Schema not read before write
Property name guessed from docs prose Types/schema not consulted
API request field named incorrectly OpenAPI spec not fetched
File format field type wrong Format spec not checked
CLI flag value format incorrect --help output not inspected

Common thread: the agent wrote to a system governed by external constraints it had not read.

Proposal

Add CONSTRAINTS-BEFORE-WRITE as a fifth probe category to the PROBE-BEFORE-BUILD gate proposed in #1001. Before the first write to any external system, the agent identifies and reads the constraint model of that system.

Probe Categories (extending #1001's four)

# Category Question How to probe
1 Tool/binary existence (#1001) Does it exist? which <tool>, python3 -c "import <lib>"
2 Data format sampling (#1001) What does real data look like? Fetch one sample, inspect
3 DOM/page structure (#1001) What selectors actually work? Inspect live page
4 API/auth validation (#1001) Is the endpoint reachable and authed? Lightweight ping
5 Constraint model (this proposal) What are the rules I must follow when writing? Read schema / fetch spec / run validator

Constraint Source Priority (for Type 5)

  1. Local schema file — Zod, JSON Schema, TypeScript types → Read directly
  2. Validate subcommand — run the tool's own validator before and after (openclaw config validate, ajv validate, etc.)
  3. CLI introspection--help, --schema, config schema subcommand
  4. Online docsWebFetch if no local schema exists
  5. Working example — read an existing correct config and pattern-match structure

Never guess. If no constraint source can be found in 30 seconds, document that explicitly and proceed with maximum caution.

The validate / verify distinction (worth encoding explicitly)

Both steps are required after a write. They answer different questions:

Term Question When Tool
Validate Is the written artifact structurally correct? Immediately after write Schema validator, type checker, linter, config validate
Verify Does the running system behave as expected? After restart/reload Observe logs, test the feature, check status endpoint

A config can pass schema validation and still have semantically wrong values. A service can restart without error and still silently ignore a misplaced key. Both checks are necessary.

Suggested Output Format (Type 5 probe)

🔒 CONSTRAINTS CHECK:
 🔒 Target: [config name / API name / format]
 🔒 Source: [schema file path | docs URL | --help | working example]
 🔒 Key constraints: [required nesting, property names, types, enums]
 🔒 Validate command: [command to run after write]
 🔒 Verify signal: [what to observe after restart to confirm correctness]

PRD Integration

Constraint checks map naturally to atomic ISC criteria:

- [ ] ISC-N: [Config name] schema read before any property written
- [ ] ISC-N: [Config name] validates cleanly after write
- [ ] ISC-N: [Service] behavior verified after restart/reload

The validate and verify criteria should always appear as a pair — one without the other is incomplete.

Relationship to #1001

This proposal is a direct extension of #1001, not a competing proposal. The PROBE-BEFORE-BUILD gate's placement options (end of PLAN, start of BUILD, or explicit new PROBE phase) apply equally to Type 5 — wherever #1001 lands, CONSTRAINTS-BEFORE-WRITE fits into the same gate.

The two probes address adjacent gaps in the PLAN→BUILD transition:

I would not have known how to frame this with the precision this repo deserves without #1001 as a reference — the reflection-mining methodology and the gate placement options table were exactly the right scaffolding. Thank you to @catchingknives for that issue.

Impact

  • Pattern prevented: Write → silent failure → debug cycle on config/API/format writes
  • Iterations saved per incident: 2–5
  • Scope: Any Algorithm run that includes a write to an external system with a defined schema, spec, or constraint model
  • Implementation effort: Low — Algorithm text addition in the PROBE section + output format extension

Inspired by the PROBE-BEFORE-BUILD pattern in #1001. Submitted as a complementary extension rather than a separate gate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions