Skip to content

Add fuzz testing for validate-fixtures.mjs against randomly malformed JSON #36

Description

@knytcomics-ui

Problem

The unit tests proposed elsewhere in this batch cover known, hand-picked error cases. They don't protect against the validator crashing (uncaught exception, non-zero-but-wrong exit behavior) on unanticipated malformed input — e.g. null where an object is expected, deeply nested unexpected types, non-string keys. A validator whose job is to be the last line of defense on data correctness should degrade to a clear error message, never an unhandled stack trace, on literally any input shape.

Requirements

  • Add a property-based/fuzz test (hand-rolled random-mutation generator is fine — no need for a heavy fuzzing dependency) that takes the current valid destinations.json/scores.json and applies randomized structural mutations (delete a random key, replace a random value with null/a wrong type/an empty array, duplicate a random entry) across many iterations.
  • Assert, for every mutated input, that the validator (using the pure-function refactor from the unit-testing issue) either returns a non-empty errors array or throws a caught, descriptive error — it must never throw an uncaught TypeError/undefined is not a function-style crash.
  • Seed the random mutations deterministically (fixed seed) so failures are reproducible in CI logs.

Acceptance Criteria

  • Running the fuzz test surfaces at least one current crash-on-malformed-input bug in the existing validator (there almost certainly is one, e.g. d.label on a non-object array element), which this issue's implementer should also fix as part of landing it.
  • Fuzz test is deterministic and runs in CI without flaking.

Part of the Gryd-lock organization. This repo supplies the labelled testnet fixtures and stub scores consumed by grydlock-oracle-adapter's StubOracle and evaluated end-to-end by grydlock-research.

Metadata

Metadata

Assignees

No one assigned

    Labels

    GrantFox OSSIssue tracked in GrantFox OSSMaybe RewardedIssue may be eligible for a GrantFox rewardOfficial Campaign | FWC26Campaign: Official Campaign | FWC26advancedRequires deep domain knowledge or cross-repo contextarea: testingTest infrastructureenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions