Skip to content

test: add CLI black-box integration tests from kael.factory#15

Merged
beogip merged 2 commits into
mainfrom
test/cli-integration-tests
May 24, 2026
Merged

test: add CLI black-box integration tests from kael.factory#15
beogip merged 2 commits into
mainfrom
test/cli-integration-tests

Conversation

@beogip

@beogip beogip commented May 24, 2026

Copy link
Copy Markdown
Owner

Summary

  • Migrates end-to-end CLI integration tests from kael.factory/tests/tool-base.test.ts that exercise Tool.run() as a child process
  • Adds tests/fixtures/dummy-tool.ts fixture with 8 subcommands covering sync, async, defaults, throws, ToolError, badShape, l3Output, and side-effect paths
  • Adds tests/cli.test.ts with 38 black-box tests covering: defaults applied, handler non-invocation on validation failure (security contract), l3Output pipeline, --subcommand normalization, global --help/--schema override, and exit-code-always-0 umbrella assertion

Test plan

  • bun test tests/cli.test.ts — 38 tests pass
  • bun test — all 108 tests pass (no regressions)

Migrates the end-to-end CLI tests from kael.factory's tool-base.test.ts
that were missing in this repo. These cover Tool.run() behavior as a
child process (exit codes, stdout JSON, stderr markers) complementing
the existing in-process invoke() tests.
@beogip beogip self-assigned this May 24, 2026
@greptile-apps

greptile-apps Bot commented May 24, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR migrates end-to-end CLI integration tests from kael.factory into the main repo, adding tests/cli.test.ts (38 tests) and the tests/fixtures/dummy-tool.ts fixture that exercises Tool.run() as a child process. Previously flagged issues — parseOutput crashing on non-JSON stdout and missing error-envelope assertions for throws/businessError — are both resolved in this version.

  • tests/cli.test.ts: Nine describe blocks cover sync/async dispatch, defaults, input-validation gating, unexpected throws, ToolError, output schema violations, l3Output pipeline, --subcommand normalization, global --help/--schema override, and an umbrella exit-code-always-0 sweep.
  • tests/fixtures/dummy-tool.ts: Eight subcommands built with l1Output, l2Output, l3Output, raw z.object, and ToolError; intentional mismatches are documented with @ts-expect-error.

Confidence Score: 5/5

Pure test addition with no changes to production code; all 108 tests pass and the new tests cover the full range of Tool.run() behaviors.

Only production-safe test files are added. The helpers correctly guard against spawn failures and malformed stdout, and every error envelope path now has dedicated assertions. No regressions are introduced.

No files require special attention.

Important Files Changed

Filename Overview
tests/cli.test.ts Adds 38 black-box integration tests across 9 describe blocks; runTool and parseOutput helpers now guard against spawn failures and empty/malformed stdout. One dead-code branch (sub === "") in the exit-code umbrella callback is never reachable.
tests/fixtures/dummy-tool.ts 8-subcommand fixture covering sync, async, defaults, throws, ToolError, output schema violation, l3Output pipeline, and stderr side-effect detection; all intentional mismatches are clearly marked.

Sequence Diagram

sequenceDiagram
    participant T as bun:test
    participant R as runTool()
    participant S as spawnSync
    participant F as dummy-tool.ts
    participant TR as Tool.run()
    participant H as subcommand handler
    T->>R: runTool(subcommand, ...args)
    R->>S: spawnSync bun FIXTURE args
    S->>F: spawn child process
    F->>TR: tool.run(argv)
    alt input valid
        TR->>H: invoke handler
        H-->>TR: result or throw
        TR-->>F: JSON envelope stdout
    else input invalid
        TR-->>F: ok:false envelope stdout
    end
    F-->>S: exit 0
    S-->>R: stdout stderr status
    R-->>T: stdout stderr exitCode
    T->>T: parseOutput then assertions
Loading

Reviews (2): Last reviewed commit: "fix(tests): address Greptile review — sp..." | Re-trigger Greptile

Comment thread tests/cli.test.ts
Comment thread tests/cli.test.ts
… assertions

- runTool now throws on spawn failure instead of silently passing
- parseOutput gives clear error on empty/malformed stdout
- Added dedicated test blocks for throws (unexpected_error),
  businessError (ToolError envelope), and badShape (schema_violation)
@beogip beogip merged commit d5ef3c9 into main May 24, 2026
2 checks passed
@beogip beogip deleted the test/cli-integration-tests branch May 24, 2026 23:30
@github-actions

github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 0.1.2 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant