Skip to content

Conversation

Copy link

Copilot AI commented Oct 31, 2025

CLI commands (kaprese run, kaprese eval, kaprese benchmark, kaprese engine, kaprese config) lacked detailed option references. Users had examples but no complete argument specifications or expected outputs.

Changes

Documentation Structure

  • Created docs/cli.md with:
    • Global options and all command/subcommand option tables
    • Default values and required/optional indicators
    • 12 usage examples with actual command syntax
    • 8 sample output snippets showing expected results
    • Common workflows section for typical usage patterns
    • Environment variables reference (LOG_LEVEL)

Command Reference Tables

Each command now documents:

  • Option flags with long/short forms
  • Descriptions clarifying behavior (e.g., "use 'all' for all benchmarks")
  • Default values or "Required" indicator
  • Practical notes (e.g., "At least one engine must be specified")

README Updates

  • Changed "Options" section to "Commands" with corrected table header
  • Added link to CLI reference documentation

Example

The run command now has a complete reference:

Option Description Default
-e [ENGINE ...] Engine(s) to run Required
-b [BENCHMARK ...] Benchmark(s) to run All registered benchmarks
-o OUTPUT Output directory for results kaprese-out
--rebuild-runner Rebuild runner Docker image False

With usage examples:

# Run multiple engines on multiple benchmarks
kaprese run -b flint-1 spearmint-1 -e saver

# Pass extra arguments to engine
kaprese run -b flint-1 -e saver -- -j4
Original prompt

This section details on the original issue you should resolve

<issue_title>Add detailed CLI option documentation</issue_title>
<issue_description>### Description
CLI commands such as kaprese run, kaprese eval, and kaprese register lack detailed option references.
README shows examples but doesn’t list full argument sets or expected outputs.

Expected Behavior

Each command should have a concise table describing:

  • Options / Flags
  • Default values
  • Example usage and output

Suggested Fix

  • Add a “CLI Reference” section to README or docs/cli.md.
  • Include sample runs (with command + output snippet).
  • Consider generating docs automatically from argparse definitions.

Example

Command Option Description
kaprese run --benchmark <name> Specify benchmark to execute
kaprese eval --metric <type> Choose evaluation metric

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

@hyun-hyang hyun-hyang marked this pull request as ready for review October 31, 2025 04:51
Copilot AI changed the title [WIP] Add detailed CLI option documentation to README Add comprehensive CLI reference documentation Oct 31, 2025
Copilot AI requested a review from hyun-hyang October 31, 2025 04:52
@hyun-hyang hyun-hyang closed this Oct 31, 2025
@hyun-hyang hyun-hyang deleted the copilot/add-cli-option-documentation branch October 31, 2025 04:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add detailed CLI option documentation

2 participants