Skip to content

Conversation

ralphbean
Copy link
Contributor

@ralphbean ralphbean commented Oct 10, 2025

Summary

As a new user, I felt overwhelmed by the variety of subcommands. Lots of check* options mixed in with audit and verify. At a certain level, "check" and "verify" feel like synonyms. The --help didn't help me build a mental model for how the subcommands should be used in a workflow.

In this change, I organized the sourcetool CLI help output into four logical command groups to hopefully improve comprehension and user experience.

Changes

Command Groups

Verification Commands (2 commands)

  • audit - Verifies multiple commits in the branch history
  • verifycommit - Verifies the specified commit is valid

Assessment Commands (5 commands)

  • status - Check the SLSA Source status of a repo/branch
  • checklevel - Determines the SLSA Source Level of the repo
  • checklevelprov - Checks the given commit against policy using & creating provenance
  • checktag - Checks to see if the tag operation should be allowed and issues a VSA
  • prov - Creates provenance for the given commit, but does not check policy

Policy Commands (2 commands)

  • policy - tools to work with source policies
  • createpolicy - Creates a policy in a local copy of source-policies

Configuration & Setup Commands (2 commands)

  • auth - Manage user authentication
  • setup - configure SLSA source features in a repository

Implementation Details

  • Uses Cobra's AddGroup feature (supported in v1.10.1+)
  • Each command is assigned a GroupID to categorize it
  • "Assessment" encompasses both evaluation and attestation creation
  • "Verification" is reserved for commands that verify existing attestations

Rationale

Why move status to Assessment?

  • status reads GitHub API controls, not VSAs (same data sources as checklevel)
  • Functionally similar to checklevel - both evaluate controls to determine SLSA level
  • status is essentially a read-only version of checklevel
  • Makes "Verification" group more cohesive (only commands that verify existing attestations)

Why "Assessment" instead of "Evaluation"?

  • Broad enough to cover both evaluation and attestation creation
  • 4 of 5 commands evaluate policy
  • 4 of 5 commands create attestations
  • "Assessment" encompasses both aspects

Testing

  • ✅ All unit tests pass
  • ✅ golangci-lint clean (0 issues)
  • ✅ Help output verified manually

Before/After

Before:

Available Commands:
  audit          Audits the SLSA properties and controls of a repository
  auth           Manage user authentication
  checklevel     Determines the SLSA Source Level of the repo
  checklevelprov Checks the given commit against policy using & creating provenance
  checktag       Checks to see if the tag operation should be allowed and issues a VSA
  completion     Generate the autocompletion script for the specified shell
  createpolicy   Creates a policy in a local copy of source-policies
  help           Help about any command
  policy         tools to work with source policies
  prov           Creates provenance for the given commit, but does not check policy.
  setup          configure SLSA source features in a repository
  status         Check the SLSA Source status of a repo/branch
  verifycommit   Verifies the specified commit is valid

After:

Verification Commands:
  audit          Verifies multiple commits in the branch history
  verifycommit   Verifies the specified commit is valid

Assessment Commands:
  checklevel     Determines the SLSA Source Level of the repo
  checklevelprov Checks the given commit against policy using & creating provenance
  checktag       Checks to see if the tag operation should be allowed and issues a VSA
  prov           Creates provenance for the given commit, but does not check policy.
  status         Check the SLSA Source status of a repo/branch

Policy Commands:
  createpolicy   Creates a policy in a local copy of source-policies
  policy         tools to work with source policies

Configuration & Setup Commands:
  auth           Manage user authentication
  setup          configure SLSA source features in a repository

Generated with assistance from Claude Code

Organize the sourcetool CLI help output into three logical command
groups to improve discoverability and user experience:

- Verification Commands: verifycommit, audit, status
- Attestation & Evaluation Commands: checklevel, checklevelprov,
  checktag, prov
- Configuration & Setup Commands: setup, auth, policy, createpolicy

This uses Cobra's AddGroup feature to categorize commands by their
primary function, making it easier for users to find the command
they need.

Assisted-by: Claude Code
Signed-off-by: Ralph Bean <[email protected]>
Move createpolicy and policy commands from the Configuration & Setup
group into their own Policy Commands group for better organization.

The help output now shows four distinct command groups:
- Verification Commands
- Attestation & Evaluation Commands
- Policy Commands
- Configuration & Setup Commands

This makes policy management commands more discoverable and logically
separated from general configuration tasks.

Assisted-by: Claude Code
Signed-off-by: Ralph Bean <[email protected]>
Change audit command short description from "Audits the SLSA properties
and controls of a repository" to "Verifies multiple commits in the branch
history" to better describe what the command actually does.

The word "multiple" is more accurate than "all" since the command can be
limited with --depth and --ending-commit flags.

Assisted-by: Claude Code
Signed-off-by: Ralph Bean <[email protected]>
Rename "Attestation & Evaluation Commands" to "Assessment Commands"
and move status from Verification to Assessment group.

The new organization better reflects command behavior:

Verification Commands (2):
- audit: verifies multiple commits by reading existing VSAs
- verifycommit: verifies single commit by reading existing VSA

Assessment Commands (5):
- status: assesses current repository controls
- checklevel: assesses controls and creates VSA
- checklevelprov: assesses with provenance creation
- checktag: assesses tag operations
- prov: creates provenance without policy evaluation

"Assessment" encompasses both evaluation (status, checklevel) and
attestation creation (prov), making it a better umbrella term than
"Attestation & Evaluation".

Assisted-by: Claude Code
Signed-off-by: Ralph Bean <[email protected]>
Copy link
Contributor

@TomHennen TomHennen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this generally LGTM. I definitely understand the confusion around the many commands, most of that is my fault. I think @puerco has plans to eventually remove some of the redundant ones.

Does this look good to you Puerco?

Copy link
Collaborator

@puerco puerco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love how much clearer the CLI is now @ralphbean !

Even if we consolidate some of the commands later, the groups describe well the current and planned functions of the tool, so any upcoming features will fit within them. Thanks!

@puerco puerco merged commit 63c69ad into slsa-framework:main Oct 12, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants