feat(cli): add openshell conformance subcommand skeleton#2182
Draft
elezar wants to merge 2 commits into
Draft
Conversation
Adds a hidden `openshell conformance` subcommand behind a `conformance` Cargo feature flag. The subcommand connects to a registered gateway and runs a suite of driver-agnostic conformance scenarios, reporting pass/fail per scenario and exiting non-zero on any failure. Implements the `lifecycle` scenario (create → ready → delete) as the first concrete scenario. The remaining five scenarios are stubbed as "not yet implemented" placeholders. Sandbox names follow the `conformance-<scenario>-<run-id>` convention (Kubernetes RFC 1123 safe) to make orphaned sandboxes identifiable. Signed-off-by: Evan Lezar <elezar@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
…te, validate, concurrent, labels Adds five new conformance scenarios and a shared wait_for_ready helper: - not-found: verifies GetSandbox returns NOT_FOUND and DeleteSandbox returns deleted=false for a sandbox name that was never created - idempotent-delete: creates a sandbox, deletes it (deleted=true), then deletes it again and asserts deleted=false without an error - validate: asserts INVALID_ARGUMENT for spec=None and gpu.count=0, covering both the missing-spec and field-constraint validation paths - concurrent: issues two CreateSandbox calls via tokio::join!, waits for both to reach Ready in parallel, and verifies neither interferes with the other in ListSandboxes - labels: creates two sandboxes with distinct label values, filters with label_selector, and asserts only the matching sandbox is returned Also extracts wait_for_ready from scenario_lifecycle into a shared helper and adds the labels entry to all_scenarios(). Signed-off-by: Evan Lezar <elezar@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a hidden
openshell conformancesubcommand behind aconformanceCargofeature flag. The subcommand connects to a registered gateway and runs a suite
of driver-agnostic conformance scenarios, reporting pass/fail per scenario and
exiting non-zero on any failure.
Related Issue
Accompanies the driver conformance suite spike issue. This PR makes the design
concrete; it is not intended to merge as-is.
Changes
crates/openshell-cli/Cargo.toml— addsconformancefeature flagcrates/openshell-cli/src/main.rs— addsConformancecommand variant andConformanceCommandsenum (run,list)crates/openshell-cli/src/run.rs— addsconformance_run,conformance_list,run_scenario,wait_for_ready, and all implemented scenario functionsScenarios
capabilitiesGetCapabilitiesreturns non-empty driver name, version, and image (blocked: requires proto change)lifecyclenot-foundGetSandboxreturnsNOT_FOUND;DeleteSandboxreturnsdeleted=falsewithout erroringidempotent-deleteDeleteSandboxcall returnsdeleted=falsewithout erroringvalidatespec=Noneandgpu.count=0both returnINVALID_ARGUMENTbefore any sandbox is createdconcurrenttokio::join!both reach Ready without interferinglabelslabel_selectorreturns only the matching sandboxNotes
conformance-<scenario>-<run-id>convention (Kubernetes RFC 1123 safe)conformance listrequires no gateway connection — purely static metadatatokio::time::timeout, default 300swait_for_readyis a shared helper; cleanup is always the caller's responsibilitycapabilitiesremains stubbed pending a decision on whether to add a gateway passthrough RPC or accept a--driver-addrflagTesting
mise run pre-commitpasses