Skip to content

Calculate validation timeout from matrix plan - #171

Merged
almaslennikov merged 1 commit into
mainfrom
dev/validate-auto-timeout
Aug 14, 2026
Merged

Calculate validation timeout from matrix plan#171
almaslennikov merged 1 commit into
mainfrom
dev/validate-auto-timeout

Conversation

@almaslennikov

Copy link
Copy Markdown
Collaborator

Summary

  • replace the fixed five-minute connectivity deadline with an automatic budget derived from the generated matrix plan
  • account for selected test families, source-route probes, ordered pod-pair batch deadlines, setup, cleanup, and a safety margin
  • preserve a positive --connectivity-timeout as an explicit hard setup and execution deadline
  • log the automatic total budget before matrix execution, or the explicit user budget when overridden
  • document the behavior in the README, user guide, CLI reference, and bundled validate skill

For the 3-node, 4-rail quick matrix that plans 144 checks, the calculated total budget is 2h10m24s.

Testing

  • make build
  • make test
  • go test -race -count=1 ./...
  • golangci-lint v2.11.4 run ./... (0 issues)
  • git diff --check

Replace the fixed five-minute connectivity deadline with a budget derived from selected checks and ordered pod-pair batches. Preserve a positive --connectivity-timeout as an explicit hard override and log the selected total before execution.

Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
@greptile-apps

greptile-apps Bot commented Aug 14, 2026

Copy link
Copy Markdown

Greptile Summary

The PR replaces the fixed connectivity timeout with separate bounded setup and matrix-derived execution deadlines, while retaining positive user-supplied values as hard end-to-end deadlines.

  • Calculates execution allowance from selected test families, route checks, ordered RDMA batches, command limits, cleanup, and a safety margin.
  • Centralizes connectivity timeout constants and applies the calculated context across ICMP, RDMA, and GPUDirect stages.
  • Updates CLI tests, timeout-budget tests, user documentation, and the validation skill.

Confidence Score: 5/5

The PR appears safe to merge with no actionable correctness or security issues identified.

The calculated budget mirrors the serial connectivity execution graph, context transitions remain valid in both automatic and explicit modes, and duplicated documentation surfaces consistently describe the new default.

Important Files Changed

Filename Overview
pkg/networkoperatorplugin/connectivity/connectivity.go Splits automatic setup and execution contexts, applies the calculated deadline consistently to matrix stages, and preserves explicit end-to-end deadlines.
pkg/networkoperatorplugin/connectivity/timeout.go Defines centralized timeout constants and a budget calculation matching the serial route, command, and ordered-batch execution graph.
pkg/networkoperatorplugin/connectivity/rdma.go Reuses centralized command and cleanup limits without changing RDMA result semantics.
pkg/cmd/validate.go Changes the connectivity-timeout default from five minutes to automatic calculation while preserving positive overrides.
pkg/networkoperatorplugin/connectivity/timeout_test.go Covers selected-family accounting, expectation-dependent route budgets, empty plans, and the documented 144-test matrix.
docs/user/validation.md Documents automatic and explicit deadline behavior consistently with the implementation.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Start connectivity validation] --> B{Timeout positive?}
    B -->|Yes| C[Create explicit end-to-end deadline]
    B -->|No| D[Create bounded setup context]
    C --> E[Apply workload and discover pods]
    D --> E
    E --> F[Generate matrix plan]
    F --> G{Automatic mode?}
    G -->|Yes| H[Calculate execution budget from planned tests]
    G -->|No| I[Reuse explicit deadline]
    H --> J[Run route, RDMA, GPUDirect, and ICMP stages]
    I --> J
    J --> K[Best-effort cleanup with independent context]
Loading

Reviews (1): Last reviewed commit: "Calculate validation timeout from matrix..." | Re-trigger Greptile

@almaslennikov
almaslennikov merged commit 7bce0be into main Aug 14, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant