Skip to content

ci: five workflows run on every push with no path filter #977

Description

@balajinvda

Problem

Seven workflows fire on every push. Five of them have no path filter at all:

build-test.yml            runs on every push
license-dependencies.yml  runs on every push
secret-scan.yml           runs on every push
byoo-otel-collector.yml   runs on every push
bazel.yml                 runs on every push
codeql.yml                paths-filtered
openbao-jwt-plugin.yml    paths-filtered

The volume this produces is what pushes the tail of runner wait times out. Measured 2026-08-18:

day            n   median    p90     max   over 60s
Fri 08-14     32       2s     3s      5s      0%
Mon 08-17     48       1s    59s    147s      8%
Tue 08-18     37       0s    43s    606s      8%

Median is fine; the worst case went from 5 seconds on Friday to 10 minutes. At the time of measurement 15 runs were queued, the oldest for 35 minutes. 881 runs that day, with the 100 most recent all inside a single hour, against 770 for all of Friday.

The 10 minute waits cluster inside single runs (dependency licenses, go-lib codegen, docs, GitHub release helper, license headers, generated dependency docs and Secret Scan all waited about 610s together), which is a concurrency ceiling rather than a slow runner. GitHub reported all systems operational with no incidents.

One branch, nvsnap/e2e-restore-guards, produced 25 runs in a day by itself across about four pushes.

Proposal

Extend the pattern from #949 to the remaining always-on workflows: keep the job running so it always reports and branch protection is untouched, and gate only the expensive steps on a scope decision computed from the merge base.

The two constraints that made #949 non-trivial apply here too and should not be re-derived:

  • A skipped required check never reports, so on.pull_request.paths blocks the merge unless a companion no-op job is added. Gating steps rather than the job avoids this entirely.
  • The input set is usually wider than it looks. For generated dependency docs the Java runtime inventory is built through Bazel, so a Java component's BUILD.bazel counts even though no manifest changed. Each workflow needs its real input set worked out, not guessed, and the decision must fail closed.

bazel.yml is already change-aware and should stay as is; #974 narrows its one remaining over-broad edge.

Order

build-test.yml and byoo-otel-collector.yml first: they are the ones most likely to be scopeable to a subtree. secret-scan.yml should probably stay unscoped, since scanning only changed paths defeats the point of scanning.

Each one needs its own change with its own tests, in the shape of tools/ci/dependency-docs-scope and its behavioral test, so the fail-closed behaviour is verifiable rather than asserted.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions