Skip to content

feat(filter): add grid route inference routing#802

Closed
nerdalert wants to merge 2 commits into
praxis-proxy:mainfrom
nerdalert:praxis-05-grid-route
Closed

feat(filter): add grid route inference routing#802
nerdalert wants to merge 2 commits into
praxis-proxy:mainfrom
nerdalert:praxis-05-grid-route

Conversation

@nerdalert

@nerdalert nerdalert commented Jul 12, 2026

Copy link
Copy Markdown
Member

Summary

This PR adds the grid_route HTTP filter for static AI Grid inference routing.

grid_route selects an upstream cluster from a configured list of route candidates. It is intended to sit after a request-inspection filter such as json_body_field, which extracts the requested model into a header, and before load_balancer, which forwards to the selected cluster.

The filter provides the first Praxis-side routing primitive needed for Grid gateway-to-gateway inference routing:

  • matches inference requests by a configured model header, defaulting to X-Model
  • supports static inference_model route candidates with name, site, cluster, and fresh
  • prefers fresh candidates over stale candidates
  • prefers candidates on the configured local_site when freshness is equal
  • writes the selected cluster into ctx.cluster for downstream filters
  • preserves an existing ctx.cluster if an earlier filter already selected one
  • records bounded route-decision metadata under the grid.route.* namespace
  • fails closed for invalid model header values and unknown models
  • performs no request-time database, control-plane, metrics, or network lookups

This PR also adds:

  • the filter/src/builtins/http/ai/grid/ module structure
  • grid_route registration in the built-in filter registry
  • generated filter reference docs for grid_route
  • an example config showing json_body_field -> grid_route -> load_balancer
  • schema coverage for the new example config
  • focused unit coverage for descriptor validation, route selection, scoring, metadata, and error behavior

MCP tool routing and A2A routing are intentionally out of scope for this PR. This change only accepts and routes inference_model candidates.

Behavior

Routing is deterministic and local to the request.

Selection flow:

  1. If ctx.cluster is already set, grid_route does not override it.
  2. The filter reads the configured model header.
  3. Missing model header means no routing decision is made and processing continues.
  4. Blank, oversized, or invalid header values are rejected with 400.
  5. Matching inference_model candidates are scored.
  6. The highest-scoring candidate sets ctx.cluster.
  7. If no candidate matches the requested model, the request is rejected with 404.

Candidate scoring:

  • fresh candidates outrank stale candidates
  • local-site candidates outrank remote candidates when freshness is equal
  • freshness is stronger than locality
  • equal scores preserve configured candidate order

Validation

  • Added focused grid_route unit coverage for descriptor parsing, candidate validation, model-header handling, route scoring,
    selected-cluster behavior, and metadata emission.
  • Added schema coverage for examples/configs/ai/grid-route-inference.yaml.
  • Regenerated filter docs/reference entries.
  • Verified the branch uses 2026 headers for new files.
  • Verified the example and tests avoid topology-specific Grid demo fixtures.
  • This branch was included in the combined Grid gateway-to-gateway validation stack used to verify the Praxis/Grid integration path.
  • Included this branch in the combined Grid gateway-to-gateway validation stack used to verify the Praxis/Grid integration path, see the validation package in nerdalert/praxis-research-spikes.

@nerdalert nerdalert requested review from a team July 12, 2026 12:51
@nerdalert nerdalert requested a review from shaneutt as a code owner July 12, 2026 12:51
@praxis-bot-app

Copy link
Copy Markdown

PR too large: 1145 lines added (limit: 750, excludes Cargo files, tests, docs, examples, and benchmarks). Please split into smaller PRs. Add skip/pr-conventions label to override.

@nerdalert nerdalert added the skip/pr-conventions Skip conventions checks for PRs label Jul 12, 2026
@nerdalert nerdalert force-pushed the praxis-05-grid-route branch from 888f077 to 44e41fe Compare July 12, 2026 12:57
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
@nerdalert nerdalert force-pushed the praxis-05-grid-route branch from 44e41fe to 3bf742e Compare July 12, 2026 13:14
@shaneutt shaneutt self-assigned this Jul 13, 2026
@shaneutt shaneutt moved this from Next to Review in AI Gateway - Model Serving Jul 13, 2026
@shaneutt shaneutt added this to the v0.5.0 milestone Jul 13, 2026

@praxis-bot praxis-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: grid_route inference routing filter

Purpose: Adds a grid_route HTTP filter that selects an upstream cluster from a static candidate list by matching the inference model name from a configured request header.

Overall: Well-structured implementation with clean separation between descriptor validation (descriptor.rs) and route selection logic (route.rs). Candidate scoring is deterministic, well-documented, and thoroughly unit-tested. Five medium findings to address, primarily around a missing trait implementation and test coverage gaps.

Rebase required: filter/src/builtins/http/ai/mod.rs is created from scratch on this branch (the diff shows @@ -0,0), but the file already exists on current main with existing AI filter modules (agentic, inference, openai, prompt_enrich). Similarly, http/mod.rs and builtins/mod.rs have diverged. Merging without rebasing will produce conflicts that could break existing AI filters if resolved incorrectly.

Severity Count
Critical 0
Large 0
Medium 5

Comment thread filter/src/builtins/http/ai/grid/route.rs
Comment thread tests/schema/tests/suite/examples/ai/grid_route_inference.rs
Comment thread filter/src/builtins/http/traffic_management/router/mod.rs
Comment thread filter/src/registry.rs
Comment thread filter/src/builtins/http/ai/grid/descriptor.rs Outdated
@praxis-bot-app

Copy link
Copy Markdown

Missing Signed-off-by: 4609115. All commits require sign-off (via git commit --signoff).

Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
@nerdalert nerdalert force-pushed the praxis-05-grid-route branch from 4609115 to 441cfc3 Compare July 13, 2026 16:09
@nerdalert

Copy link
Copy Markdown
Member Author

Closing to move to the AI repo

@nerdalert nerdalert closed this Jul 14, 2026
@github-project-automation github-project-automation Bot moved this from Review to Done in AI Gateway - Model Serving Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip/pr-conventions Skip conventions checks for PRs

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants