Skip to content

feat(llm-d): move ext_proc compatibility into AI#334

Draft
nerdalert wants to merge 2 commits into
praxis-proxy:mainfrom
nerdalert:llmd-ext-proc-transfer
Draft

feat(llm-d): move ext_proc compatibility into AI#334
nerdalert wants to merge 2 commits into
praxis-proxy:mainfrom
nerdalert:llmd-ext-proc-transfer

Conversation

@nerdalert

@nerdalert nerdalert commented Jul 11, 2026

Copy link
Copy Markdown
Member

Summary

Moves the llm-d ext_proc compatibility layer into the AI repository under integrations/llmd/ext-proc/.

This replaces the abandoned Praxis-side crate publication approach. See the comments from @shaneutt in praxis-proxy/praxis#780 for details. I am pushing a draft as I need it for llm-d integration work in grid so this may be temporary if Shane has other plans for gating ext_proc exposure.

The crate is AI-owned, publish = false, and scoped to llm-d’s current EPP protocol rather than general-purpose Envoy ext_proc support.

Runtime registration is gated behind the praxis-ai-proxy/llmd-ext-proc feature. Default praxis-ai-proxy builds do not link, register, or expose the ext_proc filter.

What changed

  • Added praxis-ai-llmd-ext-proc at integrations/llmd/ext-proc/
  • Added llmd-ext-proc feature to praxis-ai-proxy
  • Added six llm-d environment tests using llm-d-inference-sim
  • Added docs for the support boundary and test flow
  • Temporarily pinned Praxis to 546871d8fdb85a6b0260e77ee4a63083c1c097fb until the next Praxis release includes required support APIs
  • Allowed the pinned Praxis git source in deny.toml

Why draft

Opening this as a draft because the ownership and support boundary should be reviewed before treating this as final.

The main points needing maintainer review are:

  • whether integrations/llmd/ext-proc/ is the right long-term location for the llm-d-only compatibility layer
  • whether the llmd-ext-proc feature boundary is strict enough
  • whether the public wording is clear that this is not general-purpose Envoy ext_proc support

Dependency pin

This branch temporarily pins the Praxis crates to:

546871d8fdb85a6b0260e77ee4a63083c1c097fb

The current published 0.4.0 Praxis crates do not include APIs used by this transfer, including:

  • TrustedHeaderMutation
  • pre_read_mutations
  • set_structured_metadata
  • MetricsConfig

The intent is not to keep a git dependency permanently. Once Praxis publishes a release containing those APIs, this should move back to normal released crate dependencies.

Known caveats

  • cargo deny check currently exits 2 because of pre-existing duplicate-version bans on AI main. The branch-specific Praxis git source allowance passes under sources ok.

  • The COPY examples ./examples container fix is included here so the draft branch can build images. It may be dropped during rebase if the standalone container fix lands first in fix(container): restore container image builds #333.

  • The Praxis git pin is temporary and should be replaced by released Praxis crate versions once the required APIs are published.

    Validation

    • cargo check --workspace
    • cargo check --features praxis-ai-proxy/llmd-ext-proc
    • cargo clippy --workspace --all-targets -- -D warnings
    • cargo clippy --workspace --all-targets --features praxis-ai-proxy/llmd-ext-proc -- -D warnings
    • cargo test --workspace --exclude praxis-tests-environment
    • cargo test -p praxis-tests-environment --features llmd-ext-proc -- --test-threads=1
    • RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps --document-private-items
    • make container
    • git diff --check

Move the llm-d ext_proc compatibility layer into the AI repository under integrations/llmd/ext-proc and gate runtime registration behind the llmd-ext-proc feature.

The crate is publish=false and scoped to llm-d's current EPP protocol rather than general-purpose Envoy ext_proc support. Environment tests cover the mock EPP to endpoint_selector to llm-d-inference-sim path.

Temporarily pin Praxis to 546871d until the next release includes the required support APIs.

Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant