feat(llm-d): move ext_proc compatibility into AI#334
Draft
nerdalert wants to merge 2 commits into
Draft
Conversation
Move the llm-d ext_proc compatibility layer into the AI repository under integrations/llmd/ext-proc and gate runtime registration behind the llmd-ext-proc feature. The crate is publish=false and scoped to llm-d's current EPP protocol rather than general-purpose Envoy ext_proc support. Environment tests cover the mock EPP to endpoint_selector to llm-d-inference-sim path. Temporarily pin Praxis to 546871d until the next release includes the required support APIs. Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Moves the llm-d
ext_proccompatibility layer into the AI repository underintegrations/llmd/ext-proc/.This replaces the abandoned Praxis-side crate publication approach. See the comments from @shaneutt in praxis-proxy/praxis#780 for details. I am pushing a draft as I need it for llm-d integration work in grid so this may be temporary if Shane has other plans for gating ext_proc exposure.
The crate is AI-owned,
publish = false, and scoped to llm-d’s current EPP protocol rather than general-purpose Envoyext_procsupport.Runtime registration is gated behind the
praxis-ai-proxy/llmd-ext-procfeature. Defaultpraxis-ai-proxybuilds do not link, register, or expose theext_procfilter.What changed
praxis-ai-llmd-ext-procatintegrations/llmd/ext-proc/llmd-ext-procfeature topraxis-ai-proxyllm-d-inference-sim546871d8fdb85a6b0260e77ee4a63083c1c097fbuntil the next Praxis release includes required support APIsdeny.tomlWhy draft
Opening this as a draft because the ownership and support boundary should be reviewed before treating this as final.
The main points needing maintainer review are:
integrations/llmd/ext-proc/is the right long-term location for the llm-d-only compatibility layerllmd-ext-procfeature boundary is strict enoughext_procsupportDependency pin
This branch temporarily pins the Praxis crates to:
546871d8fdb85a6b0260e77ee4a63083c1c097fbThe current published
0.4.0Praxis crates do not include APIs used by this transfer, including:TrustedHeaderMutationpre_read_mutationsset_structured_metadataMetricsConfigThe intent is not to keep a git dependency permanently. Once Praxis publishes a release containing those APIs, this should move back to normal released crate dependencies.
Known caveats
cargo deny checkcurrently exits 2 because of pre-existing duplicate-version bans on AI main. The branch-specific Praxis git source allowance passes undersources ok.The
COPY examples ./examplescontainer fix is included here so the draft branch can build images. It may be dropped during rebase if the standalone container fix lands first in fix(container): restore container image builds #333.The Praxis git pin is temporary and should be replaced by released Praxis crate versions once the required APIs are published.
Validation
cargo check --workspacecargo check --features praxis-ai-proxy/llmd-ext-proccargo clippy --workspace --all-targets -- -D warningscargo clippy --workspace --all-targets --features praxis-ai-proxy/llmd-ext-proc -- -D warningscargo test --workspace --exclude praxis-tests-environmentcargo test -p praxis-tests-environment --features llmd-ext-proc -- --test-threads=1RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps --document-private-itemsmake containergit diff --check