Provide standalone CLI/Docker image that can be run without k8s/EG

[mathetake]

Description:

Since we have a clearly separated interface between External Processor and K8s/EG specific knowledge, it should be straightforward to provide the standalone CLI or Docker image that can be run locally by users on Mac or Linux distributions.

In fact, there's already external processing stand alone end-to-end tests in tests/extproc which doesn't require any k8s/EG setup. So it should be easy to create one.

This will drive the adoption of the project as well as our development since the majority of future features like prompt guard, etc should be independent of the platform, and it would be nice to allow developers to iterate on it without the heavy setups etc.

[optional Relevant Links:]

Any extra documentation required to understand the issue.

[codefromthecrypt]

I would really love to see something like this. We could borrow or steal code from func-e if we need to provision envoy vs configure a pre-existing

[mathetake]

so basically this is about how to package https://github.com/envoyproxy/ai-gateway/tree/main/tests/extproc and how much we want to expose the public external processing config in filterconfig in that standalone version....

So basically everything needed to create is in a public package right now (filterconfig and extproc main cmd), so it is even possible to create this outside this repository ;)

[mathetake]

ok i got an idea... the cli can also runs the external processing server internally inside the process since both written in Go so if this could be combined with func-e like stuff this could be real easy.

[mathetake]

but mac envoy binary is disastrous right now (brew is 2 versions behind) so maybe simply use the docker would be simplest

[codefromthecrypt]

agree there's tension on brew versions, but also there's some benefit not requiring docker in the the example (e.g. running it inside docker).

Can we check out updating brew, they tend to bump quickly on PR, just no one in envoy looks at it by default so tends to not look after itself..

[codefromthecrypt]

in any case a docker mode is fine and a good start vs k8s

[mathetake]

Yep let’s start with docker and wait for envoy brew to be updated (this project relies on the features on latest envoy)

[mathetake]

assigning @codefromthecrypt

[mathetake]

brew is fixed now!

[arkodg]

Envoy Gateway recently added support for standalone mode https://gateway.envoyproxy.io/docs/tasks/operations/standalone-deployment-mode/ and uses func-e under the hood for this, sharing this here in case this helps with this use case