Skip to content

Commit ea3f16a

Browse files
harupyBenWilson2dbczumar
authored
MLflow Gateway (mlflow#8694)
Signed-off-by: Ben Wilson <[email protected]> Signed-off-by: Ben Wilson <[email protected]> Signed-off-by: harupy <[email protected]> Signed-off-by: Harutaka Kawamura <[email protected]> Signed-off-by: dbczumar <[email protected]> Signed-off-by: Corey Zumar <[email protected]> Co-authored-by: Ben Wilson <[email protected]> Co-authored-by: Ben Wilson <[email protected]> Co-authored-by: dbczumar <[email protected]> Co-authored-by: Corey Zumar <[email protected]>
1 parent 3c4edb1 commit ea3f16a

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

68 files changed

+6618
-12
lines changed

‎.circleci/config.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ jobs:
3737
name: Install Python dependencies
3838
command: |
3939
pip --version
40-
pip install --progress-bar off -r requirements/doc-requirements.txt pytest pytest-cov plotly .
40+
pip install --progress-bar off -r requirements/doc-requirements.txt pytest pytest-cov plotly .[gateway]
4141
- run:
4242
name: Build documentation
4343
working_directory: docs

‎.github/ISSUE_TEMPLATE/bug_report_template.yaml

+2
Original file line numberDiff line numberDiff line change
@@ -195,6 +195,8 @@ body:
195195
required: false
196196
- label: "`area/examples`: Example code"
197197
required: false
198+
- label: "`area/gateway`: AI Gateway service, Gateway client APIs, third-party Gateway integrations"
199+
required: false
198200
- label: "`area/model-registry`: Model Registry service, APIs, and the fluent client calls for Model Registry"
199201
required: false
200202
- label: "`area/models`: MLmodel format, model serialization/deserialization, flavors"

‎.github/ISSUE_TEMPLATE/feature_request_template.yaml

+2
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,8 @@ body:
6666
required: false
6767
- label: "`area/examples`: Example code"
6868
required: false
69+
- label: "`area/gateway`: AI Gateway service, Gateway client APIs, third-party Gateway integrations"
70+
required: false
6971
- label: "`area/model-registry`: Model Registry service, APIs, and the fluent client calls for Model Registry"
7072
required: false
7173
- label: "`area/models`: MLmodel format, model serialization/deserialization, flavors"

‎.github/pull_request_template.md

+1
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@ Components
4848
- [ ] `area/build`: Build and test infrastructure for MLflow
4949
- [ ] `area/docs`: MLflow documentation pages
5050
- [ ] `area/examples`: Example code
51+
- [ ] `area/gateway`: AI Gateway service, Gateway client APIs, third-party Gateway integrations
5152
- [ ] `area/model-registry`: Model Registry service, APIs, and the fluent client calls for Model Registry
5253
- [ ] `area/models`: MLmodel format, model serialization/deserialization, flavors
5354
- [ ] `area/recipes`: Recipes, Recipe APIs, Recipe configs, Recipe Templates

‎.github/workflows/gateway.yml

+31
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
name: ⛩ Gateway ⛩
2+
3+
on:
4+
pull_request:
5+
push:
6+
branches:
7+
- master
8+
- branch-[0-9]+.[0-9]+
9+
10+
concurrency:
11+
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
12+
cancel-in-progress: true
13+
14+
defaults:
15+
run:
16+
shell: bash --noprofile --norc -exo pipefail {0}
17+
18+
jobs:
19+
gateway:
20+
runs-on: ubuntu-latest
21+
timeout-minutes: 30
22+
steps:
23+
- uses: actions/checkout@v3
24+
- uses: ./.github/actions/setup-python
25+
- name: Install dependencies
26+
run: |
27+
pip install -e .[gateway]
28+
pip install pytest pytest-timeout pytest-asyncio httpx
29+
- name: Run tests
30+
run: |
31+
pytest tests/gateway

‎.gitignore

+3
Original file line numberDiff line numberDiff line change
@@ -110,3 +110,6 @@ a.py
110110

111111
# Pytest-monitor load testing DB file
112112
*.pymon
113+
114+
# Ignore a gunicorn config file
115+
gunicorn.conf.py

‎ISSUE_TRIAGE.rst

+1
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,7 @@ Components
7171
- ``area/build``: Build and test infrastructure for MLflow
7272
- ``area/docs``: MLflow documentation pages
7373
- ``area/examples``: Example code
74+
- ``area/gateway``: AI Gateway service, Gateway client APIs, third-party Gateway integrations
7475
- ``area/model-registry``: Model Registry service, APIs, and the fluent client calls for Model Registry
7576
- ``area/models``: MLmodel format, model serialization/deserialization, flavors
7677
- ``area/recipes``: Recipes, Recipe APIs, Recipe configs, Recipe Templates

‎docs/Makefile

+5-1
Original file line numberDiff line numberDiff line change
@@ -68,9 +68,13 @@ rdocs:
6868
test-examples:
6969
@pytest --no-cov .examples
7070

71+
.PHONY: gateway-api-docs
72+
gateway-api-docs:
73+
@python gateway_api_docs.py
74+
7175
# Builds only the RST-based documentation (i.e., everything but Java & R docs)
7276
.PHONY: rsthtml
73-
rsthtml:
77+
rsthtml: gateway-api-docs
7478
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
7579
@echo
7680
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."

‎docs/gateway_api_docs.py

+96
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
import json
2+
import tempfile
3+
from pathlib import Path
4+
5+
from mlflow.gateway.app import create_app_from_path
6+
7+
# This HTML was obtained by sending a request to the `/docs` route and saving the response.
8+
# To hide the "try it out" button, we set `supportedSubmitMethods` to an empty list.
9+
# The url was changed to "./openapi.json" from "/openapi.json" because `api.html` and `openapi.json`
10+
# are served from the same directory.
11+
API_HTML = """
12+
<!DOCTYPE html>
13+
<html>
14+
<head>
15+
<link
16+
type="text/css"
17+
rel="stylesheet"
18+
href="https://cdn.jsdelivr.net/npm/swagger-ui-dist@4/swagger-ui.css"
19+
/>
20+
<link
21+
rel="shortcut icon"
22+
href="../_static/favicon.ico"
23+
/>
24+
<title>MLflow Gateway API - Swagger UI</title>
25+
</head>
26+
<body>
27+
<div id="swagger-ui"></div>
28+
<script src="https://cdn.jsdelivr.net/npm/swagger-ui-dist@4/swagger-ui-bundle.js"></script>
29+
<!-- `SwaggerUIBundle` is now available on the page -->
30+
<script>
31+
const ui = SwaggerUIBundle({
32+
supportedSubmitMethods: [],
33+
url: "./openapi.json",
34+
dom_id: "#swagger-ui",
35+
layout: "BaseLayout",
36+
deepLinking: true,
37+
showExtensions: true,
38+
showCommonExtensions: true,
39+
oauth2RedirectUrl: window.location.origin + "/docs/oauth2-redirect",
40+
presets: [
41+
SwaggerUIBundle.presets.apis,
42+
SwaggerUIBundle.SwaggerUIStandalonePreset,
43+
],
44+
});
45+
</script>
46+
</body>
47+
</html>
48+
"""
49+
50+
51+
def main():
52+
config = """
53+
routes:
54+
- name: chat
55+
route_type: llm/v1/chat
56+
model:
57+
provider: openai
58+
name: gpt-3.5-turbo
59+
config:
60+
openai_api_base: https://api.openai.com/v1
61+
openai_api_key: key
62+
63+
- name: completions
64+
route_type: llm/v1/completions
65+
model:
66+
provider: openai
67+
name: gpt-3.5-turbo
68+
config:
69+
openai_api_base: https://api.openai.com/v1
70+
openai_api_key: key
71+
72+
- name: embeddings
73+
route_type: llm/v1/embeddings
74+
model:
75+
provider: openai
76+
name: text-embedding-ada-002
77+
config:
78+
openai_api_base: https://api.openai.com/v1
79+
openai_api_key: key
80+
"""
81+
with tempfile.TemporaryDirectory() as tmpdir:
82+
config_path = Path(tmpdir).joinpath("config.yaml")
83+
config_path.write_text(config)
84+
85+
app = create_app_from_path(config_path)
86+
docs_build = Path("build/html/gateway")
87+
docs_build.mkdir(parents=True, exist_ok=True)
88+
with docs_build.joinpath("openapi.json").open("w") as f:
89+
json.dump(app.openapi(), f)
90+
91+
with docs_build.joinpath("api.html").open("w") as f:
92+
f.write(API_HTML)
93+
94+
95+
if __name__ == "__main__":
96+
main()

‎docs/source/conf.py

+1
Original file line numberDiff line numberDiff line change
@@ -351,6 +351,7 @@
351351
("py:class", "scipy.sparse._csr.csr_matrix"),
352352
("py:class", "scipy.sparse._csc.csc_matrix"),
353353
("py:class", "pathlib.Path"),
354+
("py:class", "pydantic.main.BaseModel"),
354355
]
355356

356357

0 commit comments

Comments
 (0)