Skip to content

Commit 28e6937

Browse files
sunishsheth2009sunishsheth2009
authored
Initial setup to migrate pipeline to recipe (mlflow#7221)
* Initial setup to migrate pipeline to recipe Signed-off-by: Sunish Sheth <[email protected]> * Update tests Signed-off-by: Sunish Sheth <[email protected]> * Updating docs to pass build-ci Signed-off-by: Sunish Sheth <[email protected]> * Fixing tests Signed-off-by: Sunish Sheth <[email protected]> * Fixing docs test Signed-off-by: Sunish Sheth <[email protected]> Signed-off-by: Sunish Sheth <[email protected]>
1 parent 8cd2eb0 commit 28e6937

File tree

84 files changed

+2568
-2611
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

84 files changed

+2568
-2611
lines changed

‎.circleci/config.yml

+9-11
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ jobs:
6767
- store_artifacts:
6868
path: docs/build/html
6969

70-
mlflow-pipelines:
70+
mlflow-recipes:
7171
docker:
7272
- image: cimg/python:3.8
7373

@@ -84,28 +84,26 @@ jobs:
8484
pip install nbconvert ipykernel matplotlib
8585
- run:
8686
name: Run CLI
87-
working_directory: examples/pipelines/sklearn_regression_example
87+
working_directory: examples/recipes/regression
8888
environment:
89-
MLFLOW_PIPELINES_PROFILE: local
89+
MLFLOW_RECIPES_PROFILE: local
9090
command: |
91-
# Temporary patch for custom_metrics.py to make it return a scalar value
92-
cat ../../../dev/custom_metrics_patch.py > steps/custom_metrics.py
93-
mlp run
91+
mlflow recipes run
9492
- run:
9593
name: Run notebook
96-
working_directory: examples/pipelines/sklearn_regression_example
94+
working_directory: examples/recipes/regression
9795
environment:
98-
MLFLOW_PIPELINES_PROFILE: local
96+
MLFLOW_RECIPES_PROFILE: local
9997
command: |
10098
jupyter nbconvert --to html --execute notebooks/jupyter.ipynb
10199
- store_artifacts:
102-
path: ~/.mlflow/pipelines
100+
path: ~/.mlflow/recipes
103101
- store_artifacts:
104-
path: examples/pipelines/sklearn_regression_example/notebooks/jupyter.html
102+
path: examples/recipes/regression/notebooks/jupyter.html
105103

106104
workflows:
107105
build:
108106
jobs:
109107
- build_doc
110108
- build_doc_r
111-
- mlflow-pipelines
109+
- mlflow-recipes

‎.github/ISSUE_TEMPLATE/bug_report_template.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -151,7 +151,7 @@ body:
151151
required: false
152152
- label: "`area/models`: MLmodel format, model serialization/deserialization, flavors"
153153
required: false
154-
- label: "`area/pipelines`: Pipelines, Pipeline APIs, Pipeline configs, Pipeline Templates"
154+
- label: "`area/recipes`: Recipes, Recipe APIs, Recipe configs, Recipe Templates"
155155
required: false
156156
- label: "`area/projects`: MLproject format, project running backends"
157157
required: false

‎.github/ISSUE_TEMPLATE/feature_request_template.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ body:
7070
required: false
7171
- label: "`area/models`: MLmodel format, model serialization/deserialization, flavors"
7272
required: false
73-
- label: "`area/pipelines`: Pipelines, Pipeline APIs, Pipeline configs, Pipeline Templates"
73+
- label: "`area/recipes`: Recipes, Recipe APIs, Recipe configs, Recipe Templates"
7474
required: false
7575
- label: "`area/projects`: MLproject format, project running backends"
7676
required: false

‎.github/pull_request_template.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ Components
4747
- [ ] `area/examples`: Example code
4848
- [ ] `area/model-registry`: Model Registry service, APIs, and the fluent client calls for Model Registry
4949
- [ ] `area/models`: MLmodel format, model serialization/deserialization, flavors
50-
- [ ] `area/pipelines`: Pipelines, Pipeline APIs, Pipeline configs, Pipeline Templates
50+
- [ ] `area/recipes`: Recipes, Recipe APIs, Recipe configs, Recipe Templates
5151
- [ ] `area/projects`: MLproject format, project running backends
5252
- [ ] `area/scoring`: MLflow Model server, model deployment tools, Spark UDFs
5353
- [ ] `area/server-infra`: MLflow Tracking server backend

‎.github/workflows/master.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -293,4 +293,4 @@ jobs:
293293
export HADOOP_HOME=`realpath winutils/hadoop-3.2.2`
294294
export PATH=$PATH:$HADOOP_HOME/bin
295295
# Run Windows tests
296-
pytest --ignore-flavors --ignore=tests/projects --ignore=tests/examples tests --ignore=tests/pipelines
296+
pytest --ignore-flavors --ignore=tests/projects --ignore=tests/examples tests --ignore=tests/recipes

‎.github/workflows/pipeline-template.yml

+5-5
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
1-
name: Pipeline template tests
1+
name: Recipe template tests
22

33
on:
44
workflow_dispatch:
55
inputs:
66
repository:
77
description: >
8-
[Optional] Repository name with owner. For example, mlflow/mlp-regression-template.
8+
[Optional] Repository name with owner. For example, mlflow/recipes-examples.
99
required: true
10-
default: "mlflow/mlp-regression-template"
10+
default: "mlflow/recipes-examples"
1111
ref:
1212
description: >
1313
[Optional] The branch, tag or SHA to checkout. When checking out the repository that
@@ -53,7 +53,7 @@ jobs:
5353
- name: Run lint checks
5454
run: |
5555
cd ${{ github.event.inputs.repository }} && ../../dev/lint.sh
56-
pipeline:
56+
recipe:
5757
runs-on: ubuntu-latest
5858
steps:
5959
- uses: actions/checkout@v3
@@ -73,7 +73,7 @@ jobs:
7373
- name: Run tests
7474
run: |
7575
cd ${{ github.event.inputs.repository }} && pytest tests --rootdir .
76-
pipeline-windows:
76+
recipe-windows:
7777
runs-on: windows-latest
7878
steps:
7979
- uses: actions/checkout@v3

‎.github/workflows/pipeline.yml

+6-10
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: Pipeline tests
1+
name: Recipe tests
22

33
on:
44
pull_request:
@@ -39,7 +39,7 @@ env:
3939
SPARK_LOCAL_IP: localhost
4040

4141
jobs:
42-
pipelines:
42+
recipes:
4343
runs-on: ubuntu-latest
4444
steps:
4545
- uses: actions/checkout@v3
@@ -55,12 +55,10 @@ jobs:
5555
pip install 'pyspark<3.3'
5656
- name: Run tests
5757
run: |
58-
# Temporary patch for custom_metrics.py to make it return a scalar value
59-
cat dev/custom_metrics_patch.py > examples/pipelines/sklearn_regression_example/steps/custom_metrics.py
6058
export MLFLOW_HOME=$(pwd)
61-
pytest tests/pipelines
59+
pytest tests/recipes
6260
63-
pipelines-windows:
61+
recipes-windows:
6462
runs-on: windows-latest
6563
steps:
6664
- uses: actions/checkout@v3
@@ -84,8 +82,6 @@ jobs:
8482
# Set Hadoop environment variables required for testing Spark integrations on Windows
8583
export HADOOP_HOME=`realpath winutils/hadoop-3.2.2`
8684
export PATH=$PATH:$HADOOP_HOME/bin
87-
# Temporary patch for custom_metrics.py to make it return a scalar value
88-
cat dev/custom_metrics_patch.py > examples/pipelines/sklearn_regression_example/steps/custom_metrics.py
89-
# Run pipelines tests
85+
# Run recipes tests
9086
export MLFLOW_HOME=$(pwd)
91-
pytest tests/pipelines
87+
pytest tests/recipes

‎.gitmodules

+3-4
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
[submodule "examples/pipelines/sklearn_regression_example"]
2-
path = examples/pipelines/sklearn_regression_example
3-
url = https://github.com/mlflow/mlp-regression-example.git
4-
branch = sunish-pipeline-2.0
1+
[submodule "examples/recipes"]
2+
path = examples/recipes
3+
url = https://github.com/mlflow/recipes-examples.git

‎ISSUE_TRIAGE.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,7 @@ Components
7373
- ``area/examples``: Example code
7474
- ``area/model-registry``: Model Registry service, APIs, and the fluent client calls for Model Registry
7575
- ``area/models``: MLmodel format, model serialization/deserialization, flavors
76-
- ``area/pipelines``: Pipelines, Pipeline APIs, Pipeline configs, Pipeline Templates
76+
- ``area/recipes``: Recipes, Recipe APIs, Recipe configs, Recipe Templates
7777
- ``area/projects``: MLproject format, project execution backends
7878
- ``area/scoring``: MLflow Model server, model deployment tools, Spark UDFs
7979
- ``area/server-infra``: MLflow Tracking server backend

‎dev/run-python-tests.sh

+1-1
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,6 @@ err=0
66
trap 'err=1' ERR
77
export MLFLOW_HOME=$(pwd)
88

9-
pytest tests --quiet --requires-ssh --ignore-flavors --ignore=tests/examples --ignore=tests/pipelines
9+
pytest tests --quiet --requires-ssh --ignore-flavors --ignore=tests/examples --ignore=tests/recipes
1010

1111
test $err = 0

‎docs/source/index.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ Get started using the :ref:`quickstart` or by reading about the :ref:`key concep
2929
projects
3030
models
3131
model-registry
32-
pipelines
32+
recipes
3333
plugins
3434
cli
3535
search-runs

‎docs/source/python_api/mlflow.pipelines.rst

-29
This file was deleted.

‎docs/source/python_api/mlflow.recipes.rst

+29
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
.. _recipes-apis:
2+
3+
mlflow.recipes
4+
================
5+
6+
.. automodule:: mlflow.recipes
7+
8+
.. autoclass:: mlflow.recipes.Recipe()
9+
:members:
10+
:undoc-members:
11+
:special-members: __new__
12+
13+
Regression Recipe
14+
~~~~~~~~~~~~~~~~~~~
15+
16+
.. automodule:: mlflow.recipes.regression.v1.recipe
17+
18+
.. autoclass:: mlflow.recipes.regression.v1.recipe.RegressionRecipe()
19+
:members:
20+
:undoc-members:
21+
22+
Classification Recipe
23+
~~~~~~~~~~~~~~~~~~~~~~~
24+
25+
.. automodule:: mlflow.recipes.classification.v1.recipe
26+
27+
.. autoclass:: mlflow.recipes.classification.v1.recipe.ClassificationRecipe()
28+
:members:
29+
:undoc-members:

0 commit comments

Comments
 (0)