From 96849330ba0ef3d93bdecc7d509ac69d7400d2ac Mon Sep 17 00:00:00 2001 From: "promptless[bot]" <179508745+promptless[bot]@users.noreply.github.com> Date: Tue, 19 Nov 2024 18:23:22 +0000 Subject: [PATCH 1/4] Docs update (ee0512a) --- doc/source/serve/advanced-guides/dev-workflow.md | 12 ++++++++++++ doc/source/serve/key-concepts.md | 4 ++++ doc/source/serve/production-guide/index.md | 9 +++++++++ 3 files changed, 25 insertions(+) diff --git a/doc/source/serve/advanced-guides/dev-workflow.md b/doc/source/serve/advanced-guides/dev-workflow.md index de8b6745a400..5b1af58677a5 100644 --- a/doc/source/serve/advanced-guides/dev-workflow.md +++ b/doc/source/serve/advanced-guides/dev-workflow.md @@ -79,6 +79,18 @@ After you're done testing, you can shut down Ray Serve by interrupting the `serv Note that rerunning `serve run` will redeploy all deployments. To prevent redeploying those deployments whose code hasn't changed, you can use `serve deploy`; see the [Production Guide](serve-in-production) for details. +### Local Testing Mode + +For faster development iterations and easier debugging, you can enable the local testing mode by setting the environment variable `RAY_SERVE_FORCE_LOCAL_TESTING_MODE`. This mode allows for in-process deployment, which is particularly useful for writing unit tests for application and model composition logic. + +To enable local testing mode, set the environment variable before running your application: + +```bash +export RAY_SERVE_FORCE_LOCAL_TESTING_MODE=1 +serve run local_dev:app +``` + +In this mode, the `serve.run` function will use a new `Router` and `ReplicaResult` implementation to facilitate local testing. This setup enhances developer productivity by allowing for quicker feedback loops during the development process. ## Testing on a remote cluster To test on a remote cluster, you'll use `serve run` again, but this time you'll pass in an `--address` argument to specify the address of the Ray cluster to connect to. For remote clusters, this address has the form `ray://:10001`; see [Ray Client](ray-client-ref) for more information. diff --git a/doc/source/serve/key-concepts.md b/doc/source/serve/key-concepts.md index 93eefed0f6f3..752799c69e61 100644 --- a/doc/source/serve/key-concepts.md +++ b/doc/source/serve/key-concepts.md @@ -1,3 +1,7 @@ +There are no changes between the original content and the proposed changes. Therefore, the content remains the same. + +``` + (serve-key-concepts)= # Key Concepts diff --git a/doc/source/serve/production-guide/index.md b/doc/source/serve/production-guide/index.md index bc4c9fa08096..da4cfb22850c 100644 --- a/doc/source/serve/production-guide/index.md +++ b/doc/source/serve/production-guide/index.md @@ -119,6 +119,15 @@ Test the application using Python `requests`: To update the application, modify the config file and use `serve deploy` again. +### Local Testing Mode + +For faster development iterations and easier debugging, you can enable the local testing mode by setting the environment variable `RAY_SERVE_FORCE_LOCAL_TESTING_MODE`. This mode allows for in-process deployment, which can significantly speed up the testing process. To enable this mode, set the environment variable before running your application: + +```bash +export RAY_SERVE_FORCE_LOCAL_TESTING_MODE=1 +``` + +This feature is particularly useful for writing unit tests for your application and model composition logic. ## Next Steps For a deeper dive into how to deploy, update, and monitor Serve applications, see the following pages: From 74e064d0f23351c80bbd0a57d72b7366333b3a99 Mon Sep 17 00:00:00 2001 From: "promptless[bot]" <179508745+promptless[bot]@users.noreply.github.com> Date: Tue, 19 Nov 2024 20:45:54 +0000 Subject: [PATCH 2/4] Revert previous docs update --- doc/source/serve/advanced-guides/dev-workflow.md | 12 ------------ doc/source/serve/key-concepts.md | 4 ---- doc/source/serve/production-guide/index.md | 9 --------- 3 files changed, 25 deletions(-) diff --git a/doc/source/serve/advanced-guides/dev-workflow.md b/doc/source/serve/advanced-guides/dev-workflow.md index 5b1af58677a5..de8b6745a400 100644 --- a/doc/source/serve/advanced-guides/dev-workflow.md +++ b/doc/source/serve/advanced-guides/dev-workflow.md @@ -79,18 +79,6 @@ After you're done testing, you can shut down Ray Serve by interrupting the `serv Note that rerunning `serve run` will redeploy all deployments. To prevent redeploying those deployments whose code hasn't changed, you can use `serve deploy`; see the [Production Guide](serve-in-production) for details. -### Local Testing Mode - -For faster development iterations and easier debugging, you can enable the local testing mode by setting the environment variable `RAY_SERVE_FORCE_LOCAL_TESTING_MODE`. This mode allows for in-process deployment, which is particularly useful for writing unit tests for application and model composition logic. - -To enable local testing mode, set the environment variable before running your application: - -```bash -export RAY_SERVE_FORCE_LOCAL_TESTING_MODE=1 -serve run local_dev:app -``` - -In this mode, the `serve.run` function will use a new `Router` and `ReplicaResult` implementation to facilitate local testing. This setup enhances developer productivity by allowing for quicker feedback loops during the development process. ## Testing on a remote cluster To test on a remote cluster, you'll use `serve run` again, but this time you'll pass in an `--address` argument to specify the address of the Ray cluster to connect to. For remote clusters, this address has the form `ray://:10001`; see [Ray Client](ray-client-ref) for more information. diff --git a/doc/source/serve/key-concepts.md b/doc/source/serve/key-concepts.md index 752799c69e61..93eefed0f6f3 100644 --- a/doc/source/serve/key-concepts.md +++ b/doc/source/serve/key-concepts.md @@ -1,7 +1,3 @@ -There are no changes between the original content and the proposed changes. Therefore, the content remains the same. - -``` - (serve-key-concepts)= # Key Concepts diff --git a/doc/source/serve/production-guide/index.md b/doc/source/serve/production-guide/index.md index da4cfb22850c..bc4c9fa08096 100644 --- a/doc/source/serve/production-guide/index.md +++ b/doc/source/serve/production-guide/index.md @@ -119,15 +119,6 @@ Test the application using Python `requests`: To update the application, modify the config file and use `serve deploy` again. -### Local Testing Mode - -For faster development iterations and easier debugging, you can enable the local testing mode by setting the environment variable `RAY_SERVE_FORCE_LOCAL_TESTING_MODE`. This mode allows for in-process deployment, which can significantly speed up the testing process. To enable this mode, set the environment variable before running your application: - -```bash -export RAY_SERVE_FORCE_LOCAL_TESTING_MODE=1 -``` - -This feature is particularly useful for writing unit tests for your application and model composition logic. ## Next Steps For a deeper dive into how to deploy, update, and monitor Serve applications, see the following pages: From 74875ee53e040755f362d95b68ed2bed9461cd32 Mon Sep 17 00:00:00 2001 From: "promptless[bot]" <179508745+promptless[bot]@users.noreply.github.com> Date: Tue, 19 Nov 2024 23:35:56 +0000 Subject: [PATCH 3/4] Revert previous docs update From b955cdf229ecd24868a44585cded1eb049444206 Mon Sep 17 00:00:00 2001 From: "promptless[bot]" <179508745+promptless[bot]@users.noreply.github.com> Date: Tue, 19 Nov 2024 23:36:03 +0000 Subject: [PATCH 4/4] Docs update (ee0512a) --- doc/source/serve/advanced-guides/dev-workflow.md | 1 + doc/source/serve/develop-and-deploy.md | 1 + 2 files changed, 2 insertions(+) diff --git a/doc/source/serve/advanced-guides/dev-workflow.md b/doc/source/serve/advanced-guides/dev-workflow.md index de8b6745a400..fb5fe533bd06 100644 --- a/doc/source/serve/advanced-guides/dev-workflow.md +++ b/doc/source/serve/advanced-guides/dev-workflow.md @@ -12,6 +12,7 @@ Benefits: - Self-contained Python is convenient for writing local integration tests. - No need to deploy to a cloud provider or manage infrastructure. +- Supports a `_local_testing_mode` for faster development iterations and testing. Drawbacks: diff --git a/doc/source/serve/develop-and-deploy.md b/doc/source/serve/develop-and-deploy.md index 2e2bf9d2541d..3062dea117bf 100644 --- a/doc/source/serve/develop-and-deploy.md +++ b/doc/source/serve/develop-and-deploy.md @@ -6,6 +6,7 @@ The flow for developing a Ray Serve application locally and deploying it in prod * Converting a Machine Learning model into a Ray Serve application * Testing the application locally + * Use `serve.run` with the `_local_testing_mode` flag for unit testing your application logic. This allows for faster iterations by running deployments in-process. * Building Serve config files for production deployment * Deploying applications using a config file