[serve] Initial version of local_testing_mode (#48477)

[frances720]

Adds initial support for a (currently private) _local_testing_mode flag to serve.run. The intention for this flag is to enable users to write unit tests for their application and model composition logic.

User code for each deployment will be run in a background thread using the UserCallableWrapper that runs in replica actors. There is a new Router and ReplicaResult implementation that interacts with the user code to enable the existing DeploymentHandle code to work.

Before merging:

• Figure out why RAY_SERVE_FORCE_LOCAL_TESTING_MODE=1 pytest -vs tests/test_handle_cancellation.py hangs locally

Follow-ups to this PR:

• Fix blocking .result() in resolving args for composition
• Make get_replica_context() and other auxiliary APIs work
• Support FastAPI TestClient

[promptless]

📝 Documentation updates detected!

Promptless has analyzed your changes and created a documentation update PR. You can review the proposed documentation changes here:
#6

Please review the suggested updates to ensure they accurately reflect your changes.

[promptless]

📝 Documentation updates detected!

Promptless has analyzed your changes and created a documentation update PR. You can review the proposed documentation changes here:
#7

Please review the suggested updates to ensure they accurately reflect your changes.

[promptless]

📝 Documentation updates detected!

Promptless has analyzed your changes and created a documentation update PR. You can review the proposed documentation changes here:
#8

Please review the suggested updates to ensure they accurately reflect your changes.

[promptless]

📝 Documentation updates detected!

Promptless has analyzed your changes and created a documentation update PR. You can review the proposed documentation changes here:
#9

Please review the suggested updates to ensure they accurately reflect your changes.

[promptless]

📝 Documentation updates detected!

Promptless has analyzed your changes and created a documentation update PR. You can review the proposed documentation changes here:
#10

Please review the suggested updates to ensure they accurately reflect your changes.

[promptless]

✅ No documentation updates required

Promptless has analyzed your changes against existing documentation and determined that no updates are needed at this time.

If you believe documentation updates are needed, please update the relevant files manually.

[promptless]

📝 Documentation updates detected!

Promptless has analyzed your changes and created a documentation update PR. You can review the proposed documentation changes here:
#10

Please review the suggested updates to ensure they accurately reflect your changes.