[serve] Allow RAY_SERVE_THROUGHPUT_OPTIMIZED flags to be overriden (#58057)

## Description
This PR allows users to override individual flags set by
`RAY_SERVE_THROUGHPUT_OPTIMIZED`. This improves the current UX of having
to set each flag individually if any of the flags is different from what
`RAY_SERVE_THROUGHPUT_OPTIMIZED` sets.

## Related issues

## Additional information
> Optional: Add implementation details, API changes, usage examples,
screenshots, etc.

---------

Signed-off-by: akyang-anyscale <[email protected]>

Author: akyang-anyscale

doc/source/serve/advanced-guides/performance.md

@@ -152,6 +152,7 @@ You can also configure each option individually. The following table details the
 | `RAY_SERVE_REQUEST_PATH_LOG_BUFFER_SIZE=1000` | Sets the log buffer to batch writes to every `1000` logs, flushing the buffer on write. The system always flushes the buffer and writes logs when it detects a line with level ERROR.  Set the buffer size to `1` to disable buffering and write logs immediately. |
 | `RAY_SERVE_LOG_TO_STDERR=0` | Only write logs to files under the `logs/serve/` directory. Proxy, Controller, and Replica logs no longer appear in the console, worker files, or the Actor Logs section of the Ray Dashboard. Set this property to `1` to enable additional logging. |
 
+You may want to enable throughput-optimized serving while customizing the options above. You can do this by setting `RAY_SERVE_THROUGHPUT_OPTIMIZED=1` and overriding the specific options. For example, to enable throughput-optimized serving and continue logging to stderr, you should set `RAY_SERVE_THROUGHPUT_OPTIMIZED=1` and override with `RAY_SERVE_LOG_TO_STDERR=1`.
 
 ## Debugging performance issues in controller
 

python/ray/serve/_private/constants.py

@@ -501,10 +501,16 @@
 # This should be at the end.
 RAY_SERVE_THROUGHPUT_OPTIMIZED = get_env_bool("RAY_SERVE_THROUGHPUT_OPTIMIZED", "0")
 if RAY_SERVE_THROUGHPUT_OPTIMIZED:
-    RAY_SERVE_RUN_USER_CODE_IN_SEPARATE_THREAD = False
-    RAY_SERVE_REQUEST_PATH_LOG_BUFFER_SIZE = 1000
-    RAY_SERVE_RUN_ROUTER_IN_SEPARATE_LOOP = False
-    RAY_SERVE_LOG_TO_STDERR = False
+    RAY_SERVE_RUN_USER_CODE_IN_SEPARATE_THREAD = get_env_bool(
+        "RAY_SERVE_RUN_USER_CODE_IN_SEPARATE_THREAD", "0"
+    )
+    RAY_SERVE_REQUEST_PATH_LOG_BUFFER_SIZE = get_env_int(
+        "RAY_SERVE_REQUEST_PATH_LOG_BUFFER_SIZE", 1000
+    )
+    RAY_SERVE_RUN_ROUTER_IN_SEPARATE_LOOP = get_env_bool(
+        "RAY_SERVE_RUN_ROUTER_IN_SEPARATE_LOOP", "0"
+    )
+    RAY_SERVE_LOG_TO_STDERR = get_env_bool("RAY_SERVE_LOG_TO_STDERR", "0")
 
 # The maximum allowed RPC latency in milliseconds.
 # This is used to detect and warn about long RPC latencies