#98 introduced temporary endpoints for scaling task executors. However, there is no planner for deciding which GPUs to allocate when scaling up and which GPUs to remove when scaling down. As an example,
When a unit task has 4 GPUs in one node and 2 GPUs in another node, from which node should we deallocate? This requires a lot of information to do well, and will be part of the autoscaling feature.
The planner should be implemented together with #27. The planner should consider the sharing status of a task executor, the current GPU availability, and the current system load.
#98 introduced temporary endpoints for scaling task executors. However, there is no planner for deciding which GPUs to allocate when scaling up and which GPUs to remove when scaling down. As an example,
The planner should be implemented together with #27. The planner should consider the sharing status of a task executor, the current GPU availability, and the current system load.