Skip to content

Conversation

@Shirley125
Copy link
Contributor

@Shirley125 Shirley125 commented Nov 26, 2025

What this PR does / why we need it?

recompute scheduler adapt main

Does this PR introduce any user-facing change?

How was this patch tested?

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request is a large-scale adaptation of the RecomputeScheduler to align with recent changes in the main vLLM scheduler. It includes significant refactoring, such as renaming the class, updating its inheritance, and modifying many core methods like schedule and update_from_output. The changes introduce support for new features like hybrid memory allocation (HMA), encoder cache transfer (ECConnector), and the v2 model runner. It also brings substantial improvements in robustness, particularly in handling distributed KV cache transfers, preemption logic, and error recovery from invalid cache blocks.

My review has identified two critical issues that need to be addressed. First, the scheduler class was renamed in a way that will break its loading via configuration. Second, a dataclass definition contains a type mismatch and a risky default value, which could lead to runtime errors. After these issues are fixed, this PR will be a great step forward in keeping the Ascend-specific scheduler up-to-date with the core vLLM project.

"""This Scheduler extends vllm's original v1 scheduler of version 0.11
to fix recomputing bug."""

class Scheduler(SchedulerInterface):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The class has been renamed from RecomputeScheduler to Scheduler. However, the configuration file vllm_ascend/core/recompute_schedule_config.py still refers to vllm_ascend.core.recompute_scheduler.RecomputeScheduler. This will cause an ImportError when vLLM tries to load the scheduler class at runtime. To fix this, the class should be renamed back to RecomputeScheduler to match the name used in the configuration.

Suggested change
class Scheduler(SchedulerInterface):
class RecomputeScheduler(SchedulerInterface):

Comment on lines +1681 to +1683
class RecomputeSchedulerOutput(SchedulerOutput):
# requests that need to recompute kv
recomputed_reqs: list[RecomputeReqInfo]

# KV Cache Connector metadata.
kv_connector_metadata: Optional[KVConnectorMetadata] = None
recomputed_reqs: list[RecomputeReqInfo] = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The recomputed_reqs field is defined with a type hint of list[RecomputeReqInfo] but has a default value of None. This is a type mismatch and can lead to a TypeError at runtime, for instance in update_from_output where this field is iterated over without a None check. The default value should be an empty list to match the type hint and ensure safe iteration.

Suggested change
class RecomputeSchedulerOutput(SchedulerOutput):
# requests that need to recompute kv
recomputed_reqs: list[RecomputeReqInfo]
# KV Cache Connector metadata.
kv_connector_metadata: Optional[KVConnectorMetadata] = None
recomputed_reqs: list[RecomputeReqInfo] = None
class RecomputeSchedulerOutput(SchedulerOutput):
# requests that need to recompute kv
recomputed_reqs: list[RecomputeReqInfo] = []

@github-actions
Copy link

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@Shirley125 Shirley125 changed the title recompute scheduler adapt main [wip]recompute scheduler adapt main Nov 26, 2025
@github-actions
Copy link

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

  • A PR should do only one thing, smaller PRs enable faster reviews.
  • Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
  • Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant