Skip to content

Conversation

fan2956
Copy link
Contributor

@fan2956 fan2956 commented Sep 9, 2025

What this PR does / why we need it?

On main, AscendScheduler does not support Multimodels, becuse of lacking of scheduled_encoder_inputs which is need on multimodels inference

Does this PR introduce any user-facing change?

No

How was this patch tested?

vLLM version: main@93e28e6862669e3b5cf47cea9f782a65ec47e155

@fan2956 fan2956 changed the title add ascend scheduler support multimodel main add ascend scheduler support multimodel Sep 9, 2025
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for multimodal models in the AscendScheduler by implementing the scheduling of encoder inputs. The changes correctly remove the previous restriction and add the necessary logic for handling encoder inputs in both prefill and decode paths. However, I've identified a significant code duplication for the encoder cache allocation logic, which is present in both the prefill and decode scheduling loops. I've left a comment suggesting to refactor this duplicated code into a helper method to improve maintainability.

Copy link

github-actions bot commented Sep 9, 2025

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

  • A PR should do only one thing, smaller PRs enable faster reviews.
  • Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
  • Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

Copy link

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@fan2956 fan2956 changed the title main add ascend scheduler support multimodel main add ascend scheduler support multimodal Sep 10, 2025
@wangxiyuan
Copy link
Collaborator

please rebase to main to fix the merge conflict

@fan2956
Copy link
Contributor Author

fan2956 commented Sep 10, 2025

fix the merge conflict

Copy link

codecov bot commented Sep 12, 2025

Codecov Report

❌ Patch coverage is 52.00000% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.02%. Comparing base (1bbb20e) to head (d90bde6).
⚠️ Report is 28 commits behind head on main.

Files with missing lines Patch % Lines
vllm_ascend/core/scheduler.py 45.45% 12 Missing ⚠️

❌ Your patch check has failed because the patch coverage (52.00%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2844      +/-   ##
==========================================
+ Coverage   74.76%   75.02%   +0.26%     
==========================================
  Files         150      154       +4     
  Lines       20891    21297     +406     
==========================================
+ Hits        15620    15979     +359     
- Misses       5271     5318      +47     
Flag Coverage Δ
unittests 75.02% <52.00%> (+0.26%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link

This pull request has conflicts, please resolve those before we can evaluate the pull request.

zhoufan2956 and others added 9 commits September 13, 2025 17:16
Signed-off-by: fan2956 <zhoufan53@huawei.com>
Signed-off-by: fan2956 <zhoufan53@huawei.com>
Signed-off-by: fan2956 <zhoufan53@huawei.com>
Signed-off-by: fan2956 <zhoufan53@huawei.com>
Signed-off-by: fan2956 <zhoufan53@huawei.com>
Signed-off-by: fan2956 <zhoufan53@huawei.com>
Signed-off-by: fan2956 <zhoufan53@huawei.com>
Signed-off-by: fan2956 <zhoufan53@huawei.com>
Signed-off-by: fan2956 <zhoufan53@huawei.com>
@wangxiyuan wangxiyuan added ready read for review ready-for-test start test by label for PR labels Sep 13, 2025
@wangxiyuan wangxiyuan merged commit c5a502f into vllm-project:main Sep 14, 2025
44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module:tests ready read for review ready-for-test start test by label for PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants