Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ysc pluggable scheduler #78

Merged
merged 7 commits into from
Feb 20, 2025
Merged

Conversation

yannicks1
Copy link
Contributor

@yannicks1 yannicks1 commented Feb 11, 2025

This PR moves the (spyre specific) scheduler class into the plugin repo vllm-spyre.
it goes along with this ->PR<- in the vllm-spyre repository

Changes:

  • vllm/config.py: introducing new variable scheduler_cls for class SchedulerConfig (can be a path (str) or a class directly) and removing spyre specific code to 'load warmup shapes and sort by "speed"' (moved to vllm-spyre)
  • vllm/engine/llm_engine.py: importing Scheduler based on vllm_config.scheduler_config.scheduler_cls
  • vllm/core/scheduler.py: removing all spyre specific code from the scheduler logic
  • vllm/envs.py: removing all spyre related env variables (moved to vllm-spyre)
  • remove the variable: SchedulerConfig.spyre_warmup_shapes in vllm/vllm.config.py

Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
This was referenced Feb 11, 2025
@yannicks1 yannicks1 marked this pull request as ready for review February 11, 2025 16:19
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
Copy link
Member

@sducouedic sducouedic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment on lines 348 to 351
Scheduler = resolve_obj_by_qualname(
self.vllm_config.parallel_config.scheduler_cls)
self.scheduler = [
Scheduler(
Copy link
Member

@sducouedic sducouedic Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also think it is better with Scheduler (with capital letter), but I've seen elsewhere in the code that they used cls
for example when retrieving the worker class:

cls = resolve_obj_by_qualname(target_worker_config.parallel_config.worker_cls)
target_worker = cls(*args, **kwargs)

Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
@yannicks1 yannicks1 merged commit 8c3c29f into sop-plugin-refactoring Feb 20, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants