-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: PT-541: rewrite basically the whole thing & add support for modelcache #20
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pandyamarut : what's the file used for? can we delete it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This drives the config forms at quick deploy. If you need any env var that User should be able to configure, it goes here. This schema definition is parsed over to our console as form and inputs. This was created for vllm at first to support versions and changes in their configs. We're using it for all quick deploy templates in the future.
@@ -1,7 +1,7 @@ | |||
ray | |||
pandas | |||
pyarrow | |||
runpod~=1.7.0 | |||
runpod>=1.7.7 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't do this. ~= covers for 1.7.0-1.7.x. If/when we go 1.8.x it means breaking changes. We don't want this upgrading before we can test it. Revert this line please.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This drives the config forms at quick deploy. If you need any env var that User should be able to configure, it goes here. This schema definition is parsed over to our console as form and inputs. This was created for vllm at first to support versions and changes in their configs. We're using it for all quick deploy templates in the future.
PT-541:
runpod-workers/worker-sglang
: update for modelcacheTook me ages to figure out how this worked. It's not really that complicated, but it's been quite obscured by the control flow jumping around all over the place. Should be a bit easier to follow now.
To summarize the changes:
runpod-python
client to 1.7.7