Activity
Don't use self.unload model for deleting vllm instance
Don't use self.unload model for deleting vllm instance
Add domain age, increase connect timeout, miner response info
Add domain age, increase connect timeout, miner response info
Stream back Response When Single Chunk
Stream back Response When Single Chunk
2 days ago
Decrease Simultaneous Loops
Decrease Simultaneous Loops
Improving model unloading
Improving model unloading
fixing vllm loading/unloading
fixing vllm loading/unloading
Merge branch 'staging' into SN1-447-add-logits
Merge branch 'staging' into SN1-447-add-logits
Adding beginning of text token
Adding beginning of text token
Force push
Remove debug messages
Remove debug messages
Run formatter
Run formatter
Rename model lock
Rename model lock
Increase model rotation interval
Increase model rotation interval
Fix inference bug
Fix inference bug
Add sleep to prevent partial model loading
Add sleep to prevent partial model loading
Fix QA task
Fix QA task
Bump version v2.17.10
Bump version v2.17.10