Skip to content
This repository has been archived by the owner on May 28, 2024. It is now read-only.

Commit

Permalink
fixup
Browse files Browse the repository at this point in the history
Signed-off-by: Alan Guo <aguo@anyscale.com>
  • Loading branch information
alanwguo committed Jan 25, 2024
1 parent a83ac2d commit 4613e16
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions models/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,8 @@ In addition, there some configurations to control the prompt formatting behavior
* `strip_whitespace` - Whether to automatically strip whitespace from left and right of the content for the messages provided in the ChatCompletions API.


You can see an example in the [Adding a new model](#adding-a-new-model) section below.

### Scaling config

Finally, the `scaling_config` section specifies what resources should be used to serve the model - this corresponds to Ray AIR [ScalingConfig](https://docs.ray.io/en/latest/train/api/doc/ray.train.ScalingConfig.html). Note that the `scaling_config` applies to each model replica, and not the entire model deployment (in other words, each replica will have `num_workers` workers).
Expand Down

0 comments on commit 4613e16

Please sign in to comment.