Skip to content

Latest commit

 

History

History
29 lines (15 loc) · 1.33 KB

File metadata and controls

29 lines (15 loc) · 1.33 KB

Model Serving Guide

Deployment

Assuming you have already created a model in the Model Registry, a deployment can now be created to prepare a model artifact for this model and make it accessible for running predictions behind a REST or gRPC endpoint. Follow the Deployment Creation Guide to create a Deployment for your model.

Predictor

Predictors are responsible for running a model server that loads a trained model, handles inference requests and returns predictions, see the Predictor Guide.

Transformer

Transformers are used to apply transformations on the model inputs before sending them to the predictor for making predictions using the model, see the Transformer Guide.

Resource Allocation

Configure the resources to be allocated for predictor and transformer in a model deployment, see the Resource Allocation Guide.

Inference Batcher

Configure the predictor to batch inference requests, see the Inference Batcher Guide.

Inference Logger

Configure the predictor to log inference requests and predictions, see the Inference Logger Guide.

Troubleshooting

Inspect the model server logs to troubleshoot your model deployments, see the Troubleshooting Guide.