It loads versions, preprocesses inputs, schedules batches, runs inference, and returns outputs through an interface. Production serving also manages scaling, health checks, observability, access control, and rollback.
Model serving is the infrastructure that makes trained models available for prediction requests.
It loads versions, preprocesses inputs, schedules batches, runs inference, and returns outputs through an interface. Production serving also manages scaling, health checks, observability, access control, and rollback.