Amazon has announced the general availability of Provisioned Concurrency support for Amazon SageMaker Serverless Inference. In simple words, the feature allows customers to deploy models on serverless endpoints with predictable performance and high scalability.
Here are some important facts:
- You can add provisioned concurrency to your serverless endpoints, and for the pre-defined amount of provisioned concurrency SageMaker will keep the endpoints warm and ready to respond to requests instantaneously.
- Customers with predictable traffic and low throughput can benefit from Provisioned Concurrency.
- Cold starts can be reduced by enabling Provisioned Concurrency, ensuring endpoints are always ready for bursts in traffic.
- Provisioned Concurrency can be enabled through the AWS console, SDKs, or CLI.
- This feature is generally available in all AWS Regions where SageMaker Serverless Inference is available.
