AWS has announced the availability of Amazon Elastic Compute Cloud (EC2) Inf2 instances, which deliver high performance for generative AI models at a low cost.

Inf2 instances are powered by AWS Inferentia2 chips and offer up to four times higher throughput and up to 10 times lower latency than first-generation EC2 Inf1 instances. The new instances also introduce scale-out distributed inference supported by NeuronLink, a high-speed, nonblocking interconnect. Inf2 instances offer up to 2.3 petaflops of DL performance and up to 384 GB of total accelerator memory.

These instances are available in four sizes and can be used with popular machine learning frameworks such as PyTorch and TensorFlow.