AWS Adds Managed NVIDIA Dynamo to SageMaker HyperPod Inference

Amazon Web Services announced on November 17, 2026, that Amazon SageMaker HyperPod Inference now supports NVIDIA Dynamo for managed multi-node and disaggregated model serving, alongside benchmark results showing 2.05 times the highest qualifying aggregated output-token throughput per GPU under AWS’s latency targets. The addition addresses two constraints in serving large language models: fitting model weights and the reusable key-value (KV) cache into GPU memory, and keeping long prompt…

This article has been indexed from Unite.AI