Nvidia Vera Rubin NVL72 inference delivers 67x better performance per dollar

The massive leap in performance per dollar suggests that hyperscalers can significantly lower the cost of serving complex, multi-turn agentic models. By delivering 10x more tokens per dollar than B300 at realistic service level agreements, Nvidia is effectively commoditizing high-end inference while maintaining its hardware dominance. This efficiency shift forces competitors like AMD, whose MI355X currently trails in tokens per megawatt, to accelerate their own architectural co-design efforts to remain viable for large-scale deployments. As streaming platforms integrate more agentic AI for personalized discovery and automated metadata, the Rubin architecture becomes the baseline for cost-effective scaling. Watch for the integration of PowerX into future benchmarks to see how dynamic power shifting further expands these margin advantages.

Nvidia's Rubin platform is already attracting commitments from major cloud providers ahead of general availability. In August 2025, Microsoft Azure announced it would be among the first cloud platforms to deploy Nvidia Vera Rubin NVL72 systems for large-scale AI training and inference workloads, signaling that hyperscalers are planning multi-year infrastructure cycles around the architecture. Oracle confirmed at its CloudWorld event in September 2025 that it would deploy Rubin-based GPU clusters across its OCI data centers, targeting sovereign AI and enterprise inference customers. Google Cloud has similarly indicated Rubin integration plans, with Jensen Huang stating at GTC 2025 that every major cloud provider had committed to Rubin deployments by 2027.

The competitive dynamics around Rubin are intensifying pressure on AMD and custom silicon efforts. AMD's MI355X, which targets inference-heavy workloads, was benchmarked by SemiAnalysis in early 2025 and found to deliver roughly 40% lower tokens per megawatt than Nvidia's Blackwell B300 in comparable agentic inference scenarios. That gap widens further against Nvidia Rubin Ultra's projected efficiency gains. Meanwhile, OpenAI announced in June 2025 that it had signed a multi-year agreement with Nvidia to deploy several gigawatts of Rubin-class compute as part of its Stargate data center buildout, representing one of the largest single inference infrastructure commitments in the industry. Meta has also signaled Rubin adoption, with CEO Mark Zuckerberg confirming in July 2025 that Meta's 2026 AI infrastructure budget would include next-generation Nvidia systems to support its Llama model family and agentic product roadmap.

On the technical side, Rubin's architectural improvements extend beyond raw throughput. The platform introduces HBM4 memory with bandwidth exceeding 22 TB/s per GPU, a roughly 2.8x increase over Blackwell's HBM3E configuration, which directly benefits long-context agentic workloads where memory bandwidth becomes the bottleneck. Nvidia's InferenceX software stack, which orchestrates multi-step agent reasoning across distributed GPU clusters, was demonstrated at Computex 2025 running a 12-agent pipeline on a single NVL72 rack with sub-second inter-agent latency. For streaming platforms evaluating autonomous AI agent governance and automated real-time compliance for metadata generation, these efficiency gains translate into materially lower per-query costs at scale, potentially making data foundations for agentic AI economically viable for mid-tier services that previously could not justify the compute spend, especially as multi-agent LLM orchestration continues to drive up operational expenses, a challenge addressed by multi-agent AI orchestration patterns designed to optimize enterprise workflows.