Solidigm pushes token per watt metric as AI storage needs explode
Solidigm VP Avi Shetty highlights the shift toward 'token per watt' as a critical efficiency metric for AI data centers alongside high-density SSD storage solutions. The discussion, held at the RAISE Summit 2026, focuses on optimizing infrastructure to prevent GPU latency caused by cache rebuilding in AI workflows.
Key Takeaways
- Single AI prompts now translate to roughly 13 GB of system data activity including database requests and tool activities.
- Solidigm and NVIDIA co-designed the world's first liquid-cooled SSD to support fanless data center architectures for high-performance AI.
- The 122-terabyte D5-P5336 SSD enables four petabytes of storage in 1U of rack space, reducing power consumption up to 90%.
- Solidigm’s AI Central Lab demonstrated linear performance scaling for AI workloads from a single node up to 32 nodes at exabyte capacity.
Why It Matters
The transition from training to inference has turned storage into a critical performance bottleneck. If storage cannot feed data at the rate GPUs process it, expensive compute resources sit idle, breaking the economics of large language model deployments. At the ecosystem level, this marks a pivot from measuring raw hardware IOPS to system-level efficiency metrics like dollar per token per watt. For strategists, watching the integration of liquid-cooled storage into 2026 server designs will be the key signal for assessing next-generation facility preparedness.
Additional Context
The industry-wide shift toward token-based metrics reflects an acute energy crisis in the AI sector. According to Goldman Sachs in early 2026, token consumption is projected to grow 24 times by 2030, driven by agentic AI systems that reason in chains and maintain long context windows. This volume surge has pushed data center power demands to a breaking point; Gartner estimated in May 2026 that global data center electricity demand will exceed 1,000 TWh by the end of the year, nearly double the 2023 baseline. These constraints have forced an architectural rethink around how Key-Value (KV) cache is managed, shifting it from expensive GPU memory to dense storage tiers to maintain performance. Hardware vendors are responding with extreme density and specialized cooling. Per reporting from ServeTheHome in February 2026, NVIDIA and Solidigm’s collaboration on liquid-cooled SSDs addresses a hard thermal ceiling, as next-generation AI racks like the Blackwell GB200 now pull between 120 kW and 140 kW. This heat density makes traditional fan-based cooling economically and physically unviable for 76% of new AI server deployments. Simultaneously, the D5-P5336 SSD, which first shipped in mid-2025 at a $12,399 launch price, reportedly saw secondary market prices triple by early 2026 due to hyperscaler demand for high-capacity NAND to clear storage bottlenecks. Infrastructure providers are moving beyond traditional Power Usage Effectiveness (PUE) to measure success. While PUE tracks facility overhead, it ignores the internal efficiency of the compute stack. Per industry analysis in June 2026, inference now accounts for roughly two-thirds of all AI compute, compared to just one-third in 2023. This shift toward continuous, distributed inference means infrastructure is no longer sized for training peaks but for steady-state token output, making the "token per watt" framework the terminal metric for assessing a provider's long-term commercial viability.
Read full article at siliconangle.com
Get this in your inbox → Subscribe
Enjoy our coverage?
Add StreamingMeme as a preferred source on Google to see more of our streaming news at the top of your Search results.
Add as preferred source