Huawei-SJTU research group reveals dedicated hardware for video diffusion transformers
Researchers from Shanghai Jiao Tong University and Huawei have introduced Kaleido, an algorithm-hardware co-design approach for accelerating video diffusion transformers (vDiTs). By exploiting channel-wise spatio-temporal correlations in latent space, the architecture achieves up to 5.9x speedups and 16x energy savings compared to state-of-the-art accelerators.
Key Takeaways
- Kaleido architecture delivers a 5.9x speedup over current state-of-the-art accelerators by reusing latent space partial results.
- Hardware design features a systolic-array-like accelerator with reconfigurable processing elements and a lightweight data dispatcher.
- The co-design maintains high generative quality exceeding 17 dB while skipping redundant self-attention computations.
- Current state-of-the-art models like Tencent's HunyuanVideo require over 30 minutes on an Nvidia H100 to produce a 5-second 720p clip.
Why It Matters
The transition from image-based to video-based generative models has hit a hardware wall; the sheer volume of spatiotemporal tokens in transformers makes real-time 720p or 1080p generation commercially unviable on general-purpose GPUs. By shifting from generic LLM-style sparse attention to video-specific latent reuse, Kaleido demonstrates that specialized silicon can slash costs and power consumption for the next generation of creative tools. This signals a shift toward vertical integration in the streaming supply chain, where dedicated AI video chips may become as standard as hardware H.264 encoders. Watch if Huawei integrates the Kaleido design into its Ascend AI chip lineup to compete with Nvidia's dominance in creative AI workloads.
Additional Context
The introduction of Kaleido coincides with a broader push for efficiency in video generation models. In November 2025, Tencent released HunyuanVideo-1.5, which per GitHub documentation achieved a 1.87x speedup over FlashAttention-3 by implementing selective and sliding tile attention. This software-level optimization targeted similar redundancies to Kaleido but relied on existing hardware. Meanwhile, the hardware landscape has seen increased specialization; in December 2025, Shanghai Jiao Tong University announced LightGen, an all-optical computing chip designed for high-definition video generation, which per university filings reached energy efficiency levels orders of magnitude beyond traditional digital chips.
Huawei’s involvement highlights its ongoing strategy to build an independent AI ecosystem. Per Huawei's 2026 reports, the company has deepening ties with Shanghai Jiao Tong University through the Huawei ICT Academy, focusing on co-developing hardware for complex AI tasks. This partnership previously yielded optimizations for autonomous driving, and the pivot toward video diffusion suggests a strategic interest in the high-growth generative video market. As of June 2026, industry reports from ITU's Kaleidoscope event suggest more than a billion monthly users for generative AI tools, putting immense pressure on data center power grids and driving the demand for the 16x energy savings promised by the Kaleido architecture.
Read full article at arxiv.org
Enjoy our coverage?
Add StreamingMeme as a preferred source on Google to see more of our streaming news at the top of your Search results.
Add as preferred source