Visko Orbis live world model hits real-time speeds on Google TPUs
AI startup Visko collaborated with Google Cloud to deploy its Orbis live world model on Cloud TPU hardware. The project utilized JAX, torchax, and custom Pallas kernels to achieve real-time, steerable video generation at 1.07x playback speed within a 30-day development window.
Key Takeaways
- Achieved 1.07x playback speed for 1664x936 resolution video at 18 fps using BF16 precision.
- Implemented Ulysses context parallelism to distribute token sequences across TPU chips for single-stream workloads.
- Reduced audio generation latency by 5.8x by porting that specific component from torchax to native JAX.
- Integrated custom Pallas splash attention kernels from MaxDiffusion to optimize video denoising compute.
Why It Matters
This deployment demonstrates that interactive generative video can overcome the 'batch size of one' bottleneck that typically plagues real-time AI inference. By utilizing a single TPU slice for a chained multimodal pipeline, Visko establishes a cost-effective blueprint for scaling steerable environments without the latency penalties of traditional offline rendering. For the broader streaming ecosystem, this signals a shift from static content delivery toward dynamic, user-influenced video streams that require high-bandwidth memory and specialized interconnects. Watch for whether other generative video startups adopt the JAX and Pallas framework to bypass the performance overhead of standard PyTorch implementations on non-GPU hardware.
Additional Context
The industry is increasingly focused on AI production tools to optimize workflows and reduce overhead in high-end content creation.
Companies in this story
In short
Visko has successfully deployed its Orbis live world model on Google Cloud TPUs, achieving 1.07x playback speed. By utilizing a single TPU slice for a multimodal pipeline, the startup rendered interactive video at 18 fps. This breakthrough proves that generative video can overcome traditional latency bottlenecks, enabling dynamic, user-influenced streaming content.
FAQ
What performance speed did the Visko Orbis model achieve?
During a 30-day technical collaboration with Google Cloud, the Visko Orbis live world model achieved 1.07x playback speed.
What frame rate does the Orbis model support on TPUs?
The model successfully rendered interactive, steerable video at 18 frames per second (fps) with synchronized audio and super-resolution upscaling.
How did Visko reduce audio generation latency?
Visko reduced audio generation latency by 5.8x by porting the specific component from torchax to native JAX.
What technology was used to optimize video denoising?
Visko integrated custom Pallas splash attention kernels from MaxDiffusion to optimize video denoising compute.
Read full article at cloud.google.com
Enjoy our coverage?
Add StreamingMeme as a preferred source on Google to see more of our streaming news at the top of your Search results.
Add as preferred source
