Researchers from JD.com and Shanghai Jiao Tong University developed AdaCodec, a predictive visual code for video Multimodal Large Language Models (MLLMs). This new approach encodes inter-frame changes as compact P-tokens, reducing visual token usage by 84.7% and significantly lowering inference latency. AdaCodec demonstrates improved performance across various long-video and general video-understanding benchmarks compared to traditional per-frame RGB encoding.
The efficiency gains from AdaCodec represent a notable step forward for video MLLMs, addressing the core challenge of processing increasing volumes of video data more effectively. By drastically reducing token count and latency, models can tackle longer video content and complex temporal reasoning tasks with greater speed and accuracy. This development could accelerate the integration of advanced AI capabilities into video platforms, impacting areas from automated content moderation to enhanced viewer engagement analytics. What to watch next is how quickly these predictive coding principles are adopted by other major AI research labs and integrated into commercial video MLLM offerings.
Add StreamingMeme as a preferred source on Google to see more of our streaming news at the top of your Search results.
Add as preferred source