AI for Video: Generative Tools, Automation & Machine Learning News

Nebius has announced that Clarifai's core engineering and research team, including founder and CEO Matthew Zeiler, is joining Nebius, with Zeiler taking on the role of SVP, Research. Additionally, Nebius has licensed Clarifai's inference and compute orchestration technology and acquired its patent portfolio covering AI inference. This move is intended to strengthen Nebius Token Factory as a full-stack inference platform, following Nebius's acquisition of Eigen AI, by adding system-level optimization to existing model-level optimization.

Topaz Labs has introduced Hyperion, its first SDR to HDR AI model within Topaz Video AI, designed to convert standard dynamic range content to high dynamic range. This process involves Inverse Tone Mapping (ITM) to expand color data and convert BT.1886/rec709 SDR to BT.2100/PQ HDR, with adjustable exposure and saturation settings. The model is compatible with various ProRes, H264, H265, AV1, and other codecs, though some codecs have known issues.

OpenBMB has introduced MiniCPM-V 4.6, a multi-modal large language model designed for efficient image and video understanding on mobile devices. The model, built on SigLIP2-400M and Qwen3.5-0.8B LLM, offers strong multimodal capabilities and significant computation efficiency improvements, including support for mixed 4x/16x visual token compression and deployment across iOS, Android, and HarmonyOS platforms.

Jina by Elastic researchers introduced jina-embeddings-v5-omni, a suite of multimodal embedding models that extends existing text embedding models to support image, audio, and video inputs. The models utilize a "frozen-encoder model composition" approach, connecting pre-trained modality-specific encoders to a frozen text embedding model via compact projectors, resulting in competitive performance with less training. The jina-embeddings-v5-omni-small model achieves strong text-only performance and competitive scores on image and audio tasks compared to other open-weight multimodal embedding models.

SDVI and Vionlabs announced a strategic partnership, integrating Vionlabs' AI-powered content enrichment capabilities into the SDVI Rally media supply chain platform. This integration enables automated metadata extraction, ad-break detection, creative asset generation, and binge marker identification for video content, processed within the customer's infrastructure through Rally's application services.

Meta AI has introduced TRIBE v2, a new AI foundation model designed to predict how the human brain processes complex visual and auditory stimuli. This model, trained on extensive fMRI data from 700 volunteers watching varied media, aims to accelerate neuroscience research by enabling brain activity predictions with 70x higher resolution than previous models, without requiring human subjects for every experiment. Meta AI is openly releasing the model, codebase, paper, and an interactive demo to foster further research and applications in neuroscience and AI system development.

Anthropic has made its full Claude Platform, including Claude Managed Agents and various AI tools like Code Execution and Web Search, generally available to AWS customers. This offering allows access to native Claude API features and day-one feature parity, with authentication via AWS IAM and billing through AWS, distinct from Claude on Amazon Bedrock which keeps AWS as the data processor.

The article discusses how Artificial Intelligence is transforming Video On Demand (VOD) workflows. It highlights specific applications such as smarter encoding, scene metadata generation, and improved observability. The piece suggests these AI integrations can lead to measurable ROI for streaming teams.

Netflix is staffing a new internal studio called INKubator, focused on using AI to produce experimental animated short-form content with future ambitions for long-form projects. This artist-led incubator aims to integrate new tools and workflows with traditional animation creative practices, developing content based on Netflix's own intellectual property.

Marketers are implementing guardrails and increasing human oversight for AI agents used in programmatic ad buying due to concerns over accuracy and transparency. This development comes as the IAB Tech Lab launches a Programmatic Governance Council to address workflow, governance, and auction transparency in the evolving digital ad ecosystem. Companies like Kelly Scott Madison (KSM) Media and Bayer are deploying internal agents with specific rules, such as brand voice adherence and spend caps, while stressing the necessity of human supervision.

OpenAI has introduced three new audio models into its API: GPT-Realtime-2 for real-time voice intelligence, GPT-Realtime-Translate for live translation across 70+ input and 13 output languages, and GPT-Realtime-Whisper for low-latency streaming speech-to-text transcription. These models aim to enable developers to build more natural, intelligent, and responsive voice applications by improving reasoning, context handling, and real-time processing capabilities.

ComfyStudio Pro has launched as an AI video workstation designed to help creators generate, direct, edit, upscale, and finish videos using ComfyUI workflows. It supports local, cloud, and hybrid generation options, integrating various AI models and offering features like Director Mode, Flow AI, and a timeline editor for video production and export.

AtlasCloudAI has launched a GitHub repository for 'Awesome Seedance 2.0 Prompts,' offering a curated collection of prompts and preview videos for their multimodal video generation AI, Seedance 2.0. The repository details Seedance 2.0's capabilities in handling text, image, video, and audio inputs for tasks like reference-driven shots, advanced camera language, and video remixing, categorizing 70 prompts across various applications.

This article is a landing page for VentureBeat's technology coverage, primarily focusing on artificial intelligence, security, and enterprise AI applications. It features headlines related to AI models for video analysis, AI security flaws, AI agent orchestration, and AI infrastructure challenges. Several articles mention AI's application in video-related tasks such as auto-clipping sports highlights and real-time AI voice/video conversation.

Dreamina has launched an AI-powered video generator designed to create realistic baseball broadcast and fan cam style videos from text prompts. The tool, powered by Seedance 2.0, aims to simulate live sports broadcasting visuals, including camera movements, crowd dynamics, and lighting, without requiring video editing skills.

ZEGOCLOUD released a detailed guide on building interactive AI avatars with real-time voice interaction, demonstrating how to orchestrate ASR, LLM, TTS, and digital human rendering with WebRTC for sub-1.5-second latency. The guide provides architecture, code examples, and steps for server-side API authentication and client-side streaming using their Conversational AI platform and Express SDK. This enables developers to deploy lifelike voice-interactive digital humans for applications like customer service and live commerce.

A research paper titled "Farewell to Item IDs: Unlocking the Scaling Potential of Large Ranking Models via Semantic Tokens" introduces a new framework, TRM, that uses semantic tokens instead of traditional item IDs for ranking models in recommendation systems and search engines. TRM reportedly reduces sparse storage by 33% and improves AUC by 0.85%, demonstrating consistent outperformance at scale and leading to A/B test improvements of 0.26% in user active days and 0.75% in change query ratio in personalized search engines.

A2e.ai offers a suite of AI-powered video and image generation tools, including features for animating still images, multi-modal video generation up to 1080P, cinematic video creation with synced audio, and various image editing capabilities. The platform commercializes video models from OpenAI (Sora 2 Pro), Google (Veo 3.1), and other developers like Kling 3.0 and Wan 2.7.

Modal, in collaboration with Pipecat, detailed how they achieved near one-second voice-to-voice latency for AI chatbots using open-weight models (Parakeet-tdt-v3 for STT, Qwen3-4B-Instruct-2507 + vLLM for LLM, KokoroTTS for TTS) hosted on Modal's platform. The article explains the architectural approach, including the use of WebRTC, Modal Tunnels for low-latency communication, and RAG with ChromaDB for knowledge retrieval, demonstrating optimized performance through geographic co-location of services.

An analysis of over 30 stack benchmarks details the challenges and progress in achieving sub-second voice loops for AI agents, identifying that LLM Time to First Token (TTFT) and TTS Time to First Byte (TTFB) are the primary latency contributors. The research highlights various STT, LLM, and TTS combinations, with GPT-4 Nano + Cartesia Sonic-Turbo achieving the lowest first-byte latency of 0.73-0.75 seconds, noting that English still outperforms Spanish in speed and advocating for modular pipeline designs. The article predicts continued model shrinkage and emergence of joint LLM-TTS training for further latency reduction.

Topaz Labs has released a new extension for Adobe Premiere Pro, enabling users to apply AI-powered video and image enhancement models directly within their editing timeline. This includes features like upscaling, denoising, slow-motion, and frame interpolation, with processing handled via Topaz's cloud rendering services.

Google I/O 2026 will feature announcements for Gemini 3.2 Flash and Gemma 4 open-weights, alongside updates to Android 17 and Android XR glasses. These developments involve AI models and a new operating system with extended reality capabilities.

Google plans to integrate Gemini Intelligence into Android, turning it into an agentic AI system for task automation. These AI-driven features were announced at the Android Show 2026.

Adobe has launched GenStudio, a new offering integrating generative AI into enterprise marketing workflows for marketers and agencies. This platform aims to unify and accelerate the content supply chain by leveraging AI capabilities.

The article discusses the emergence of agentic media planning and buying as the next evolution of AI. It emphasizes that the crucial work in this field begins with human strategic intelligence before automation. The core idea is that AI agents are tools for executing human-defined strategies, not replacing the strategic thinking itself.

The article discusses the emergence of agentic media planning and buying as the next evolution of AI in early 2026. It emphasizes that the real work for agentic media, involving strategic thinking and content contextualization, begins prior to automation.

The article from Broadcom discusses the increasing demands of AI workloads on private infrastructure, specifically highlighting the need for enhanced compute, security, governance, and cost control. It positions private AI infrastructure as a solution for securely and scalably supporting enterprise AI applications.

Nebius has integrated Clarifai's core team and licensed its inference IP to enhance the Nebius Token Factory, focusing on system-level inference optimization and production AI. This move aims to strengthen Nebius's managed inference platform and global capacity for AI applications.

Red Hat's latest AI platform, AI 3.4, introduces Model-as-a-Service capabilities and enhanced agent management. The update also includes tighter integration with Nvidia technologies.

An article from AICerts discusses the challenges of verifying multimodal AI, specifically in the context of rumored Google Gemini 3.1 releases and Ultra versions. It reviews aspects such as specifications, costs, safety, and enterprise guidance related to the practical adoption of multimodal AI.

New research indicates that traditional marketing tactics are ineffective on AI shopping agents, which are becoming a significant portion of online shoppers.

Google is promoting its Agent Development Kit (ADK) for building long-running AI agents capable of managing complex enterprise workflows. The ADK facilitates the development of production-grade AI agents that can pause, resume, and maintain context.

Surf AI founder Yair Grindlinger published an opinion piece arguing that the primary challenge for AI deployments is orchestration, rather than the development of new AI models, highlighting the need for efficient management of AI systems.

Nebius Group N.V., an AI infrastructure company, has recruited the core engineering team from Clarifai, an AI orchestration software firm, as well as acquiring its compute orchestration technology. This move aims to enhance Nebius's AI inference capabilities. The acquisition specifically targets a team specializing in efficient utilization of AI computing resources.

Ahead of the Cannes film festival opening, actress Demi Moore commented on the increasing presence of AI in filmmaking, describing the fight against it as a "losing battle." The festival itself prohibits generative AI in competitive categories, yet AI's role in the film industry remains a prominent discussion.

Canadian broadcaster Sportsnet removed a Mother's Day interview segment featuring Canadiens center Nick Suzuki after it was reported that AI-generated family images were used in the clip. This incident highlights issues related to content authenticity and ethical considerations in media production using artificial intelligence.

This article discusses the application of economic principles, specifically game theory and mechanism design, to improve AI content moderation systems. Researchers are exploring how system design can incentivize users to report problematic content accurately and disincentivize bad actors from posting it. The approach aims to create a more efficient and effective moderation environment by understanding the strategic interactions between platforms, users, and content creators.

Taja has launched an AI video clipping tool designed to convert long-form recordings into multiple short-form videos optimized for social media platforms. The system automatically formats the content for various social platforms.

Stats Perform has launched Opta Pulse, an AI-assisted video creation and distribution solution for sports highlights. The tool is designed for leagues, rights holders, and broadcasters to streamline the creation and distribution of video content.

The article discusses how AI video editing is changing content creation, noting advancements in the creator economy over the last three years.

Samsung is expanding its Vision AI Companion feature across its entire 2026 TV lineup, encompassing new Micro RGB, OLED, Neo QLED, and audio models. This integration represents a broader application of AI capabilities within consumer streaming platforms.

The Indian film industry is adopting AI technologies in production workflows at a faster pace than Hollywood, utilizing AI for applications such as creating alternate endings, de-aging actors, and fully AI-generated content. This rapid integration highlights a growing trend within the country's entertainment sector.

At the 79th Cannes Film Festival, actress Demi Moore stated that the film industry should not attempt to combat AI, suggesting an acceptance of the technology.

TikTok has initiated the integration of ByteDance's next-generation artificial intelligence (AI) video model into its Symphony suite. This development is aimed at offering AI-powered video capabilities for brands and creators using the platform.

Google is reportedly developing a new AI video generation tool called Gemini Omni, which would enable users to create and edit realistic videos via chat. This potential tool is anticipated to be revealed at Google I/O 2026.

Nota, an AI model lightweighting and optimization company, announced that its "NVA" video control solution received a global Edge AI award. The solution focuses on on-device video control.

ShengShu Technology launched Vidu Claw, an AI CMO designed to generate complete ad campaigns from a single brief. The system utilizes Vidu Q3 in its Video Plan feature, streamlining ad creation without per-tool pricing or complex workflows.