AI for Video: Generative Tools, Automation & Machine Learning News

MathWorks' Image Processing Toolbox provides algorithms and apps for image processing, analysis, and visualization, supporting both deep learning and traditional techniques. It can process 2D, 3D, and large images, with capabilities such as image segmentation, enhancement, noise reduction, and geometric transformations. The toolbox also supports C/C++ code generation for embedded vision systems and GPU/multicore acceleration.

Ericsson and Qualcomm published a report exploring the evolving demands of AI-enabled Extended Reality (XR) on mobile networks, from 5G to 6G. The article highlights significant increases in uplink traffic for consumer and enterprise XR applications and the need for enhanced network capabilities to support these data-intensive, low-latency experiences. This trend impacts future mobile network infrastructure and the development of immersive streaming technologies.

Spheron, a GPU cloud provider, details a three-stage disaggregated architecture for vLLM-Omni, a multimodal model serving large language models with separate encoder, prefill, and decode GPU pools. This architecture significantly boosts throughput for image and audio-heavy workloads, especially at scale, by optimizing GPU types for each stage's bottleneck. The article includes a full deployment walkthrough on Spheron GPU Cloud with recommendations for GPU sizing and cost optimization.

Researchers have introduced VisualClaw, a real-time personalized AI agent designed to filter visual evidence, reason with cloud VLMs, and evolve skills, significantly reducing video processing costs by up to 99.3%. It employs hybrid encoding and self-evolving skill banks to improve accuracy and cost efficiency in multimodal agentic workflows while addressing deployment gaps like expensive video frames and static model scaffolds. The system includes VisualClawArena, a benchmark for evaluating visual evidence use in executable multimodal workflows with 200 scenarios.

OpenAI has updated its ChatGPT app with 70+ language voice input and launched its developer-focused GPT-Realtime model suite. This suite contains GPT-Realtime-Translate and GPT-Realtime-Whisper, enabling low-latency streaming speech translation and automated live captioning workflows.

Digital forensics expert Hany Farid warned that generative AI video has advanced to a point where human experts cannot reliably detect deepfakes. This shift is driving interest in AI-based detection software and cryptographic authenticity standards, such as the Content Credentials provenance system, to verify video history and prevent digital scams.

AMD announced that its FSR 4.1 AI upscaling technology will have different models for RDNA 3 and RDNA 2 GPUs due to varying hardware capabilities. RDNA 3 will use a requantized INT8 model instead of FP8, while RDNA 2 requires significant optimization for FSR 4.1 support due to a lack of AI accelerators. This impacts AI upscaling performance and adoption across different GPU generations, with RDNA 2 support delayed until early 2027.

Google has launched Gemini 3.5 Live Translate, a new speech-to-speech AI model supporting over 70 languages which is rolling out across consumer, enterprise, and developer products. The model will expand real-time translation features in Google Meet for Workspace customers and the Google Translate mobile app. It is also available via the Gemini Live API, allowing developers to build multilingual voice applications for various use cases including live broadcasting, with partners like Agora and LiveKit already integrating it.

New AI tools are enabling creators to generate talking videos from single images and audio files, significantly lowering production barriers for character-driven content and podcasts. This technology also facilitates AI dubbing with synchronized lip movements, allowing creators to reach global audiences more easily. The trend is creating a creative explosion, particularly benefiting independent creators and gaming communities.

Studio Freewillusion Inc. announced new performance results for its AI-powered automatic dubbing technology, TailorDub, which significantly outperformed a global AI dubbing SaaS platform in speech-pacing stability and other metrics. Unlike conventional text-to-speech tools, TailorDub uses an audio-based pipeline to align translated dialogue with original speech segments, preserving emotion and intonation. The company has also presented related academic research on its technology at computer science conferences.

Chinese film industry experts at the Shanghai International Film Festival highlighted computing power, distribution models, and directable generative video AI as major challenges for future filmmaking. Speakers emphasized the need for greater compute infrastructure to refine AI models and more 'obedient' AI that aligns with specific artistic visions. The discussion also touched upon the emergence of new jobs like AI directing assistants, suggesting that advanced tools elevate the importance of human skills in filmmaking.

Ion Video has filed a patent application in Australia for technology designed to verify the authenticity of video content by examining binary samples, rather than pixels or metadata, to detect AI-generated or manipulated footage. The system creates a cryptographic record and uses an authentication policy engine that combines multiple verification methods, including C2PA content credentials and camera sensor fingerprinting. This development aims to provide a more robust method for proving video authenticity amid the rise of generative AI and manipulated media.

Luma AI, a generative AI video startup valued at $4 billion, is strategically focusing its tools on creative professionals rather than direct consumers. The company's COO, Caroline Ingeborn, stated that the aim is to address production problems for filmmakers, advertisers, and creative teams in Hollywood. This approach contrasts with the 'movie-in-a-prompt' fantasy previously envisioned by companies like OpenAI with Sora, and has reportedly led to significant growth in Luma's valuation and team size since 2024.

StFX graduate Moayadeldin Hussain and Dr. Iker Gondra won the Best Paper in Computer Vision Award at CRV 2026 for their research on temporal action localization. Their work, titled "CtxPL: Context-based Prototype Learning for Weakly-Supervised Temporal Action Localization," offers advancements in AI-driven video analysis. This technology has applications in video summarization and intelligent surveillance systems.

A joint research team from KAIST, MIT, and Microsoft developed 'Upsample Anything' technology, a training-free upsampling method that improves GPU memory efficiency for AI computer vision by up to 16 times. This innovation allows AI to process visual information more precisely with limited computational resources, crucial for on-device AI applications like smartphones, humanoid robots, and autonomous driving. The technology was recognized at CVPR 2026 for its efficiency and responsible AI research principles.

The article discusses the increasing use of AI in press releases and marketing within the media and entertainment industry, leading to confusing jargon. This trend coincides with a broader shift towards digitalization and software applications over hardware. The author expresses concern about the lack of clarity in communication due to AI-generated content, despite embracing technological evolution.

3 Screen Solutions (3SS) has launched new AI-powered tools, 3Ready Hero Search and 3Ready Hero for Editors, within its 3Ready platform. These features aim to improve content discovery for streaming service users and enhance editorial workflows for operators and automotive OEMs. The tools leverage AI to understand user intent, aggregate content sources, and provide platform-specific assistance to editorial teams.

The 2026 APOS summit focuses on the evolving $180 billion Asian screen entertainment market, emphasizing the impact of AI on cost reduction and production timelines. Discussions will cover premium streaming, the rise of microdramas, and the continued growth of K-content and anime. Key questions revolve around the future of premium streaming, India's JioStar competing with YouTube, microdrama profitability, and how AI can collapse costs without cutting out human involvement in production.

Cineverse President Erick Opeka detailed how AI currently drives revenue through improved metadata, ad decisioning, and content discovery, citing over $54 million for "Terrifier 3" using an AI-powered theatrical strategy. The company's "Hex" system uses AI to classify emotional content, enhancing programming and advertising for specific fandoms. Cineverse is also selling this AI-powered machinery as a product, making technology platforms a significant revenue driver.

Google has released Android 17 and Wear OS 7, showcasing new AI models like Lyria 3 for music generation and Gemini Omni for video editing on Pixel devices. The updates integrate advanced AI features for content creation, communication, and device experiences, including a new feature for recording simultaneous selfie camera and screen videos for social platforms, and parental controls.

Hydrolix now offers high-speed analytics for petabyte-scale agentic AI through AWS Marketplace, providing sub-second query responses from complete datasets. This solution, featuring a built-in stream processing engine, is particularly beneficial for media clients managing CDNs and web applications, as demonstrated by a case study with Nvidia to resolve a CDN incident using natural language querying.

TwelveLabs has expanded its video AI models, Marengo, Pegasus, and Rodeo, onto AWS Marketplace, enabling enhanced video search and editing capabilities for the telco, media, entertainment, games, and sports (TMEGS) sectors. This move aims to broaden the reach of its multimodal AI technology for analyzing and interacting with video content for streaming industry professionals. Danny Nicolopoulos, head of strategic partnerships at TwelveLabs, highlighted the demand from companies monetizing video for these AI tools, which facilitate use cases like creating highlight reels and accelerating video editing workflows.

Qualcomm unveiled Snapdragon Reality Elite, a rebranded and enhanced version of its XR chip line, positioning it as an AI platform with significantly improved neural processing units. This new chip aims to power next-generation immersive VR/MR experiences and support on-device large language model processing for smart glasses and other wearables. Qualcomm is also launching Snapdragon START, a turnkey solution with reference designs for manufacturing smart devices, akin to its role in the Android handset era.

NVIDIA's Blackwell platform achieved a clean sweep in the MLPerf Training v6.0 benchmarks, demonstrating industry-leading performance and scale for training AI models like DeepSeek-V3 and GPT-OSS-20B. The company showcased significant software optimizations, including full-iteration CUDA graphs and CuTe DSL kernel fusions, which contribute to a continuous improvement in training throughput for generative AI workloads. This performance is critical for rapidly evolving streaming AI applications.

NVIDIA has released XR AI, an open-source library in public beta, for developing intelligent AI agents for AR/XR devices. This platform helps integrate live media streams and multimodal AI models to create contextual AI experiences in hands-on environments. It provides a reusable foundation for connecting XR devices to GPU-accelerated AI services across various deployment environments.

Nvidia's Q1 FY2027 report indicates a significant market shift by separately forecasting edge computing, with edge revenue reaching $6.4 billion and growing 29%. This signals a move of AI systems from centralized cloud infrastructure to real-time edge processing for low latency and data security needs, affecting various industries including defense and automotive.

Qwen has released Qwen-RobotSuite, a collection of three embodied AI models (RobotManip, RobotWorld, RobotNav) designed for robotics applications. These models address challenges in manipulation, video world modeling, and navigation, built on Qwen vision-language backbones and demonstrating strong benchmark results. The suite aims to overcome the fragmentation of robotics data and provides a framework for scaling manipulation data, using language as a universal action interface, and offering a controllable interface for navigation.

Modulate has developed Velma, an AI voice intelligence platform that analyzes acoustic signals beyond simple transcription to detect risks like fraud, churn, deepfakes, and compliance issues in real-time conversations. The platform and its associated APIs (including Deepfake Detection and PII Redaction) are designed for enterprise use, offering features for fraud prevention, AI agent guardrails, and trust & safety in various voice-enabled environments.

Adobe has upgraded the underlying object detection model for its Object Masking tool, leading to more detected objects, complete boundaries, and faster mask creation in their video editing software. This enhancement aims to provide professionals with more precise and efficient video editing and masking effects, though hardware limitations exist for older systems.

Meltwater has introduced an add-on broadcast monitoring feature that integrates Australian and New Zealand television and radio content into its platform. This new capability allows users to search, clip, share, and analyze broadcast media, including ad detection, and enables the exclusion or targeting of commercial advertisements. The feature aims to expand media monitoring beyond online content for Meltwater's customers.

Researchers have developed ITW-SM, a new dataset of 10,000 real and AI-generated social media images, to improve the accuracy of AI-generated image detection in real-world scenarios. Their study identifies key factors like training data, backbone architectures, cropping methods, and data augmentations that significantly impact detector performance, leading to a 26.87% average AUC improvement. This research is crucial for streaming professionals to identify and combat synthetic media, which poses risks like disinformation and fraud.

ElevenLabs has launched Dubbing v2, an AI model designed to translate video and audio into over 90 languages while preserving the original speaker's performance, tone, and pacing. This new version supports uploads up to 2 GB and 180 minutes, aiming to provide creators with a more accessible and high-fidelity localization solution compared to traditional methods.

OpenMusic AI has launched an AI Story Video Generator that transforms music into cinematic narratives automatically. The platform allows users to upload audio, customize elements like actors and video styles, and then generates beat-synced videos with automatic subtitles and continuous shot effects. This tool aims to provide professional-quality story videos without the need for extensive film production experience.

The article outlines five key AI video trends for enterprise marketing teams in 2026, focusing on workflows for repurposing long-form content, leveraging AI studios for video creation, adopting multi-model strategies, ensuring consistent volume publishing, and integrating brand consistency into AI governance. Vizard is presented as a solution unifying these workflows, offering access to various text-to-video models like Seedance 2.0, Veo 3, and Kling, and automating resizing and subtitling for multi-platform distribution.

Researchers from Iqra University and Ulster University have developed a temporally aware, intra-video Retrieval-Augmented Generation (RAG) framework to improve VideoQA accuracy for lecture videos. This framework aligns speech transcripts and visual captions to temporal boundaries, and refines retrieved segments with a cross-encoder before answer generation. The method was evaluated on the LectQA-Vid dataset, demonstrating improved factual alignment and robustness over non-temporal baselines.

Google's Gemini Enterprise Agent Platform has detailed its image understanding capabilities, outlining supported models (Gemini 3.5 Flash, 3.1 Pro, etc.), various image formats (PNG, JPEG, WebP, HEIC, HEIF) and capacities up to 30 MB. The documentation provides specifics on resolution options, tokenization methods, and best practices for developers to integrate image analysis into their applications using Gemini models for tasks like object detection and understanding image content.

GeiserX has developed WhisperSubs, a Jellyfin plugin that automatically generates subtitles for media libraries using local AI models. The plugin supports fully local audio transcription via whisper.cpp, automatic language detection, forced subtitles, and GPU acceleration. It includes features like a priority queue, real-time progress updates, and scheduled tasks for seamless integration with Jellyfin's admin UI.

Interprefy published a guide on real-time captioning services for global enterprise events, detailing how AI and human-powered options enhance accessibility and comprehension for diverse audiences. The article explains the technology behind real-time captions, including audio capture, speech recognition, text refinement, and translation, as well as crucial factors for event teams when choosing a captioning partner. It highlights the growing importance of multilingual and hybrid captioning for various event types, from conferences to internal meetings.

A new review outlines how advanced generative AI models, such as OpenAI Sora and Tencent HunyuanVideo, are eroding traditional deepfake detection methods by producing high-resolution, temporally consistent videos. It proposes new detection paradigms focusing on assumption-aware and generalization-focused frameworks, emphasizing the urgent need for robust strategies to combat synthetic media threats due to societal impacts like the rise of deepfake pornography crimes.

Synthesia, an AI video platform, announced a partnership with Cinder to enhance its trust and safety stack for AI video content moderation. This collaboration aims to increase product velocity and improve the handling of new types of harm, which is critical for enterprise customers in regulated industries. The partnership will integrate Cinder's infrastructure to act as a second pass on model decisions and help Synthesia's team build faster against new harm types.

Neymar Jr. has partnered with COL Group's FlareFlow platform to launch a 16-title AI-powered live-action microdrama franchise, with the first six titles debuting on June 19. This collaboration leverages AI creative workflows to produce premium vertical storytelling for a global audience, aligning with FlareFlow's 'Vertical 2.0' strategy.

Director Aitore Zholdaskali's debut film 'Sicko' screens at the Transilvania International Film Festival after grossing $2 million in Kazakhstan. Zholdaskali highlights how AI technologies, such as those used with startup Higgsfield to create 'Hell Grind' in two weeks for $500,000, are democratizing filmmaking and offering new creative freedoms.

Microsoft is promoting various events, including the Microsoft AI Tour, Virtual Training Days, and the Microsoft Agent-a-Thon. These events focus on AI, Cloud, and Data technologies, offering training and networking opportunities for professionals. The initiative aims to help attendees develop skills and explore Microsoft's solutions in these areas.

Apple has released a rebuilt Siri AI in the iOS 27 developer beta, which, while not groundbreaking, brings its functionality closer to competitors like ChatGPT. Early testing by Mark Gurman suggests Siri AI can now perform complex tasks like managing calendar appointments and retrieving recommendations from messages. This development positions Apple's AI six months behind industry leaders but with a 'completely modern architecture' for future enhancements.

Adobe has updated five Creative Cloud applications, including Premiere and After Effects, with new AI-assisted tools. These enhancements focus on streamlining video and image editing workflows through features like AI-powered rotoscoping, reflection removal, and AI-generated motion from photos. Tools such as Object Matte in After Effects and Reflection Removal in Photoshop leverage AI to improve efficiency and creative production for professionals.

This article explains AI inference engineering, focusing on optimizing Large Language Model (LLM) operations in production for efficiency. It details techniques like batching, quantization, and disaggregation to improve latency, throughput, and cost, driven by the shift towards self-hosting open AI models. The piece highlights the importance of understanding the prefill-decode split in LLM inference for effective optimization.

Google's Gemini Enterprise Agent Platform has introduced implicit and explicit context caching for its Gemini models. This feature is designed to reduce costs and latency for AI requests containing repeated content, offering up to a 90% discount on cached tokens for certain models. This is particularly useful for scenarios such as chatbots and repetitive analysis of large video or document files.