Skip to content

Multimodal

Beyond text: vision, mixed image-text, audio and video input and output in models and products.

Latest picks

361–380 of 592

May 18Monday

Latent Space

The Autonomous Drone Tech Stack and Economics of Drones — Yaroslav Azhnyuk

Latent Space interviewed The Fourth Law founder Yaroslav Azhnyuk for a two-hour episode covering FPV drones, five levels of autonomy, eight dimensions of the autonomous battlefield, and China’s manufacturing advantage; the transcript claims Ukraine produced 4 million FPV drones last year and discusses a hypothetical Chinese capacity of 4 billion.

Why it matters: The Fourth Law founder Yaroslav Azhnyuk explains on a podcast how FPV drones became a battlefield staple, and lays out autonomy levels and cost structure.

r/LocalLLaMA

Qwen 3.6 27B on 24GB VRAM: backend comparisons, quant choice, and settings

The author tested Qwen 3.6 27B on an RTX 3090 24GB and kept ik_llama.cpp with Qwen3.6-27B-MTP-IQ4_KS.gguf; at 156k context with q8_0 KV and MTP, a ~5.9k-token prompt plus 1024-token output reached about 1261 tok/s prefill and 72.9 tok/s decode, while vLLM lacked a clean single-card long-context run.

Why it matters: The author benchmarked several backends and quantization schemes for Qwen 3.6 27B on a single 24GB RTX 3090, settling on ik_llama.cpp with IQ4_KS, as a single-card comparison for local deployment.

AI HOT (Curated Pool)

Grok Now Supports Video Understanding and Analysis

Grok now supports full-video uploads for real-time analysis, summarization, translation, scene explanation, and context extraction; the post does not disclose duration limits, supported formats, or rollout scope.

Why it matters: Grok added video understanding, letting users upload a whole video for frame analysis, summarization, translation and scene explanation, extending its multimodal reach from images and text to full video.

AI HOT (Curated Pool)

Alibaba Cloud launches HappyHorse video generation model

Alibaba Cloud launched HappyHorse on Model Studio, with prompt-to-1080p multi-shot video generation in one workflow; the post lists a limited-time 20% discount but does not disclose pricing, model parameters, or availability terms.

Why it matters: Alibaba Cloud put the HappyHorse video model on Model Studio, generating 1080p multi-shot video straight from text, with a limited 20% discount, useful for anyone picking video tools and weighing cloud costs.

Google DeepMind

Google DeepMind releases Gemini Omni Flash video model

Google DeepMind released Gemini Omni Flash, the first model in the Gemini Omni family. It combines image, audio, video and text inputs to generate high-quality video, and supports multi-turn editing in natural language.

Why it matters: One model now handles image, audio, video and text input plus multi-turn editing, and it ships inside Gemini, Flow and Shorts.

May 17Sunday

Bloomberg Technology

Apple’s New ChatGPT-Like Siri App Will Have Auto-Deleting Chats

The title says Apple’s ChatGPT-like Siri app will support auto-deleting chats; the RSS snippet only adds that iOS 27 will include a Genmoji upgrade, and the post does not disclose retention periods, release timing, or feature details.

Why it matters: Bloomberg reports iOS 27 will bring a standalone ChatGPT-like Siri app with auto-deleting chats, making the privacy handling the most concrete detail in the leak.

Google DeepMind

Google expands content provenance and verification tools across Search, Gemini, Chrome and Pixel

Google is widening its content transparency and verification tools across Search, Gemini, Chrome, Pixel and Cloud, and deepening industry partnerships. SynthID has watermarked over 100 billion images and videos plus 60,000 years of audio. SynthID verification in the Gemini app has been used 50 million times, and the capability reaches Search today, with Chrome in the coming weeks.

Why it matters: SynthID verification now reaches Search and Chrome, plus C2PA checks, so AI content labels cover more of the browsing path.

QbitAI · WeChat

TGO Aligns Visual Generative Models with Scalar Feedback Without Preference Pairs | ICML 2026

NUS proposed Threshold-Guided Optimization, which converts scalar feedback into positive or negative updates through a score-distribution threshold and was accepted by ICML 2026; experiments cover Stable Diffusion v1.5, FLUX, Wan 1.3B, and Meissonic across image and video generation settings.

Why it matters: NUS's TGO, accepted at ICML 2026, skips preference-pair construction and aligns visual generative models directly from scalar scores like aesthetics or image-text matching by thresholding.

QbitAI · WeChat

A Robot Dog Challenges Nvidia's Compute Lead

Weilan Technology unveiled BabyAlpha A3, a consumer quadruped robot using a six-chip heterogeneous cluster that runs a 7B-parameter model on-device at 280 TPS; the article says it has 66MP vision, 2.232 million point-cloud samples per second, and a planned Q3 launch.

Why it matters: Azurion runs a 7B model locally on a consumer robot dog using a six-chip heterogeneous cluster, claiming 280 TPS and giving on-device compute a concrete sample.

AI HOT (Curated Pool)

Grok Imagine image generation is officially released

Grok Imagine is now available on X for all users, with text-to-image generation for realistic images and multiple aspect ratios; the post does not disclose model parameters, pricing, or regional limits.

Why it matters: Grok Imagine opens text-to-image generation to all users on X, supports multiple aspect ratios and produces images inside the social timeline, a contrast with the entry points of Midjourney and DALL·E.

Financial Times · Technology

Chinese AI Groups Pull Ahead of US Rivals in Video Generation Race

FT says Chinese AI groups have moved ahead of US rivals in video generation; the RSS snippet names ByteDance and Kuaishou and says they outshine western competitors in advertising and entertainment quality, but the post does not disclose benchmark metrics or model details.

Why it matters: ByteDance and Kuaishou are said to lead US rivals in video generation, with better results in ads and entertainment, though the report gives no models, benchmarks or sample sizes.

Synced · WeChat

What Are World Models? Their History and the $10 Billion Bet

Jiqizhixin translated a MoE Capital blog tracing two world-model lineages. The article says more than $10 billion entered the category over 18 months, and cites DreamDojo as using 44,711 hours of first-person video pretraining to reach r=0.995 correlation with real-world robot policy outcomes.

Why it matters: Maps two routes to world models, one from AI planning research, one from game and film 3D generation, with over $10B invested in 18 months.

May 16Saturday

Hacker News front page

SANA-WM, a 2.6B open-source world model for 1-minute 720p video

SANA-WM’s title says the project is a 2.6B open-source world model for 1-minute 720p video; the RSS body only lists the project URL, Hacker News comments URL, 9 points, and 8 comments, and the post does not disclose training data, license terms, inference cost, evaluation setup, or benchmark results.

Why it matters: NVIDIA open-sources SANA-WM, a 2.6B world model that turns one image plus a camera trajectory into a minute of 720p video, with a two-stage pipeline for long-video quality and motion consistency.

Synced · WeChat

Why Robots Need World Models: Top Institutions Release Joint Survey

NTU MARS Lab and collaborators released a 43-page survey on robot world models, covering definitions, architectures, applications, benchmarks, and challenges around action-conditioned consistency, inference efficiency, and physical grounding.

Why it matters: NTU MARS Lab and partner institutions release a 43-page survey on robot world models, covering how they let robots predict physical outcomes before acting, as background on simulation training for embodied AI.

QbitAI · WeChat

Zhejiang University and Microsoft use 3,000 text prompts to improve video 3D consistency with World-R1

Zhejiang University and Microsoft introduced World-R1, training Wan 2.1 with about 3,000 text-only prompts, Flow-GRPO, and a four-part reward; the 1.3B version improves PSNR over the baseline by 10.23 dB.

Why it matters: World-R1 from Zhejiang University and Microsoft aligns Wan 2.1 video generation using about 3,000 text-only prompts and no 3D annotations, a cheap route for teams without labeled data.

Google DeepMind

Google DeepMind releases Gemini 3.5 Flash

Google DeepMind released the Gemini 3.5 model family, with the first model, Gemini 3.5 Flash, available the same day in the Gemini app, Google Search AI Mode, Google Antigravity, the Gemini API and Gemini Enterprise.

Why it matters: Gemini 3.5 Flash beats Gemini 3.1 Pro on coding and agent benchmarks and outputs 4x faster than other frontier models, shipping the same day in the Gemini app, Search AI Mode and the Gemini API.

AI HOT (Curated Pool)

Runway Agent Generates Complete Ads in One Session

Runway Agent turns product photos and ideas into fully produced ads in one session; the post does not disclose the model, pricing, generation length, or regional availability.

Why it matters: Runway's Agent turns a product photo and an idea into a finished ad in one chat window, so short-video ad teams can gauge how far their production pipeline can shrink.

May 15Friday

r/LocalLLaMA

Fully Offline Suitcase Robot Built Around Jetson Orin NX SUPER 16GB

CreativelyBankrupt built Sparky as a fully offline suitcase robot on Jetson Orin NX SUPER 16GB, running Gemma 4 E4B Q4_K_M via llama.cpp with q8_0 KV cache, about 200 ms cached TTFT, 14-15 tok/s sustained output, 12K context, 30+ sensors, and no WiFi, Bluetooth, or cellular interface.

Why it matters: A solo developer runs Gemma 4 E4B fully offline on a Jetson Orin NX SUPER 16GB mounted on a suitcase, with about 200ms cached TTFT and 14 to 15 tokens per second.

MIT Technology Review · AI

The Download: China’s AI Drama Factory and the WHO’s Missing Health Targets

China’s short-drama industry released an average of 470 AI-generated short dramas per day in January, while production timelines fell from months to weeks and costs dropped by up to 90%.

Why it matters: China puts out 470 AI vertical dramas a day, with production cut from months to weeks and costs down 90%, and scripts now driven by play data.

MIT Technology Review · AI

How Chinese Short Dramas Became AI Content Machines

Chinese short-drama companies are using AI for full-series production, with DataEye counting an average of 470 AI-generated short dramas released per day in January 2026, while FlexTV says production time fell from three to four months to under one month and North American per-series costs can drop by 80% to 90%.

Why it matters: Shows how far AI has moved into full production pipelines, with DataEye citing 470 AI short dramas a day and costs down 80 to 90 percent.