Skip to content

Multimodal

Beyond text: vision, mixed image-text, audio and video input and output in models and products.

Latest picks

241–260 of 592

Jun 4Thursday

AI HOT (Curated Pool)

Ideogram 4.0 Open-Source Text-to-Image Model Released

Ideogram released Ideogram 4.0, an open-source text-to-image model with a 9.3B-parameter core, a single-stream DiT architecture, Qwen3-VL-8B-Instruct text encoder, and a No. 4 ranking in DesignArena human evaluation.

Why it matters: Ideogram 4.0 ships open source at 9.3B parameters, running text and images through one pipeline, with long-text rendering and prompt-specified layout as its main tricks.

Financial Times · Technology

MP sues Musk’s xAI in UK test case over fake sexual images

UK MP Jess Asato sued Musk’s xAI over fake sexual images, using the claim to test whether AI model makers are liable for system outputs; the post does not disclose the model, generation mechanism, damages sought, or court timetable.

Why it matters: UK Labour MP Jess Asato is suing Musk's xAI over fake sexual images of her, a test case on whether model developers are liable for what their models output.

Hacker News front page

Gemma 4 12B: A Unified, Encoder-Free Multimodal Model

Google’s title introduces Gemma 4 12B as a unified, encoder-free multimodal model; the RSS snippet only lists 137 Hacker News points and 48 comments, and the post does not disclose architecture details, training setup, pricing, release terms, or benchmark results.

Why it matters: Google released Gemma 4 12B with a unified multimodal design that drops the separate vision encoder, letting one model handle text and images directly.

Jun 3Wednesday

r/LocalLLaMA

google/gemma-4-12B on Hugging Face

Google DeepMind released Gemma 4 open-weight models in five sizes, with the 12B variant supporting text, image, and audio input, instruction-tuned and pre-trained variants, native system prompts, function calling, and a context window of up to 256K tokens.

Why it matters: Google DeepMind posted open weights for five Gemma 4 sizes on Hugging Face, with the 12B version taking text, images and audio and a context window up to 256K tokens.

MIT Technology Review · AI

The Download: Trump’s New AI Order, and Smart Glasses for Warfare

President Donald Trump signed a new AI order asking companies to voluntarily submit frontier models for government review 30 days before release, without mandatory licensing; the newsletter also says Anduril and Meta are prototyping a military AR headset that envisions drone-strike orders through eye tracking and voice commands.

Why it matters: Trump signed a new AI executive order requiring tech companies to voluntarily submit frontier models for government review 30 days before release, shorter than the shelved 90-day version.

Synced · WeChat

RSS 2026: Ant Lingbo Proposes Autoregressive Causal World Model for Robot Manipulation with 50 Demos

Ant Lingbo and HKUST introduced LingBot-VA, an autoregressive video-action world model that unifies visual dynamics prediction and action inference, and the paper reports fine-tuning with 50 real-world demonstrations per task plus 92.0% and 91.1% success on RoboTwin 2.0 Easy and Hard settings.

Why it matters: Ant Group's Lingbo and HKUST fold frame prediction and action decisions into one autoregressive model, fine-tuned on just 50 real demos per task.

Latent Space

[AINews] Microsoft Build: MAI-Thinking-1 and MAI Family Models

Microsoft announced seven MAI models at Build, with MAI-Thinking-1 described as a 35B-active-parameter MoE with a 256K context window, and released a 109-page technical report covering training, data lineage, and performance claims.

Why it matters: Microsoft launched seven MAI models at Build, led by the 35B-active-parameter reasoning model MAI-Thinking-1, and published a 109-page technical report stating the training data used no third-party model distillation.

QbitAI · WeChat

Daxiao Robot and NTU Release PhysX-Omni for Unified Physical 3D Generation

Daxiao Robot and NTU introduced PhysX-Omni, a unified simulation-ready physical 3D generation framework for rigid, deformable, and articulated objects, with PhysXVerse covering 8.7K assets across 2.9K categories and PhysX-Bench evaluating six dimensions including geometry, scale, material, affordance, kinematics, and description.

Why it matters: One pipeline generates rigid, soft and articulated 3D assets, so teams skip separate processing and manual mesh fixes.

AI HOT (Curated Pool)

xAI releases Grok Imagine 1.5 preview image-to-video model

xAI released grok-imagine-video-1.5-preview via its API, letting users turn one still image into 720p video while controlling camera movement, pacing, and sound effects with natural-language prompts.

Why it matters: xAI's image-to-video preview in its API turns one still plus a text prompt into a 720p clip, a first look at its video plans.

AI HOT (Curated Pool)

Runway API adds Aleph 2.0 video editing

Runway API now provides Aleph 2.0 video editing for integration into apps, products, and platforms, supporting precise edits on multi-shot videos up to 30 seconds at 1080p while changing only selected portions; the post does not disclose pricing, rate limits, latency, or model availability by region.

Why it matters: Runway opened Aleph 2.0's video editing as an API, letting developers make local edits to multi-shot clips up to 30 seconds at 1080p and embed the capability in their own apps.

r/LocalLLaMA

Using Gemma 4 E4B with LiteRT: about 2.4× faster text generation than Q4 GGUF

The author tested Gemma 4 E4B on an RTX 4060 Ti 16GB, where LiteRT averaged 157.2 tok/s for text generation versus 66.3 tok/s for llama.cpp Q4 GGUF; image captioning on 111 full-resolution images improved only 1.1×, at about 72 seconds versus 80 seconds.

Why it matters: A hands-on RTX 4060 Ti test puts LiteRT text generation at roughly 2.4x llama.cpp Q4 GGUF, with image processing about even.

The Verge · AI

Microsoft’s Project Solara is an OS for AI agent gadgets

Microsoft announced Project Solara at Build 2026 as an Android-based OS for AI agent gadgets, not Windows, and the post discloses two concept devices: a desk device with facial recognition and a wearable badge with a camera and fingerprint scanner.

Why it matters: Project Solara runs on Android but targets hardware built to run AI agents, shown with a desktop device and a wearable badge.

Jun 2Tuesday

AI HOT (Curated Pool)

StepFun releases Step 3.7 Flash as an open-weight model for agentic coding

StepFun released the open-weight Step 3.7 Flash model for fast agentic coding, with tool calling and multimodal understanding, and the model is already available in Kilo alongside MiniMax M3.

Why it matters: StepFun opens the Step 3.7 Flash weights for agentic coding with tool calling and multimodal input, and MiniMax open-sources M3 the same day; both run on the Kilo platform.

r/LocalLLaMA

NVIDIA releases Cosmos 3 Omnimodal world models on Hugging Face

NVIDIA released Cosmos 3 on Hugging Face with Nano at 16B parameters and Super at 64B parameters; the post says the models generate video, images, audio, and action commands from text, image, video, and action-trajectory inputs.

Why it matters: NVIDIA open-sourced its Cosmos 3 world models on Hugging Face in 16B and 64B parameter versions, taking mixed text, image, video and action-trajectory inputs and outputting video, images, audio and action commands.

QbitAI · WeChat

Jensen Huang Brings NVIDIA CPUs Into the PC Market

NVIDIA RTX Spark will ship in Windows PCs this fall with 1 petaflop of AI compute and 128GB unified memory. The platform combines a Blackwell RTX GPU, a 20-core Arm-based Grace CPU, and NVLink-C2C, and NVIDIA says it can run 1-million-token-context, 120B-parameter language models locally.

Why it matters: NVIDIA packs a 20-core Grace CPU and a Blackwell GPU into a Windows machine over NVLink-C2C with 128GB of shared unified memory, a hardware path to running large models locally that differs from discrete GPUs.

Latent Space

[AINews] NVIDIA Cosmos 3, Nemotron 3 Ultra, and RTX Spark

NVIDIA released Cosmos 3 and Nemotron 3 Ultra; Cosmos 3 uses a Mixture-of-Transformers design with 16B Nano and 64B Super variants, while Nemotron 3 Ultra is described as a 550B-A55B open-weight model.

Why it matters: Nvidia used Computex to put out a world model, a large model and local inference hardware, with Cosmos 3's hybrid Transformer and 16B/64B sizes as the concrete anchors.

AI HOT (Curated Pool)

NVIDIA Cosmos 3 Tops Open-Weight Image and Video Generation Rankings

NVIDIA Cosmos 3 ranked first in Artificial Analysis’s open-weight text-to-image and image-to-video categories, with 16B Nano and 64B Super variants, and the release includes weights, code, curated datasets, and fine-tuning recipes under the OpenMDW 1.1 license.

Why it matters: NVIDIA's Cosmos 3 took first on Artificial Analysis's open-weights board for both text-to-image and image-to-video, with a Mixture-of-Transformers design pairing an autoregressive reasoner with a diffusion generator.

AI HOT (Curated Pool)

Gemini Omni Supports Creating Personal Digital Avatars

Gemini App says Gemini Omni can add users to video creation by generating a digital avatar that resembles their appearance and voice; the post does not disclose rollout scope, pricing, or safety mechanisms.

Why it matters: A Gemini App demo uses Gemini Omni to generate a digital double with the user's own face and voice for video creation, an official feature reveal showing product progress in video and avatars.

Jun 1Monday

AI HOT (Curated Pool)

MiniMax Releases Open-Source M3 with Coding, Long-Context, and Multimodal Capabilities

MiniMax released the open-source M3 model with coding, a 1M-token context window, and native multimodal support; M3 scores 59.0% on SWE-Bench Pro, 83.5% on BrowseComp, and costs about one-twelfth per token versus GPT-5.5.

Why it matters: MiniMax has open-sourced M3, putting code, million-token long context and native multimodality in one model, scoring 59.0% on SWE-Bench Pro against GPT-5.5's 58.6%.

Latent Space

Why Video Agent Models Are Next — Ethan He on xAI Grok Imagine

Ethan He says a small xAI team built Grok Imagine from zero to one in 3 months, and the episode discusses video agents, audio-video alignment, inference speedups, and the storage, egress, and GPU-hour costs behind large video datasets.

Why it matters: xAI's Ethan He built Grok Imagine with a small team in three months and argues video-model intelligence comes mostly from language models.