Skip to content

Multimodal

Beyond text: vision, mixed image-text, audio and video input and output in models and products.

Latest picks

261–280 of 592

Jun 1Monday

AI HOT (Curated Pool)

NVIDIA Open-Sources Cosmos 3, Its First Generalist Model for Physical AI

NVIDIA open-sourced Cosmos 3 at GTC Taipei, releasing two variants, Super 32B and Nano 8B, with model weights, code, and datasets made available.

Why it matters: NVIDIA open-sourced all of Cosmos 3's weights, code and datasets in 32B and 8B versions, giving robotics work a ready physical-world understanding and action-generation model.

Xinzhiyuan · WeChat

400 tokens/s: StepFun Step 3.7 Flash cuts Agent task costs

StepFun released Step 3.7 Flash, a sparse MoE model with 196B parameters plus a 1.8B ViT, activating 11B parameters per inference and reaching up to 400 tokens per second.

Why it matters: StepFun released Step 3.7 Flash, 196B total parameters with only 11B activated per inference, claiming agent task costs are a fraction of Claude's.

Synced · WeChat

OpenAI recruits for robotics team led by Sora creator Aditya Ramesh

OpenAI has listed more than a dozen San Francisco robotics roles for OpenAI Robotics, a team that evolved from Aditya Ramesh’s Worldsim work, with the actuator design engineer role offering $342,000 to $445,000 in base cash pay plus PPU incentives.

Why it matters: OpenAI is hiring more than ten robotics roles in San Francisco under Sora lead Aditya Ramesh, with actuator design engineers paid $342,000 to $445,000 in base cash.

Synced · WeChat

World models get a “save state”: VAST releases Project Eden

VAST released Project Eden, a three-layer world-model architecture that separates persistent state evolution from visual rendering, and disclosed nearly $200 million across its A+ and A++ funding rounds.

Why it matters: VAST's Project Eden splits a scene's persistent state from its rendering, so a generated 3D world can be saved and reopened with changes intact.

QbitAI · WeChat

How Cloud Models Reach the Physical World: CMG Lion Rock AI Lab Uses LiOS for Embodied AI

CMG Lion Rock AI Lab released the LiOS edge-cloud architecture for embodied robotics, reporting about 30 ms one-way latency from local camera to cloud GPU memory in cross-machine tests, and open-sourced the low-latency video transmission module plus the LeFold laundry-folding dataset.

Why it matters: China Merchants Group's Lion Rock Lab open-sourced the LiOS edge-cloud stack, measuring about 30 ms one-way latency from a local camera to cloud GPU memory, plus a low-latency video module and the LeFold clothes-folding dataset.

QbitAI · WeChat

VAST Raises Nearly $200M and Discloses Its Project Eden World Model Roadmap

VAST raised nearly $200 million in A+ and A++ rounds and disclosed Project Eden, a world model architecture that separates state evolution from visual rendering through a structured state layer, a conditional interface layer, and a generative rendering layer.

Why it matters: VAST published its Project Eden world model architecture alongside a nearly $200 million raise, splitting state inference from rendering into a state layer, a conditioning interface and a generative renderer.

AI HOT (Curated Pool)

Introducing Cosmos Coalition

Runway joined Cosmos Coalition as a founding member and will co-develop the first open world-model foundation model for physical AI with NVIDIA.

Why it matters: Runway joined the Cosmos Coalition as a founding member to build open physical AI world models with NVIDIA and other labs, and its jointly developed foundation model is the alliance's first project.

AI HOT (Curated Pool)

Cosmos 3 Released: First Open Physical AI Generalist Model

NVIDIA released Cosmos 3 as an open physical AI generalist model with native visual reasoning, world generation, and action generation, offering two variants: Super at 32B parameters and Nano at 8B parameters.

Why it matters: Nvidia open-sourced Cosmos 3 for physical AI in two sizes, a 32B Super and an 8B Nano, folding visual reasoning, 3D world-state generation and action output into one model.

AI HOT (Curated Pool)

MiniMax M3: Frontier coding, 1M-token context, and native multimodal model

MiniMax released M3 as an open-source unified model with coding, agent, and native multimodal capabilities, supporting a 1M-token context window and using MiniMax Sparse Attention to cut per-token compute at 1M context to 1/20 of its predecessor, with over 9x faster prefill and over 15x faster decoding.

Why it matters: MiniMax open-sourced M3 with published weights, combining coding, a 1M-token context and native multimodality in one model, scoring 59.0% on SWE-Bench Pro.

AI HOT (Curated Pool)

Qwen3.7-Plus: Multimodal Agent Intelligence

Qwen Studio lists seven capability areas: chatbots, image and video understanding, image generation, document processing, web search integration, tool use, and artifact generation; the post does not disclose Qwen3.7-Plus parameters, pricing, or release timing.

Why it matters: Qwen folds vision and language into one model that reads screens, drives apps and writes frontend code from screenshots, scoring 70.3 on Terminal Bench 2.0.

Bloomberg Technology

Gen-Z Gamer’s 3D-Model Startup Becomes China’s Latest AI Unicorn

Vast raised nearly $200 million and reached a $1 billion valuation; the RSS snippet says the 3D-modeling startup was founded by a 29-year-old gamer but does not disclose investors or product specifications.

Why it matters: A 29-year-old gamer's AI 3D-model startup raised nearly $200M at a $1B valuation, showing how hard capital is betting on AI-generated 3D content.

OpenAI News

OpenAI bans PRC-linked accounts using ChatGPT to generate comments on US tech policy and tariffs

OpenAI's June 2026 threat report details a banned cluster of ChatGPT accounts likely originating in China. The operators used Simplified Chinese prompts and VPNs to generate English comments and political cartoons criticizing US tariffs, rare earths, AI, and 5G policy. They instructed the model to depict only Trump, not Xi Jinping or China. The same cluster produced Chinese-language water army content attacking the US and Israel, amplifying anti-Jewish tropes, and harassing dissidents. OpenAI also linked these accounts to a separate X network that falsely claimed ChatGPT user data was compromised. The post does not disclose the exact number of banned accounts or the operators' specific institutional affiliation.

Why it matters: OpenAI's threat report says it banned accounts likely from China that used Simplified Chinese prompts to mass-produce English comments and cartoons aimed at US tariff and tech policy debates.

May 31Sunday

Xinzhiyuan · WeChat

Fudan-Linked Team Releases STI-WM Spatiotemporally Integrated World Model

MouShen Intelligence released STI-WM, a spatiotemporally integrated world-action model for robotics, claiming support for RGB, point-cloud, and proprioceptive inputs, hundred-second task planning, and disclosing five funding rounds in six months plus a RMB 300 million Pre-A round.

Why it matters: A Fudan-linked team's world action model fuses RGB, point clouds and robot proprioception, planning tasks over a hundred seconds long.

QbitAI · WeChat

Robot-Native World Action Model Debuts With Spatiotemporal Architecture From Fudan-Linked Team

Moushen Intelligence released STI-WM, a spatiotemporally integrated world action model for robotics, with RGB, depth point cloud, and proprioceptive inputs; the post says it supports hundred-second-scale long-horizon task rollout and closed-loop replanning, but does not disclose benchmark scores or deployment costs.

Why it matters: Muchen Intelligence released STI-WM, feeding RGB, depth point clouds and robot joint states into one spatiotemporal architecture, supporting long-task rollouts over a hundred seconds and replanning after errors.

May 30Saturday

AI HOT (Curated Pool)

AI Scammers Create Fake Black Personas to Sell Low-Quality Shein Goods

Sellers use AI-generated Black personas on TikTok, Facebook, and Instagram to pose as handmade creators and sell mass-produced dropshipped goods; the snippet cites one fake persona, “Aliyah,” selling fictional handmade belt buckles.

Why it matters: A Verge reporter found AI-generated fictional Black creator accounts on TikTok and other platforms, using a fake artisan persona to sell $9 Shein belt buckles for $40.

AI HOT (Curated Pool)

Nano Banana Pro and Nano Banana 2 officially released

Google AI Developers released Nano Banana Pro and Nano Banana 2, mapped to gemini-3-pro-image and gemini-3.1-flash-image. The post says both are production-ready through the Gemini API, but does not disclose pricing, benchmarks, or runtime limits.

Why it matters: Google put its gemini-3-pro-image and gemini-3.1-flash-image models on the Gemini API under the names Nano Banana Pro and Nano Banana 2, so developers can call them directly.

QbitAI · WeChat

Key Gemini IMO Gold Contributor Nearly Became a Professional Pianist

Yi Tay served as a modeling co-captain for Gemini Deep Think when it reached IMO gold-medal level, co-founded Reka AI in 2023, and returned to Google DeepMind after 639 days, while the article also notes his 2012 Trinity classical piano associate diploma.

Why it matters: Yi Tay was co-lead of modeling when Gemini Deep Think reached IMO gold-level results, co-founded Reka AI, and returned to Google DeepMind after 639 days.

Synced · WeChat

Apple Uses AI to Rework Image Compression: Same Visual Quality at One-Third the File Size

Apple’s team published PICO, a perceptual image codec that uses 57%-70% fewer bits than AV1, VVC, and JPEG AI at the same subjective visual quality, while encoding a 12MP photo in 230 ms and decoding it in 150 ms on an iPhone 17 Pro Max.

Why it matters: Apple's paper claims 57% to 70% bitrate savings over AV1, VVC and JPEG AI at similar subjective quality, with on-device timings on iPhone 17 Pro Max.

Synced · WeChat

NVIDIA and Tsinghua Team's Gamma-World Tops Hugging Face Daily Chart

NVIDIA, Tsinghua, University of Toronto, and Vector Institute released Gamma-World, a multi-agent world model using simplex-based positional encoding and hub tokens to cut interaction cost from quadratic to linear, with 8-player latency dropping from 17.6 ms to 4.5 ms.

Why it matters: NVIDIA and Tsinghua researchers propose the multi-agent world model γ-World, using simplex positional encoding and hub tokens to cut inter-agent interaction cost from quadratic to linear.

AI HOT (Curated Pool)

OpenAI launches real-time translation model with 70+ input languages

OpenAI launched gpt-realtime-translate, a speech translation model that accepts 70+ input languages and outputs speech in 13 target languages; the post says the feature is running on smart glasses.

Why it matters: OpenAI's gpt-realtime-translate takes speech input in over 70 languages and outputs 13, and it already runs on smart glasses, showing what a dedicated translation model trades off.