Skip to content

#多模态

4 today

Jun 3Wednesday

Synced · WeChat

RSS 2026: Ant Lingbo Proposes Autoregressive Causal World Model for Robot Manipulation with 50 Demos

Ant Lingbo and HKUST introduced LingBot-VA, an autoregressive video-action world model that unifies visual dynamics prediction and action inference, and the paper reports fine-tuning with 50 real-world demonstrations per task plus 92.0% and 91.1% success on RoboTwin 2.0 Easy and Hard settings.

Why it matters: Ant Group's Lingbo and HKUST fold frame prediction and action decisions into one autoregressive model, fine-tuned on just 50 real demos per task.

Latent Space

[AINews] Microsoft Build: MAI-Thinking-1 and MAI Family Models

Microsoft announced seven MAI models at Build, with MAI-Thinking-1 described as a 35B-active-parameter MoE with a 256K context window, and released a 109-page technical report covering training, data lineage, and performance claims.

Why it matters: Microsoft launched seven MAI models at Build, led by the 35B-active-parameter reasoning model MAI-Thinking-1, and published a 109-page technical report stating the training data used no third-party model distillation.

QbitAI · WeChat

Daxiao Robot and NTU Release PhysX-Omni for Unified Physical 3D Generation

Daxiao Robot and NTU introduced PhysX-Omni, a unified simulation-ready physical 3D generation framework for rigid, deformable, and articulated objects, with PhysXVerse covering 8.7K assets across 2.9K categories and PhysX-Bench evaluating six dimensions including geometry, scale, material, affordance, kinematics, and description.

Why it matters: One pipeline generates rigid, soft and articulated 3D assets, so teams skip separate processing and manual mesh fixes.

AI HOT (Curated Pool)

xAI releases Grok Imagine 1.5 preview image-to-video model

xAI released grok-imagine-video-1.5-preview via its API, letting users turn one still image into 720p video while controlling camera movement, pacing, and sound effects with natural-language prompts.

Why it matters: xAI's image-to-video preview in its API turns one still plus a text prompt into a 720p clip, a first look at its video plans.

AI HOT (Curated Pool)

Runway API adds Aleph 2.0 video editing

Runway API now provides Aleph 2.0 video editing for integration into apps, products, and platforms, supporting precise edits on multi-shot videos up to 30 seconds at 1080p while changing only selected portions; the post does not disclose pricing, rate limits, latency, or model availability by region.

Why it matters: Runway opened Aleph 2.0's video editing as an API, letting developers make local edits to multi-shot clips up to 30 seconds at 1080p and embed the capability in their own apps.

r/LocalLLaMA

Using Gemma 4 E4B with LiteRT: about 2.4× faster text generation than Q4 GGUF

The author tested Gemma 4 E4B on an RTX 4060 Ti 16GB, where LiteRT averaged 157.2 tok/s for text generation versus 66.3 tok/s for llama.cpp Q4 GGUF; image captioning on 111 full-resolution images improved only 1.1×, at about 72 seconds versus 80 seconds.

Why it matters: A hands-on RTX 4060 Ti test puts LiteRT text generation at roughly 2.4x llama.cpp Q4 GGUF, with image processing about even.

The Verge · AI

Microsoft’s Project Solara is an OS for AI agent gadgets

Microsoft announced Project Solara at Build 2026 as an Android-based OS for AI agent gadgets, not Windows, and the post discloses two concept devices: a desk device with facial recognition and a wearable badge with a camera and fingerprint scanner.

Why it matters: Project Solara runs on Android but targets hardware built to run AI agents, shown with a desktop device and a wearable badge.

Jun 2Tuesday

AI HOT (Curated Pool)

StepFun releases Step 3.7 Flash as an open-weight model for agentic coding

StepFun released the open-weight Step 3.7 Flash model for fast agentic coding, with tool calling and multimodal understanding, and the model is already available in Kilo alongside MiniMax M3.

Why it matters: StepFun opens the Step 3.7 Flash weights for agentic coding with tool calling and multimodal input, and MiniMax open-sources M3 the same day; both run on the Kilo platform.

r/LocalLLaMA

NVIDIA releases Cosmos 3 Omnimodal world models on Hugging Face

NVIDIA released Cosmos 3 on Hugging Face with Nano at 16B parameters and Super at 64B parameters; the post says the models generate video, images, audio, and action commands from text, image, video, and action-trajectory inputs.

Why it matters: NVIDIA open-sourced its Cosmos 3 world models on Hugging Face in 16B and 64B parameter versions, taking mixed text, image, video and action-trajectory inputs and outputting video, images, audio and action commands.

QbitAI · WeChat

Jensen Huang Brings NVIDIA CPUs Into the PC Market

NVIDIA RTX Spark will ship in Windows PCs this fall with 1 petaflop of AI compute and 128GB unified memory. The platform combines a Blackwell RTX GPU, a 20-core Arm-based Grace CPU, and NVLink-C2C, and NVIDIA says it can run 1-million-token-context, 120B-parameter language models locally.

Why it matters: NVIDIA packs a 20-core Grace CPU and a Blackwell GPU into a Windows machine over NVLink-C2C with 128GB of shared unified memory, a hardware path to running large models locally that differs from discrete GPUs.

Latent Space

[AINews] NVIDIA Cosmos 3, Nemotron 3 Ultra, and RTX Spark

NVIDIA released Cosmos 3 and Nemotron 3 Ultra; Cosmos 3 uses a Mixture-of-Transformers design with 16B Nano and 64B Super variants, while Nemotron 3 Ultra is described as a 550B-A55B open-weight model.

Why it matters: Nvidia used Computex to put out a world model, a large model and local inference hardware, with Cosmos 3's hybrid Transformer and 16B/64B sizes as the concrete anchors.

AI HOT (Curated Pool)

NVIDIA Cosmos 3 Tops Open-Weight Image and Video Generation Rankings

NVIDIA Cosmos 3 ranked first in Artificial Analysis’s open-weight text-to-image and image-to-video categories, with 16B Nano and 64B Super variants, and the release includes weights, code, curated datasets, and fine-tuning recipes under the OpenMDW 1.1 license.

Why it matters: NVIDIA's Cosmos 3 took first on Artificial Analysis's open-weights board for both text-to-image and image-to-video, with a Mixture-of-Transformers design pairing an autoregressive reasoner with a diffusion generator.

AI HOT (Curated Pool)

Gemini Omni Supports Creating Personal Digital Avatars

Gemini App says Gemini Omni can add users to video creation by generating a digital avatar that resembles their appearance and voice; the post does not disclose rollout scope, pricing, or safety mechanisms.

Why it matters: A Gemini App demo uses Gemini Omni to generate a digital double with the user's own face and voice for video creation, an official feature reveal showing product progress in video and avatars.

Jun 1Monday

AI HOT (Curated Pool)

MiniMax Releases Open-Source M3 with Coding, Long-Context, and Multimodal Capabilities

MiniMax released the open-source M3 model with coding, a 1M-token context window, and native multimodal support; M3 scores 59.0% on SWE-Bench Pro, 83.5% on BrowseComp, and costs about one-twelfth per token versus GPT-5.5.

Why it matters: MiniMax has open-sourced M3, putting code, million-token long context and native multimodality in one model, scoring 59.0% on SWE-Bench Pro against GPT-5.5's 58.6%.

Latent Space

Why Video Agent Models Are Next — Ethan He on xAI Grok Imagine

Ethan He says a small xAI team built Grok Imagine from zero to one in 3 months, and the episode discusses video agents, audio-video alignment, inference speedups, and the storage, egress, and GPU-hour costs behind large video datasets.

Why it matters: xAI's Ethan He built Grok Imagine with a small team in three months and argues video-model intelligence comes mostly from language models.

AI HOT (Curated Pool)

NVIDIA Open-Sources Cosmos 3, Its First Generalist Model for Physical AI

NVIDIA open-sourced Cosmos 3 at GTC Taipei, releasing two variants, Super 32B and Nano 8B, with model weights, code, and datasets made available.

Why it matters: NVIDIA open-sourced all of Cosmos 3's weights, code and datasets in 32B and 8B versions, giving robotics work a ready physical-world understanding and action-generation model.

Xinzhiyuan · WeChat

400 tokens/s: StepFun Step 3.7 Flash cuts Agent task costs

StepFun released Step 3.7 Flash, a sparse MoE model with 196B parameters plus a 1.8B ViT, activating 11B parameters per inference and reaching up to 400 tokens per second.

Why it matters: StepFun released Step 3.7 Flash, 196B total parameters with only 11B activated per inference, claiming agent task costs are a fraction of Claude's.

Synced · WeChat

OpenAI recruits for robotics team led by Sora creator Aditya Ramesh

OpenAI has listed more than a dozen San Francisco robotics roles for OpenAI Robotics, a team that evolved from Aditya Ramesh’s Worldsim work, with the actuator design engineer role offering $342,000 to $445,000 in base cash pay plus PPU incentives.

Why it matters: OpenAI is hiring more than ten robotics roles in San Francisco under Sora lead Aditya Ramesh, with actuator design engineers paid $342,000 to $445,000 in base cash.

Synced · WeChat

World models get a “save state”: VAST releases Project Eden

VAST released Project Eden, a three-layer world-model architecture that separates persistent state evolution from visual rendering, and disclosed nearly $200 million across its A+ and A++ funding rounds.

Why it matters: VAST's Project Eden splits a scene's persistent state from its rendering, so a generated 3D world can be saved and reopened with changes intact.

QbitAI · WeChat

How Cloud Models Reach the Physical World: CMG Lion Rock AI Lab Uses LiOS for Embodied AI

CMG Lion Rock AI Lab released the LiOS edge-cloud architecture for embodied robotics, reporting about 30 ms one-way latency from local camera to cloud GPU memory in cross-machine tests, and open-sourced the low-latency video transmission module plus the LeFold laundry-folding dataset.

Why it matters: China Merchants Group's Lion Rock Lab open-sourced the LiOS edge-cloud stack, measuring about 30 ms one-way latency from a local camera to cloud GPU memory, plus a low-latency video module and the LeFold clothes-folding dataset.

QbitAI · WeChat

VAST Raises Nearly $200M and Discloses Its Project Eden World Model Roadmap

VAST raised nearly $200 million in A+ and A++ rounds and disclosed Project Eden, a world model architecture that separates state evolution from visual rendering through a structured state layer, a conditional interface layer, and a generative rendering layer.

Why it matters: VAST published its Project Eden world model architecture alongside a nearly $200 million raise, splitting state inference from rendering into a state layer, a conditioning interface and a generative renderer.

AI HOT (Curated Pool)

Introducing Cosmos Coalition

Runway joined Cosmos Coalition as a founding member and will co-develop the first open world-model foundation model for physical AI with NVIDIA.

Why it matters: Runway joined the Cosmos Coalition as a founding member to build open physical AI world models with NVIDIA and other labs, and its jointly developed foundation model is the alliance's first project.

AI HOT (Curated Pool)

Cosmos 3 Released: First Open Physical AI Generalist Model

NVIDIA released Cosmos 3 as an open physical AI generalist model with native visual reasoning, world generation, and action generation, offering two variants: Super at 32B parameters and Nano at 8B parameters.

Why it matters: Nvidia open-sourced Cosmos 3 for physical AI in two sizes, a 32B Super and an 8B Nano, folding visual reasoning, 3D world-state generation and action output into one model.

AI HOT (Curated Pool)

MiniMax M3: Frontier coding, 1M-token context, and native multimodal model

MiniMax released M3 as an open-source unified model with coding, agent, and native multimodal capabilities, supporting a 1M-token context window and using MiniMax Sparse Attention to cut per-token compute at 1M context to 1/20 of its predecessor, with over 9x faster prefill and over 15x faster decoding.

Why it matters: MiniMax open-sourced M3 with published weights, combining coding, a 1M-token context and native multimodality in one model, scoring 59.0% on SWE-Bench Pro.

AI HOT (Curated Pool)

Qwen3.7-Plus: Multimodal Agent Intelligence

Qwen Studio lists seven capability areas: chatbots, image and video understanding, image generation, document processing, web search integration, tool use, and artifact generation; the post does not disclose Qwen3.7-Plus parameters, pricing, or release timing.

Why it matters: Qwen folds vision and language into one model that reads screens, drives apps and writes frontend code from screenshots, scoring 70.3 on Terminal Bench 2.0.

Bloomberg Technology

Gen-Z Gamer’s 3D-Model Startup Becomes China’s Latest AI Unicorn

Vast raised nearly $200 million and reached a $1 billion valuation; the RSS snippet says the 3D-modeling startup was founded by a 29-year-old gamer but does not disclose investors or product specifications.

Why it matters: A 29-year-old gamer's AI 3D-model startup raised nearly $200M at a $1B valuation, showing how hard capital is betting on AI-generated 3D content.

OpenAI News

OpenAI bans PRC-linked accounts using ChatGPT to generate comments on US tech policy and tariffs

OpenAI's June 2026 threat report details a banned cluster of ChatGPT accounts likely originating in China. The operators used Simplified Chinese prompts and VPNs to generate English comments and political cartoons criticizing US tariffs, rare earths, AI, and 5G policy. They instructed the model to depict only Trump, not Xi Jinping or China. The same cluster produced Chinese-language water army content attacking the US and Israel, amplifying anti-Jewish tropes, and harassing dissidents. OpenAI also linked these accounts to a separate X network that falsely claimed ChatGPT user data was compromised. The post does not disclose the exact number of banned accounts or the operators' specific institutional affiliation.

Why it matters: OpenAI's threat report says it banned accounts likely from China that used Simplified Chinese prompts to mass-produce English comments and cartoons aimed at US tariff and tech policy debates.

May 31Sunday

Xinzhiyuan · WeChat

Fudan-Linked Team Releases STI-WM Spatiotemporally Integrated World Model

MouShen Intelligence released STI-WM, a spatiotemporally integrated world-action model for robotics, claiming support for RGB, point-cloud, and proprioceptive inputs, hundred-second task planning, and disclosing five funding rounds in six months plus a RMB 300 million Pre-A round.

Why it matters: A Fudan-linked team's world action model fuses RGB, point clouds and robot proprioception, planning tasks over a hundred seconds long.

QbitAI · WeChat

Robot-Native World Action Model Debuts With Spatiotemporal Architecture From Fudan-Linked Team

Moushen Intelligence released STI-WM, a spatiotemporally integrated world action model for robotics, with RGB, depth point cloud, and proprioceptive inputs; the post says it supports hundred-second-scale long-horizon task rollout and closed-loop replanning, but does not disclose benchmark scores or deployment costs.

Why it matters: Muchen Intelligence released STI-WM, feeding RGB, depth point clouds and robot joint states into one spatiotemporal architecture, supporting long-task rollouts over a hundred seconds and replanning after errors.

May 30Saturday

AI HOT (Curated Pool)

AI Scammers Create Fake Black Personas to Sell Low-Quality Shein Goods

Sellers use AI-generated Black personas on TikTok, Facebook, and Instagram to pose as handmade creators and sell mass-produced dropshipped goods; the snippet cites one fake persona, “Aliyah,” selling fictional handmade belt buckles.

Why it matters: A Verge reporter found AI-generated fictional Black creator accounts on TikTok and other platforms, using a fake artisan persona to sell $9 Shein belt buckles for $40.

AI HOT (Curated Pool)

Nano Banana Pro and Nano Banana 2 officially released

Google AI Developers released Nano Banana Pro and Nano Banana 2, mapped to gemini-3-pro-image and gemini-3.1-flash-image. The post says both are production-ready through the Gemini API, but does not disclose pricing, benchmarks, or runtime limits.

Why it matters: Google put its gemini-3-pro-image and gemini-3.1-flash-image models on the Gemini API under the names Nano Banana Pro and Nano Banana 2, so developers can call them directly.

QbitAI · WeChat

Key Gemini IMO Gold Contributor Nearly Became a Professional Pianist

Yi Tay served as a modeling co-captain for Gemini Deep Think when it reached IMO gold-medal level, co-founded Reka AI in 2023, and returned to Google DeepMind after 639 days, while the article also notes his 2012 Trinity classical piano associate diploma.

Why it matters: Yi Tay was co-lead of modeling when Gemini Deep Think reached IMO gold-level results, co-founded Reka AI, and returned to Google DeepMind after 639 days.

Synced · WeChat

Apple Uses AI to Rework Image Compression: Same Visual Quality at One-Third the File Size

Apple’s team published PICO, a perceptual image codec that uses 57%-70% fewer bits than AV1, VVC, and JPEG AI at the same subjective visual quality, while encoding a 12MP photo in 230 ms and decoding it in 150 ms on an iPhone 17 Pro Max.

Why it matters: Apple's paper claims 57% to 70% bitrate savings over AV1, VVC and JPEG AI at similar subjective quality, with on-device timings on iPhone 17 Pro Max.

Synced · WeChat

NVIDIA and Tsinghua Team's Gamma-World Tops Hugging Face Daily Chart

NVIDIA, Tsinghua, University of Toronto, and Vector Institute released Gamma-World, a multi-agent world model using simplex-based positional encoding and hub tokens to cut interaction cost from quadratic to linear, with 8-player latency dropping from 17.6 ms to 4.5 ms.

Why it matters: NVIDIA and Tsinghua researchers propose the multi-agent world model γ-World, using simplex positional encoding and hub tokens to cut inter-agent interaction cost from quadratic to linear.

AI HOT (Curated Pool)

OpenAI launches real-time translation model with 70+ input languages

OpenAI launched gpt-realtime-translate, a speech translation model that accepts 70+ input languages and outputs speech in 13 target languages; the post says the feature is running on smart glasses.

Why it matters: OpenAI's gpt-realtime-translate takes speech input in over 70 languages and outputs 13, and it already runs on smart glasses, showing what a dedicated translation model trades off.

The Verge · AI

Tech companies desperately want to film you doing chores

Shift said it would clean New Yorkers’ homes for free if it can film cleaners doing chores such as washing dishes, wiping counters, dusting tables, and mopping floors, creating domestic robot training data; the snippet does not disclose consent terms, data retention, pricing, or expansion timing for cities such as London.

Why it matters: Shift trades free house cleaning for video of New York households washing dishes and mopping floors, footage meant to train home robots, with plans to expand to London and other cities.

May 29Friday

The Verge · AI

This AI startup will clean your home for free to train future robots

Shift offers free home cleaning and records cleaners scrubbing, vacuuming, dusting, tidying, and washing to collect robot training footage; the RSS snippet does not disclose service cities, privacy terms, consent mechanics, or dataset scale.

Why it matters: Shift trades free house cleaning for footage of the work, turning ordinary chores into a robot-training data pipeline whose data value covers the service cost.

The Verge · AI

Adobe’s Conversational AI Agent Is a Mediocre Design Intern

The Verge tested Adobe Firefly AI Assistant in beta. It can operate Adobe design apps as a conversational middleman, rather than only generating images or video. The post says it explains edit steps clearly, but the results were not impressive. The RSS snippet does not disclose pricing, release timing, or the full list of supported apps.

Why it matters: A Verge editor tries Adobe's Firefly AI assistant beta, which drives design software through conversation instead of generating images, and finds the output mediocre.

AI HOT (Curated Pool)

Xiaomi Open-Sources Controllable Video Foley Model ControlFoley

Xiaomi’s large model application team open-sourced ControlFoley, a controllable video Foley model supporting three tasks: text-guided video dubbing, text-controlled video dubbing, and reference-audio-controlled video dubbing, with code, model weights, and an online demo released.

Why it matters: Xiaomi open-sources ControlFoley, letting video sound effects be specified by text or reference audio instead of guessed by the model, while staying aligned to on-screen motion.

AI HOT (Curated Pool)

Google DeepMind CEO Demis Hassabis Says AGI Could Arrive Within Three Years

Demis Hassabis predicts AGI could arrive around 2029 to 2030, with mature multimodal capabilities and autonomous decision-making as key conditions, while warning that society remains underprepared and needs rules and safeguards before deployment.

Why it matters: Hassabis moves his AGI timeline up to 2029-2030 and calls independently working agents its preview, a notably faster estimate than before.