Skip to content

Multimodal

Beyond text: vision, mixed image-text, audio and video input and output in models and products.

Latest picks

381–400 of 592

May 15Friday

QbitAI · WeChat

Understand LeCun’s JEPA World Model in 160 Lines of Code

A developer released the keon/jepa teaching repository with five JEPA variants implemented as standalone PyTorch files, ranging from 160 to 278 lines, depending only on PyTorch and torchvision; the post reports iJEPA runs on CIFAR-10 for 100 epochs and reaches 52.7% linear-probe accuracy, while V-JEPA, C-JEPA, and LeWorldModel use toy or synthetic datasets.

Why it matters: The open teaching repo keon/jepa breaks LeCun's JEPA world model into five single-file PyTorch implementations, the shortest 160 lines, with iJEPA reaching 52.7% linear probe accuracy on CIFAR-10 after 100 epochs.

Xinzhiyuan · WeChat

Hassabis Praises Google DeepMind's AI-enabled Pointer Powered by Gemini

Google DeepMind released a Gemini-powered AI-enabled pointer and opened two demos in Google AI Studio: image editing and place finding on maps, while the post says Chrome pointer selection and a Googlebook Magic Pointer are planned product paths.

Why it matters: Google DeepMind put Gemini on the mouse pointer, so pointing at something lets the AI read the context and edit an image or find a place, with two demos live in AI Studio.

May 14Thursday

r/LocalLLaMA

Open-source one-prompt-to-cinematic-reel pipeline on one GPU with FLUX.2 and Wan2.2-I2V

The developer open-sourced StudioMI300, an 8-stage sequential pipeline that turns one English sentence into a 720p MP4 on a single AMD Instinct MI300X, cutting end-to-end time from 25.9 minutes to 10.4 minutes per clip.

Why it matters: StudioMI300 runs the whole 8-step path from one prompt to a 720p short film on a single MI300X, with a vision critic and auto-retry step holding output quality in check.

MIT Technology Review · AI

The Shock of Seeing Your Body Used in Deepfake Porn

MIT Technology Review documents Jennifer and other adult content creators whose bodies were used in NCII deepfakes, with examples spanning Jennifer’s circa-2013 video and the 2017 Reddit “deepfakes” uploads involving celebrity face swaps.

Why it matters: MIT Technology Review follows adult content creator Jennifer after she found an old video face-swapped, her body intact and her face someone else's, putting non-consensual synthetic porn on one person's actual experience.

Xinzhiyuan · WeChat

Anthropic Overtakes OpenAI in Enterprise AI Adoption After Three Years

Ramp says Anthropic reached 34.4% enterprise adoption, surpassing OpenAI at 32.3% for the first time; the index is based on credit-card and invoice spending from more than 50,000 companies.

Why it matters: Ramp's tally of card and invoice spending across more than 50,000 companies puts Anthropic at 34.4% of enterprise share and OpenAI at 32.3%, the first time OpenAI has lost this lead in three years.

QbitAI · WeChat

Alexandr Wang Responds to LeCun, Manus, and Meta AI Rebuild

Alexandr Wang said Meta rebuilt its pretraining, reinforcement learning, and data stacks in nine months, while Muse Spark remains closed because it triggered safety checks in areas including biosecurity, cyber capability, and loss of control.

Why it matters: Alexandr Wang answers the LeCun and Manus disputes in an interview and confirms Meta rewrote three systems, pretraining, reinforcement learning and data processing, in nine months.

Synced · WeChat

China in Focus: PsiBot Uses 100,000 Hours of Human Data for Embodied AI

PsiBot says it uses 100,000 hours of human operation data to train robot policies, with the W0 world model acting only as a training-time transfer module while deployment runs R2 alone.

Why it matters: Lingchu claims 100,000 hours of human operation data trained its robot policy, with the W0 world model used only in training; the data collection and task results can't be confirmed.

AI HOT (Curated Pool)

Best Practices for Computer and Browser Use with Claude

Anthropic published guidance for Claude computer and browser use, with Claude 4.6 API screenshots capped at a 1,568-pixel long edge and 1.15 million total pixels, while Opus 4.7 raises the limits to 2,576 pixels and 3.75 million total pixels.

Why it matters: Anthropic gives developers screenshot resolution limits for Claude controlling a computer and browser, comparing Claude 4.6 and Opus 4.7 settings on detail visibility versus inference cost.

AI HOT (Curated Pool)

Introducing Runway Agent

Runway launched Runway Agent, a video creation agent that turns one natural-language conversation into multi-scene videos with narration, dialogue, and music; new free-plan users receive 1,500 credits for their first video.

Why it matters: Runway launched an Agent that generates video with multiple shots, narration, dialogue and music from one conversation, aimed at brand ads, social content and indie shorts that are slow and costly to produce.

r/LocalLLaMA

sensenova/SenseNova-U1-A3B-MoT · Hugging Face

SenseNova published SenseNova-U1-A3B-MoT on Hugging Face; the post lists A3B MoT, 8B MoT, and 0.4B LoRA weight links, and says the NEO-unify architecture unifies multimodal understanding, reasoning, and generation in one model family.

Why it matters: SenseTime released the SenseNova-U1-A3B-MoT weights on Hugging Face, using the NEO-unify architecture to fold multimodal understanding, reasoning and generation into one model family, plus 0.4B LoRA weights.

May 13Wednesday

r/LocalLLaMA

AIDC-AI/Ovis2.6-80B-A3B on Hugging Face

AIDC-AI released Ovis2.6-80B-A3B, a multimodal MoE model with 80B total parameters and about 3B active parameters at inference, supporting a 64K-token context window and images up to 2880×2880 resolution.

Why it matters: Alibaba's DAMO Academy open-sourced Ovis2.6-80B-A3B, a multimodal MoE with 80B total parameters that activates about 3B at inference, with 64K context and 2880x2880 image input.

QbitAI · WeChat

ByteDance Proposes Generative Refinement Networks as a Third Route for Visual Generation

ByteDance’s commercial technology team proposed GRN, a visual generation architecture using HBQ, global refinement, and complexity-aware sampling to address quantization loss, error accumulation, and fixed-step inference; on a 130M model, adaptive sampling reduced inference from 50 steps to an average of 24, while gFID changed from 3.56 to 3.79.

Why it matters: ByteDance's GRN drafts a coarse image then refines it, with HBQ layered binary quantization cutting loss in discrete codes, a concrete third path beyond diffusion and autoregression.

Synced · WeChat

Lin Junyang Reportedly Starts New AI Lab Seeking $2 Billion Valuation

The Information says Lin Junyang is raising several hundred million dollars for a new AI Lab at a potential $2 billion post-financing valuation, while the lab’s research direction and final valuation remain undisclosed.

Why it matters: Lin Junyang, former tech lead of Alibaba's Qwen, is reported to be raising for a new AI lab at a possible $2 billion post-money valuation, showing the capital heat around top AI talent starting companies.

AI HOT (Curated Pool)

SenseNova-U1 Technical Report Released: Guide to Native Multimodal Model Building

SenseTime released the SenseNova-U1 technical report, covering six-stage training, RL post-training, and distillation; the open-source SenseNova-U1-A3B-MoT uses an MoE architecture and activates only 3 billion parameters.

Why it matters: SenseTime's SenseNova-U1 technical report lays out the six-step pipeline for a native multimodal model, from unified modeling to RL post-training and distillation, and open-sources a 3B-active-parameter MoE version.

Xinzhiyuan · WeChat

Tsinghua-affiliated team open-sources MiniCPM-V 4.6, a 1.3B model tunable on one RTX 4090

ModelBest, Tsinghua University, and OpenBMB open-sourced MiniCPM-V 4.6, a 1.3B multimodal model that supports full fine-tuning on one RTX 4090 and offers 4x/16x visual token compression for accuracy or speed trade-offs.

Why it matters: ModelBest, Tsinghua and OpenBMB open-source MiniCPM-V 4.6 at 1.3B parameters, small enough to fully fine-tune on one RTX 4090, with 4x and 16x visual token compression modes.

AI HOT (Curated Pool)

Step Image Edit 2 image model released with leading performance and efficiency

StepFun released the 3.5B-parameter Step Image Edit 2 model, which ranks first in KRIS-Bench overall, factual, and conceptual categories, and is now available on the Stepfun Open Platform.

Why it matters: StepFun released Step Image Edit 2 at 3.5 billion parameters, topping all three KRIS-Bench instruction-editing categories ahead of models five to six times larger.

May 12Tuesday

Synced · WeChat

ByteDance Open-Sources DreamLite for Offline Mobile Image Generation and Editing

ByteDance open-sourced DreamLite, a 0.39B-parameter unified diffusion model that generates or edits a 1024×1024 image on an iPhone 17 Pro in about 3 seconds, using 4-step DMD2 distillation and on-device offline inference without cloud dependency.

Why it matters: ByteDance open-sourced DreamLite, a 390M-parameter unified diffusion model that generates or edits a 1024×1024 image locally on an iPhone 17 Pro in about 3 seconds, using DMD2 distillation to cut inference to 4 steps.

Xinzhiyuan · WeChat

The Largest Single Industrial Product in History Is Entering Mass Production in China

AgiBot says it had shipped 10,000 general-purpose embodied robots by the end of March, and its humanoid robots worked eight continuous hours on a Nanchang 3C production line, completing 2,283 tasks with zero errors under formal line-cycle requirements.

Why it matters: Shows a humanoid robot's real production-line pace and reliability: 2,283 tasks over 8 hours with no errors.

Latent Space

Thinking Machines' Native Interaction Models: TML-Interaction-Small 276B-A12B Advances Realtime Voice

Thinking Machines released TML-Interaction-Small, a 276B-parameter MoE model with 12B active parameters, and the post says it advances realtime voice through 200ms time-aligned microturns, encoder-free early fusion for audio and images under 200ms, and benchmark wins over GPT-Realtime-2 and Gemini 3.1-Flash.

Why it matters: Thinking Machines released TML-Interaction-Small, a 276B-parameter MoE with 12B active, building realtime voice interaction into the model instead of bolting on separate speech recognition and synthesis modules.

QbitAI · WeChat

OpenClaw quietly updates with Peekaboo v3 for Mac computer use

OpenClaw-related Peekaboo v3 adds Mac agent capabilities for pixel-level screenshots, UI position reading, clicks, text input, hotkeys, scrolling, and drag-and-drop, with MCP server integration for Cursor, Claude Code, and Codex.

Why it matters: OpenClaw's Peekaboo v3 gives Macs pixel-level screenshots, UI targeting and simulated click input, and wires its MCP service into Cursor, Claude Code and Codex so models can drive desktop apps directly.