Skip to content

Multimodal

Beyond text: vision, mixed image-text, audio and video input and output in models and products.

Latest picks

121–140 of 592

Aug 15Saturday

AI HOT (Curated Pool)

MOSS-VL: An open VLM family that treats real-time interaction as a first-class capability

Fudan's MOSS-VL makes real-time interaction—perceiving while speaking—a first-class capability. Gated cross-attention keeps visual tokens outside the decoded sequence, giving it a 2.8× to 5.1× time-to-first-token advantage over same-backbone Qwen3-VL-8B. MOSS-VL-Realtime tops three of four streaming benchmarks, hitting 66.0 vs. 37.5 on OmniMMI Proactive Alerting. The offline variant leads temporal-reasoning video sets at comparable scale. All five checkpoints, the training curriculum, and inference code are open.

Why it matters: Fudan's open-source MOSS-VL treats real-time interaction as an architecture goal, with gated cross-attention handling visual information outside the decode sequence, cutting first-token latency 2.8 to 5.1 times versus Qwen3-VL-8B on the same base.

Computing Life · Share · Yage

Give DeepSeek V4 a 3B vision front-end — it works today

DeepSeek V4's API rejects image input, but Liquid AI's new LFM2.5-VL-3B can serve as a perception layer. The author tested it zero-shot on garage camera data — 94.3% accuracy for door open/close, 1.5s locally. Perception runs on-device, images never leave, only structured text hits the cloud for reasoning. The latency, privacy, and cost advantages of this split architecture hold even if DeepSeek adds native vision later. One gap: extracting a reliable confidence signal from natural-language output — the post doesn't detail a calibration method.

Why it matters: A 3B LFM2.5-VL-3B runs as a vision front end for DeepSeek V4, which takes no images, judging a garage door zero-shot at 94.3% accuracy in 1.5 seconds locally, with images never leaving the device.

Aug 14Friday

AI HOT (Curated Pool)

Qwen releases Qwen3.8 series: a 27B dense multimodal model and open weights for a 2.4T-A95B Max variant

Qwen delivered on its open-source promise with the Qwen3.8 series. Qwen3.8-27B is a natively multimodal dense model that beats Qwen3.7-Plus at only 27B parameters, supports 262K context natively and up to 1M tokens via YaRN, under Apache 2.0. Open weights for the Max-tier Qwen3.8-2.4T-A95B are also available. The post doesn't cover training data, inference cost, or release timeline details.

Why it matters: Qwen3.8 ships open source, with a 27B dense multimodal model beating the previous Qwen3.7-Plus and a 2.4T-parameter MoE released under Apache 2.0.

Aug 13Thursday

AI Chat-Group Daily (群聊日报)

Closed-source reasoning chains extracted at scale; Coze CLI hijacks AI tools

The big one today: researchers extracted hidden reasoning chains from Anthropic, OpenAI, and Google models at scale. The trick is absurdly simple—take Opus 4.8's encrypted CoT and feed it to Haiku 4.5, which decodes it verbatim. All three API families were broken, and decoding 10K trajectories costs about $720. A separate paper shows you can even reverse-engineer reasoning from public outputs alone using a 1.5B-param model. Separately, Coze CLI was caught silently scanning local Codex and Claude Code directories and injecting its own skills into workflows. On the engineering side, the group discussed how prompt debt now rivals traditional code debt—old rules pile up, evals lag behind model iterations, and nobody dares delete anything.

Why it matters: A cheap same-vendor model decodes another model's encrypted reasoning, and the trick works across Anthropic, OpenAI and Google APIs.

Aug 12Wednesday

Google DeepMind

Google DeepMind releases SL2T sign language-to-text model, first in Pixel 11 Gboard and Live Transcribe

Google DeepMind released SL2T, a multilingual sign language-to-text model, bringing sign language AI into consumer products for the first time. On Pixel 11, Gboard and Live Transcribe support American Sign Language (ASL) to English dictation, with more devices and languages to follow.

Why it matters: Google DeepMind put its multilingual sign language to text model SL2T into Gboard and Live Transcribe on Pixel 11, supporting American Sign Language dictation in English.

AI HOT (Curated Pool)

Google Gemini app hits 1B monthly users, matching ChatGPT's June milestone

Sundar Pichai announced on X that the standalone Gemini app surpassed 1 billion monthly active users—Google's 14th product to hit that mark. The figure excludes AI Mode in Search and other channels. ChatGPT reached 1B MAU in June; Gemini is now keeping pace. Usage stats: 63% of users have tried the voice feature, the app generates over 150 million images daily, and iOS has more than 100 million active users. The post doesn't disclose paid user share or revenue.

Why it matters: Google's CEO said the standalone Gemini app passed 1 billion monthly users, making it Google's 14th product at that scale; the figure counts only the standalone app and matches where ChatGPT stood in June.

Aug 11Tuesday

New York Times Chinese

Meta releases open-weight Muse Glimmer, a free version of its paid Muse Spark model

Meta released Muse Glimmer on Monday, an open-weight AI model nearly identical to its paid, closed-source Muse Spark launched in July—capable of generating code, text, and images. Mark Zuckerberg also published a 14-page essay arguing superintelligence should not be concentrated in a few companies, and announced a $1 billion fund for communities hosting its data centers. Muse Glimmer is open-weight, not fully open-source; the underlying code isn't fully public. Meta also teased a more powerful model codenamed Watermelon but didn't disclose whether it will be open or closed.

Why it matters: Meta puts nearly the same capabilities from last month's paid closed model Muse Spark into open-weight Muse Glimmer, downloadable and self-deployable, though the training code stays closed.

Aug 10Monday

AI HOT (Curated Pool)

SGLang adds Day-0 inference support for Meta's local agent model Muse Glimmer

Meta released Muse Glimmer, a 30B multimodal model built for local agentic workflows. SGLang ships Day-0 support with dedicated optimizations: on a single RTX 5090 with NVFP4 quantization and DFlash speculative decoding, per-user decode hits 236 tok/s and total throughput reaches 1,452 tok/s. The model uses a hybrid of sliding-window and full-sequence attention with a 128k+ context window. Apple Silicon is supported via the MLX backend, though speculative decoding isn't available there yet.

Why it matters: SGLang shipped support the day Meta released Muse Glimmer and tuned inference for local agent workflows, hitting 1,452 tokens per second on one RTX 5090.

Hacker News front page

Meta open-sources Muse Glimmer, a 30B agentic model that runs locally on a single GPU

Meta released Muse Glimmer weights under Apache 2.0. It's a 30B model built for always-on local agent workflows, small enough to run on a Mac or PC with a single consumer GPU. 4-bit quantization shrinks it below 20 GB, leaving room for KV cache and the vision encoder within a 24 GB or 32 GB envelope. Training used logit distillation from a larger Muse Spark teacher, followed by mid-training on long-context agent data and post-training with SFT, on-policy distillation, and RL. Meta's benchmarks show it outperforming Gemma4-31B and Qwen3.6-27B on agentic, coding, multimodal, and safety evals. The post doesn't disclose specific latency numbers, only that inference optimizations were applied to keep it responsive.

Why it matters: Meta open-sourced a 30B model under Apache 2.0 that fits under 20 GB at 4-bit, so a single consumer GPU can host a local agent workflow.

Aug 6Thursday

AI HOT (Curated Pool)

Alibaba Cloud launches Qwen-Image-3.0 with high-res image generation starting at $0.03

Qwen-Image-3.0 is pitched as production-ready: 4.5K-token prompts, 100%+ text accuracy with no broken logos, and native support for 12 languages. High-res generation starts at $0.03. The post only provides a headline and links—no model architecture, inference speed, or benchmarks are disclosed, so I'd hold off on the accuracy claim until third-party tests appear.

Why it matters: Alibaba Cloud released Qwen-Image-3.0 for production use, with 4,500-token prompts, 12 languages and high-resolution generation starting at $0.03 per image on pay-as-you-go.

Product Hunt · AI

Mistral launches Shieldstral: a 3B open-weight multimodal guardrail that defines safety policies in natural language at inference

Mistral AI launched Shieldstral on Product Hunt this week—a 3B open-weight multimodal guardrail. You define safety policies in natural language at inference time, and it evaluates text, images, or both with a single token output. It runs locally on one 16GB GPU. The Product Hunt page shows the basics and screenshots, but doesn't disclose latency, accuracy benchmarks, or comparisons with other guardrails like Llama Guard. Treat it as a lightweight, self-hostable compliance component for now; real-world performance needs community benchmarks.

Why it matters: Mistral open-sourced Shieldstral, a 3B-parameter multimodal safety guardrail that takes safety rules in natural language and runs locally on a 16GB consumer GPU.

AI HOT (Curated Pool)

Simon Willison one-shots a full 3D Raccoon Heist game with Claude Fable 5

Simon Willison fed a 2022 tweet and two concept images to Claude Fable 5 and let it build a playable browser 3D game with zero further input. The model chose Three.js, called OpenAI's gpt-image-2 for textures, and added mechanics like a patrol dog with scent tracking. The whole project was done on mobile, deployed via GitHub Pages. The gameplay is basic, but the zero-intervention workflow is the real story.

Why it matters: Given only an old tweet and two concept images, Claude Fable 5 picked Three.js on its own and called an image model for textures, finishing and deploying the whole thing from a phone.

Aug 5Wednesday

Hacker News front page

Qwen Image 3.0 Pro targets production use with 4.5k-token layouts, 10px text, and realistic detail

Qwen Image 3.0 Pro handles up to 4,500 input tokens and generates dense layouts—newspapers, storyboards, menus—in one pass. It reliably renders text down to 10px across 12 languages and 20+ fonts, and reproduces micro-expressions, pores, and hair strands at near-photographic quality. Output pricing is $0.04 per 1K image and $0.075 per 2K image, but the rate limit is just 1 request per minute, so high-throughput use cases are off the table for now.

Why it matters: Qwen released Image 3.0 Pro, which renders dense layouts like newspapers and menus and keeps text legible down to 10 pixels, at $4 per thousand 1K images.

Hacker News front page

Mistral releases Shieldstral: a 3B open-weights model for multimodal moderation

Mistral introduced Shieldstral, a 3B-parameter open-weights model built to moderate both text and image content. It can run locally or on edge devices, checking user inputs and model outputs for policy violations. The post doesn't disclose benchmark scores, latency figures, or pricing—only that it's positioned as a safety filter. I'd wait for third-party evals, but the 3B size is genuinely lightweight for self-hosted moderation.

Why it matters: Mistral released Shieldstral, a 3B open-weights model that checks text and images for policy violations and runs locally or on edge devices, a lightweight option for self-hosted moderation.

Aug 4Tuesday

AI HOT (Curated Pool)

SenseTime open-sources SenseNova U1: unified reasoning and image generation in one model

SenseTime open-sourced SenseNova U1, a model that handles reasoning and image generation in a single pipeline. It can turn a prompt into a structured slide deck or generate step-by-step illustrated content, like a six-step dragon drawing tutorial. Available on HuggingFace, GitHub, and SenseNova Studio. The post doesn't disclose parameter count, training data, or benchmarks.

Why it matters: SenseNova U1 is open source, with code and weights on HuggingFace, GitHub and SenseNova Studio, and it puts logical reasoning and image generation in one pipeline.

AI Chat-Group Daily (群聊日报)

Qwen 3.8 Max matches Fable 5 at 2.4T params, open weights next week

Qwen 3.8 Max launched with Terminal Bench 2.1 score 86.6 and PaperBench 93.0, beating Fable 5's 88.8. A 500-yuan token plan burned out in one day; the model lands between Luna and Terra, with price as the main draw. Open weights for both Qwen 3.8 Max and Qwen 3.8-27B drop next week. DS V4 Flash hit 8T tokens consumed in a single day, topping weekly charts—the group sees tokens becoming a commodity. On tools: an M5Stick voice dongle turns a keychain into an agent remote, LoopX keeps agent state across 200+ hours, and reverse-skill injects reverse-engineering toolchain knowledge into coding agents. The wildest methodology story: an agent autonomously downloaded a local Qwen mid-translation task and auto-installed Whisper when its API key ran out of funds.

Why it matters: Qwen 3.8 Max launched at 2.4 trillion parameters, scored 93.0 on PaperBench against Fable 5's 88.8, and weights open next week.

Latent Space

Alibaba Qwen drops Qwen3.8-Max and 27B, open weights coming next week

Alibaba Qwen announced Qwen3.8-Max, a 2.4T-parameter model, and Qwen3.8-27B, both promised as open weights. Max claims 10+ days of autonomous coding, a 125-hour self-directed research loop beating the original paper by 2.71 points, and a 4.16x return in a 365-day e-commerce sim. API pricing is $2/M input, $6/M output. I'd hold the champagne: the post doesn't include standard academic benchmarks, and the exact open-weight date and license aren't specified.

Why it matters: Alibaba released Qwen3.8-Max at 2.4 trillion parameters, aimed at coding and long autonomous tasks, with official figures of over ten days of continuous coding and a 125-hour research run.

AI HOT (Curated Pool)

EU AI Act transparency rules kick in, with fines up to €15M for non-compliance

As of today, tech companies in the EU must label AI-generated content and disclose when users interact with a machine. Fines reach €15 million or 3% of global annual turnover. The rules cover chatbots, deepfakes, and AI-generated audio, video, and text. The EU also published official label icons companies can use instead of designing their own.

Why it matters: EU AI transparency rules now apply, so any company shipping AI in Europe must label generated content and disclose bots, with fines up to €15M or 3% of global revenue.

Aug 3Monday

Computing Life · Share · Yage

Google Earth pulled its AI generation feature in one day—interface trust travels farther than watermarks

Google added an AI generation button to Google Earth on July 30, 2026, letting users create synthetic images on real satellite basemaps with Nano Banana 2, then pulled it within a day. The core issue: screenshots shared on social media lost AI watermarks and metadata, but Google Earth's 20-year reputation as a 'window on reality' traveled with them. OSINT analyst Henk van Ess generated fake craters, flooded landmarks, and destroyed sites, noting the fakes inherited the map's credibility. The article contrasts Wikipedia banning AI edits, Snapchat embracing AI lenses, and LinkedIn removing AI writing while adding a slop-report button, arguing that platform attitudes hinge on the interface promise made to users—factual archive vs. playground. Provenance tech like SynthID and C2PA degrades across platforms; labels alone barely shift user belief; outright bans and combo strategies (labeling + demonetization + downranking) are the main responses.

Why it matters: Google pulled an AI image-generation button from Google Earth within a day, a case that maps the gap between what a product interface promises and how AI content is governed.

Aug 1Saturday

Hacker News front page

Explorative Modeling: A Third Pretraining Axis That Also Enables End-to-End Generation

Alexi Gladstone introduces Explorative Modeling (XM): generate K candidates per step, train only on the best. This adds a third pretraining axis beyond data and parameters. More exploration monotonically improves image, video, and language models, with gains growing at scale—7%→36% with more data, 13%→23% with more parameters. XM achieves 6.2× sample efficiency, 4.1× FLOP efficiency, and 47% better parameter efficiency. As an end-to-end generator, XM matches diffusion on control tasks using up to 256× less inference compute. The post does not disclose specific model names or training costs.

Why it matters: Alexi Gladstone's explorative modeling generates K candidates per step and trains only on the best, adding a scaling axis beyond data and parameters.