Skip to content

#多模态

4 today

Apr 29Wednesday

QbitAI · WeChat

Avenir-Web Open-Sources Web Agent Harness With 53.7% on ONLINE-MIND2WEB

UCL, Princeton, and Edinburgh open-sourced Avenir-Web, reaching 53.7% success on ONLINE-MIND2WEB. The training-free harness uses EIP, MoGE, checklists, and adaptive memory across 136 sites and 300 live tasks. The key signal: with Gemini 3 Pro, it beats Claude Computer Use 3.7 at 47.3%.

Why it matters: Avenir-Web from UCL, Princeton and Edinburgh plugs into models without extra training and hits 53.7% success on ONLINE-MIND2WEB, 136 sites and 300 real tasks.

QbitAI · WeChat

DeepSeek’s multimodal AI has entered testing

DeepSeek researchers confirmed V4 vision mode is in gray testing, with an image-recognition mode on the homepage. A screenshot shows it identified drinks and cup types in a non-text-heavy image after 4 seconds. The post does not disclose rollout scope, API access, or pricing.

Why it matters: DeepSeek researchers confirm the V4 vision version is in gray-scale testing, with an image recognition entry on the homepage and a screenshot showing drink and cup types identified in 4 seconds.

QbitAI · WeChat

ShengShu Technology Claims MotuBrain, a Dual-Benchmark Robot Brain for Long-Horizon Tasks

ShengShu Technology claimed MotuBrain on April 29 after it topped WorldArena and RoboTwin2.0 in mid-April. It scored 95.8 and 96.1 in RoboTwin2.0 Clean and Randomized settings, and a demo used 3 humanoid robots across 5 tasks. The key detail is its World Action Model: a video-action-language MoT design for cross-embodiment tasks beyond 10 atomic actions.

Why it matters: Shengshu Technology claims MotuBrain, which topped WorldArena and RoboTwin2.0, and shows three robots handing off a long-horizon task, a concrete look at a video company moving into embodied AI.

r/LocalLLaMA

SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-Unify Architecture

SenseNova released SenseNova-U1 with 4 MoT multimodal models. The post lists 8B and A3B variants with GitHub and HuggingFace weight links. The key claim is a monolithic architecture instead of adapters; benchmarks are not disclosed.

Why it matters: SenseNova-U1 handles text and images inside the model with no external image adapter, and releases 8B and A3B MoT weights, with A3B smaller and suited to local runs.

Bloomberg Technology

Apple Readies Photo-Editing Overhaul With New AI Tools in iOS 27

Apple plans to overhaul built-in photo editing for iPhone, iPad, and Mac in iOS 27 with AI tools. The RSS snippet says it targets Android competition; the post does not disclose features, models, timing, or supported devices.

Why it matters: Bloomberg reports Apple's planned AI photo tools for iOS 27 and macOS 27, with expand, enhance and recompose pointing to where system-level editing is headed.

NVIDIA Blog

NVIDIA Launches Nemotron 3 Nano Omni for Vision, Audio, and Language Agents

NVIDIA launched Nemotron 3 Nano Omni, claiming up to 9x higher throughput at the same interactivity. It uses a 30B-A3B hybrid MoE with Conv3D, EVS, and 256K context, taking text, images, audio, video, documents, charts, and GUIs as input. Open weights, datasets, and training methods arrive April 28, 2026 on Hugging Face, OpenRouter, build.nvidia.com, and 25+ platforms.

Why it matters: NVIDIA open-sourced Nemotron 3 Nano Omni, one model handling image, text, audio and video with 3B active parameters and 256K context, giving teams building multimodal agents an open path.

Apr 28Tuesday

QbitAI · WeChat

ModelBest Releases MiniCPM-o 4.5 Technical Report for Consumer-GPU Deployment

ModelBest, OpenBMB, Tsinghua THUNLP and THUMAI released the MiniCPM-o 4.5 technical report, covering a roughly 9B-parameter model. It supports video, audio and text streams; a 12GB RTX 5070 runs full-duplex mode at RTF 0.4. The key mechanism is Omni-Flow: a unified timeline with time-division multiplexing, without external VAD.

Why it matters: ModelBest with Tsinghua and others published the MiniCPM-o 4.5 technical report, a roughly 9B-parameter model using Omni-Flow to time-share audio, video and text on one timeline, running full-duplex on a 12GB consumer GPU.

QbitAI · WeChat

Open-source SenseNova-U1 unifies image understanding and generation

SenseTime open-sourced two SenseNova-U1 models: an 8B version and a 38B-total MoE version using NEO-unify. The architecture removes VE and VAE, processes pixels directly, and generates 2048×2048 images in about 9 seconds on one H100/H200 node. The key item is interleaved text-image reasoning; 32K context, long-text rendering, and beta interleaved creation remain limits.

Why it matters: SenseTime open-sourced two SenseNova-U1 models using a NEO-unify architecture that drops the vision encoder and VAE to work on pixels directly, generating a 2048x2048 image in about 9 seconds on one H100 or H200.

Synced · WeChat

Open-source medical video understanding system uAI-NEXUS-MedVLM released

United Imaging Intelligence released uAI-NEXUS-MedVLM for medical video understanding, with a CVPR 2026 paper. MedVidBench has 532k video-instruction pairs across 8 medical sources and 8 tasks. Qwen2.5-VL-7B SFT reached 89.4% CVS accuracy; GPT-5.4 scored 16.4%.

Why it matters: United Imaging Intelligence open-sourced uAI-NEXUS-MedVLM, a large model for ultrasound, endoscopy and other medical video, plus MedVidBench, an evaluation set of 532,000 video instruction pairs.

Xinzhiyuan · WeChat

NUS and NTU Release Pask with Streaming Intent Detection and Persistent Memory

NUS and NTU released Pask, with paper arXiv:2604.08000. Pask uses DD, MM, and PAS, with IntentFlow detecting intent in 1.5 seconds. The key bet is real-time intent detection, not longer execution chains.

Why it matters: Pask, from NUS and NTU, uses its IntentFlow component to detect user intent in real time within 1.5 seconds and adds permanent memory, offering proactive AI a path based on real-time intent recognition.

r/LocalLLaMA

Microsoft Presents TRELLIS.2: Open-Source 4B Image-to-3D Model

Microsoft’s title says TRELLIS.2 is an open-source 4B image-to-3D model. The title lists 1536³ PBR assets, native 3D VAEs, and 16× spatial compression; the Reddit body is blocked by 403 and discloses no license or benchmarks.

Why it matters: Microsoft open-sourced TRELLIS.2, a 4B-parameter image-to-3D model that outputs assets up to 1536³ with PBR materials, worth a look for anyone tracking open 3D generation.

Apr 27Monday

The Verge · AI

Canva apologizes after its AI tool replaces ‘Palestine’ in designs

Canva said Magic Layers replaced “Palestine” with “Ukraine” in designs. The tool should split flat images into editable layers, not alter visible content; X user @ros_ie9 said “Gaza” was unaffected. Canva says it fixed the issue; the post does not disclose the trigger mechanism.

Why it matters: Canva's Magic Layers swapped the word Palestine for Ukraine in design text; Canva confirmed a fix but did not explain what triggered it, a concrete case of how AI editing tools handle political terms.

Xinzhiyuan · WeChat

Five Months After Altman’s Code Red, GPT Image 2 Tops Arena Image Rankings

GPT Image 2 topped three Arena image charts within 12 hours, scoring 1512 in text-to-image and beating Nano Banana 2 by 241 points. Arena calls it the largest Image Arena gap, with 93% blind-test wins and a 316-point text-rendering gain. The key shift is native thinking: planning, self-checking, web search, and 8 coherent images per run.

Why it matters: Within half a day of launch, GPT Image 2 took three first places on the Arena image leaderboard, scoring 1512 in text-to-image, 241 points above second, with a 93 percent blind-test win rate.

QbitAI · WeChat

Meshy tops 10M users and moves into 3D printing as ARR rises 14x

Meshy says it passed 10M registered users, reached $40M ARR, and grew 2025 revenue 14x year over year. Meshy Creative Lab supports keychain, magnet, and keycap design; physical ordering is not live yet. The key signal is print fit: 97% slice-pass rate in Bambu Studio across 75 tested models.

Why it matters: Meshy passed 10 million registered users and $40 million ARR, and its new Creative Lab generates models for small objects like keychains and fridge magnets, showing AI 3D content moving into physical consumer goods.

Synced · WeChat

Apple paper asks: What do your logits know?

Apple researchers posted an arXiv paper testing whether VLM top-k logits leak image details. Using CLEVR, MSCOCO, and probes, 30–80 logits recover noise, target traits, and some background attributes. The key risk is gray-box APIs exposing top-k log probabilities.

Why it matters: Apple's experiments on CLEVR and MSCOCO found that the top 30 to 80 logits alone can recover image noise, object features and some background attributes, giving quantifiable evidence of output-interface risk in vision-language models.

Synced · WeChat

From 99 Lines of Frozen Code to Meshy AI’s 3D Momentum in the West

Meshy AI released Meshy 6 and claims over 60% share in developed Western markets. The post says it has 10M+ users, $40M+ ARR, and 100M+ AI-generated 3D models in three years. The key signal is workflow fit: 37Games reports 30–40% less base sculpting work.

Why it matters: Meshy 6 ships with the company claiming over 60% of the US and European 3D AI market and a 30% to 40% cut in base sculpting work at 37 Interactive.

Apr 25Saturday

Hacker News front page

Google Flow Music

Google Flow Music launched a web creation entry with six sections: songs, playlists, Spaces, videos, projects, and Turntable. The page says Producer creates full songs with Lyria 3, and AI music videos use Veo. Pricing, regions, model specs, and rights terms are not disclosed.

Why it matters: Google launched the web version of Flow Music, putting Lyria 3 song generation and Veo music videos behind one entry point, with the conversational Producer tool making full tracks with vocals.

TechCrunch · AI

ComfyUI hits $500M valuation as creators seek more control over AI-generated media

ComfyUI raised $30 million at a $500 million valuation. The RSS snippet says its tools give creators more control over AI image, video, and audio generation; the post does not disclose investors, round stage, pricing, or release timing. The real signal is workflow control, not another model vendor.

Why it matters: ComfyUI raised $30 million at a $500 million valuation, betting that creators want to wire up node-based image and video workflows themselves for finer control over the output.

Hacker News front page

TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment

Google DeepMind released TIPSv2 with 3 pretraining changes for CVPR 2026. iBOT++ applies self-distillation to masked and visible patches, adding 14.1 mIoU on ADE150; Head-only EMA cuts training parameters by 42%. The key signal is visible-token supervision, not a larger teacher model.

Why it matters: Google DeepMind's TIPSv2 adds self-distillation losses on both masked and visible patches, gaining 14.1 mIoU in zero-shot segmentation and cutting 42% of training parameters with Head-only EMA.

The Verge · AI

How Project Maven taught the military to love AI

In the first 24 hours of the assault on Iran, the US military struck more than 1,000 targets, with targeting accelerated by AI systems including Maven Smart System. The snippet says this was nearly 2x the scale of Iraq's “shock and awe” attack over 20 years ago, and Katrina Manson's new book traces Project Maven from its 2017 start in computer vision for drone footage; the post does not disclose model details, later contractors, or current deployment scope.

Why it matters: Katrina Manson's book traces Maven from 2017 drone-video analysis to Maven Smart System, whose Iran strike support hit over 1,000 targets in 24 hours, about twice Shock and Awe.

Apr 24Friday

Synced · WeChat

Remember more, answer faster, use less: HERMES speeds real-time streaming video understanding by 10x

Fudan University, Shanghai Academy of AI for Science, and NUS proposed HERMES, a training-free framework that turns KV cache into hierarchical memory for streaming video understanding and cuts TTFT by up to 10x. The post lists three mechanisms: hierarchical cache management, cross-layer memory smoothing, and position re-indexing; it reports 68% fewer video tokens with comparable or better results, and Qwen2.5-VL-7B on StreamingBench rising from 73.31% to 79.44%. What matters for practitioners: it answers without external retrieval, with TTFT around 27/29/28 ms at 16/64/256 frames.

Why it matters: Fudan University and other teams propose HERMES, a training-free framework that organizes the KV cache into three memory layers to speed up streaming video understanding, cutting first-token latency by up to 10x.

Synced · WeChat

After robots beat humans in marathon times: hardware nears its limit, intelligence becomes the second half

Honor's humanoid robot Lightning ran 50:26 at the 2026 Beijing Yizhuang half marathon, faster than the men's human world record of 57:20; the post also says Unitree H1 did a 1.9 km winding course in 4:13. The post cites nearly 200 embodied-AI financings and over RMB 30 billion in Q1 2026, plus Spirit AI's $455 million Pre-A on April 16. The real signal is capital shifting from robot hardware to model-centric 'brains.'

Why it matters: Honor's humanoid ran a half marathon in 50:26, faster than the human men's world record, while embodied AI drew nearly 200 funding deals worth over 30 billion yuan in Q1.

Xinzhiyuan · WeChat

Google's Vision Banana aims to unify vision tasks with a single pixel-generation interface

Google DeepMind and collaborators including Kaiming He introduced Vision Banana, claiming one pixel-generation interface can cover detection, segmentation, generation, and editing. The RSS snippet gives two head-to-head numbers versus Nano Banana Pro: 53.5% human win rate on GenAI-Bench and 47.8% on ImgEdit; it says only a small amount of reversible-format task data was mixed in, while data scale and full benchmark tables are not disclosed in the post.

Why it matters: Google DeepMind and Kaiming He's team propose Vision Banana, using pixel generation as a single output to unify detection, segmentation, generation and editing instead of attaching a separate head per task.

Hacker News front page

GPT-5.5: Mythos-Like Hacking, Open to All

XBOW says GPT-5.5 cut miss rate to 10% on its real-vulnerability benchmark, versus 40% for GPT-5 and 18% for Opus 4.6. It scored 97.5% on visual acuity and used about half the login iterations of the next-best model. The key point is black-box testing: GPT-5.5 without source beat GPT-5 with source.

Why it matters: Security firm XBOW tested GPT-5.5 against a real vulnerability set, and its black-box miss rate fell to 10%, below GPT-5 white-box at 40% and Opus 4.6 at 18%.

Apr 23Thursday

QbitAI · WeChat

Qwen3.6-27B open-weights, beats its 397B flagship predecessor on agentic coding

Qwen released Qwen3.6-27B and says it beats Qwen3.5-397B on 4 agentic coding benchmarks with about 1/15 the parameters. The post cites SkillsBench rising from 30.0 to 48.2, GPQA Diamond at 87.8, and AIME26 at 94.1; it uses a dense architecture, Thinking Preservation, and Gated DeltaNet, with weights on Hugging Face and ModelScope.

Why it matters: Qwen released Qwen3.6-27B, a 27B dense model it says beats its own 397B previous flagship on four agentic coding benchmarks.

Xinzhiyuan · WeChat

Tashi Zhihang raises $455.0 million in a Pre-A round, with Sequoia China and Hillhouse jointly leading

Tashi Zhihang said on April 16 it closed a $455.0 million Pre-A round led by Sequoia China, Hillhouse Ventures, and Meituan, which the post says set China records for embodied AI single-round and Pre-A financing. The post also says its AWE3.0 four-modal model lifted unseen-view task success by 3x and cut execution jitter by about 45%, and that its A1 robot set a Guinness record in sub-millimeter wire-harness assembly within one hour. What matters is whether model, data, and deployment keep reproducing; the post does not disclose valuation or deal terms.

Why it matters: Tashi Zhihang's $455m Pre-A, led by Sequoia China, Hillhouse and Meituan, sets a domestic record for a single embodied-AI round and comes with AWE3.0 and A1 deployment figures.

Hacker News front page

Website streamed live directly from a model

Flipbook generates an entire clickable website in real time with an image model, where each page is a pixel image and every click spawns a deeper image. The post says all on-screen text is drawn by the image model with no HTML or text overlays, and content comes from agentic web search plus model knowledge. The key point is the interaction model, not a standard generative UI; the live video stream remains an experimental, resource-heavy toggle.

Why it matters: Flipbook turns a webpage into a live-generated pixel image, where clicking anywhere generates a deeper new image and text is drawn by the image model itself.

Apr 22Wednesday

Hacker News front page

Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

Qwen released the open-weight 27B dense model Qwen3.6-27B and made it available in Qwen Studio. It scores 77.2 on SWE-bench Verified vs. 76.2 for Qwen3.5-397B-A17B, and 59.3 on Terminal-Bench 2.0 under a 256K context and 3-hour timeout. The real takeaway is deployment: this is not a larger MoE, but a denser 27B model with stronger coding results.

Why it matters: Qwen open-sourced Qwen3.6-27B, a 27B dense coding model that scores 77.2 on SWE-bench Verified, beating the previous 397B-parameter MoE flagship, making the dense-versus-MoE comparison the draw.

QbitAI · WeChat

SenseAuto's Sage with 3B active params claims to beat GPT-5.4 and Opus 4.6 in cars

SenseAuto released Sage, an in-car multimodal edge model with 32B total params and 3B active params, and says it scored 94% on PinchBench, above Claude Opus 4.6 at 93.3% and GPT-5.4 at 90.5%. The post says Sage runs on Nvidia OrinX with about 0.5s TTFT, 0.03s TPOT, and 80 tok/s throughput; its SCOUT training method cuts GPU hours by about 60%, and ERL raises complex-task completion by 20%. The key point is not the headline race but whether a 3B-active model can sustain multi-step tool use on device.

Why it matters: SenseAuto's Sage is an on-car multimodal model with 32B total parameters that activates 3B per pass, scoring 94% on PinchBench ahead of Claude Opus 4.6 and GPT-5.4, with OrinX latency and throughput numbers.

Latent Space

OpenAI launches GPT-Image-2

OpenAI shipped GPT-Image-2 in ChatGPT, Codex, and the API. It has thinking and non-thinking variants, with stronger text, layout, editing, multilingual output, and QR codes. Arena ranks it first on 3 Image Arena boards, with 1512 Elo in text-to-image and a +242 lead.

Why it matters: OpenAI put GPT-Image-2 into ChatGPT, Codex and the API, with better text rendering, layout editing and multilingual output for slides, infographics and UI mockups.

X · @dotey

OpenAI launches ChatGPT Images 2.0, available to all ChatGPT and Codex users starting today

OpenAI made ChatGPT Images 2.0 available today to all ChatGPT and Codex users, and also opened the gpt-image-2 API. The RSS snippet says it supports up to 2K output, aspect ratios from 3:1 to 1:3, and more reliable non-English text rendering. In thinking mode, it can search the web, generate multiple styles, and self-check outputs; that tier is limited to Plus, Pro, and Business, with Enterprise not yet available.

Why it matters: ChatGPT image generation reaches 2.0 with free-tier access, steadier complex layouts and multilingual text, up to 2K output and aspect ratios from 3:1 to 1:3.

X · @OpenAI

Introducing ChatGPT Images 2.0

OpenAI introduced ChatGPT Images 2.0 as an image model for complex visual tasks and directly usable visuals. The RSS snippet cites sharper editing, richer layouts, and “thinking-level intelligence,” but the post does not disclose model size, pricing, latency, or rollout scope.

Why it matters: OpenAI's ChatGPT Images 2.0 announcement claims finer edits, richer layouts and thinking-level intelligence, but gives no model size, pricing or rollout details.

Bloomberg Technology

OpenAI unveils new image model that is better at charts and diagrams

OpenAI released an update to its image generation software to produce more accurate, complex charts and scientific diagrams. The RSS snippet does not disclose the model name, launch timing, pricing, benchmarks, or technical method. The real signal is a push into professional use cases, not generic image quality.

Why it matters: OpenAI's image model update targets accuracy on complex charts and scientific diagrams rather than general image quality.

The Verge · AI

OpenAI’s updated image generator can now pull information from the web

OpenAI said ChatGPT Images 2.0 can pull information from the web when a thinking model is selected, helping generate multiple images from one prompt. It runs on GPT Image 2 and is available to ChatGPT Plus, Pro, Business, and Enterprise users; the post does not disclose rollout timing, usage limits, or pricing changes. The key shift is web-grounded multi-image generation, not just image quality.

Why it matters: ChatGPT Images 2.0 can now search the web before generating a set of images, available to Plus, Pro, Business and Enterprise users.

TechCrunch · AI

Report says Clarifai deleted 3 million photos OkCupid provided to train facial recognition AI

Clarifai deleted 3 million photos from OkCupid after an FTC settlement, and the images had been used to train facial recognition AI. The RSS snippet says the data sharing request dates to 2014 and OkCupid executives had invested in Clarifai. The post does not disclose the settlement terms, deletion verification, or model impact.

Why it matters: Clarifai's FTC settlement forced it to delete 3 million OkCupid user photos and the facial recognition model trained on them, data OkCupid handed over in 2014 while an executive was investing in Clarifai.

Apr 21Tuesday

Ben's Bites

That's My Designer - Claude

Anthropic added a Design tab to Claude that asks 5-10 interactive questions, then builds wireframes or high-fidelity prototypes. The post says image-to-design works well; in research preview it has separate limits, and the $20 plan appears to allow only 2-3 large generations per week. The sharper point is usability: the author says Claude Cowork depends on connectors and plugins that average users may not find.

Why it matters: Anthropic added a design tab to Claude that turns 5 to 10 rounds of questions into wireframes or high-fidelity prototypes, with usage limits disclosed for the research preview.

Synced · WeChat

Monet: Enabling multimodal LLMs to reason in latent visual space

Monet trains Qwen2.5-VL-7B into Monet-7B to reason with continuous latent visual embeddings instead of external tools; the work is accepted by CVPR 2026 and releases paper, code, model, and a 125K SFT dataset. The method uses three-stage SFT plus VLPO reinforcement learning; the post reports 3% to 9.75% gains on in-distribution tasks and 2.31% on out-of-distribution abstract visual reasoning versus the base model. The key detail is the VLPO mechanism and dataset construction; the post does not disclose one unified table of absolute headline scores.

Why it matters: Monet adapted Qwen2.5-VL-7B to do abstract reasoning in a continuous visual latent space, training in two stages of supervised fine-tuning and VLPO reinforcement learning, gaining 3% to 9.75% on in-distribution tasks.

OpenAI News

Introducing ChatGPT Images 2.0

OpenAI introduced ChatGPT Images 2.0 as a new image generation model, highlighting better text rendering, multilingual support, and visual reasoning. The RSS snippet names only these three upgrades; the post does not disclose architecture, resolution, pricing, latency, or availability. What matters is whether text fidelity and multilingual consistency improve in real use; for now, only headline-level details are disclosed.

Why it matters: OpenAI released ChatGPT Images 2.0, claiming gains in text accuracy, multilingual layout and visual reasoning on complex prompts, with sample posters, comics and magazine pages.

Latent Space

Moonshot Kimi K2.6 open-weight model refresh aims to catch Opus 4.6

Moonshot released Kimi K2.6, a 1T-parameter MoE with 32B active and 256K context. The post cites 58.6 on SWE-Bench Pro, 4,000+ tool calls, 12+ hour runs, and 300 parallel sub-agents. The key signal is long-horizon agent execution, not only open-model scores.

Why it matters: Moonshot AI released Kimi K2.6, an open model with 1 trillion total parameters built for long multi-step agent tasks, with the company claiming runs of over 12 hours and 300 sub-agents managed in parallel.

Hacker News front page

Expansion Artifacts

Matt Ström-Awn argues that flaws in LLM outputs are “expansion artifacts,” not compression artifacts, and cites 2024 evidence that they can be tracked. He notes Stanford researchers estimated AI-drafted text in 17.5% of recent CS papers and 16.9% of peer reviews from post-ChatGPT word-frequency shifts, and contrasts this with a JPG after 10,000 recompressions reaching PSNR 14.59. The point for practitioners is forensic: these artifacts expose both model aesthetics and generation provenance.

Why it matters: Matt Ström-Awn groups LLM clichés, six-fingered hands and clipping video as expansion artifacts, filling in high-frequency patterns at decompression time, a different explanation from compression distortion.