Skip to content

Multimodal

Beyond text: vision, mixed image-text, audio and video input and output in models and products.

Latest picks

481–500 of 592

Apr 21Tuesday

Ben's Bites

That's My Designer - Claude

Anthropic added a Design tab to Claude that asks 5-10 interactive questions, then builds wireframes or high-fidelity prototypes. The post says image-to-design works well; in research preview it has separate limits, and the $20 plan appears to allow only 2-3 large generations per week. The sharper point is usability: the author says Claude Cowork depends on connectors and plugins that average users may not find.

Why it matters: Anthropic added a design tab to Claude that turns 5 to 10 rounds of questions into wireframes or high-fidelity prototypes, with usage limits disclosed for the research preview.

Synced · WeChat

Monet: Enabling multimodal LLMs to reason in latent visual space

Monet trains Qwen2.5-VL-7B into Monet-7B to reason with continuous latent visual embeddings instead of external tools; the work is accepted by CVPR 2026 and releases paper, code, model, and a 125K SFT dataset. The method uses three-stage SFT plus VLPO reinforcement learning; the post reports 3% to 9.75% gains on in-distribution tasks and 2.31% on out-of-distribution abstract visual reasoning versus the base model. The key detail is the VLPO mechanism and dataset construction; the post does not disclose one unified table of absolute headline scores.

Why it matters: Monet adapted Qwen2.5-VL-7B to do abstract reasoning in a continuous visual latent space, training in two stages of supervised fine-tuning and VLPO reinforcement learning, gaining 3% to 9.75% on in-distribution tasks.

OpenAI News

Introducing ChatGPT Images 2.0

OpenAI introduced ChatGPT Images 2.0 as a new image generation model, highlighting better text rendering, multilingual support, and visual reasoning. The RSS snippet names only these three upgrades; the post does not disclose architecture, resolution, pricing, latency, or availability. What matters is whether text fidelity and multilingual consistency improve in real use; for now, only headline-level details are disclosed.

Why it matters: OpenAI released ChatGPT Images 2.0, claiming gains in text accuracy, multilingual layout and visual reasoning on complex prompts, with sample posters, comics and magazine pages.

Latent Space

Moonshot Kimi K2.6 open-weight model refresh aims to catch Opus 4.6

Moonshot released Kimi K2.6, a 1T-parameter MoE with 32B active and 256K context. The post cites 58.6 on SWE-Bench Pro, 4,000+ tool calls, 12+ hour runs, and 300 parallel sub-agents. The key signal is long-horizon agent execution, not only open-model scores.

Why it matters: Moonshot AI released Kimi K2.6, an open model with 1 trillion total parameters built for long multi-step agent tasks, with the company claiming runs of over 12 hours and 300 sub-agents managed in parallel.

Hacker News front page

Expansion Artifacts

Matt Ström-Awn argues that flaws in LLM outputs are “expansion artifacts,” not compression artifacts, and cites 2024 evidence that they can be tracked. He notes Stanford researchers estimated AI-drafted text in 17.5% of recent CS papers and 16.9% of peer reviews from post-ChatGPT word-frequency shifts, and contrasts this with a JPG after 10,000 recompressions reaching PSNR 14.59. The point for practitioners is forensic: these artifacts expose both model aesthetics and generation provenance.

Why it matters: Matt Ström-Awn groups LLM clichés, six-fingered hands and clipping video as expansion artifacts, filling in high-frequency patterns at decompression time, a different explanation from compression distortion.

Latent Space

Training Transformers to Address the 95% Failure Rate in Cancer Trials — Noetik

Noetik uses TARIO-2 to predict tumor spatial transcriptomics, targeting a 95% cancer-trial failure rate. GSK signed a $50M technology deal, and TARIO-2 predicts a ~19,000-gene spatial map from routine H&E assays. The key issue is patient-tumor-treatment matching, not the claim that AI cures cancer.

Why it matters: Noetik trained an autoregressive Transformer to predict spatial expression of about 19,000 genes from routine H&E slides, tracing cancer trials' high failure rate to patient, tumor and drug pairing.

Apr 20Monday

r/LocalLLaMA

TRELLIS.2 image-to-3D now runs on Mac (Apple Silicon) with no NVIDIA GPU required

A developer ported Microsoft's TRELLIS.2 to Apple Silicon and reports generating ~400K-vertex meshes from one photo in about 3.5 minutes on an M4 Pro with 24GB. The port replaces five CUDA-only extensions with PyTorch MPS and custom backends; texture baking takes about 18 seconds, removing the NVIDIA and cloud requirement.

Why it matters: A developer ported Microsoft TRELLIS.2 to Apple Silicon, swapping five CUDA extensions for PyTorch MPS, generating a roughly 400,000-vertex mesh in about three and a half minutes on a 24GB M4 Pro.

QbitAI · WeChat

Sudo, valued above $2 billion, unveils embodied model Sudo R1 with zero real-robot data and ~98% first-try grasp success

Sudo unveiled embodied model Sudo R1 and says it achieved about 98% first-try grasp success in 200+ zero-shot tests with zero real-robot training data, nearing 100% within two attempts. The post says the 60-minute run covered 100+ unseen objects, including transparent, metallic, soft, and reflective items, using integrated world-model and reinforcement-learning training on a high-fidelity simulator. It also says Sudo is valued above $2 billion and is working with CATL, but the post does not disclose round size, benchmark protocol, or third-party validation.

Why it matters: Sudo Technology released the embodied model Sudo R1, which without real-robot training data passed over 200 zero-shot grasping tests on 100-plus unseen objects at about 98% first-try success.

Synced · WeChat

In the first year of “deployment mode,” AgiBot expanded its rollout plans to seven solutions

AgiBot said at its April 17 Shanghai event that it released 4 robots, 6 AI models, and 7 standardized deployment solutions, and framed 2026 as the first year of embodied AI “deployment mode.” The post cites concrete metrics: Expedition A3 runs 8-10 hours, WITA Omni 1.0 targets sub-500ms interaction latency, and BFM was trained on 100 million-plus frames and 700 hours of motion-capture data; it also claims 5,100-plus shipments and 39% share in 2025, with the 10,000th robot rolling off in March 2026. The real point for practitioners is repeatable delivery rather than launch volume: the post lists 7 scenarios from 3C line loading to patrol, but independent validation details are not disclosed.

Why it matters: Zhiyuan unveiled four robots, six AI models and seven standardized deployment packages in Shanghai, with concrete battery, interaction-latency and training-data figures, framing 2026 as the first year of deployment.

Hacker News front page

Show HN: TRELLIS.2 image-to-3D running on Apple Silicon, no Nvidia GPU needed

Developer shivampkumar ported Microsoft's 4B-parameter TRELLIS.2 to Apple Silicon with PyTorch MPS for single-image 3D generation. He replaced flash_attn, nvdiffrast, and custom sparse conv kernels with pure PyTorch sparse 3D conv, SDPA attention, and Python mesh extraction. On an M4 Pro with 24GB, it generates ~400K-vertex meshes in about 3.5 minutes; slower than H100 seconds, but fully offline.

Why it matters: A developer ported Microsoft's 4B-parameter TRELLIS.2 to Apple silicon via PyTorch's MPS backend, giving Mac users without an Nvidia GPU local image-to-3D.

Apr 19Sunday

QbitAI · WeChat

Did Musk Really Sell Lao Gan Ma on Douyin?

QbitAI says the shown “Musk selling Lao Gan Ma on Douyin” and “GTA-6 crossover” images were generated by OpenAI GPT Image 2; the claimed 100K+ live viewers were part of fake visuals. The post argues Image 2 can render realistic posters, game screenshots, and readable long text, and links that to Codex-style UI workflows; the post does not disclose pricing, rollout scope, or launch timing. The real issue is verification: image realism is eroding “photo as evidence.”

Why it matters: Fake images of Musk livestreaming on Douyin show GPT Image 2 can render readable text and viewer counts convincingly.

QbitAI · WeChat

Amap unveiled ABot, its first full-stack embodied AI stack for AGI, and claimed 15 SOTA results

Amap unveiled embodied AI stack ABot and claimed SOTA on 15 metrics. The post says ABot-3DGS builds 10k-scale 3D scenes from centimeter-level map data, while ABot-PhysWorld uses a 14B DiT and 3M real manipulation videos. What matters is the interactive world model and VLA loop; the post does not disclose the 15 benchmarks, exact metrics, or the open-source timeline and scope.

Why it matters: Amap published its full-stack embodied intelligence system ABot, whose ABot-PhysWorld trains a 14B-parameter diffusion model on 3 million real robot-arm manipulation videos so robots simulate an action before executing it.

Apr 18Saturday

Synced · WeChat

Claude Design enters research preview for generating mockups, prototypes, and slides

Anthropic launched Claude Design in research preview for Claude Pro, Max, Team, and Enterprise users, covering mockups, prototypes, slides, and one-pagers. Powered by Claude Opus 4.7, it can ingest codebases, images, DOCX, PPTX, XLSX, and web captures, then export to Canva, PDF, PPTX, and HTML; the headline cites Figma and Adobe stock drops, but the post does not disclose the moves. The real signal is the workflow link from design system ingestion to handoff into Claude Code.

Why it matters: Anthropic launched a Claude Design research preview on paid Claude accounts that reads codebases, images and several document types, and exports to Canva, PDF, PPTX and HTML.

Bloomberg Technology

OpenAI’s Former Product Chief and Sora Head Leave Company

OpenAI is losing two leaders: its former product chief and the head of Sora; the title confirms the count is two. The post does not disclose timing, reasons, successors, or names; the key watchpoint is whether the Sora org changes as well.

Why it matters: OpenAI's former product chief and its Sora head left together, with no reason or succession plan disclosed, leaving the Sora team's next step as the open thread.

X · @dotey

Anthropic launches Claude Design, a conversational design generation product

Anthropic released Claude Design in research preview and is rolling it out to Pro, Max, Team, and Enterprise subscribers. Powered by Claude Opus 4.7, it can start from text, images, docs, or web clips, then iterate via chat, comments, direct edits, and sliders. On first use it reads a team's codebase and design files to build a design system; outputs export to Canva, PDF, PPTX, or standalone HTML, with one-click handoff to Claude Code.

Why it matters: Anthropic's Claude Design turns conversation into design files, and on first use reads the codebase and design files to build a full system of brand colors, fonts and components.

Apr 17Friday

Hacker News front page

Introducing Claude Design by Anthropic Labs

Anthropic launched Claude Design on April 17, 2026, in research preview for Claude Pro, Max, Team, and Enterprise subscribers. Powered by Claude Opus 4.7, it generates designs from text, images, DOCX, PPTX, XLSX, and codebases, and exports to Canva, PDF, PPTX, or HTML. The key detail is its one-step handoff bundle to Claude Code; the post does not disclose standalone pricing beyond existing plan limits.

Why it matters: Anthropic opened a research preview of Claude Design, where Pro, Max, Team and Enterprise users turn text, images, documents or a codebase into designs and prototypes.

X · @claudeai

Introducing Claude Design by Anthropic Labs: make prototypes, slides, and one-pagers by talking to Claude

Anthropic Labs launched Claude Design in research preview for Pro, Max, Team, and Enterprise plans, letting users create prototypes, slides, and one-pagers by talking to Claude. The post says it runs on Claude Opus 4.7, Anthropic’s most capable vision model; the post does not disclose pricing, output constraints, or a detailed rollout schedule. The thing to watch is the interactive design workflow, not just another writing surface.

Why it matters: Anthropic Labs released Claude Design as a research preview for Pro, Max, Team and Enterprise users, generating prototypes, slides and one-pagers from conversation and running on Opus 4.7.

Xinzhiyuan · WeChat

AgiBot says robots have entered the deployment phase with 8-hour continuous factory work

At APC 2026 on April 17, AgiBot defined 2026 as year one of the “deployment phase” and said its robots had run for 8 hours on a real production line. The clearest case in the post is Genie G2 at Longcheer’s Nanchang factory: 2,283 loading tasks, over 99.5% success, and 18-20 seconds per cycle; these figures are company disclosures, and the post does not disclose independent audit results. The real signal is scale and line integration: AgiBot said it shipped over 5,100 units in 2025 and reached 10,000 cumulative units by March 2026, while Longcheer plans nearly 1,000 deployments.

Why it matters: Zhiyuan published figures for Genie G2 at Longqi's Nanchang plant: 8 hours of continuous operation, 2283 feeding cycles and a success rate above 99.5%, all self-reported.

MIT Technology Review · AI

How robots learn: A brief, contemporary history

Companies and investors put $6.1 billion into humanoid robots in 2025, 4x 2024, and MIT Technology Review attributes the surge to a shift in how robots learn. The piece highlights two mechanisms: around 2015, simulation plus reward signals enabled millions of trial-and-error runs; after ChatGPT in 2022, robotics models took images, sensors, and joint states to predict dozens of motor commands per second. The key change is data-driven learning over hand-written rules; the provided text is truncated, so later examples are not fully disclosed.

Why it matters: Traces how humanoid robot learning moved from hand-written rules to simulation trial and error, against $6.1 billion of 2025 investment, four times the 2024 figure.

X · @dotey

Seedance 2.0 API is now available on Volcano Engine and BytePlus

Volcano Engine has released the Seedance 2.0 API for enterprises, individual developers, and overseas users via BytePlus; China pricing is RMB 46 per million tokens, or about RMB 1 per second for pure video generation. The post says it supports text, image, audio, and video inputs, plus face verification, portrait authorization, and 10,000+ preset avatars for workflow automation; overseas pricing is not disclosed here. The part to watch is orchestration: the post cites up to 10x efficiency gains, but does not disclose a common benchmark or model specs.

Why it matters: Volcano Engine opened Seedance 2.0's video API at 46 yuan per million tokens, with overseas access through BytePlus and built-in likeness licensing.