Skip to content

Multimodal

Beyond text: vision, mixed image-text, audio and video input and output in models and products.

Latest picks

61–80 of 592

Oct 13Tuesday

AI HOT · Models

Meta launches Muse Wonder and Muse Wonder Full, intelligence index up on the previous generation

Meta released Muse Wonder and Muse Wonder Full, which score 52 and 56 on the Artificial Analysis Intelligence Index v4.3 at maximum reasoning effort, above the 48 of the previous generation, Muse Spark 1.3.

Why it matters: Muse Wonder gains 4 index points over its predecessor, but per-task cost runs 2.8x the previous generation, so the added capability comes with a higher bill.

The Decoder

Meta launches Muse Wonder series, Full version reaches the top tier of frontier models

Meta released Muse Wonder and Muse Wonder Full, scoring 52 and 56 on the Artificial Analysis Intelligence Index v4.3, up from 48 for the previous Muse Spark 1.3.

Why it matters: Muse Wonder improves coding and agent tasks, but factual knowledge still trails rivals and cost per task is nearly triple the previous generation.

Oct 8Thursday

AI HOT · Models

Anthropic releases Claude Haiku 5.5, scoring 43 on Artificial Analysis

Anthropic released Claude Haiku 5.5, which scores 43 on the Artificial Analysis Intelligence Index, 26 points above the previous Haiku generation from a year earlier. Pricing is tiered by prompt length: past 100k tokens, both input and output rates rise 5x.

Why it matters: Haiku 5.5's tiered pricing means long-context prompts cost 5x more per token once they pass 100k.

TechCrunch · AI

ChatGPT adds Intelligent UI with interactive visuals inside conversations

OpenAI launched Intelligent UI for ChatGPT, adding clickable buttons, task-specific calculators, interactive charts and editable graphics alongside GPT-6. Demos cover wing lift diagrams, recipes, bicycle mechanics and hiking maps, and users can dial down how many visual elements appear in answers. The feature rolled out Wednesday to Pro, Plus, Business and Enterprise users worldwide, and Thursday to free and cheaper Go plan users.

Why it matters: OpenAI added interactive charts and task calculators to ChatGPT, letting explanations and everyday tasks lean on visuals users can actually operate.

OpenAI News

OpenAI brings GPT-6 and Intelligent UI to more ChatGPT users

OpenAI released GPT-6 to more ChatGPT users, adding Intelligent UI that combines text, graphics, buttons, forms and interactive tools inside answers. The model can answer first and keep thinking to add more, and in internal evals GPT-6 Instant started answering an average 44% earlier on questions needing web search, compared with GPT-5.6 Instant.

Why it matters: OpenAI folded interface generation into GPT-6's answer flow, so users adjust inputs, explore diagrams and use throwaway tools right in the chat.

Oct 7Wednesday

Ars Technica · AI

Google upgrades SynthID AI content detector, opens it to all via standalone site

Google upgraded its SynthID detector to read watermarks from all partners and opened it to everyone through SynthID.com starting today. Previously, users outside the trusted tester program could only check watermarks through Gemini, and the detector could not read SynthID watermarks in ChatGPT images. The new version removes that limit. SynthID embeds invisible signals in the pixels of AI-generated images and videos and in audio waveforms.

Why it matters: Google widened SynthID detection to all partners and opened a standalone site, so users no longer have to go through Gemini to check content watermarks.

Hacker News front page

OpenAI Decisions API opens public beta for text and image decisions

OpenAI's Decisions API is in public beta. It takes text, images or both and returns typed answers, and the docs put it at roughly 10x the speed of the Responses API. It offers conditional probabilities, fixed-choice selection and ordered rating scales. Only gpt-6-luna is supported for now, called via POST /v1/decisions, with general availability expected in the coming weeks.

Why it matters: OpenAI limits the decisions endpoint to conditional probabilities, fixed choices and ordered ratings, drawing a clear line between classification and ranking tasks and free-form structured generation.

Google DeepMind

Google DeepMind releases EmbeddingGemma 2 for on-device multimodal embeddings

Google DeepMind released EmbeddingGemma 2, built on the Gemma 4 architecture, which maps text, code, images, video and audio into one shared embedding space. The model has 740 million parameters, ships under the Apache 2.0 license, and supports an 8K token context window for fully offline cross-modal search and RAG. Text-only tasks need just 270M parameters, and output vectors can be cut from 768 dimensions to 128.

Why it matters: EmbeddingGemma 2 runs cross-modal retrieval on local devices, so text, image, audio and video search never has to send data to the cloud.

Oct 6Tuesday

The Decoder

Mistral Large 4 hits 1T parameters, takes on US models that refuse security tasks

Mistral opened a public preview of Mistral Large 4, a trillion-parameter multimodal model, with API access through Mistral Studio. Model weights are expected at the end of October.

Why it matters: Mistral Large 4 leads vulnerability reproduction and patching tests, though rivals refusing those tasks makes the comparison a mix of safety policy and model capability.

Mistral AI

Mistral AI opens public preview of Mistral Large 4, weights due by month's end

Mistral AI has put Mistral Large 4 (ML4), its native multimodal model, into public preview. It can be tried now through the Mistral Studio API, and the company plans to release the weights by the end of this month.

Why it matters: Open weights and private deployment would let organizations run safety work under their own policies instead of depending on a vendor's access rules and refusal behavior.

Oct 3Saturday

AI HOT · Models

Anthropic releases Claude Opus 5.5, first with an ECI score of 167

Anthropic released the closed-weight model Claude Opus 5.5 on September 22, 2026, with an ECI score of 167 that ranks first among 253 tracked models. It scores 77% on the software engineering benchmark MirrorCode and 83% on the multimodal Furniture Assembly Benchmark. Pricing is $4 per 1M input tokens and $20 per 1M output tokens.

Why it matters: Claude Opus 5.5 beats GPT-6 Astra on software engineering but trails it on the hardest math tests.

Oct 2Friday

Google Blog · Gemini

Google launches Gemini Live Guided Vision as an accessibility aid on Android

Google launched Guided Vision in Gemini Live, which uses real-time visual description and spoken framing prompts to help blind and low-vision users and anyone who needs visual assistance.

Why it matters: Google adds real-time visual description and spoken framing prompts to Gemini Live, letting blind and low-vision users get everyday visual information through conversation.

Oct 1Thursday

Google DeepMind

Google DeepMind releases Gemini 4 Argon frontier model

Google DeepMind released Gemini 4 Argon, its new frontier model. It goes first to trusted cyber-defense teams through the Fairwind Program, then widens to developers, enterprises and consumers. The release notes list an output cap, pricing and several benchmark scores.

Why it matters: Gemini 4 Argon opens first to trusted cyber defenders, with pricing of $2 per million input tokens and $10 per million output tokens, and an output cap raised from 64K to 1M.

Google Blog · Gemini

Google releases Gemini 4 Argon, first to trusted cyber defense teams

Google announced a new model, Gemini 4 Argon, rolling out gradually through the Fairwind Program to selected trusted cyber defense teams. It targets coding, enterprise knowledge work and cyber defense.

Why it matters: Google raised Gemini 4 Argon's output limit from 64K to 1M tokens, giving complex tasks more room for long single-pass reasoning.

Sep 29Tuesday

AI HOT (Curated Pool)

Anthropic launches Claude Sonnet 5.5, now the free-tier default on claude.ai

Claude Sonnet 5.5 beats Sonnet 5 on every benchmark, runs 30%+ faster, and costs up to 30% less for most work. The big move: it's now the free-tier default on claude.ai, which Simon Willison tested and got a solid WebGL 3D pelican on a bicycle. The 'max' thinking effort still hits the same bug as Opus 5.5—128K tokens of thought with no output, costing $1.28. 'xhigh' delivered a decent SVG in 41 seconds for 5.74 cents. Anthropic says Haiku 5.5 is coming in weeks; Simon hopes it's price-competitive with GPT-6 Luna.

Why it matters: Anthropic made Sonnet 5.5 the default model on the claude.ai free tier. Pricing matches Sonnet 5, but Anthropic says it is over 30% faster and cuts cost by up to 30% on most work.

AI HOT (Curated Pool)

Anthropic releases Claude Sonnet 5.5, over 30% faster than Sonnet 5

Anthropic launched Claude Sonnet 5.5, claiming over 30% speed gains and clearer writing for fast-turnaround tasks like bug fixes, docs, and slide decks. Opus 5.5 targets complex judgment work, and Haiku 5.5 is coming in a few weeks. The post doesn't disclose pricing or latency numbers.

Why it matters: Claude Sonnet 5.5 runs 30%+ faster than the previous generation, and Anthropic spells out how it splits work with Opus 5.5 between fast daily tasks and complex judgment calls.

Sep 28Monday

AI HOT (Curated Pool)

Human contractors are reviewing Microsoft Copilot user prompts and uploaded images

404 Media obtained internal documents showing Microsoft hires at least hundreds of contractors to review Copilot users' prompts and uploaded images. They are not filtering for safety—they judge output quality, such as whether AI-enlarged breasts are big enough. Reviewers are flooded with sexual requests: shortening skirts, foot fetish images of children's cartoon characters, pro-anorexia content. Uploaded faces are never blurred, and many prompts are dubiously consensual. One contractor said it's hard to take the work seriously when the focus is which model generated the right bust size.

Why it matters: 404 Media reports, based on internal documents, that Microsoft contract reviewers look at Copilot users' prompts and uploaded images to grade generated image quality, laying out how user data reaches human review.

Sep 25Friday

Google DeepMind

Google DeepMind releases Gemini 3.8 Live with Live Avatar

Google DeepMind released Gemini 3.8 Live with Live Avatar, adding near-real-time video generation to its native real-time conversation model. The result is a dynamic visual avatar with lip sync, natural expressions and smooth turn-taking.

Why it matters: Gemini 3.8 Live adds near-real-time video generation to a live conversation model, with lip sync, switching across 97 languages and asynchronous tool calls mid-dialogue, showing what enterprise multimodal interaction looks like now.

Sep 24Thursday

Google Blog · Gemini

Google launches Gemini 3.8 Live with Live Avatar in Gemini Enterprise

Google launched Gemini 3.8 Live with Live Avatar, pairing near-real-time video generation with native live conversation, available now in Gemini Enterprise. It handles visual and audio input at once, calls tools asynchronously during an ongoing conversation, and syncs lip and facial movement while switching across 97 languages. Enterprises can use preset avatars or customize one from a reference image, though custom creation is limited to an enterprise allowlist. Generated audio and video carry SynthID watermarks.

Why it matters: Google combines live voice conversation with a dynamic avatar and background tool calls, so enterprise support and guided flows don't have to interrupt the conversation to fetch data.

The Verge · AI

Meta unveils Muse Charm, a standalone AI gadget that looks like a strapless smartwatch

Meta teased the Muse Charm at the end of Connect, a dedicated hardware device for its Muse AI agent. It resembles a chunky strapless smartwatch with a lanyard. A fingerprint sensor on the top right activates voice input; the front has at least three mic holes and a small camera. Zuckerberg noted you don't need to unlock a phone to use it. The post doesn't disclose pricing, battery life, or a release date.

Why it matters: Meta closed its Connect event with Muse Charm, a standalone AI device where a fingerprint press starts a conversation with the Muse agent without unlocking a phone.