Skip to content

Multimodal

Beyond text: vision, mixed image-text, audio and video input and output in models and products.

Latest picks

481–500 of 514

Dec 3, 2025Wednesday

Mistral AI

Mistral releases the Mistral 3 family, including 675B-parameter Mistral Large 3

Mistral AI released the Mistral 3 family: three dense models at 14B, 8B and 3B, plus Mistral Large 3, which uses a sparse MoE architecture with 41B active and 675B total parameters. All are open-sourced under Apache 2.0.

Why it matters: Mistral 3 ships an Apache 2.0 family from 3B to 675B in one release, a useful read on where open weights now stand for on-device and frontier capability.

Oct 21, 2025Tuesday

Hugging Face Blog

Unlock the power of images with AI Sheets

Hugging Face added vision support to its open-source AI Sheets, letting users analyze images, extract data, generate visuals, and edit images inside a spreadsheet. The post says AI Sheets uses Inference Providers to access thousands of open models, and manual edits plus thumbs-up feedback become few-shot examples; outputs can be exported as CSV or Parquet. What matters is the unified data workflow, not a standalone demo.

Why it matters: Direct-source Hugging Face product update with concrete mechanics: AI Sheets now handles OCR, image understanding, generation, and editing in one spreadsheet flow, and corrections become few-shot examples. HKR-H and HKR-K pass; HKR-R is weaker because the impact is workflow-level

Sep 30, 2025Tuesday

OpenAI News

Sora 2 System Card

OpenAI published the Sora 2 System Card on September 30, 2025, and said the video-audio generation model will launch first via limited invites on sora.com and a standalone iOS app. The post confirms no video uploads and no image uploads with photorealistic people at launch; API timing, pricing, and benchmark scores are not disclosed.

Why it matters: This lands in the 78–84 band. HKR-H comes from the Sora 2 + iOS app hook; HKR-K from concrete launch limits and safety rules; HKR-R from competition and likeness-abuse nerves. It stays below P1 because price, eval scores, context details, and API timing are not disclosed.

OpenAI News

Sora 2 is here

OpenAI released Sora 2 on September 30, 2025 and launched a social iOS app called Sora built on the model. The post says it generates video with synced dialogue and sound effects, and its “characters” feature uses a one-time video and audio recording to verify identity and insert a real person’s likeness; pricing, generation limits, and rollout regions are not disclosed. The key shift is from model demo to a consumer app with a feed, teen limits, and parental controls.

Why it matters: This is a same-day write: OpenAI shipped a flagship video/audio model and attached it to a standalone app, so HKR-H/K/R all clear. The post gives real product facts like synced dialogue and sound effects, but missing price, duration caps, and rollout details keeps it below 90.

Sep 29, 2025Monday

OpenAI News

Introducing parental controls

OpenAI launched parental controls for all ChatGPT users on September 29, 2025, letting parents link with teen accounts and manage usage settings from their own account. Linked teen accounts get stronger content safeguards by default, and parents can set quiet hours, disable voice, memory, image generation, and opt out of model training. The key mechanism is the alert flow: suspected self-harm signals trigger human review, and acute distress leads to email, SMS, and push notifications to parents.

Why it matters: OpenAI rolled parental controls to all ChatGPT users and disclosed a concrete self-harm escalation flow: system detection, human review, then email/SMS/push alerts to parents. HKR-K and HKR-R are strong; this is a substantive safety product update, but not a model-level launch,so

OpenAI News

Combating online child sexual exploitation & abuse

OpenAI said on September 29, 2025 it bans any sexualized content involving people under 18, and reports accounts that generate or upload CSAM/CSEM to NCMEC. The post names hash matching, Thorn’s CSAM classifier, and OpenAI models for monitoring text, image, audio, video, and uploads; the key signal is that OpenAI says it has observed users uploading abusive material and asking for detailed descriptions.

Why it matters: HKR-K and HKR-R pass: OpenAI discloses a concrete moderation stack across uploads and admits observed abuse patterns. HKR-H is weak because the title is a direct safety policy note, so this fits the 72–77 featured band.

Aug 6, 2025Wednesday

OpenAI News

Providing ChatGPT to the Entire U.S. Federal Workforce

OpenAI partnered with the U.S. General Services Administration to offer ChatGPT Enterprise to the full federal executive workforce for $1 per agency for 1 year. Participating agencies also get 60 days of unlimited advanced models and features, including Deep Research and Advanced Voice Mode; federal business data will not be used for training. The key signal is centralized procurement access, while the post does not disclose agency count, budget size, or exact model list.

Why it matters: OpenAI’s GSA deal turns ChatGPT Enterprise into a federal-wide procurement channel at $1 for one year, which is a real distribution signal, not a routine discount. HKR-H/K/R all pass, but the post omits agency count, budget size, and full model scope, so I keep it at 84.

Jul 17, 2025Thursday

Mistral AI

Mistral adds Deep Research, voice mode and more to Le Chat

Mistral rolled out a batch of new Le Chat features: a preview Deep Research mode, a voice mode powered by the new Voxtral speech model, a multilingual thinking mode backed by the Magistral reasoning model, Projects for organizing conversations, and advanced image editing built with Black Forest Labs.

Why it matters: Mistral announced five Le Chat features at once, so readers can see how its research, voice and image-editing abilities fit together.

Jun 1, 2025Sunday

OpenAI News

OpenAI bans China-origin accounts using ChatGPT to generate US polarization content

OpenAI banned a set of China-origin ChatGPT accounts, dubbed 'Uncle Spam,' after a tip from Meta. The accounts used models to generate pro- and anti-tariff posts, create fake US veteran profile images, and write code to scrape user data from X and Bluesky. The content pushed both sides of divisive topics but got almost no real engagement—most posts had zero likes or reposts. OpenAI rates the impact as Category 2 on the Brookings Breakout Scale: multi-platform activity with no breakout.

Why it matters: Official OpenAI disclosure with a codename and behavioral specifics, not a generic threat report. Hits all three HKR axes, but it's a safety incident notice rather than a product/model update, so it lands in the 78-84 'worth recommending' band.

May 8, 2025Thursday

OpenAI News

Introducing data residency in Asia

OpenAI launched data residency on May 7, 2025 in Japan, India, Singapore, and South Korea for ChatGPT Enterprise, ChatGPT Edu, and the API Platform. Eligible API customers must create a new Project and pick a country; new Enterprise and Edu workspaces can store customer content at rest in-region, including chats, uploads, and text, vision, and image data. The key limit: the post only states at-rest storage and does not disclose whether inference stays fully local.

May 7, 2025Wednesday

Mistral AI

Mistral AI releases Mistral Medium 3, targeting low cost and enterprise deployment

Mistral AI released Mistral Medium 3, which it says reaches or exceeds 90% of Claude Sonnet 3.7 across benchmarks. Pricing is $0.4 per million input tokens and $2 per million output tokens.

Why it matters: Mistral Medium 3 benchmarks against Claude Sonnet 3.7 at lower cost, and the post gives a path to private enterprise deployment and customization.

Apr 23, 2025Wednesday

OpenAI News

Introducing our latest image generation model in the API

OpenAI added gpt-image-1 to the Images API on April 23, 2025, after ChatGPT image generation reached 130 million users and 700 million images in its first week. Pricing is token-based: $5 per 1M text input tokens, $10 per 1M image input tokens, and $40 per 1M image output tokens, or about $0.02, $0.07, and $0.19 per square image by quality. The part to watch is operational: it keeps 4o image safety guardrails, adds C2PA metadata, and does not train on customer API data by default.

Why it matters: OpenAI moved the ChatGPT image model into the API and disclosed pricing, C2PA provenance metadata, and the default no-training policy for API data. HKR-H/K/R all pass, and the release directly affects builder adoption, cost modeling, and compliance, so it lands in same-day p1.

Apr 16, 2025Wednesday

OpenAI News

Introducing OpenAI o3 and o4-mini

OpenAI released o3 and o4-mini on April 16, 2025, and said its reasoning models can now use ChatGPT tools together, including web search, Python, files, and images. The post says o3 makes 20% fewer major errors than o1 in expert evals, while o4-mini reaches 99.5% pass@1 and 100% consensus@8 on AIME 2025 with Python. The real shift is RL-trained tool use, not just two new model names.

Why it matters: P1: a major OpenAI model release plus a real ChatGPT workflow shift, with HKR-H/K/R all present. The story includes concrete claims (-20% major errors vs o1; 99.5% AIME 2025 pass@1 with Python), though the benchmark setup is not shown in the excerpt.

OpenAI News

Thinking with images

OpenAI said on April 16, 2025 that o3 and o4-mini can process user images inside their internal reasoning chain, with native crop, zoom, and rotation actions. The post shows o3 taking 20 seconds to read upside-down handwriting and 1m44s to solve a maze and draw a path; it claims strong multimodal benchmark results, but the provided body does not disclose the scores. The key point is that image manipulation is folded into the same reasoning stack, not handed off to a separate vision model.

Why it matters: OpenAI confirms a meaningful capability step: o3 and o4-mini manipulate images inside the same reasoning process, so HKR-H/K/R all pass. I kept it below p1 because the provided text gives demo timings, but not the benchmark scores or rollout scope.

Mar 25, 2025Tuesday

OpenAI News

Introducing 4o Image Generation

OpenAI integrated 4o image generation into GPT-4o on March 25, 2025, focusing on native multimodal generation, accurate text rendering, and multi-turn image editing in chat. The post points to joint training on image-text distributions and shows a “transformer → diffusion → pixels” pipeline; examples are labeled best of 1, best of ~8, or best of 8. The real signal is consistency and editability, while pricing, API details, and quotas are not disclosed.

Why it matters: This is a major ChatGPT capability update: native image generation lands inside GPT-4o with explicit claims on text rendering and multi-turn editing. HKR-H/K/R all pass; price, API details, and quotas are not disclosed, so it stays below the top of the band.

OpenAI News

Addendum to GPT-4o System Card: 4o image generation

OpenAI published a GPT-4o system card addendum on March 25, 2025, covering 4o image generation capabilities and marginal risks. The post confirms native GPT-4o integration, photorealistic output, image-to-image edits, and reliable text rendering; specific eval scores and mitigations are not disclosed in the post.

Why it matters: This official OpenAI addendum sits near the major-product-update band for native GPT-4o image generation. HKR-H/K/R all pass on the multimodal hook and concrete capability facts, but missing eval scores and mitigation detail keep it below P1.

Mar 20, 2025Thursday

OpenAI News

Introducing next-generation audio models in the API

OpenAI released three API audio models on March 20, 2025: gpt-4o-transcribe, gpt-4o-mini-transcribe, and gpt-4o-mini-tts. The post says the STT models beat Whisper v2 and v3 on FLEURS and other benchmarks across 100+ languages, while the TTS model adds style control but stays limited to monitored preset synthetic voices. The key shift is controllable TTS plus lower WER; the post does not disclose pricing or latency figures.

Why it matters: OpenAI shipped 3 API audio models with concrete benchmark and mechanism details, so HKR-H/K/R all pass and it clears featured. I kept it at 84, not 85+, because price, latency, and a fuller benchmark table are not disclosed.

Mar 17, 2025Monday

Mistral AI

Mistral AI releases Mistral Small 3.1 with multimodal support and 128k context

Mistral AI released Mistral Small 3.1, which improves text performance and multimodal understanding over Mistral Small 3 and extends the context window to 128k tokens. Inference runs at 150 tokens per second, and the model is open-sourced under Apache 2.0.

Why it matters: Mistral gives the multimodal, 128k-context and 150 tokens/s figures for Mistral Small 3.1, letting readers compare it with small models of the same class.

Mar 12, 2025Wednesday

Hugging Face Blog

Welcome Gemma 3: Google's all new multimodal, multilingual, long context open LLM

Google announced an open LLM called Gemma 3 and named three traits in the title: multimodal, multilingual, and long context. The RSS snippet has no body, so parameter size, context length, license, and benchmark results are not disclosed. Watch the full post or model card; “open” does not equal open source from the title alone.

Why it matters: A new Gemma release from Google is inherently newsy, and the multimodal/long-context/open framing hits HKR-H and HKR-R. HKR-K misses because the feed gives no specs, context window, license, or benchmark data, so this lands at the low end of featured.

Mar 7, 2025Friday

Mistral AI

Mistral AI releases document-understanding OCR API Mistral OCR

Mistral AI released Mistral OCR, an optical character recognition API that takes images and PDFs and outputs interleaved text and images in order. It handles complex layouts such as tables, formulas and LaTeX.

Why it matters: Mistral gives benchmark comparisons, multilingual performance and pricing for Mistral OCR, showing how usable document parsing is in a RAG pipeline.