Product updates

OpenAI added automatic prompt caching to GPT-4o, GPT-4o mini, o1-preview, and o1-mini API models, giving a 50% discount on recently reused input prefixes. Caching starts at 1,024 tokens and grows in 128-token increments; caches are often cleared after 5-10 minutes of inactivity and always within 1 hour of last use. The field to watch is cached_tokens in the API usage response.

OpenAI released omni-moderation-latest on September 26, 2024, a GPT-4o-based Moderation API model for text and image inputs that is free for all developers. It adds illicit and illicit/violent text categories, supports image moderation in 6 subcategories, and improves 42% on an internal 40-language eval, with gains in 98% of languages tested.

OpenAI released o1-mini on Sept. 12, 2024 for Tier 5 API users at 80% lower cost than o1-preview. The post reports 70.0% on AIME and 1650 Codeforces Elo, close to o1 at 74.4% and 1673, with about 3-5x faster answers than o1-preview in one word-reasoning example. The key tradeoff is explicit: it targets STEM reasoning, while non-STEM factual knowledge is only comparable to small models like GPT-4o mini.

OpenAI has opened GPT-4o fine-tuning to developers on all paid tiers, with 1M free training tokens per org per day through September 23. Training costs $25 per 1M tokens, and inference costs $3.75 per 1M input tokens and $15 per 1M output tokens on gpt-4o-2024-08-06. The signal for practitioners: partners reported 43.8% on SWE-bench Verified and 71.83% on BIRD-SQL with fine-tuned GPT-4o.

OpenAI released Structured Outputs on Aug 6, 2024, making model outputs conform to developer-supplied JSON Schemas; `gpt-4o-2024-08-06` scored 100% on complex schema-following evals versus under 40% for `gpt-4-0613`. The feature is enabled with `strict: true` in function calling and works on tool-supporting models including `gpt-4-0613`, `gpt-3.5-turbo-0613`, and later. The key shift is constrained decoding plus schema training, not just valid JSON from JSON mode.

DeepSeek API 上线硬盘缓存技术，价格再降一个数量级。更新详情见文档 https://api-docs.deepseek.com/zh-cn/updates#api-上线硬盘缓存技术 。

OpenAI began testing the SearchGPT prototype on July 25, 2024 with a small group of users and publishers. It answers with real-time web information, named inline citations, source links in a sidebar, and follow-up queries in shared context. The key detail is scope: this is a temporary prototype planned for future ChatGPT integration; the post does not disclose the model, rollout size, or commercial timeline.

Meta released Llama 3.1 with 405B, 70B, and 8B sizes, and the title says it adds multilingual support and long context. Only the title is available; the post does not disclose context length, languages, license terms, or benchmark results. Watch the 405B release terms and real inference cost.

OpenAI released GPT-4o mini on July 18, 2024 at $0.15 per 1M input tokens and $0.60 per 1M output tokens, replacing GPT-3.5 in ChatGPT. It supports text and vision, offers a 128K context window and 16K max output, scores 82.0% on MMLU and 87.2% on HumanEval. The key detail for builders is that its API version is the first to use instruction hierarchy against jailbreaks and prompt injection.

OpenAI on July 18, 2024 launched an Enterprise Compliance API, eight third-party compliance integrations, and SCIM user management for ChatGPT Enterprise. The post confirms timestamped exports for conversations, files, GPT configs, memories, and users, plus support for Okta, Microsoft Entra ID, Google Workspace, and Ping; details on “Expanded GPT controls” are not disclosed in the provided body.

OpenAI says it is bringing GPT-4o and more tools to ChatGPT free users, with free-tier access as the stated condition. Only the title is available; the post does not disclose tool list, usage limits, rollout timing, or regions.

OpenAI says the GPT-4 API is generally available and older models in the Completions API will be deprecated. Only the title confirms these two facts; the post body is empty and does not disclose scope, timeline, or affected model names. The real issue to watch is migration cost: this is both an API and model transition.

OpenAI says ChatGPT is getting memory and new controls, with 2 changes disclosed in the title. The body is empty, so default state, opt-out scope, and user-tier availability are not disclosed. The key issue is control granularity; the title alone is not enough to judge product impact.

OpenAI says ChatGPT now supports seeing, hearing, and speaking. The post body is empty, so it does not disclose model versions, rollout timing, regional limits, pricing, or API scope. The real watchpoints are voice latency, vision limits, and access paths.