Skip to content

Deployment & engineering

Running models in practice: inference optimization, memory and cost, serving architecture and infrastructure choices.

Latest picks

441–455 of 455

Aug 7, 2025Thursday

OpenAI News

GPT-5 System Card

OpenAI published the GPT-5 System Card on Aug. 7, 2025, stating GPT-5 combines gpt-5-main, gpt-5-thinking, and a real-time router, with mini models used after limits are hit. The API exposes gpt-5-thinking, gpt-5-thinking-mini, and gpt-5-thinking-nano, while ChatGPT adds gpt-5-thinking-pro; the post does not disclose pricing, context window, or benchmark scores. The key signal is safety: OpenAI classifies gpt-5-thinking as High capability in biological and chemical domains and applies the related safeguards.

Why it matters: This system card for OpenAI’s flagship model discloses GPT-5’s routed architecture, mini fallback, and direct access to thinking variants. HKR-H/K/R all pass; the High bio/chem capability rating makes this a same-day safety and deployment story, not routine documentation.

Aug 5, 2025Tuesday

OpenAI News

Introducing gpt-oss

OpenAI released gpt-oss-120b and gpt-oss-20b under Apache 2.0, with the 120B model running on one 80GB GPU and the 20B model on devices with 16GB memory. Both are MoE Transformers with 117B and 21B total parameters, 5.1B and 3.6B active params per token, 128k context, and support for the Responses API and Structured Outputs. The part that matters is the lower deployment bar plus open weights; the post excerpt claims strong reasoning, but full benchmark scores are not disclosed here.

Why it matters: Same-day write. OpenAI moving into Apache 2.0 open weights is a strategy story, not a routine update; HKR-H lands on the unexpected move, HKR-K on concrete deployment specs, and HKR-R on cost and open-vs-closed debates. Not 95+ because the excerpt does not disclose full benchmark

Jul 31, 2025Thursday

OpenAI News

Introducing Stargate Norway

OpenAI said Stargate Norway, its first European AI data center project, is planned for 230MW with a further 290MW expansion target. Nscale and Aker will build it in Narvik, aiming for 100,000 NVIDIA GPUs by end-2026, using renewable power and closed-loop direct-to-chip liquid cooling. The key detail is allocation: OpenAI is an initial offtaker, while surplus capacity is intended for users in Norway, the UK, the Nordics, and Northern Europe; the post does not disclose capex or exact GPU models.

Jul 30, 2025Wednesday

Mistral AI

Mistral ships Codestral 25.08 and an enterprise coding stack

Mistral AI released Codestral 25.08 along with a full enterprise coding stack: Codestral, Codestral Embed, Devstral and a Mistral Code IDE plugin.

Why it matters: The post gives Codestral 25.08's completion gains and how the enterprise stack is deployed, so you can judge whether a private coding setup is viable.

Jul 22, 2025Tuesday

OpenAI News

Stargate advances with 4.5 GW partnership with Oracle

OpenAI and Oracle agreed to add 4.5 GW of Stargate data center capacity in the U.S., bringing capacity under development to over 5 GW and more than 2 million chips. OpenAI says this advances its January pledge to build 10 GW of U.S. AI infrastructure with $500 billion over four years, and it now expects to exceed that target. The concrete signal is deployment: Stargate I in Abilene has started receiving Nvidia GB200 racks and is already running early training and inference workloads.

Why it matters: This clears HKR-H/K/R: the hook is the sheer 4.5GW scale, the post includes concrete capacity numbers, and compute supply is a live industry nerve. At 88, this is a same-day infrastructure story with strategic impact, below only top-tier model or executive news.

Jun 4, 2025Wednesday

Mistral AI

Mistral AI launches Mistral Code enterprise coding assistant

Mistral AI released Mistral Code, an enterprise AI coding assistant that combines four models: Codestral, Codestral Embed, Devstral and Mistral Medium. It runs in the cloud, on dedicated capacity or on air-gapped local GPUs, so code stays inside the company's boundary.

Why it matters: Mistral lays out the model mix, deployment options and customer cases for an enterprise coding assistant, showing one path to private coding setups.

May 22, 2025Thursday

OpenAI News

Introducing Stargate UAE

OpenAI, with G42, Oracle, NVIDIA, Cisco, and SoftBank, will deploy a 1GW Stargate UAE cluster in Abu Dhabi, with 200MW expected online in 2026. The project is the first OpenAI for Countries deal; OpenAI says the UAE will be the first country with nationwide ChatGPT access, and the site can serve a 2,000-mile radius. What matters is sovereign compute tied to U.S. coordination; the post does not disclose capex split, GPU counts, or how nationwide ChatGPT access will work.

Why it matters: This clears HKR-H/K/R: the first overseas Stargate is a strong hook, the post includes 1GW and 200MW-by-2026 specifics, and sovereign compute will drive discussion. It stops short of a higher score because funding split, chip count, and the ChatGPT access mechanism are not yet in

May 8, 2025Thursday

OpenAI News

OpenAI’s response to the Department of Energy on AI infrastructure

On May 7, 2025, OpenAI submitted AI infrastructure proposals to the US Department of Energy, urging federal land use, faster permitting, and financial incentives for AI supercomputer hubs. The post says the first Stargate campus is underway in Abilene, Texas, and more sites are being evaluated in Texas and other states; it does not disclose specific tax, power-pricing, or lease terms. The real signal is policy positioning: OpenAI is framing data centers, energy, and permitting as a national industrial agenda.

Why it matters: This is a primary-source policy filing, not a product update, but HKR-H/K/R all land because it connects federal land, permitting, and power to AI compute expansion. The DOE proposal and Abilene Stargate construction are concrete; undisclosed tax, power-price, and lease terms cap

May 7, 2025Wednesday

Mistral AI

Mistral AI launches Le Chat Enterprise, powered by Mistral Medium 3

Mistral AI released Le Chat Enterprise, an enterprise AI assistant powered by the new Mistral Medium 3 model. It includes enterprise search, an agent builder, connectors for custom data and tools, a document library, custom models and hybrid deployment. All features will roll out over the next two weeks.

Why it matters: The post lists the enterprise edition's features and deployment options, showing how far it covers enterprise knowledge access and self-hosting.

Mistral AI

Mistral AI releases Mistral Medium 3, targeting low cost and enterprise deployment

Mistral AI released Mistral Medium 3, which it says reaches or exceeds 90% of Claude Sonnet 3.7 across benchmarks. Pricing is $0.4 per million input tokens and $2 per million output tokens.

Why it matters: Mistral Medium 3 benchmarks against Claude Sonnet 3.7 at lower cost, and the post gives a path to private enterprise deployment and customization.

Mar 31, 2025Monday

OpenAI News

OpenAI raises $40 billion at a $300 billion post-money valuation

OpenAI said it raised $40 billion at a $300 billion post-money valuation. The post names SoftBank Group as a partner and says the funds will expand compute infrastructure and support tools for ChatGPT's 500 million weekly users. The AGI framing is broad; the post does not disclose deal structure, funding timing, or product roadmap details.

Why it matters: HKR-H lands on the $40B/$300B hook; HKR-K on the disclosed financing and 500M weekly users; HKR-R on the capital and compute race. The post omits structure and funding timing, but this is still p1-scale financing news.

Oct 23, 2024Wednesday

OpenAI News

Simplifying, stabilizing, and scaling continuous-time consistency models

OpenAI introduced sCM and scaled continuous-time consistency models to 1.5B parameters on ImageNet at 512×512. The post says sCM reaches sample quality comparable to leading diffusion models in 2 sampling steps, with about 50x wall-clock speedup. Its largest model generates one sample in 0.11s on a single A100 at batch size 1 without inference optimization.

Why it matters: This clears HKR-H/K/R: the hook is 2-step sampling with diffusion-like quality, and the paper gives concrete numbers—1.5B params, ImageNet 512x512, ~50x wall-clock speed, and 0.11s per sample on one A100. Strong research release, but not a shipped product, so featured fits better

Oct 2, 2024Wednesday

OpenAI News

New funding to scale the benefits of AI

OpenAI said it raised $6.6B at a $157B post-money valuation. The post says the money will fund frontier AI research, more compute, and product tools; ChatGPT has over 250M weekly users. The investor list, ownership terms, and added compute scale are not disclosed.

Why it matters: Official OpenAI funding post with hard numbers: $6.6B, $157B post-money, and 250M weekly ChatGPT users. HKR-H/K/R all pass because the scale is newsy, the facts are concrete, and the story speaks directly to the capital-compute race; missing investor and structure details keep it

Oct 1, 2024Tuesday

OpenAI News

Prompt Caching in the API

OpenAI added automatic prompt caching to GPT-4o, GPT-4o mini, o1-preview, and o1-mini API models, giving a 50% discount on recently reused input prefixes. Caching starts at 1,024 tokens and grows in 128-token increments; caches are often cleared after 5-10 minutes of inactivity and always within 1 hour of last use. The field to watch is cached_tokens in the API usage response.

Why it matters: A substantive OpenAI API update: not a new model, but it ships a 50% input discount, a 1,024-token threshold, 128-token cache steps, and cached_tokens telemetry, so HKR-H/K/R all pass. It is highly relevant to builder cost and latency, strong enough for featured, but not a same‑y

Aug 6, 2024Tuesday

OpenAI News

Introducing Structured Outputs in the API

OpenAI released Structured Outputs on Aug 6, 2024, making model outputs conform to developer-supplied JSON Schemas; `gpt-4o-2024-08-06` scored 100% on complex schema-following evals versus under 40% for `gpt-4-0613`. The feature is enabled with `strict: true` in function calling and works on tool-supporting models including `gpt-4-0613`, `gpt-3.5-turbo-0613`, and later. The key shift is constrained decoding plus schema training, not just valid JSON from JSON mode.

Why it matters: HKR-H/K/R all pass: OpenAI moves from 'valid JSON' to strict schema adherence and publishes a 100% vs <40% reliability gap. I keep it at 84 because this is a high-value API capability update, not a new frontier-model launch or company-level industry event.