Skip to content

Deployment & engineering

Running models in practice: inference optimization, memory and cost, serving architecture and infrastructure choices.

Latest picks

581–591 of 591

May 8, 2025Thursday

OpenAI News

OpenAI’s response to the Department of Energy on AI infrastructure

On May 7, 2025, OpenAI submitted AI infrastructure proposals to the US Department of Energy, urging federal land use, faster permitting, and financial incentives for AI supercomputer hubs. The post says the first Stargate campus is underway in Abilene, Texas, and more sites are being evaluated in Texas and other states; it does not disclose specific tax, power-pricing, or lease terms. The real signal is policy positioning: OpenAI is framing data centers, energy, and permitting as a national industrial agenda.

Why it matters: OpenAI asks the Department of Energy for federal land, faster permitting and power tax credits, with the Abilene Stargate campus already under construction.

May 7, 2025Wednesday

Mistral AI

Mistral AI launches Le Chat Enterprise, powered by Mistral Medium 3

Mistral AI released Le Chat Enterprise, an enterprise AI assistant powered by the new Mistral Medium 3 model. It includes enterprise search, an agent builder, connectors for custom data and tools, a document library, custom models and hybrid deployment. All features will roll out over the next two weeks.

Why it matters: Mistral AI launches Le Chat Enterprise, an enterprise assistant powered by Mistral Medium 3 with enterprise search, agent building and self-hosted deployment, all due within two weeks.

Mistral AI

Mistral AI releases Mistral Medium 3, targeting low cost and enterprise deployment

Mistral AI released Mistral Medium 3, which it says reaches or exceeds 90% of Claude Sonnet 3.7 across benchmarks. Pricing is $0.4 per million input tokens and $2 per million output tokens.

Why it matters: Mistral Medium 3 benchmarks against Claude Sonnet 3.7 at lower cost, and the post gives a path to private enterprise deployment and customization.

Mar 31, 2025Monday

OpenAI News

OpenAI raises $40 billion at a $300 billion post-money valuation

OpenAI said it raised $40 billion at a $300 billion post-money valuation. The post names SoftBank Group as a partner and says the funds will expand compute infrastructure and support tools for ChatGPT's 500 million weekly users. The AGI framing is broad; the post does not disclose deal structure, funding timing, or product roadmap details.

Why it matters: OpenAI raised $40 billion at a $300 billion post-money valuation, with SoftBank as the key partner, to fund compute infrastructure and ChatGPT development.

Feb 7, 2025Friday

Mistral AI

Mistral AI launches new le Chat with mobile apps and Pro, Team plans

Mistral AI released a new version of le Chat with iOS and Android apps, Pro and Team tiers, and an Enterprise private preview. It adds a code interpreter, document analysis and image generation, and Flash Answers is in open preview for all users at a claimed ~1000 words per second. Most features are free; Pro starts at $14.99 per month.

Why it matters: Mistral AI is giving le Chat both a mobile entry point for daily use and enterprise deployment options for work inside a company's own environment.

Jan 30, 2025Thursday

Mistral AI

Mistral AI releases Mistral Small 3, a 24B model under Apache 2.0

Mistral AI released Mistral Small 3, a 24B model built for low-latency tasks, with pretrained and instruction-tuned checkpoints under Apache 2.0. The company says it matches Llama 3.3 70B instruct, runs more than 3x faster on the same hardware, and scores above 81% on MMLU. The model was not trained with RL or synthetic data.

Why it matters: Mistral AI benchmarks Small 3 against Llama 3.3 70B and reports more than 3x the speed on the same hardware.

Oct 23, 2024Wednesday

OpenAI News

Simplifying, stabilizing, and scaling continuous-time consistency models

OpenAI introduced sCM and scaled continuous-time consistency models to 1.5B parameters on ImageNet at 512×512. The post says sCM reaches sample quality comparable to leading diffusion models in 2 sampling steps, with about 50x wall-clock speedup. Its largest model generates one sample in 0.11s on a single A100 at batch size 1 without inference optimization.

Why it matters: OpenAI scales continuous-time consistency models to 1.5B parameters, matching diffusion quality on ImageNet 512×512 in two sampling steps.

Oct 2, 2024Wednesday

OpenAI News

New funding to scale the benefits of AI

OpenAI said it raised $6.6B at a $157B post-money valuation. The post says the money will fund frontier AI research, more compute, and product tools; ChatGPT has over 250M weekly users. The investor list, ownership terms, and added compute scale are not disclosed.

Why it matters: Sets the scale of OpenAI's war chest and the user base behind it.

Oct 1, 2024Tuesday

OpenAI News

Prompt Caching in the API

OpenAI added automatic prompt caching to GPT-4o, GPT-4o mini, o1-preview, and o1-mini API models, giving a 50% discount on recently reused input prefixes. Caching starts at 1,024 tokens and grows in 128-token increments; caches are often cleared after 5-10 minutes of inactivity and always within 1 hour of last use. The field to watch is cached_tokens in the API usage response.

Why it matters: Halves input cost for repeated long prefixes, which changes how long-context apps are priced.

Aug 6, 2024Tuesday

OpenAI News

Introducing Structured Outputs in the API

OpenAI released Structured Outputs on Aug 6, 2024, making model outputs conform to developer-supplied JSON Schemas; `gpt-4o-2024-08-06` scored 100% on complex schema-following evals versus under 40% for `gpt-4-0613`. The feature is enabled with `strict: true` in function calling and works on tool-supporting models including `gpt-4-0613`, `gpt-3.5-turbo-0613`, and later. The key shift is constrained decoding plus schema training, not just valid JSON from JSON mode.

Why it matters: Structured Outputs makes models follow a developer's JSON Schema exactly, and the accuracy gap between new and old models shows why schema adherence is a model capability, not a prompt trick.

Aug 2, 2024Friday

DeepSeek · API updates

DeepSeek API adds disk caching, cutting prices by another order of magnitude

DeepSeek API 上线硬盘缓存技术,价格再降一个数量级。更新详情见文档 https://api-docs.deepseek.com/zh-cn/updates#api-上线硬盘缓存技术 。

Why it matters: DeepSeek's API now uses disk caching and prices fell by another order of magnitude, so the direct effect of this caching update lands on API usage cost.