Skip to content

Deployment & engineering

Running models in practice: inference optimization, memory and cost, serving architecture and infrastructure choices.

Latest picks

421–440 of 455

Mar 7Saturday

Bloomberg Technology

US Considers Permits for Global Nvidia, AMD AI Chip Sales | Bloomberg Tech 3/6/2026

The US Commerce Department has reportedly drafted rules that would require American approval before Nvidia and AMD AI chips ship anywhere globally. The RSS snippet also says Oracle plans thousands of job cuts amid cash strain from AI data center expansion, and the Pentagon told lawmakers Anthropic poses a US supply-chain risk. The post does not disclose permit thresholds, layoff details, or the basis for the Anthropic finding.

Why it matters: The core policy angle is major: a global permit regime for Nvidia and AMD AI chip exports would have industry-wide impact. HKR-H/K/R all pass, but this is a video roundup page with thin disclosed detail—scope, thresholds, and timing are not clear—so it stays high featured, not p1

Bloomberg Technology

OpenAI, Oracle Won't Expand Flagship AI Data Center in Texas

OpenAI and Oracle have scrapped plans to expand a flagship AI data center in Texas after financing talks dragged and OpenAI's needs changed. The RSS snippet confirms only the Texas site; the post does not disclose the facility name, target capacity, capex, or revised timeline. The signal to watch is shifting compute demand, not just a stalled real estate project.

Why it matters: Bloomberg reports OpenAI and Oracle dropped a flagship Texas data-center expansion, citing financing delays and shifting OpenAI demand. HKR-H/K/R all pass and source authority helps, but missing capacity, capex, and timeline details keep it in the low 80s.

Bloomberg Technology

Oracle and OpenAI End Plans to Expand Flagship Data Center

Oracle and OpenAI ended plans to expand a flagship AI data center in Texas. The RSS snippet says talks dragged over financing and OpenAI’s changing needs; the post does not disclose the site’s size, budget, or timeline. The real signal is financing friction plus a demand reassessment.

Why it matters: Bloomberg reports a meaningful infrastructure reversal, so HKR-H and HKR-R land: it is unexpected and it hits compute-supply and capex concerns around OpenAI. HKR-K is limited because the writeup omits size, spend, and timing, keeping this near the featured threshold.

Feb 28Saturday

36Kr (direct RSS)

Qwen plans AI glasses, earbuds, and rings as tech giants race for a new AI entry point

A report says Alibaba's Qwen plans AI glasses, earbuds, and rings for a global launch in 2026; the glasses are slated for MWC 2026, with reservations opening on March 2. The post adds that Qwen app functions like food delivery and ride hailing will move to these devices, and cites Qwen3.5-Plus with 60% lower memory use, up to 19x inference throughput, and RMB 0.8 per million tokens. The real point is distribution: if the hardware connects Alipay, Amap, and Taobao, Alibaba is chasing the consumer AI entry layer, not just device sales.

Why it matters: This is a distribution-entry story for Alibaba/Qwen, not a routine accessory refresh. HKR-H/K/R all pass: the multi-device bet is a strong hook, the report includes launch timing and model economics, and it hits the ecosystem-front-end nerve; but it is still a media exclusive, so

36Kr (direct RSS)

36Kr 9AM Briefing: Lynk apologizes after voice command headlight crash; OpenAI raises $110B; miHoYo reports employee death

OpenAI said it raised $110B, with $30B each from SoftBank and NVIDIA and $50B from Amazon, at a $730B pre-money valuation. The post adds a strategic partnership with Amazon and a next-gen inference compute deal with NVIDIA.

Why it matters: HKR-H/K/R all pass: this combines a record-scale $110B raise, a $730B pre-money valuation, and deal terms that tie capital to inference compute and cloud distribution. This changes market structure, not just OpenAI's cash position.

Feb 20Friday

Hugging Face Blog

GGML and llama.cpp join Hugging Face to support the long-term progress of Local AI

Hugging Face said the GGML and llama.cpp team is joining the company, while Georgi Gerganov’s team will still spend 100% of its time maintaining llama.cpp. The post says the project remains 100% open source and community driven, with full technical and community autonomy. The key angle is tighter delivery from transformers model definitions into llama.cpp, aiming for near “single-click” shipping; the post does not disclose timeline, team size, or deal terms.

Why it matters: This is a meaningful local-AI infrastructure move: HF brings in the GGML/llama.cpp team, so HKR-H/K/R all pass. I kept it at 78 because the post confirms staffing and integration direction, but not a ship date, team size, or deal terms.

Feb 6Friday

Dwarkesh Patel

Elon Musk: “In 36 months, the cheapest place to put AI will be space”

Elon Musk predicts that in 30–36 months, space will become the cheapest place to deploy AI compute. He cites flat power growth, permitting bottlenecks, and roughly 5x better solar output in space without batteries; the interview does not disclose a cost model or validation data.

Why it matters: This clears the featured line as source-authority commentary: HKR-H comes from the stark 36-month space-cost claim, and HKR-R from the power bottleneck every AI infra team watches. HKR-K fails because the transcript gives heuristics, not a disclosed cost model or serviceability/​

Jan 30Friday

Bloomberg Technology

Perplexity Inks Microsoft AI Cloud Deal Amid Dispute With Amazon

Perplexity signed a $750 million Azure cloud deal with Microsoft while facing a legal dispute with its longtime cloud partner Amazon. The RSS snippet discloses the deal size, cloud provider, and dispute context, but not the contract term, compute scale, or lawsuit details. The key signal is a cloud supply rebalance that can affect training and inference costs.

Why it matters: HKR-H/K/R all pass: a $750M Azure deal signed during an Amazon dispute is clicky, concrete, and discussable. It stays below 85 because the story gives price and counterpart, but not term, compute volume, or migration scope.

Bloomberg Technology

Amazon in Talks to Invest Up to $50 Billion in OpenAI and Expand Ties

Amazon is in talks to invest up to $50 billion in OpenAI and expand their existing relationship. The RSS snippet says the tie-up includes Amazon selling compute to OpenAI; the post does not disclose deal structure, timing, or whether talks will close. The key signal is compute linkage, not just capital.

Why it matters: HKR-H lands on the sheer $50B number and the unexpected Amazon-OpenAI tie-up; HKR-K lands on the reported compute-sales linkage. HKR-R is strong because cloud alignment and OpenAI's supply stack are core industry nerves, but key deal terms remain undisclosed, so this stays below

Jan 12Monday

36Kr (direct RSS)

He Xiaopeng: The best AI companies in the future will build their own chips

He Xiaopeng said XPeng's four 2026 vehicle models will use its Turing AI chip, and Ultra SE and Ultra trims will run a second-gen VLA model for entry-level L4-assisted driving. The post says MAX uses one 750 TOPS chip, Ultra SE uses two, and Ultra uses three; XPeng has entered 60 countries and regions, and VLA 2.0 is already being road-tested in Europe. The real signal is that automakers are pulling chips, models, and deployment in-house as a ceiling-on-performance play, not just a cost move.

Why it matters: The signal is not the slogan but the concrete roadmap: 4 cars, 750 TOPS per chip, 1/2/3-chip trims, and VLA 2.0 road tests. HKR-H/K/R all pass, but this is still a roadmap disclosure rather than a shipped AI-industry event, so it sits at the low end of featured.

Jan 6Tuesday

NVIDIA Blog

NVIDIA RTX Accelerates 4K AI Video Generation on PC With LTX-2 and ComfyUI Upgrades

NVIDIA said GeForce RTX and related devices can run LTX-2 and updated ComfyUI for local AI video generation up to 3x faster with up to 60% lower VRAM use. The post attributes this to PyTorch-CUDA optimizations, native NVFP4/FP8 support in ComfyUI, and an RTX Video 4K upscaling node due next month; LTX-2 open weights are available now and the workflow ships next month. The real signal for AI builders is that local 4K video is shifting from VRAM-bound demos to usable RTX workflows.

Why it matters: HKR-H/K/R all pass: the story has a sharp hook, concrete mechanisms, and clear resonance for local-inference users. I keep it at 76 because this is a vendor-blog ecosystem optimization update, not a major model launch or broad platform shift.

NVIDIA Blog

NVIDIA presents Rubin platform, open models and autonomous driving roadmap at CES

At CES 2026, NVIDIA said its six-chip Rubin AI platform is now in full production and cuts token generation cost to about one-tenth of the prior platform. The post cites 50 petaflops NVFP4 inference for Rubin GPUs, 5x gains from its KV-cache storage tier, and the new open autonomous-driving model family Alpamayo; the key signal is production status and cost curve, not the “AI everywhere” framing.

Why it matters: HKR-H lands because Rubin is in production, not just on a roadmap. HKR-K is strong with ~1/10 token cost, 50 PFLOPS NVFP4, and 5x long-context throughput; HKR-R lands because NVIDIA still sets the tone on inference economics, though the company-blog framing keeps it below 90.

NVIDIA Blog

NVIDIA DGX SuperPOD Sets the Stage for Rubin-Based Systems

NVIDIA introduced Rubin-based DGX SuperPOD systems, with DGX Vera Rubin NVL72 and DGX Rubin NVL8 slated for the second half of this year. One DGX SuperPOD can combine eight NVL72 systems for 576 Rubin GPUs, 28.8 exaflops FP4, and 600TB memory; NVIDIA says inference token cost drops by up to 10x versus the prior generation. The key detail is rack-scale design: 260TB/s NVLink per rack, which the post says removes model partitioning.

Why it matters: This is a substantive NVIDIA infra roadmap with hard numbers: 576 Rubin GPUs, 28.8 exaflops FP4, 600TB memory, 260TB/s NVLink, and up to 10x lower token cost. HKR-H/K/R all pass, but it is still a vendor roadmap post rather than a shipping model or broad product release, so it is

Jan 5Monday

Import AI (Jack Clark)

Import AI 439: AI kernels; decentralized training; and universal representations

Meta says KernelEvolve cut kernel development from weeks to hours and delivered up to 17x over PyTorch baselines in production tests. The system uses Llama, GPT, and Claude to generate kernels, validates them, and feeds results into a knowledge base across NVIDIA, AMD, and MTIA; the post also says decentralized training is growing 20x per year but still uses about 1000x less compute than frontier runs. The real signal is continuous self-optimizing infra in production, while decentralized training matters if that 1000x gap keeps shrinking.

Why it matters: HKR-H/K/R all pass: the kernel-writing angle is novel, the post includes concrete numbers and mechanism, and the decentralization thread hits cost and power-concentration nerves. I stop at 80 because this is a newsletter synthesis of technical work, not a single industry-defining

Dec 17, 2025Wednesday

Mistral AI

Mistral releases OCR 3 with better forms and handwriting, plus Document AI Playground

Mistral released Mistral OCR 3, which wins 74% of head-to-head comparisons against Mistral OCR 2 on forms, scanned documents, complex tables and handwriting. Mistral says its accuracy beats enterprise document-processing tools and AI-native OCR options.

Why it matters: Mistral OCR 3's upgrades on forms, handwriting and complex tables, plus its $2 per 1,000 pages pricing, are useful when evaluating document parsing options.

Oct 24, 2025Friday

Mistral AI

Mistral AI launches Mistral AI Studio production platform

Mistral AI released Mistral AI Studio, a production-grade AI platform for enterprise teams, built on three pillars: Observability, Agent Runtime and AI Registry.

Why it matters: The post lays out the three pillars of enterprise AI production and a private beta entry point, enough to judge how it differs from existing MLOps tools.

Oct 13, 2025Monday

OpenAI News

OpenAI and Broadcom announce collaboration to deploy 10 gigawatts of OpenAI-designed AI accelerators

OpenAI and Broadcom announced a multi-year deal to deploy 10 gigawatts of OpenAI-designed AI accelerators, with rack deployments starting in H2 2026 and completing by the end of 2029. OpenAI will design the accelerators and systems, while Broadcom provides accelerator deployment plus Ethernet, PCIe, and optical networking for OpenAI sites and partner data centers. The key signal is OpenAI's custom-chip plus Ethernet cluster path, but the post does not disclose process node, chip specs, or capex.

Why it matters: Not a routine partnership story: OpenAI put a 10GW custom-chip plan and a 2026-2029 deployment schedule on record. HKR-H/K/R all pass, but process node, per-chip specs, and capex are still undisclosed, so this lands in p1 rather than 95+.

Oct 1, 2025Wednesday

OpenAI News

Samsung and SK join OpenAI’s Stargate initiative to expand global AI infrastructure

OpenAI said on Oct. 1, 2025 that Samsung and SK joined Stargate, with the partnership centered on Korea’s AI chip supply and data center expansion. The post gives one hard target: Samsung Electronics and SK hynix plan to scale advanced memory output to 900,000 DRAM wafer starts per month, while OpenAI also signed Korean data center exploration agreements with MSIT, SK Telecom, and Samsung affiliates. The key gap is execution detail: the post does not disclose investment size, timeline, or facility scale.

Why it matters: OpenAI adding Samsung and SK to Stargate is more than a routine partnership: the post gives a 900k DRAM wafer-start target and concrete data-center assessment ties. HKR-H/K/R all pass, but missing capex, timeline, and site scale keeps it featured, not p1.

Sep 23, 2025Tuesday

OpenAI News

OpenAI, Oracle, and SoftBank expand Stargate with five new AI data center sites

OpenAI, Oracle, and SoftBank announced five new U.S. Stargate AI data center sites, bringing planned capacity to nearly 7 GW and investment to over $400 billion in three years. The post says this keeps Stargate on track to reach its full $500 billion, 10 GW commitment by the end of 2025; Oracle-linked sites account for over 5.5 GW, while two SoftBank-OpenAI sites can scale to 1.5 GW in 18 months. The key signal is supply progress: Abilene is already running early training and inference workloads with first NVIDIA GB200 racks delivered in June.

Sep 22, 2025Monday

OpenAI News

OpenAI and NVIDIA announce strategic partnership to deploy 10 gigawatts of NVIDIA systems

OpenAI and NVIDIA signed a letter of intent to deploy at least 10 gigawatts of NVIDIA systems for OpenAI’s next-generation AI infrastructure. NVIDIA plans to invest up to $100 billion into OpenAI as each gigawatt is deployed, and the first 1 GW phase is targeted for H2 2026 on the Vera Rubin platform. The key detail is execution: this is still an LOI, and final terms are not yet closed.

Why it matters: Strong HKR-H/K/R: the official post discloses 10 GW, millions of GPUs, up to $100B intended investment, and a first 1 GW phase in H2 2026 on Vera Rubin. It is still a letter of intent, not a signed final deal, so it stays below the 95+ band; the scale still makes it p1.