Skip to content

Deployment & engineering

Running models in practice: inference optimization, memory and cost, serving architecture and infrastructure choices.

Latest picks

21–40 of 609

Nov 11Wednesday

Simon Willison

Anthropic IPO roadshow video details multi-chip strategy and compute efficiency

Anthropic began its IPO roadshow yesterday, and the roadshow video covers how Claude is trained and deployed across multiple chips, plus methods for getting more out of compute.

Why it matters: The roadshow video compares a multi-chip adaptation that took dozens of engineers months with the same job done by Mythos in three days.

The Decoder

Anthropic says its AI moved Claude to AMD GPUs in 3 days, undercutting CUDA's moat

Anthropic says its own model, Mythos, got Claude running on AMD GPUs in 3 days with almost no human intervention. The model ported or wrote nearly 100% of the kernels, measured on prompt processing on AMD MI355X. Adapting a new accelerator by conventional means takes 6 to 12 months.

Why it matters: Anthropic claims Mythos cut new-hardware adaptation from 6 to 12 months to 3 days, comparing model-written kernels with conventional engineering.

The Decoder

Claroc finds Grok-built apps exposing patient data, plaintext passwords and SpaceXSI sales pipeline

Security firm Claroc checked 37,571 Grok-generated apps that are still online. Of those, 35,578 could be opened without logging in, exposing patient records, plaintext passwords and SpaceXSI internal sales data. Claroc traces the leaks to default sharing settings and broken permission checks.

Why it matters: Claroc traced the exposure to default sharing settings and permission-check bugs, and outside accounts could read, change and delete team data after logging in.

Computing Life · Share · Yage

How the Jensen Huang controversy could affect Nvidia's China chip lobbying and overseas compute rentals

The article argues that the Jensen Huang controversy may weaken Nvidia's ability to win looser China chip rules through the White House, based on public reporting and official disclosures. It further reasons that congressional export-license reviews and chip location verification could raise the compliance bar and cost for Chinese research teams renting training clusters in Southeast Asia. The analysis was done by AI, and the intelligence identity of those involved and the alleged illegal chip transfers are not judicially confirmed.

Why it matters: It links the Huang controversy to congressional chip export limits and asks how a narrower White House lobbying channel could raise compliance barriers for Chinese teams renting compute abroad.

Nov 10Tuesday

Hacker News front page

Fireworks AI security incident may affect under 1% of Notion workspaces

Notion's investigation confirmed that a Fireworks AI security incident may affect under 1% of Notion workspaces, with some Notion AI chat records possibly exposed. The records carry no user or workspace labels and contain no tool-call results, and other workspace data is unaffected. Notion has disconnected from Fireworks and disabled Insights.

Why it matters: The exposed chat records lack user and workspace labels, but the text itself may hold sensitive credentials, so affected customers should treat it as exposed data.

OpenAI News

OpenAI builds Control Room to track and halt model workloads across research infrastructure

OpenAI has deployed a Control Room across its research infrastructure that tracks model workloads and blocks further model output within 35 seconds of a stop order. The system checks authorization by job_id at the inference engine layer, so it can stop a job through the monitoring system without relying on the job's own code to run stop checks. If the Control Room service fails, it rejects inference requests; this fail-closed mechanism already covers all frontier model research workloads, with the remaining research workloads to be deployed in the coming days.

Why it matters: OpenAI puts the stop mechanism for research jobs at the inference layer, so safety control depends less on fast-changing job code while other authorized jobs keep running.

Hacker News front page

Google launches TPU 8tx and TPU 8i Max for confidential inference and ultra-long context

Google expanded its eighth-generation TPU lineup with TPU 8tx for secure confidential inference and TPU 8i Max for ultra-long context. TPU 8tx uses the same chip as TPU 8t and adds hardware-level protection for data and model weights, cutting latency to as little as half of TPU 8t. It supports fine-tuning on private data, knowledge distillation and reinforcement learning.

Why it matters: Google built specialized TPU systems by reusing an existing chip and changing security, networking and memory settings, shortening hardware adaptation from years to months.

claude.dev Blog

Anthropic's Claude model guide: choosing between Fable, Opus, Sonnet and Haiku

Anthropic's selection guide says most coding and knowledge work should start with Claude Opus 5.5 at medium thinking effort, moving to Sonnet 5.5, Haiku 5.5 or Fable 5.2 as task difficulty changes. It frames model choice around the cost of finishing a task: set up automated acceptance checks first, then adjust thinking effort or step up a model, and hand easy-to-verify side tasks to smaller models.

Why it matters: Anthropic splits Claude selection into model, thinking effort and cost per finished task, and proposes automated acceptance checks plus reruns on failure to control spend.

Computing Life · Share · Yage

Agent harness techniques tested: which framework designs raise scores and which just raise cost

Controlled experiments from Surge AI and FireFlash show coding agents can improve through deeper single-model reasoning, test and goal-check gates, and a reworked terminal pipeline. In Surge AI's run, Compound Engineering lifted cost to 8x baseline while the score moved only from 31.1 to 32.9. Claude Opus 5.5 with higher thinking settings reached 42.4 at 3.5x baseline cost.

Why it matters: The two teams break down the cost and payoff of agent harness techniques, comparing multi-agent collaboration, deeper single-model thinking and terminal pipeline changes.

Nov 9Monday

Computing Life · Share · Yage

Kimi and Zhipu align hosted agent APIs with Claude's, so is the protocol race already over?

Kimi Hosted Agents and Zhipu's Z Managed Agents both follow the four-part control plane of Claude Managed Agents, splitting role, environment, session and event. The article reads this as protocol convergence among hosted agents and weighs what it means for picking a vendor.

Why it matters: Kimi and Zhipu adopting Claude's four-part hosted interface gives long-running tasks a client abstraction that carries across vendors.

Latent Space

Stanford's AI-native engineering course: Mihail Eric on CS146S and Monaco's agent setup

Mihail Eric explains how Stanford's CS146S trains AI-native software engineers and how the course was designed alongside Monaco's team practice. The 10-week course covers coding agent internals, context engineering, skills, code review and security, and adds an open-source contribution assignment where maintainers review student patches. Monaco picked Devin after evaluating code review products, and built its own agent workspace on its cloud so background agents run the product, reproduce bugs and test patches.

Why it matters: Eric put open-source contribution into the Stanford course, with maintainers reviewing student patches so design skills are trained on real project discussion and code feedback.

Nov 8Sunday

Computing Life · Share · Yage

GDPx real-workstation review: six engineering lessons from AI running 8 hours straight

SemiAnalysis's GDPx benchmark has models use vendor agent frameworks to solve 206 real business problems in up to 8 hours. The strongest model averages about 12%. The article draws six engineering lessons from the failures: build a document authority map, set an independent rollback gate, write deterministic measurement scripts, keep an external evidence ledger, check the eval environment, and route by total cost per task.

Why it matters: It turns long-horizon failures on GDPx into engineering practice, folding document authority tagging, independent rollback checks and an external evidence ledger into agent workflows.

Nov 7Saturday

The Verge · AI

AI has scrambled cybersecurity defenses. How does Alex Stamos see the next step?

In a Decoder interview, Cognition chief security officer Alex Stamos said AI is lowering the bar for attackers and speeding up exploit development. Cybersecurity faces a stretch of adapting to new attack and defense conditions.

Why it matters: Stamos proposes a layered model for handling security alerts and limiting agent permissions, putting defense cost control and automated response risk into one operating plan.

Hacker News front page

Zvi Mowshowitz on Anthropic's S-1: founder control and compute pledges

Patrick McKenzie and Zvi Mowshowitz walk through Anthropic's S-1, covering founder control, charitable matching and future compute commitments. The material says a single Class F share carries 50.1% of the vote, employee donations are matched 3:1 with no cap, and future compute commitments total $895 billion.

Why it matters: Two podcast guests read Anthropic's filing for what it says about founder control, including a single share class holding a majority of votes.

Nov 6Friday

The Verge · AI

Democrats win control of Congress, putting AI regulation and data center buildout under pressure

Democrats won control of the House and Senate on Tuesday, leaving the AI industry facing tighter regulation, slower data center construction and pressure for safety legislation. Of 44 local ballot measures to pause data center construction, 43 passed. Texas governor-elect Gina Hinojosa promised to pause all data center approvals until the state legislature passes laws protecting residents. Experts surveyed expect federal attempts to block state-level AI limits to get harder, and tech accountability and AI safety legislation to keep advancing.

Why it matters: Voters passed 43 of 44 local data center pause measures, so AI companies' compute expansion plans face more direct local approval constraints.

Mistral AI

Mistral Studio brings Le Chonk to enterprise agents, workflows and apps

Mistral Studio now extends to hosted agents and apps, with updates to workflows, enterprise context, agent observability and evaluation. Mistral Large 4, codenamed Le Chonk, is put to work on enterprise business tasks.

Why it matters: Hosted agents, workflows and apps on shared infrastructure cut the engineering work enterprise teams would otherwise repeat to stand up a runtime.

Nov 5Thursday

The Decoder

After midterm defeats, OpenAI offers community pledges to win support for AI infrastructure

OpenAI issued a voluntary community pledge after US voters rejected new data centers in 40 of 41 local referendums. It covers future company-owned sites: OpenAI pays all power and grid-upgrade costs, cuts use at peak times, and publishes build plans, noise studies and water data. Whether partner-built facilities such as Oracle's are covered is unclear. The announcement came two days before the midterms, and OpenAI says its own projects may face delays.

Why it matters: OpenAI's pledges to cover power costs and publish water data apply only to new sites it owns and operates, leaving partner-held infrastructure unclear.

OpenAI News

OpenAI details Linux kernel bug hunt: 948 local privilege-escalation flaws confirmed, patches generated

OpenAI built a pipeline that finds, exploits and patches vulnerabilities on its own. In Linux kernel research it confirmed 948 unique local privilege-escalation bugs and produced candidate patches, dynamically reproducing 1,724 vulnerabilities at a total API cost of $1,213,430, or $1,279 per local privilege-escalation exploit on average. The pipeline uses a cybersecurity model OpenAI has not yet released; results went to maintainers after human review, and hundreds of severe bug reports and patches have been sent out.

Why it matters: After an agent succeeds at privilege escalation, OpenAI adds a separate check to confirm which bug was actually exploited, extending validation to the attack path and cutting misattribution.

MIT Technology Review · AI

Texas AI data center boom may stall after election result

Democrats won in Texas, and governor-elect Gina Hinojosa backs a full pause on data center construction. That could slow existing AI projects and block new ones, though local permitting power and building unions' positions will shape how far a state-level limit goes.

Why it matters: A winning Texas Democrat wants a data center construction pause, and local permitting plus union stances will decide how much of it lands.

Hacker News front page

Kimi K3.1 Pro releases open model weights with 1M-token context

Kimi K3.1 Pro is now open-weight: a native multimodal agent model for text and images with a 1-million-token context window. It uses a MoE architecture with 2.8T total parameters and 104B active, post-trained from Kimi K3 with large-scale joint reinforcement learning. Moonshot AI says coding, 3D creation, financial research and factual accuracy improved.

Why it matters: The release sets out what multi-turn tool calling must send back, so integrations need to keep full reasoning content and tool-call records to hold the conversation context.