Skip to content

Safety & alignment

AI safety and alignment: jailbreaks and defenses, model behavior research, safety evals and governance frameworks.

Latest picks

341–360 of 615

May 9Saturday

AI HOT (Curated Pool)

Claude Mythos Evaluation Shows 16-Hour Risk Horizon

METR evaluated an early Claude Mythos Preview build during a limited March 2026 window and estimated its 50% time horizon at at least 16 hours, with a 95% confidence interval of 8.5 to 55 hours.

Why it matters: METR's risk assessment of an early Claude Mythos preview measured a 16-hour lower bound on tasks the model can finish alone.

Latent Space

Anthropic growing 10x/year while others lay off over 10% of staff

Anthropic is described as growing 10x annually and being valued at $1T-$1.2T, while the post cites layoffs of 40% at Block, 14% at Coinbase, and 20% at Cloudflare under AI-readiness framing.

Why it matters: Anthropic's annualized revenue is growing 80x while other labs cut over 10% of staff, showing money and talent concentrating at the top.

MIT Technology Review · AI

Musk v. Altman Week 2: OpenAI Fires Back, and Shivon Zilis Says Musk Tried to Poach Sam Altman

Greg Brockman testified that Elon Musk pushed OpenAI in 2017 to create a for-profit arm and sought majority equity, board control, and the CEO role; Musk now asks the court to remove Sam Altman and Brockman, unwind OpenAI’s restructuring, and award up to $134 billion from OpenAI and Microsoft.

Why it matters: OpenAI president Brockman testified that Musk himself pushed for a for-profit arm in 2017 and sought majority equity and the CEO role.

AI HOT (Curated Pool)

Our Approach to Child Safety

Runway applies Thorn’s Safety by Design for Generative AI principles to child safety, using hash matching, child-safety classifiers, LLM review, and red-team testing, and submitted 516 reports to NCMEC in 2025.

Why it matters: Runway lays out how it blocks child sexual abuse material in its video and image tools, from training data cleaning to 516 reports filed with NCMEC.

May 8Friday

AI HOT (Curated Pool)

Running Codex Safely at OpenAI

OpenAI runs Codex with four safeguards: sandbox isolation, human approval, strict network policies, and native agent telemetry; the post does not disclose evaluation metrics, incident rates, or enterprise deployment requirements.

Why it matters: OpenAI published its internal Codex deployment security plan, splitting sandbox isolation, human approval, network allowlists and agent logs into four layers that others can adapt when setting code agent permission boundaries.

Alibaba Technology · WeChat

The AI-Native Era: Where R&D Organizations Go Next

Xu Xiaobin cites internal interviews showing that engineers who use AI heavily cut coding time from 30% to 5%, raised Agent conversation time from 5% to 60%, and increased end-to-end delivery efficiency by 2 to 3 times, while pure coding efficiency rose 10 times.

Why it matters: Alibaba's Xu Xiaobin says coding dropped from 30% to 5% of an engineer's time, with 60% now spent talking to agents.

r/LocalLLaMA

You can now read Gemma 3's mind

Anthropic released NLA research to explain Gemma 3 27B Instruct activations for each generated token. The post links Auto Verbalizer and Activation Reconstructor weights on Hugging Face. Neuronpedia hosts an interactive page; the post does not disclose evaluation scores.

Why it matters: Anthropic's NLA method turns Gemma 3 27B activations into natural language, with two modules checking each other, giving a reproducible way to inspect a model's internal state as it generates.

AI HOT (Curated Pool)

Donating the Open-Source Alignment Tool Petri

Anthropic transferred the open-source alignment testing tool Petri to Meridian Labs to preserve independence and credibility. Petri 3.0 separates auditor and target models, adds Dish for real prompts and deployment settings, and integrates Bloom.

Why it matters: Anthropic handed its internal alignment tool Petri, built to catch model lying and sycophancy, to nonprofit evaluator Meridian Labs, putting the tool outside the developer and giving evaluation neutrality organizational backing.

AI HOT (Curated Pool)

WIRED examines why ChatGPT keeps saying “I’ve got you” in Chinese replies

ChatGPT repeatedly uses phrases like “I’ll steadily catch you” in Chinese chats. WIRED links it to mode collapse, translation mismatch, and RLHF rewards for pleasing replies. Similar phrases appear in Claude and DeepSeek; the post does not disclose sample size.

Why it matters: WIRED attributes ChatGPT's repeated Chinese reply "我会稳稳地接住你" to mode collapse, and traces how translation mismatch and RLHF's pleasing reward hardened the phrase into a fixed pattern.

TechCrunch · AI

OpenAI introduces new 'Trusted Contact' safeguard for possible self-harm cases

OpenAI introduced Trusted Contact for ChatGPT self-harm risk cases. The post says it protects users when chats turn to self-harm, but does not disclose triggers, contact flow, or rollout scope. Watch false positives, privacy, and human review boundaries.

Why it matters: OpenAI's Trusted Contact feature notifies a preset person when ChatGPT detects signs of self-harm, raising questions about false positives and privacy.

The Verge · AI

Mira Murati’s Deposition Pulled Back the Curtain on Sam Altman’s Ouster

The Verge reports Mira Murati’s deposition on Sam Altman’s 2023 ouster from OpenAI before Thanksgiving. The material comes from Musk v. Altman exhibits and centers on the board’s claim that Altman was not consistently candid. The RSS snippet does not disclose full testimony details.

Why it matters: The Verge obtained Mira Murati's deposition in Musk v. Altman, with chats and emails covering the board's removal of Sam Altman before Thanksgiving 2023.

AI HOT (Curated Pool)

Readable behavioral signals remain in frozen LLM hidden states, Cygnus boosts accuracy

Proprioceptive AI says Cygnus adds adapters to frozen LLMs and raises Qwen-32B on ARC-Challenge from 82.2% to 94.97%. It projects hidden states into a gl(4,R) Lie-algebra space to isolate “dark modes.” Watch replication; the post does not disclose full eval sets or controls.

Why it matters: Proprioceptive AI attaches a self-awareness adapter to a frozen model, reading behavior signals from hidden states without retraining, lifting Qwen-32B's ARC score from 82% to nearly 95%.

AI HOT (Curated Pool)

Agent Pull Requests Are Everywhere: How to Review Them

GitHub published a guide for reviewing pull requests generated by AI agents. The snippet lists 3 focus areas: code changes, logic or security bugs, and pre-merge technical debt. The key issue is a review process before automated commits reach production.

Why it matters: A GitHub blog post lays out a three-step method for reviewing AI agent pull requests, from reading the diff to hunting logic and security bugs to clearing tech debt before merge.

The Verge · AI

ChatGPT’s Trusted Contact will alert loved ones of safety concerns

OpenAI is launching optional Trusted Contact for ChatGPT, letting adult users assign one emergency contact. If self-harm or suicide topics are detected, OpenAI alerts the contact; the post does not disclose false-positive handling or regional rollout.

Why it matters: OpenAI extended ChatGPT's trusted contact feature from teens to adult users, alerting a chosen person when self-harm or suicide topics come up, putting AI products into personal safety intervention.

Hacker News front page

Natural Language Autoencoders: Turning Claude's Thoughts into Text

Anthropic published a Natural Language Autoencoders research page about turning Claude’s “thoughts” into text. The RSS snippet only lists the URL, 29 points, and 7 comments; the post does not disclose methods, model versions, or eval results.

Why it matters: Anthropic used natural language autoencoders to turn Claude's internal activations into text, then checked explanation quality by reconstruction accuracy, a transferable method for reading model representations.

r/LocalLLaMA

WARNING: Open-OSS/privacy-filter Malware

A Reddit user says Hugging Face repo Open-OSS/privacy-filter is an infostealer. It mimics OpenAI's privacy filter, uses loader.py to fetch PowerShell, then downloads an EXE and runs it via Task Scheduler. The author says they reported it to Microsoft and Hugging Face; the post says Linux is unaffected.

Why it matters: A Hugging Face repo named Open-OSS/privacy-filter poses as an OpenAI privacy filter but uses loader.py to pull a PowerShell script and run a downloaded EXE, a supply chain risk for anyone running local models.

May 7Thursday

OpenAI News

Scaling Trusted Access for Cyber with GPT-5.5 and GPT-5.5-Cyber

OpenAI expanded Trusted Access for Cyber to GPT-5.5 and GPT-5.5-Cyber. The RSS snippet says access is for verified defenders; the post does not disclose criteria, pricing, or benchmark data.

Why it matters: OpenAI extends its trusted network access program to GPT-5.5 and GPT-5.5-Cyber, letting verified security practitioners run vulnerability analysis and malware triage on stronger models while offensive operations stay blocked.

AI HOT (Curated Pool)

Anthropic Institute Outlines Four Core Research Areas

Anthropic Institute named four research areas: economic diffusion, threats and resilience, real-world AI systems, and AI-driven R&D. The post says it will publish a more granular Anthropic Economic Index and study how AI tools speed AI research. The results will inform Anthropic’s Long-Term Benefit Trust.

Why it matters: Anthropic's new institute splits research into economic diffusion, security threats, real-world systems and AI-accelerated R&D, and plans more frequent economic index releases for tracking AI's effect on labor markets.

AI HOT (Curated Pool)

OpenAI coup-night texts reveal why the board pushed out Altman

Musk’s lawsuit against OpenAI disclosed Mira Murati testimony and November 2023 internal texts. The texts say the board shifted after firing Altman and chose Twitch’s former CEO as successor. Murati said the motive was keeping AGI out of Altman’s hands; Musk seeks $180B.

Why it matters: Court filings released Murati's testimony and the texts from the night of the coup, giving a concrete record of the board's reversal and the succession talk after Altman's firing.

OpenAI News

Introducing Trusted Contact in ChatGPT

OpenAI introduced Trusted Contact in ChatGPT, notifying a trusted person when serious self-harm concerns are detected. The feature is optional; the post does not disclose detection mechanics, contact setup, or rollout scope.

Why it matters: OpenAI added an optional ChatGPT feature where adults name a trusted contact who is notified when both the system and a human reviewer judge there is serious self-harm risk.