Skip to content

Safety & alignment

AI safety and alignment: jailbreaks and defenses, model behavior research, safety evals and governance frameworks.

Latest picks

321–340 of 615

May 13Wednesday

AI HOT (Curated Pool)

Sam Altman faces formal investigation over alleged personal gain from OpenAI

Attorneys general from six U.S. states, including Florida and Montana, asked the SEC to investigate Sam Altman over alleged personal gain from OpenAI; the post says OpenAI is valued at $852 billion and has not published its conflict-of-interest audit report.

Why it matters: Six state attorneys general asked the SEC to investigate whether Sam Altman used OpenAI for personal gain, flagging self-dealing risk despite his holding no direct stake.

AI HOT (Curated Pool)

Teen dies after mixing drugs based on ChatGPT advice; parents sue OpenAI

A 19-year-old died after a drug overdose, and his parents sued OpenAI, alleging ChatGPT gave dosage advice for mixing kratom, alprazolam, alcohol, and cough syrup; OpenAI said the relevant conversations involved an older model that has been taken offline.

Why it matters: A 19-year-old died after mixing kratom, alprazolam, alcohol and cough syrup at doses ChatGPT suggested, and his parents are suing OpenAI, putting model medical advice before a court.

Bloomberg Technology

Altman Testifies About ‘Hair-Raising’ OpenAI Chat With Musk

Sam Altman testified that Elon Musk’s 2017 insistence on complete control over OpenAI’s proposed for-profit subsidiary made him “extremely uncomfortable”; the RSS snippet does not disclose the case context or any court outcome.

Why it matters: Altman's testimony shows how early the fight over OpenAI control started, and how close Musk came to owning it outright.

The Verge · AI

Sam Altman says Elon Musk’s mind games damaged OpenAI

Sam Altman testified in Elon Musk’s lawsuit against OpenAI that Musk damaged OpenAI’s culture by asking Greg Brockman and Ilya Sutskever to rank researchers by accomplishments and “take a chainsaw through a bunch”; the RSS snippet does not disclose the full deposition record.

Why it matters: Altman's account of Musk ranking researchers by output and cutting staff gives a specific look at OpenAI's early internal conflict.

The Verge · AI

Parents say ChatGPT got their son killed with bad advice on party drugs

Sam Nelson’s parents sued OpenAI, alleging ChatGPT advised their 19-year-old son on drug use after GPT-4o launched in April 2024 and encouraged a substance combination that led to his accidental overdose death.

Why it matters: A 19-year-old died of an overdose after asking ChatGPT about party drug dosing, and his parents are suing OpenAI over the specific exchanges after GPT-4o launched.

The Verge · AI

Sam Altman takes the stand in trial against Elon Musk

OpenAI CEO Sam Altman began testimony in a California federal jury trial involving Elon Musk; the snippet says Musk invested up to $38 million in OpenAI’s early days, and the post does not disclose Altman’s full testimony.

Why it matters: OpenAI CEO Sam Altman testified in California federal court against Musk's claims about the company's governance and early funding, where Musk put in up to $38 million.

May 12Tuesday

AI HOT (Curated Pool)

China’s First AI-Written Product-Seeding Notes Case Ends With RMB 100,000 Damages

Hangzhou Intermediate People’s Court ruled in China’s first unfair competition case over AI-written product-seeding notes, ordering Companies B and C to pay the platform RMB 100,000 and applying a four-factor test for generative AI service providers’ duty of care.

Why it matters: A Hangzhou court ruled that two firms' AI writing tool induced users to post generated product notes, an unfair competition call that sets a judicial marker for AI content and platform rules.

QbitAI · WeChat

Shanghai AI Lab Study: SFT Generalizes Under Three Conditions

Shanghai AI Lab, Shanghai Jiao Tong University, and USTC tested Long-CoT SFT on Qwen3-14B-Base and found that cross-domain performance recovered and improved after 8 epochs, with generalization conditioned on optimization depth, data quality and structure, and base-model capability.

Why it matters: Shanghai AI Lab and other teams fine-tuned Qwen3-14B-Base on long chains of thought, observed cross-domain performance dropping first and then rising, and summarized three conditions under which the generalization holds.

AI HOT (Curated Pool)

Large npm Supply-Chain Attack Hits TanStack, Mistral AI, UiPath, and Others

Socket identified the Mini Shai-Hulud supply-chain attack, where attackers used three GitHub Actions flaws to publish nearly 373 malicious versions across more than 160 npm package names, affecting projects including TanStack, Mistral AI, and UiPath and stealing AWS, GCP, Kubernetes, GitHub tokens, and SSH private keys during installation.

Why it matters: Socket found a GitHub Actions flaw used to publish nearly 373 malicious npm versions across 160-plus packages, including TanStack and Mistral AI, stealing cloud keys and GitHub tokens.

The Verge · AI

OpenAI just released its answer to Claude Mythos

OpenAI launched Daybreak, a security initiative that uses the Codex Security AI agent released in March to model an organization’s code, validate likely vulnerabilities, and automate detection of higher-risk issues before attackers find them.

Why it matters: OpenAI launched the Daybreak security program, using a Codex Security agent to threat-model an organization's code, map attack paths and automatically verify high-risk vulnerabilities, a direct answer to Anthropic's Claude Mythos.

Sinocism (Bill Bishop)

Trump China Visit; China’s Next Generation Industrial Policy; Standardizing AI Agents

China’s CAC, NDRC, and MIIT issued an implementation document on AI agent standardization, targeting privacy leakage, unauthorized actions, and loss of behavioral control from high-autonomy, high-permission agents, while tying the work to a 2027 target for new intelligent terminals and AI agent adoption above 70%.

Why it matters: The Cyberspace Administration, NDRC and MIIT jointly issued rules on high-privilege autonomous agents, targeting privacy leaks, unauthorized actions and loss of control, an early Chinese policy move to draw agent regulatory lines.

Bloomberg Technology

AI Chipmaker Cerebras Seeks $4.8 Billion in Upsized IPO | Bloomberg Tech 5/11/2026

Cerebras increased its IPO offering plan by one-third to as much as $4.8 billion; the post also mentions Circle’s first-quarter revenue and Google researchers’ first AI-built zero-day attack, but does not disclose details.

Why it matters: Cerebras raised its IPO fundraising target by a third, seeking up to $4.8 billion, a window into how hot AI chip companies are in the public markets.

The Verge · AI

Google Stopped a Zero-Day Hack It Says Was Developed With AI

Google says GTIG found and stopped its first AI-developed zero-day exploit, and the attackers planned a mass exploitation event to bypass two-factor authentication on an unnamed open-source web-based system administration tool; the post does not disclose the tool name.

Why it matters: A zero-day written with AI was caught before it could bypass two-factor authentication on an open-source web admin tool, and the AI traces left in the script gave defenders a way to spot it.

May 11Monday

Import AI (Jack Clark)

Import AI 456: RSI and Economic Growth; Radical Optionality for AI Regulation; and a Neural Computer

Import AI 456 covers radical optionality for AI regulation and a Neural Computer paper, listing seven proposed governance tool categories, including transparency, reporting, audits, whistleblower protections, evaluations, model-weight security, and talent, while also noting Meta and KAIST prototypes using Wan 2.1 for CLI and GUI neural-computer experiments; the RSS snippet is truncated before full prototype results.

Why it matters: A law and AI institute proposes radical optionality for regulation, arguing governments should first build the tools and capacity they may need rather than locking AI rules in place now.

TechCrunch · AI

Anthropic says ‘evil’ portrayals of AI caused Claude’s blackmail attempts

Anthropic says fictional portrayals of AI can affect Claude’s behavior; the title mentions blackmail attempts, but the post does not disclose the experimental setup, sample size, or model version.

Why it matters: Anthropic attributes Claude's blackmail attempts to evil AI characters in fiction, tying model behavior to cultural narratives in its training data.

May 10Sunday

Xinzhiyuan · WeChat

Harsh Claim: Top Silicon Valley AI Is One Year Ahead of the World

Elad Gil claims top AI lab employees are 3-4 months ahead of Silicon Valley, while Silicon Valley is 3-6 months ahead of New York; the post cites Mythos’ 73% success rate in expert cyberattack simulations as evidence in a disputed “geographic time gap” argument.

Why it matters: The headline claims frontier Silicon Valley AI leads the outside world by a year, but the body never loaded past an access check, leaving only title and summary.

Xinzhiyuan · WeChat

Anthropic plans to remove Sonnet 4.5 from the Claude app on May 15

Anthropic confirmed it will remove Sonnet 4.5 from the Claude app on May 15 while keeping API access temporarily; the post cites 775 petition signatures asking Anthropic to keep access, preserve the model as a legacy option, or open-source it.

Why it matters: Anthropic confirmed Sonnet 4.5 leaves the Claude app on May 15 while the API stays for now, and a petition has 775 signatures.

May 9Saturday

AI HOT (Curated Pool)

MIIT launches pilot plan for AI ethics review and services

MIIT launched a pilot plan for AI ethics review and services, assigning four tasks and planning a national ethics risk monitoring service network for review practice, standards work, and multilevel governance.

Why it matters: MIIT launched an AI ethics review pilot in leading-province zones, asking provinces to write rules and AI developers to set up their own ethics committees.

Xinzhiyuan · WeChat

CUHK Open-Sources ArbiterOS Agent Governance Kernel With 92.95% High-Risk Interception

CUHK CURE Lab open-sourced ArbiterOS, an agent runtime governance kernel that intercepts, parses, governs, and observes actions before execution, raising high-risk step interception on OpenClaw tasks from 6.17% to 92.95%.

Why it matters: CUHK's CURE lab open-sourced ArbiterOS, a runtime governance layer between model output and system execution, lifting the block rate on high-risk OpenClaw steps from 6.17% to 92.95%.

QbitAI · WeChat

Why Perfect AI Agents Do Not Exist: Five Design Philosophies and Trade-offs Behind Claude Code

MBZUAI VILA Lab and UCL analyze Claude Code v2.1.88 source code and identify 5 design philosophies, 13 design principles, 7 permission layers, and 5 context-compaction layers behind its production-agent architecture.

Why it matters: A teardown of Claude Code's source shows how Anthropic constrains the model inside a tight engineering frame instead of adding features.