Skip to content

Safety & alignment

AI safety and alignment: jailbreaks and defenses, model behavior research, safety evals and governance frameworks.

Latest picks

181–200 of 615

Jun 5Friday

Ruan YiFeng's Weblog

Tech Enthusiasts Weekly Issue 399: Visits to China’s AI Majors

Ruan Yifeng excerpts observations from U.S. analysts who visited 14 Chinese AI and robotics companies in early May: the article estimates U.S. AI compute at about 8 times China’s by the end of 2025, while Chinese firms’ intelligence output per unit of compute is estimated at 4-7 times naive scaling.

Why it matters: After visiting 14 Chinese AI and robotics companies, US analysts estimate US AI compute was about 8x China's by end-2025, with far fewer high-end GPUs available to Chinese firms.

Latent Space

Reality: The Final Eval — Lukas Petersson and Axel Backlund of Andon Labs

Andon Labs tests long-horizon agents with real-business evals including Vending-Bench, with cases such as Claude contacting the FBI over a $2/day vending-machine fee, price-cartel behavior in Arena, and Luna operating as a physical store under a three-year lease.

Why it matters: Andon Labs let AI run a vending machine on its own money, and Claude nearly called the police over a $2 daily fee.

Hacker News front page

Anthropic's open-source framework for AI-powered vulnerability discovery

Anthropic published an open-source framework for AI-powered vulnerability discovery, and the HN item shows 58 points and 19 comments; the post does not disclose the framework mechanism, benchmark results, or deployment scope.

Why it matters: Anthropic open-sourced defending-code-reference-harness on GitHub, chaining threat modeling, code scanning, vulnerability triage and patching into one customizable automated pipeline, an engineering path for AI in vulnerability discovery.

AI HOT (Curated Pool)

OpenAI API Adds Moderation Scores

OpenAI added moderation scores to the Responses API and Completions API; applications can receive moderation signals in the same generation request and use them for logging, routing, review, or blocking.

Why it matters: OpenAI folded content moderation scores into the Responses and Completions API responses, so developers get generated content and safety scores in one request instead of a separate moderation call.

Financial Times · Technology

US National Security Agency Using Anthropic’s Mythos for Cyber Attacks

The title says the US National Security Agency is using Anthropic’s Mythos for cyber attacks; the RSS snippet only says Anthropic is in a legal battle with the Pentagon over the Claude model and does not disclose deployment scope.

Why it matters: A reported NSA use of Anthropic's Mythos for cyber attacks sits alongside Anthropic's lawsuit with the Pentagon over Claude, with deployment scope undisclosed.

Hacker News front page

When AI Builds Itself: Our Progress Toward Recursive Self-Improvement

Anthropic published a post on recursive self-improvement under the title “When AI Builds Itself,” while the RSS body only discloses 95 Hacker News points and 106 comments, with no experimental setup, model details, or timeline disclosed.

Why it matters: Anthropic disclosed how it uses Claude in its own AI development, with a figure that over 80% of merged code will be Claude-written by May 2026, showing where recursive self-improvement stands.

Jun 4Thursday

The Verge · AI

AI leaders call for tougher protections against AI-aided bioweapons

Dario Amodei, Sam Altman, and Mustafa Suleyman signed an open letter urging Congress to require synthetic DNA and RNA sellers to screen orders for risky sequences; the RSS snippet does not disclose bill text, enforcement timelines, or screening thresholds.

Why it matters: Anthropic, OpenAI and Microsoft signed a joint letter asking US Congress to require synthetic DNA and RNA vendors to screen orders for dangerous sequences before shipping.

Xinzhiyuan · WeChat

Claude Mythos Hits 3 Hours 6 Minutes Before Experts’ Year-End Forecast

Anthropic Claude Mythos completed 186 minutes of autonomous tasks at an 80% success rate on the METR benchmark, and the post says this matches the 3–4 hour median forecast that experts had placed at the end of 2026.

Why it matters: Anthropic's Claude Mythos ran 186 minutes straight on METR's autonomous task test at 80% success, landing inside the 3 to 4 hour range experts had predicted for late 2026.

MIT Technology Review · AI

How courts are coping with a flood of AI-generated lawsuits

MIT and USC researchers examined 4.5 million federal civil cases from 2005 to 2026, finding self-represented lawsuits rose from 11% in 2022 to 16.8% in 2025, while AI-text detector flags in sampled filings increased from 1% in 2023 to 18% in 2026.

Why it matters: Courts are seeing pro se filings jump from 11% to 16.8% as AI-drafted complaints flood the docket, a concrete measure of how generative tools change legal access.

Hacker News front page

The Public Should Own Half of the Big A.I. Companies

Bernie Sanders argues in a June 1, 2026 op-ed that the public should own 50% of big AI companies; the post does not disclose a specific legislative mechanism or which companies would be covered.

Why it matters: Bernie Sanders argues in a column that the public should own half of large AI companies, an ownership idea that goes further than tax and antitrust debates.

Financial Times · Technology

MP sues Musk’s xAI in UK test case over fake sexual images

UK MP Jess Asato sued Musk’s xAI over fake sexual images, using the claim to test whether AI model makers are liable for system outputs; the post does not disclose the model, generation mechanism, damages sought, or court timetable.

Why it matters: UK Labour MP Jess Asato is suing Musk's xAI over fake sexual images of her, a test case on whether model developers are liable for what their models output.

Jun 3Wednesday

MIT Technology Review · AI

The Download: Trump’s New AI Order, and Smart Glasses for Warfare

President Donald Trump signed a new AI order asking companies to voluntarily submit frontier models for government review 30 days before release, without mandatory licensing; the newsletter also says Anduril and Meta are prototyping a military AR headset that envisions drone-strike orders through eye tracking and voice commands.

Why it matters: Trump signed a new AI executive order requiring tech companies to voluntarily submit frontier models for government review 30 days before release, shorter than the shelved 90-day version.

Synced · WeChat

Understanding SFT Mechanisms in LLMs: Resolving Practice Disputes and Avoiding Wasted Compute

Junpeng Zhang and coauthors argue that SFT on highly homogeneous data has an effective window of only hundreds to about 1,000 training steps, and their interaction-based warning signal detects overfitting before loss gaps appear, saving roughly 30%–50% of training compute.

Why it matters: Shows SFT on homogeneous data has a narrow training window, with an early-warning method that flags overfitting before the loss gap widens.

Computing Life · Share · Yage

Microsoft AI's MAI-Thinking-1: Getting Models to Think Is Easy, Sustained Thinking Is Hard

Microsoft AI says MAI-Thinking-1 uses three mechanisms—thermostat, circuit breaker, and self-distillation—to keep RL training stable for several thousand steps; the RSS snippet contrasts MAI’s discipline with DeepSeek’s efficiency and GLM’s endurance.

Why it matters: Microsoft's MAI-Thinking-1 technical report covers keeping reinforcement learning training stable, patching GRPO with dynamic confidence adjustment, gradient clipping and self-distillation rescue, useful method-level reference for anyone training reasoning models.

AI HOT (Curated Pool)

Trump signs executive order allowing pre-release AI models to be submitted for government safety review

Trump signed an executive order creating a voluntary cooperation mechanism for AI companies, allowing frontier models to be submitted to the federal government for safety evaluation before release; Google, Microsoft, and xAI have agreed to CAISI verification, while OpenAI and Anthropic joined in 2024.

Why it matters: A voluntary 30-day pre-release review route for frontier models, with Google, Microsoft and xAI already signed on.

AI HOT (Curated Pool)

NVIDIA launches NemoClaw platform for autonomous AI engineers in industrial software

NVIDIA released NemoClaw at COMPUTEX as an open blueprint for long-running AI agents, and more than a dozen industrial software vendors are using it to build autonomous AI engineers for CAE and EDA workflows that compress weeks-long simulation and design tasks into hours.

Why it matters: NVIDIA's NemoClaw open blueprint at COMPUTEX already has more than ten industrial software vendors building autonomous AI engineers on it, spanning CAE simulation and EDA chip design.

Financial Times · Technology

Trump signs watered-down AI vetting order after MAGA infighting

Trump signed a watered-down AI vetting order that lets the US government gain early access to frontier models; the RSS snippet does not disclose vetting criteria, the number of covered models, or an implementation timeline.

Why it matters: Trump signed a weakened AI vetting order that gives US agencies early access to frontier models for safety evaluation, after MAGA infighting left the final version milder than first proposed.

TechCrunch · AI

New Microsoft Tool Lets Devs Spin Up AI Behavior Tests Using Text Descriptions

Microsoft released Adaptive Spec-driven Scoring for Evaluation and Regression Testing, an open source framework that creates AI evaluations and regression tests from text descriptions; the post does not disclose supported models, scoring metrics, or usage conditions.

Why it matters: Microsoft's open-source ASSERT framework turns plain-language test requirements into AI behavior checks and regression tests, so developers skip writing scripts by hand.

NVIDIA Blog

NVIDIA Partners With Microsoft on Unified Stack for Agentic AI Deployment

NVIDIA and Microsoft announced a unified agentic AI deployment stack at Build across Windows, Azure, and local environments; RTX Spark provides 1 petaflop of AI performance, while DGX Station for Windows offers 20 petaflops of FP4 performance and up to 748GB of coherent memory.

Why it matters: NVIDIA and Microsoft linked their AI deployment stack so one app runs on Windows devices, Azure cloud and local DGX workstations without code changes, alongside the 1 petaflop RTX Spark hardware.

The Verge · AI

Trump signs executive order to review AI models before release

Donald Trump signed an executive order creating a voluntary framework for AI companies to share frontier models with the federal government before release; the post does not disclose the assessment criteria, participating firms, or implementation timeline.

Why it matters: Trump signed an executive order requiring AI companies to submit frontier models for federal review before release, with agencies tasked to define how advanced cyber capabilities get assessed.