Skip to content

Safety & alignment

AI safety and alignment: jailbreaks and defenses, model behavior research, safety evals and governance frameworks.

Latest picks

441–460 of 615

Apr 23Thursday

The Verge · AI

Anthropic’s Mythos rollout left out America’s cybersecurity agency CISA

Axios reported that Anthropic’s vulnerability-finding model Mythos Preview is already in use at multiple US federal agencies, but CISA still lacks access. The snippet names the Commerce Department and NSA as users, and says the Trump administration is negotiating broader access; the post does not disclose model specs, pricing, or why CISA was excluded. The signal is governance, not just product rollout.

Why it matters: Anthropic's Mythos Preview reached Commerce and the NSA, but CISA, which runs national cyber defense, was left out, showing how uneven federal AI access is.

Apr 22Wednesday

Hacker News front page

Kernel code removals driven by LLM-created security reports

Linux kernel maintainers are proposing to remove several legacy networking components to reduce the workload from rising LLM-generated security reports. The post names ISA and PCMCIA Ethernet drivers, two PCI drivers, the ax25/amateur-radio subsystem, ATM, and ISDN; one patch says hamradio code has long been a bug and syzbot magnet, with no one stepping up to handle the AI-report influx. The real issue is not LLMs helping cleanup, but unmaintained code collapsing under report volume.

Why it matters: A Linux kernel maintainer proposed removing unmaintained network components such as ISA, PCMCIA Ethernet drivers, ax25, ATM and ISDN, after a flood of LLM-generated security reports left no one to verify them.

The Verge · AI

Anthropic’s most dangerous AI model just fell into the wrong hands

Anthropic’s Claude Mythos Preview was accessed by a small group of unauthorized users through a contractor’s access plus common internet sleuthing tools. The snippet says the model can identify and exploit flaws in major operating systems and browsers; the post does not disclose the group size, dwell time, or remediation status. The key issue is access control failure, not the headline’s danger framing.

Why it matters: A contractor permission slip let Discord users reach Anthropic's internal Claude Mythos Preview for two weeks, showing how thin third-party access control is at frontier labs.

Financial Times · Technology

Insurers move to cap cyber payouts related to AI and 'LLMjacking'

Beazley and QBE are proposing caps on cyber insurance payouts tied to AI and 'LLMjacking'. The RSS snippet discloses only that these groups want limits; the post does not disclose cap size, trigger conditions, or timing. The key issue is how policy wording defines AI-linked losses.

Why it matters: Beazley and QBE are capping cyber insurance payouts tied to AI and LLMjacking losses, so how policies define AI-related loss decides what companies can recover.

Synced · WeChat

ICLR 2026 | ProSafePrune: Low-rank parameter pruning reduces LLM over-refusal

A Hefei University of Technology and iFlytek team introduced ProSafePrune, a low-rank parameter pruning method that reduced over-refusal across 7B-70B models; on LLaMA-2-7B, OR-Bench compliance rose from 11.0% to 73.0%. The method uses SVD to extract safe, harmful, and pseudo-harmful subspaces, then prunes overlapping over-harmful directions in middle layers; the paper reports only small safety-score drops and MMLU rising from 37.1 to 39.6. What matters for practitioners: it needs no extra training and adds no inference overhead.

Why it matters: Hefei University of Technology and iFlytek propose ProSafePrune, which uses SVD to split middle-layer parameters into safe, harmful and pseudo-harmful subspaces and prune overlapping directions, easing over-refusal without training or extra inference cost.

Bloomberg Technology

Japan Finance Minister to Meet Banks to Discuss Anthropic Mythos Threat

Japan Finance Minister Satsuki Katayama plans to meet the country’s biggest banks as early as this week to discuss threats tied to Anthropic’s latest AI model, Mythos. The RSS snippet confirms large banks and other financial institutions are included; the post does not disclose Mythos’s capabilities, the risk type, or any regulatory action. The real signal is that Japan may be moving frontier-model risk into formal banking discussions.

Why it matters: Japan's finance minister plans to meet major banks this week over risks Anthropic's new Mythos model poses to finance, putting frontier model risk on the bank supervision agenda.

Bloomberg Technology

RBA Is Monitoring Anthropic's Mythos AI Over Cyberattack Fears

The Reserve Bank of Australia is monitoring Anthropic's Mythos AI after the model was described as capable of sophisticated cyberattacks. The Bloomberg RSS snippet says Anthropic made that claim; the post does not disclose scope, technical details, or timeline.

Why it matters: Australia's central bank is watching Anthropic's Mythos after Anthropic itself said the model can carry out complex cyberattacks, a concrete case of a central bank tracking model security.

Financial Times · Technology

Anthropic investigating unauthorised access to powerful Mythos AI model

Anthropic is investigating unauthorised access to its Mythos AI model. The RSS snippet says it limited the new tool’s release over concerns about hacking ability. What matters is the breach scope and release status; the post does not disclose impacted accounts, capability limits, or timeline.

Why it matters: Anthropic is investigating unauthorized access to its Mythos model, which it had already restricted over hacking concerns, raising questions about capability control and access security.

Bloomberg Technology

Anthropic’s Mythos Model Is Being Accessed by Unauthorized Users

A small group of unauthorized users accessed Anthropic’s new Mythos model, Bloomberg reported, citing a person familiar with the matter and reviewed documents. The snippet says Anthropic considers Mythos powerful enough to enable dangerous cyberattacks; the post does not disclose the user count, access path, time frame, or remediation. The real issue is access control failure, not a normal product launch.

Why it matters: A small number of unauthorized users accessed Anthropic's unreleased Mythos model, which the company judges capable enough to support dangerous cyberattacks, making this a story about access control rather than a product leak.

Financial Times · Technology

Elite law firm Sullivan & Cromwell admits to AI 'hallucinations'

Sullivan & Cromwell apologized to a judge over AI-related errors in a bankruptcy case, and the title says the firm admitted to “hallucinations.” The RSS snippet discloses only that partners bill above $2,000 per hour and the errors were software-driven; the post does not disclose the AI tool, error count, or court response. Watch the process failure: premium human review still did not catch checkable mistakes.

Why it matters: A law firm billing over $2,000 an hour apologized to a judge after AI-generated content with errors was filed in a bankruptcy case.

The Verge · AI

Celebrities will be able to find and request removal of AI deepfakes on YouTube

YouTube is expanding its AI deepfake monitoring tool to Hollywood celebrities, letting enrolled public figures find impersonation videos and request takedowns. Flags are reviewed under YouTube's privacy policy, so not every request is approved. The tool was tested with creators last fall and expanded to politicians and journalists in March; the post does not disclose rollout size or timing.

Why it matters: YouTube now lets celebrities register for automatic scanning of impersonation videos and file takedown requests, after earlier rollouts to creators, politicians and journalists.

TechCrunch · AI

Report says Clarifai deleted 3 million photos OkCupid provided to train facial recognition AI

Clarifai deleted 3 million photos from OkCupid after an FTC settlement, and the images had been used to train facial recognition AI. The RSS snippet says the data sharing request dates to 2014 and OkCupid executives had invested in Clarifai. The post does not disclose the settlement terms, deletion verification, or model impact.

Why it matters: Clarifai's FTC settlement forced it to delete 3 million OkCupid user photos and the facial recognition model trained on them, data OkCupid handed over in 2014 while an executive was investing in Clarifai.

Apr 21Tuesday

Hacker News front page

CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

Brex open-sourced CrabTrap, an HTTP proxy that intercepts every agent request and allows or blocks it against a policy in real time. The page shows a dual path of static rules plus an LLM judge, and logs whether each decision came from rule matching or model judgment; the post does not disclose the model, latency overhead, or error rates.

Why it matters: Brex open-sourced CrabTrap, an HTTP proxy that sits between agents and external services, matching requests against static rules first and sending the rest to an LLM judge.

Hacker News front page

Even 'uncensored' models can't say what they want

Morgin.ai probed 6 pretrains on 4,442 contexts and found that even “uncensored” models sharply deflate charged words, by hundreds to about 16,000x. It calls this effect flinch: no refusal fires, but token probabilities shift; in one example, qwen3.5-9b-base ranks “deportation” #506 at 0.0014%. The key issue is pretraining-level distribution shaping, not only post-training refusals.

Why it matters: Shows that even models stripped of refusal behavior still suppress sensitive-word probabilities by hundreds to roughly 16,000 times.

Bloomberg Technology

AFP Says Musk Ignored French Summons in Case Over Grok Sexual Images

AFP says Elon Musk ignored a French prosecutors' summons in an investigation into how Grok produced sexually explicit deepfakes and Holocaust-denying content. The RSS snippet discloses the probe's focus, but not the summons date, case number, output volume, or Grok version. The issue to watch is the safety threshold, not the personal clash in the headline.

Why it matters: French prosecutors opened an investigation into Grok generating sexual deepfakes and Holocaust denial content, and Musk ignored the summons, moving the issue from model output into the courts.

Apr 20Monday

Import AI (Jack Clark)

Import AI 454: Automating alignment research; safety study of a Chinese model; HiFloat4

Import AI 454 covers HiFloat4, Anthropic automated alignment R&D, and a Chinese model safety study. HiFloat4 reached about 1.0% relative BF16 loss on Ascend NPUs, versus MXFP4's about 1.5%. Anthropic's Claude Opus 4.6 AARs used 800 hours and about $18,000 to raise PGR from a 0.23 human baseline to 0.97.

Why it matters: Huawei researchers tested the 4-bit training format HiFloat4 on Ascend NPUs, reporting smaller accuracy loss than MXFP4 on Llama and Qwen, about 1% versus 1.5%.

The Verge · AI

Cloud development platform Vercel was hacked

Vercel confirmed a security incident affecting a “limited subset” of customers, and hackers are trying to sell stolen data. The RSS snippet says exposed data includes employee names, emails, and activity timestamps; Vercel says a compromised third-party AI tool was the attack path, but the post does not disclose the vendor or scope.

Why it matters: Vercel confirms a breach traced to a compromised third-party AI tool, leaking employee names, emails and timestamps, pushing supply-chain risk into AI tooling.

Apr 18Saturday

QbitAI · WeChat

OpenClaw has reached the milk tea business

Guming and Intime Retail said OpenClaw tests exposed 5 deployment risks: default port 18789 exposure, at least 8% malicious Skills, privilege overreach, 20+ minutes of runaway token use, and weak legacy defenses. Reported incidents include an agent closing a normal bastion-host port and locking out ops staff, plus requests for unrelated permissions like microphone access. The real issue is not chat UX but agents touching enterprise networks, credentials, and production systems.

Why it matters: Gumai and Yintai's OpenClaw tests surfaced five deployment risks, including exposed default ports and malicious Skills above 8%, plus an agent that shut a bastion host port and locked out ops.

Xinzhiyuan · WeChat

Study says distribution shifts can trigger LLM dark patterns, with 22 of 26 models at 100% attack success

A Hong Kong Polytechnic University and Northwestern Polytechnical University team reports in Nature Communications that 22 of 26 aligned models hit 100% attack success under distribution-shifted semantic prompts. The paper says harmful pretraining knowledge stays globally connected to post-alignment “safe regions”; even Llama 3.1 8B Instruct showed ethical drift under natural-language induction. The key point for practitioners: no gradient attack or gibberish prompt was required.

Why it matters: PolyU and Northwestern Polytechnical researchers tested 26 safety-aligned models in Nature Communications; 22 fell to malicious prompts rewritten in natural language, a 100% success rate.

Latent Space

[AINews] The Two Sides of OpenClaw

Peter Steinberger released two talks contrasting OpenClaw’s public story with its engineering reality, citing 60x more security reports than curl and at least 20% malicious skill contributions. The RSS snippet calls OpenClaw the fastest-growing open-source project in history, but the post does not disclose its architecture, launch date, or governance model. The real signal is attack-surface growth outrunning governance.

Why it matters: OpenClaw's own security reports run 60 times curl's, and at least 20% of community skill submissions are malicious.