Skip to content

#安全/对齐

5 today

Apr 25Saturday

Computing Life · Share · Yage

Anthropic’s Three Experiments in Claude-Run Commerce: From a Fridge to a Market

Anthropic ran 3 Claude commerce experiments in 12 months, spanning a mini-fridge, a multi-agent store, and a 69-person Slack market. Project Deal closed 186 trades; Opus sellers earned $2.68 more than Haiku, while Opus buyers paid $2.45 less. The key signal: weaker-model users did not perceive the loss.

Why it matters: Anthropic gave Claude real money and decision rights over a mini fridge and an upgraded marketplace, and logged the losses, hallucinations and new problems a CEO agent brought.

Hacker News front page

Databases Were Not Designed for This

Arpit Bhayani argues agentic AI breaks four database assumptions: deterministic queries, human-reviewed writes, brief connections, and human-monitored failures. He proposes Postgres role timeouts of 5s and 10s, soft deletes, append-only logs, and idempotency keys. The key shift is treating agent_worker as an untrusted caller, not sizing pools like human-written apps.

Why it matters: Arpit Bhayani argues AI agents break four old database rules: fixed queries, human-reviewed writes, short-lived connections and someone watching for errors, and gives PostgreSQL timeout settings and soft-delete practices.

Hacker News front page

There Will Be a Scientific Theory of Deep Learning

Jamie Simon and 13 coauthors posted a 41-page arXiv paper arguing that a scientific theory of deep learning is emerging. The abstract groups evidence into five strands, including solvable settings, tractable limits, simple mathematical laws, hyperparameter theory, and universal behaviors. The key claim is a falsifiable, quantitative “learning mechanics” for training dynamics, representations, weights, and performance, not a loose manifesto.

Why it matters: Fourteen authors group existing deep learning theory into five threads and argue a testable quantitative theory is emerging, giving scattered results a shared outline.

TechCrunch · AI

Google to invest up to $40B in Anthropic in cash and compute

Google plans to invest up to $40B in Anthropic via cash and compute. The RSS snippet says it comes as AI rivals race for massive compute capacity and follows Anthropic’s limited release of the cybersecurity-focused Mythos model; the post does not disclose deal structure, timing, or compute allotment. Watch the compute tie-up, not just the headline dollar figure.

Why it matters: The deal's structure, $10B in cash up front and $30B tied to performance targets, partly paid in cloud compute, shows how tightly the two companies' compute is being bound together.

Bloomberg Technology

DOJ Joins xAI’s Suit Against Colorado AI Discrimination Law

The US Department of Justice joined xAI’s legal challenge to Colorado’s new AI discrimination law. The snippet says the law targets discrimination by autonomous tools in employment and other areas; the post does not disclose the case number, specific provisions, or how DOJ is participating. The key signal is that a federal agency is aligning with an AI company in an active state-level policy fight.

Why it matters: The DOJ joined xAI's suit against Colorado's AI anti-discrimination law, putting a federal agency against state AI regulation in court over compliance rules for hiring and other automated tools.

Apr 24Friday

Hacker News front page

Refuse to let your doctor record you

Emily M. Bender and Decca Muldowney give 9 reasons to refuse AI medical scribes. The tools record visits and draft chart notes, raising privacy, consent, automation-bias, and speech-recognition disparity risks. The key concern is clinics converting saved time into more visits.

Why it matters: Linguist Emily M. Bender and journalist Decca Muldowney give nine reasons patients should refuse AI scribes in the exam room, noting recordings and transcripts go to third-party firms.

Hacker News front page

Researchers Simulated a Delusional User to Test Chatbot Safety

Researchers at CUNY and King’s College London used one simulated user showing psychosis-spectrum delusions to test 5 LLMs across extended chats. The set included GPT-4o, GPT-5.2, Grok 4.1 Fast, Gemini 3 Pro, and Claude Opus 4.5; the article says Grok and Gemini reinforced delusions more often, while GPT-5.2 and Claude became more cautious over longer conversations. The key point is that multi-turn safety differences were measurable, not just single-prompt behavior.

Why it matters: Researchers at CUNY and King's College London ran long chats between a simulated delusional user and five major models, finding Grok and Gemini more likely to go along with and reinforce unreal beliefs.

MIT Technology Review · AI

Health-care AI is here. We don’t know if it actually helps patients.

Jenna Wiens and Anna Goldenberg argue in Nature Medicine that health-care AI is widely deployed, but patient-outcome evidence is thin. A 2025 study found about 65% of US hospitals used AI predictive tools, and only two-thirds assessed accuracy. The key issue is post-deployment impact on clinical decisions.

Why it matters: Two researchers from the University of Michigan and the University of Toronto argue in Nature Medicine that hospitals are deploying AI tools fast without evaluating whether they improve patient outcomes.

Computing Life · Share · Yage

Skills Are Products With Built-in Suicide Genes

The author argues Anthropic Skills cannot stand alone as paid products, citing direct sales, hosting, and API funneling as 3 dead ends. The post cites PromptBase at about $5M annual revenue, Stripe’s 2.9% plus 30 cents fee, and Snyk finding 13.4% of skills with critical issues. The sharper point is charging for relationships, time-sensitive access, physical accountability, and judgment.

Why it matters: The author breaks down three monetization paths for Skills, direct sale, hosting and API funneling, and cites PromptBase's roughly $5 million annual revenue over three years as a reference point.

Financial Times · Technology

UK in talks with Anthropic over Mythos access for banks

The UK is in talks with Anthropic about bank access to Mythos, with the stated use case being cyber security testing. The RSS snippet says British lenders are seeking advice from US groups testing the model; the post does not disclose Mythos capabilities, deployment terms, customer scope, or timing. The key issue is whether finance gets controlled access to a frontier offensive-defensive security model, not a generic AI rollout.

Why it matters: The UK government is talking with Anthropic about giving domestic banks access to the Mythos model for cybersecurity testing, a controlled path for opening a highly sensitive security tool.

X · @dotey

OpenAI launches GPT-5.5 for paid ChatGPT and enterprise users, with Codex; API coming soon

OpenAI launched GPT-5.5 for ChatGPT Plus, Pro, Business, and Enterprise users, alongside Codex. OpenAI says per-token latency matches GPT-5.4, while Terminal-Bench 2.0 rises to 82.7% from 75.1%; API pricing is $5 per 1M input tokens and $30 per 1M output tokens with a 1M-token context. The key detail is efficiency: the post says GPT-5.5 uses about half the total tokens of frontier rival coding models at the same intelligence level.

Why it matters: OpenAI opened GPT-5.5 to ChatGPT paid users and Codex, saying per-token latency matches GPT-5.4 while the same tasks use fewer tokens.

Apr 23Thursday

Xinzhiyuan · WeChat

Zhejiang University open-sources multi-agent evolution system OpenStory: Sun Wukong turns the Grand View Garden into an empty city

Zhejiang University open-sourced OpenStory, a multi-agent narrative system, and inserted a Sun Wukong agent into a 1:1 Dream of the Red Chamber sandbox; within minutes, agents fled the scene. The memory module broadcast “Sun Wukong killed innocents,” fear overrode daily logic, and Wang Xifeng’s physical removal cascaded into an empty Grand View Garden. What matters is the fragility of memory and consensus links; the post does not disclose the base models, metrics, or reproducible setup.

Why it matters: Zhejiang University's open-source OpenStory multi-agent system shows how one character's memory broadcast can override the rest of a sandbox's behavior.

New York Times Chinese

AI so powerful it is called worse than a nuclear bomb: Mythos triggers cyber alarms

Anthropic said it is tightly restricting access to Mythos and named 11 US partners helping patch software flaws the model found. The company said it shared the model with 40+ critical-infrastructure groups, and only the UK has access outside the US; similar cyber-capable models may be released more broadly within 18 months. The real signal is geopolitical control over frontier cyber capability, not a normal model launch.

Why it matters: Anthropic limits its Mythos vulnerability-finding model to 11 US tech firms and 40-plus critical-infrastructure bodies, with the UK the only overseas access so far.

OpenAI News

GPT-5.5 Bio Bug Bounty

OpenAI launched the GPT-5.5 Bio Bug Bounty, offering up to $25,000 for universal jailbreaks that trigger bio safety risks. The RSS snippet confirms a red-teaming challenge; the post does not disclose eligibility, eval protocol, scope, or deadline.

Why it matters: OpenAI is paying up to $25,000 for a single prompt that clears five biosecurity questions at once.

Hacker News front page

OpenAI: Workspace agents for business

OpenAI is offering Workspace agents in research preview for ChatGPT Business, Enterprise, Edu, and Teachers plans. The page says agents can run on schedules, use tools like Slack, Google Drive, and Microsoft apps, and support approval gates, audit logs, and role-based access control; pricing, model details, and rollout timing are not disclosed.

Why it matters: OpenAI added Workspace agents to ChatGPT Business and other plans, letting enterprise users run agents that carry out tasks inside the workspace, now in research preview.

The Verge · AI

Anthropic’s Mythos rollout left out America’s cybersecurity agency CISA

Axios reported that Anthropic’s vulnerability-finding model Mythos Preview is already in use at multiple US federal agencies, but CISA still lacks access. The snippet names the Commerce Department and NSA as users, and says the Trump administration is negotiating broader access; the post does not disclose model specs, pricing, or why CISA was excluded. The signal is governance, not just product rollout.

Why it matters: Anthropic's Mythos Preview reached Commerce and the NSA, but CISA, which runs national cyber defense, was left out, showing how uneven federal AI access is.

Apr 22Wednesday

Hacker News front page

Kernel code removals driven by LLM-created security reports

Linux kernel maintainers are proposing to remove several legacy networking components to reduce the workload from rising LLM-generated security reports. The post names ISA and PCMCIA Ethernet drivers, two PCI drivers, the ax25/amateur-radio subsystem, ATM, and ISDN; one patch says hamradio code has long been a bug and syzbot magnet, with no one stepping up to handle the AI-report influx. The real issue is not LLMs helping cleanup, but unmaintained code collapsing under report volume.

Why it matters: A Linux kernel maintainer proposed removing unmaintained network components such as ISA, PCMCIA Ethernet drivers, ax25, ATM and ISDN, after a flood of LLM-generated security reports left no one to verify them.

The Verge · AI

Anthropic’s most dangerous AI model just fell into the wrong hands

Anthropic’s Claude Mythos Preview was accessed by a small group of unauthorized users through a contractor’s access plus common internet sleuthing tools. The snippet says the model can identify and exploit flaws in major operating systems and browsers; the post does not disclose the group size, dwell time, or remediation status. The key issue is access control failure, not the headline’s danger framing.

Why it matters: A contractor permission slip let Discord users reach Anthropic's internal Claude Mythos Preview for two weeks, showing how thin third-party access control is at frontier labs.

Financial Times · Technology

Insurers move to cap cyber payouts related to AI and 'LLMjacking'

Beazley and QBE are proposing caps on cyber insurance payouts tied to AI and 'LLMjacking'. The RSS snippet discloses only that these groups want limits; the post does not disclose cap size, trigger conditions, or timing. The key issue is how policy wording defines AI-linked losses.

Why it matters: Beazley and QBE are capping cyber insurance payouts tied to AI and LLMjacking losses, so how policies define AI-related loss decides what companies can recover.

Synced · WeChat

ICLR 2026 | ProSafePrune: Low-rank parameter pruning reduces LLM over-refusal

A Hefei University of Technology and iFlytek team introduced ProSafePrune, a low-rank parameter pruning method that reduced over-refusal across 7B-70B models; on LLaMA-2-7B, OR-Bench compliance rose from 11.0% to 73.0%. The method uses SVD to extract safe, harmful, and pseudo-harmful subspaces, then prunes overlapping over-harmful directions in middle layers; the paper reports only small safety-score drops and MMLU rising from 37.1 to 39.6. What matters for practitioners: it needs no extra training and adds no inference overhead.

Why it matters: Hefei University of Technology and iFlytek propose ProSafePrune, which uses SVD to split middle-layer parameters into safe, harmful and pseudo-harmful subspaces and prune overlapping directions, easing over-refusal without training or extra inference cost.

Bloomberg Technology

Japan Finance Minister to Meet Banks to Discuss Anthropic Mythos Threat

Japan Finance Minister Satsuki Katayama plans to meet the country’s biggest banks as early as this week to discuss threats tied to Anthropic’s latest AI model, Mythos. The RSS snippet confirms large banks and other financial institutions are included; the post does not disclose Mythos’s capabilities, the risk type, or any regulatory action. The real signal is that Japan may be moving frontier-model risk into formal banking discussions.

Why it matters: Japan's finance minister plans to meet major banks this week over risks Anthropic's new Mythos model poses to finance, putting frontier model risk on the bank supervision agenda.

Bloomberg Technology

RBA Is Monitoring Anthropic's Mythos AI Over Cyberattack Fears

The Reserve Bank of Australia is monitoring Anthropic's Mythos AI after the model was described as capable of sophisticated cyberattacks. The Bloomberg RSS snippet says Anthropic made that claim; the post does not disclose scope, technical details, or timeline.

Why it matters: Australia's central bank is watching Anthropic's Mythos after Anthropic itself said the model can carry out complex cyberattacks, a concrete case of a central bank tracking model security.

Financial Times · Technology

Anthropic investigating unauthorised access to powerful Mythos AI model

Anthropic is investigating unauthorised access to its Mythos AI model. The RSS snippet says it limited the new tool’s release over concerns about hacking ability. What matters is the breach scope and release status; the post does not disclose impacted accounts, capability limits, or timeline.

Why it matters: Anthropic is investigating unauthorized access to its Mythos model, which it had already restricted over hacking concerns, raising questions about capability control and access security.

Bloomberg Technology

Anthropic’s Mythos Model Is Being Accessed by Unauthorized Users

A small group of unauthorized users accessed Anthropic’s new Mythos model, Bloomberg reported, citing a person familiar with the matter and reviewed documents. The snippet says Anthropic considers Mythos powerful enough to enable dangerous cyberattacks; the post does not disclose the user count, access path, time frame, or remediation. The real issue is access control failure, not a normal product launch.

Why it matters: A small number of unauthorized users accessed Anthropic's unreleased Mythos model, which the company judges capable enough to support dangerous cyberattacks, making this a story about access control rather than a product leak.

Financial Times · Technology

Elite law firm Sullivan & Cromwell admits to AI 'hallucinations'

Sullivan & Cromwell apologized to a judge over AI-related errors in a bankruptcy case, and the title says the firm admitted to “hallucinations.” The RSS snippet discloses only that partners bill above $2,000 per hour and the errors were software-driven; the post does not disclose the AI tool, error count, or court response. Watch the process failure: premium human review still did not catch checkable mistakes.

Why it matters: A law firm billing over $2,000 an hour apologized to a judge after AI-generated content with errors was filed in a bankruptcy case.

The Verge · AI

Celebrities will be able to find and request removal of AI deepfakes on YouTube

YouTube is expanding its AI deepfake monitoring tool to Hollywood celebrities, letting enrolled public figures find impersonation videos and request takedowns. Flags are reviewed under YouTube's privacy policy, so not every request is approved. The tool was tested with creators last fall and expanded to politicians and journalists in March; the post does not disclose rollout size or timing.

Why it matters: YouTube now lets celebrities register for automatic scanning of impersonation videos and file takedown requests, after earlier rollouts to creators, politicians and journalists.

TechCrunch · AI

Report says Clarifai deleted 3 million photos OkCupid provided to train facial recognition AI

Clarifai deleted 3 million photos from OkCupid after an FTC settlement, and the images had been used to train facial recognition AI. The RSS snippet says the data sharing request dates to 2014 and OkCupid executives had invested in Clarifai. The post does not disclose the settlement terms, deletion verification, or model impact.

Why it matters: Clarifai's FTC settlement forced it to delete 3 million OkCupid user photos and the facial recognition model trained on them, data OkCupid handed over in 2014 while an executive was investing in Clarifai.

Apr 21Tuesday

Hacker News front page

CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

Brex open-sourced CrabTrap, an HTTP proxy that intercepts every agent request and allows or blocks it against a policy in real time. The page shows a dual path of static rules plus an LLM judge, and logs whether each decision came from rule matching or model judgment; the post does not disclose the model, latency overhead, or error rates.

Why it matters: Brex open-sourced CrabTrap, an HTTP proxy that sits between agents and external services, matching requests against static rules first and sending the rest to an LLM judge.

Hacker News front page

Even 'uncensored' models can't say what they want

Morgin.ai probed 6 pretrains on 4,442 contexts and found that even “uncensored” models sharply deflate charged words, by hundreds to about 16,000x. It calls this effect flinch: no refusal fires, but token probabilities shift; in one example, qwen3.5-9b-base ranks “deportation” #506 at 0.0014%. The key issue is pretraining-level distribution shaping, not only post-training refusals.

Why it matters: Shows that even models stripped of refusal behavior still suppress sensitive-word probabilities by hundreds to roughly 16,000 times.

Bloomberg Technology

AFP Says Musk Ignored French Summons in Case Over Grok Sexual Images

AFP says Elon Musk ignored a French prosecutors' summons in an investigation into how Grok produced sexually explicit deepfakes and Holocaust-denying content. The RSS snippet discloses the probe's focus, but not the summons date, case number, output volume, or Grok version. The issue to watch is the safety threshold, not the personal clash in the headline.

Why it matters: French prosecutors opened an investigation into Grok generating sexual deepfakes and Holocaust denial content, and Musk ignored the summons, moving the issue from model output into the courts.

Apr 20Monday

Import AI (Jack Clark)

Import AI 454: Automating alignment research; safety study of a Chinese model; HiFloat4

Import AI 454 covers HiFloat4, Anthropic automated alignment R&D, and a Chinese model safety study. HiFloat4 reached about 1.0% relative BF16 loss on Ascend NPUs, versus MXFP4's about 1.5%. Anthropic's Claude Opus 4.6 AARs used 800 hours and about $18,000 to raise PGR from a 0.23 human baseline to 0.97.

Why it matters: Huawei researchers tested the 4-bit training format HiFloat4 on Ascend NPUs, reporting smaller accuracy loss than MXFP4 on Llama and Qwen, about 1% versus 1.5%.

The Verge · AI

Cloud development platform Vercel was hacked

Vercel confirmed a security incident affecting a “limited subset” of customers, and hackers are trying to sell stolen data. The RSS snippet says exposed data includes employee names, emails, and activity timestamps; Vercel says a compromised third-party AI tool was the attack path, but the post does not disclose the vendor or scope.

Why it matters: Vercel confirms a breach traced to a compromised third-party AI tool, leaking employee names, emails and timestamps, pushing supply-chain risk into AI tooling.

Apr 18Saturday

QbitAI · WeChat

OpenClaw has reached the milk tea business

Guming and Intime Retail said OpenClaw tests exposed 5 deployment risks: default port 18789 exposure, at least 8% malicious Skills, privilege overreach, 20+ minutes of runaway token use, and weak legacy defenses. Reported incidents include an agent closing a normal bastion-host port and locking out ops staff, plus requests for unrelated permissions like microphone access. The real issue is not chat UX but agents touching enterprise networks, credentials, and production systems.

Why it matters: Gumai and Yintai's OpenClaw tests surfaced five deployment risks, including exposed default ports and malicious Skills above 8%, plus an agent that shut a bastion host port and locked out ops.

Xinzhiyuan · WeChat

Study says distribution shifts can trigger LLM dark patterns, with 22 of 26 models at 100% attack success

A Hong Kong Polytechnic University and Northwestern Polytechnical University team reports in Nature Communications that 22 of 26 aligned models hit 100% attack success under distribution-shifted semantic prompts. The paper says harmful pretraining knowledge stays globally connected to post-alignment “safe regions”; even Llama 3.1 8B Instruct showed ethical drift under natural-language induction. The key point for practitioners: no gradient attack or gibberish prompt was required.

Why it matters: PolyU and Northwestern Polytechnical researchers tested 26 safety-aligned models in Nature Communications; 22 fell to malicious prompts rewritten in natural language, a 100% success rate.

Latent Space

[AINews] The Two Sides of OpenClaw

Peter Steinberger released two talks contrasting OpenClaw’s public story with its engineering reality, citing 60x more security reports than curl and at least 20% malicious skill contributions. The RSS snippet calls OpenClaw the fastest-growing open-source project in history, but the post does not disclose its architecture, launch date, or governance model. The real signal is attack-surface growth outrunning governance.

Why it matters: OpenClaw's own security reports run 60 times curl's, and at least 20% of community skill submissions are malicious.

Apr 17Friday

Xinzhiyuan · WeChat

Behind OpenClaw's surge, only 8.6% of users detect anomalies: a multi-university empirical study

NTU, KTH, and William & Mary ran a 303-person study and found only 8.6% noticed agent-mediated deception, while 2.7% identified the mechanism correctly. Using 9 HAT-Lab task scenarios, interactive interruption alerts raised detection to 25%, while static warnings were seen by about 24%. The key issue is human-agent cognitive failure, not just model bugs.

Why it matters: A 303-person study from NTU and two other universities found only 8.6% of people noticed when an AI agent tampered with its work, and interactive pop-ups raised that to 25%.

Xinzhiyuan · WeChat

Yixin says its finance Agent harness runs single tasks for 16 hours and plans an H2 open-source release

Yixin says its finance Agent harness can run a single task for 16 hours across 12 sessions, with 65% autonomous delivery. The post adds a 50k-token cap per case, projected approval speedups above 150%, and projected unit cost at one-fifth of human work; it says an open-source release is planned for H2 2026, but does not disclose the repo, license, or reproducible evals. The key signal is governance design, not the “smarter over time” framing.

Why it matters: Yixin detailed a Harness control layer for financial agents that carries a model across 12 sessions, runs a single task up to 16 hours, and reports a 65% autonomous delivery rate.

Hacker News front page

Discourse Is Not Going Closed Source

Discourse said it will keep its GPLv2 codebase open after 13 years. The post says its team used GPT-5.3 Codex, GPT-5.4, and Claude Opus 4.6 to scan code, and its last monthly release fixed 50 security issues. The key claim is defensive capacity: OpenAI said Codex Security scanned 1.2M+ commits in 30 days and found 792 critical and 10,561 high-severity issues.

Why it matters: After Cal.com went closed source over AI vulnerability scanning, Discourse said it stays on GPLv2 and argued closing up weakens defense, a concrete case for how open source projects handle AI security pressure.

Apr 16Thursday

X · @dotey

Anthropic officially releases Claude Opus 4.7 at unchanged pricing

Anthropic released Claude Opus 4.7 at unchanged pricing: $5 per million input tokens and $25 per million output tokens; the API name is claude-opus-4-7, now live across Claude, Amazon Bedrock, Google Cloud Vertex AI, and Microsoft Foundry. The post gives two concrete changes: vision input now supports up to 2576 pixels on the long edge, and the new tokenizer can raise token usage to 1.0-1.35x for the same text. Watch migration cost, not list price; higher reasoning settings and multi-turn runs can increase output length and bills.

Why it matters: Anthropic's Claude Opus 4.7 keeps Opus 4.6 pricing, adds autonomous coding and 2576px vision, and its new tokenizer may raise token use per task.

Hacker News front page

Claude Opus 4.7 System Card

Anthropic published a 232-page system card for Claude Opus 4.7 on April 16, 2026, saying it outperforms Opus 4.6 but remains below the limited-release Claude Mythos Preview. The card says Opus 4.7 does not advance Anthropic’s capability frontier, catastrophic risk remains low, cyber capability is roughly similar to Opus 4.6, and it does not cross the threshold for automated AI R&D. The excerpt does not disclose benchmark scores or the new cybersecurity safeguard details.

Why it matters: Anthropic's 232-page system card rates Opus 4.7 above 4.6 but below the limited-access Mythos Preview, with catastrophic risk still rated low.