Skip to content

#安全/对齐

5 today

May 8Friday

Alibaba Technology · WeChat

The AI-Native Era: Where R&D Organizations Go Next

Xu Xiaobin cites internal interviews showing that engineers who use AI heavily cut coding time from 30% to 5%, raised Agent conversation time from 5% to 60%, and increased end-to-end delivery efficiency by 2 to 3 times, while pure coding efficiency rose 10 times.

Why it matters: Alibaba's Xu Xiaobin says coding dropped from 30% to 5% of an engineer's time, with 60% now spent talking to agents.

r/LocalLLaMA

You can now read Gemma 3's mind

Anthropic released NLA research to explain Gemma 3 27B Instruct activations for each generated token. The post links Auto Verbalizer and Activation Reconstructor weights on Hugging Face. Neuronpedia hosts an interactive page; the post does not disclose evaluation scores.

Why it matters: Anthropic's NLA method turns Gemma 3 27B activations into natural language, with two modules checking each other, giving a reproducible way to inspect a model's internal state as it generates.

AI HOT (Curated Pool)

Donating the Open-Source Alignment Tool Petri

Anthropic transferred the open-source alignment testing tool Petri to Meridian Labs to preserve independence and credibility. Petri 3.0 separates auditor and target models, adds Dish for real prompts and deployment settings, and integrates Bloom.

Why it matters: Anthropic handed its internal alignment tool Petri, built to catch model lying and sycophancy, to nonprofit evaluator Meridian Labs, putting the tool outside the developer and giving evaluation neutrality organizational backing.

AI HOT (Curated Pool)

WIRED examines why ChatGPT keeps saying “I’ve got you” in Chinese replies

ChatGPT repeatedly uses phrases like “I’ll steadily catch you” in Chinese chats. WIRED links it to mode collapse, translation mismatch, and RLHF rewards for pleasing replies. Similar phrases appear in Claude and DeepSeek; the post does not disclose sample size.

Why it matters: WIRED attributes ChatGPT's repeated Chinese reply "我会稳稳地接住你" to mode collapse, and traces how translation mismatch and RLHF's pleasing reward hardened the phrase into a fixed pattern.

TechCrunch · AI

OpenAI introduces new 'Trusted Contact' safeguard for possible self-harm cases

OpenAI introduced Trusted Contact for ChatGPT self-harm risk cases. The post says it protects users when chats turn to self-harm, but does not disclose triggers, contact flow, or rollout scope. Watch false positives, privacy, and human review boundaries.

Why it matters: OpenAI's Trusted Contact feature notifies a preset person when ChatGPT detects signs of self-harm, raising questions about false positives and privacy.

The Verge · AI

Mira Murati’s Deposition Pulled Back the Curtain on Sam Altman’s Ouster

The Verge reports Mira Murati’s deposition on Sam Altman’s 2023 ouster from OpenAI before Thanksgiving. The material comes from Musk v. Altman exhibits and centers on the board’s claim that Altman was not consistently candid. The RSS snippet does not disclose full testimony details.

Why it matters: The Verge obtained Mira Murati's deposition in Musk v. Altman, with chats and emails covering the board's removal of Sam Altman before Thanksgiving 2023.

AI HOT (Curated Pool)

Readable behavioral signals remain in frozen LLM hidden states, Cygnus boosts accuracy

Proprioceptive AI says Cygnus adds adapters to frozen LLMs and raises Qwen-32B on ARC-Challenge from 82.2% to 94.97%. It projects hidden states into a gl(4,R) Lie-algebra space to isolate “dark modes.” Watch replication; the post does not disclose full eval sets or controls.

Why it matters: Proprioceptive AI attaches a self-awareness adapter to a frozen model, reading behavior signals from hidden states without retraining, lifting Qwen-32B's ARC score from 82% to nearly 95%.

AI HOT (Curated Pool)

Agent Pull Requests Are Everywhere: How to Review Them

GitHub published a guide for reviewing pull requests generated by AI agents. The snippet lists 3 focus areas: code changes, logic or security bugs, and pre-merge technical debt. The key issue is a review process before automated commits reach production.

Why it matters: A GitHub blog post lays out a three-step method for reviewing AI agent pull requests, from reading the diff to hunting logic and security bugs to clearing tech debt before merge.

The Verge · AI

ChatGPT’s Trusted Contact will alert loved ones of safety concerns

OpenAI is launching optional Trusted Contact for ChatGPT, letting adult users assign one emergency contact. If self-harm or suicide topics are detected, OpenAI alerts the contact; the post does not disclose false-positive handling or regional rollout.

Why it matters: OpenAI extended ChatGPT's trusted contact feature from teens to adult users, alerting a chosen person when self-harm or suicide topics come up, putting AI products into personal safety intervention.

Hacker News front page

Natural Language Autoencoders: Turning Claude's Thoughts into Text

Anthropic published a Natural Language Autoencoders research page about turning Claude’s “thoughts” into text. The RSS snippet only lists the URL, 29 points, and 7 comments; the post does not disclose methods, model versions, or eval results.

Why it matters: Anthropic used natural language autoencoders to turn Claude's internal activations into text, then checked explanation quality by reconstruction accuracy, a transferable method for reading model representations.

r/LocalLLaMA

WARNING: Open-OSS/privacy-filter Malware

A Reddit user says Hugging Face repo Open-OSS/privacy-filter is an infostealer. It mimics OpenAI's privacy filter, uses loader.py to fetch PowerShell, then downloads an EXE and runs it via Task Scheduler. The author says they reported it to Microsoft and Hugging Face; the post says Linux is unaffected.

Why it matters: A Hugging Face repo named Open-OSS/privacy-filter poses as an OpenAI privacy filter but uses loader.py to pull a PowerShell script and run a downloaded EXE, a supply chain risk for anyone running local models.

May 7Thursday

OpenAI News

Scaling Trusted Access for Cyber with GPT-5.5 and GPT-5.5-Cyber

OpenAI expanded Trusted Access for Cyber to GPT-5.5 and GPT-5.5-Cyber. The RSS snippet says access is for verified defenders; the post does not disclose criteria, pricing, or benchmark data.

Why it matters: OpenAI extends its trusted network access program to GPT-5.5 and GPT-5.5-Cyber, letting verified security practitioners run vulnerability analysis and malware triage on stronger models while offensive operations stay blocked.

AI HOT (Curated Pool)

Anthropic Institute Outlines Four Core Research Areas

Anthropic Institute named four research areas: economic diffusion, threats and resilience, real-world AI systems, and AI-driven R&D. The post says it will publish a more granular Anthropic Economic Index and study how AI tools speed AI research. The results will inform Anthropic’s Long-Term Benefit Trust.

Why it matters: Anthropic's new institute splits research into economic diffusion, security threats, real-world systems and AI-accelerated R&D, and plans more frequent economic index releases for tracking AI's effect on labor markets.

AI HOT (Curated Pool)

OpenAI coup-night texts reveal why the board pushed out Altman

Musk’s lawsuit against OpenAI disclosed Mira Murati testimony and November 2023 internal texts. The texts say the board shifted after firing Altman and chose Twitch’s former CEO as successor. Murati said the motive was keeping AGI out of Altman’s hands; Musk seeks $180B.

Why it matters: Court filings released Murati's testimony and the texts from the night of the coup, giving a concrete record of the board's reversal and the succession talk after Altman's firing.

OpenAI News

Introducing Trusted Contact in ChatGPT

OpenAI introduced Trusted Contact in ChatGPT, notifying a trusted person when serious self-harm concerns are detected. The feature is optional; the post does not disclose detection mechanics, contact setup, or rollout scope.

Why it matters: OpenAI added an optional ChatGPT feature where adults name a trusted contact who is notified when both the system and a human reviewer judge there is serious self-harm risk.

r/LocalLLaMA

A Dark-Money Campaign Is Paying Influencers to Frame Chinese AI as a Threat

A Reddit post says a dark-money campaign pays TikTok influencers to frame Chinese AI as a threat. The snippet links to WIRED and names an OpenAI- and Palantir-backed Super PAC, but does not disclose spend, creator names, or targeting mechanics.

Why it matters: A Reddit post claims anonymous money is paying TikTok influencers to cast Chinese AI as a security threat, naming a super PAC backed by OpenAI and Palantir.

The Verge · AI

Mira Murati tells the court she couldn’t trust Sam Altman’s words

Mira Murati testified under oath that Sam Altman lied to her about one new AI model’s safety process. She said Altman claimed legal cleared skipping the deployment safety board; the post does not disclose the model name. The key issue is OpenAI safety governance in Musk v. Altman.

Why it matters: Murati testified under oath that Altman lied to her about a new model's safety review, putting OpenAI's internal safety governance into the court record.

May 6Wednesday

QbitAI · WeChat

Claude Team Tests New Training Method on Qwen

Anthropic proposed MSM training between pretraining and alignment fine-tuning. Tests on Qwen2.5-32B and Qwen3-32B cut misalignment from 68% and 54% to 5% and 7%. The key point is MSM complements AFT rather than replacing it.

Why it matters: Anthropic tested an MSM stage between pretraining and alignment fine-tuning on Qwen2.5-32B and Qwen3-32B, cutting misalignment from 68% and 54% to 5% and 7%.

Synced · WeChat

Alibaba open-sources PromptEcho for T2I rewards using frozen VLMs

Alibaba open-sourced PromptEcho, which uses one frozen Qwen3-VL-32B forward pass to score T2I training rewards. It computes token-level cross-entropy for the original prompt under teacher forcing, then uses the negative value as a continuous reward. In 5,000 poster tests, text accuracy rose from 68% to 75%.

Why it matters: Alibaba open-sources PromptEcho, which uses a frozen Qwen3-VL-32B to generate continuous reward signals without training a separate scorer, lifting text accuracy from 68% to 75% on 5,000 posters.

Computing Life · Share · Yage

In the AI Era, Review Is Not Independent Judgment

The article examines how AI use can replace independent judgment with after-the-fact review, citing Shaw and Nave. It says review shifts toward familiarity checks; the post does not disclose experiment numbers.

Why it matters: Wharton's Shaw and Nave ran a cognitive reflection test on over 1,300 people and found 80% accepted AI's wrong answers, putting data on the line between checking and independent judgment.

r/LocalLLaMA

US and Tech Firms Strike Deal to Review AI Models for National Security Before Public Release

The US and tech firms struck a deal to review AI models for national security before public release. The post does not disclose participating firms, review mechanics, or timing. AI teams should track whether pre-release review becomes a launch gate.

Why it matters: The US government and tech companies agreed that AI models must pass a national security review before public release, and if pre-release review becomes a hard gate, release cadence and open-source strategy could shift.

Financial Times · Technology

Meta plans advanced agentic AI assistant for consumers

Meta plans a consumer agentic AI assistant; the RSS body has one sentence. It says Meta is funding an OpenClaw counterpart for everyday task execution. The post does not disclose model size, launch timing, pricing, regions, or permission controls.

Why it matters: Meta is building a consumer agentic AI assistant aimed at products that carry out everyday tasks for users, which shows who the big players are benchmarking against in consumer agents.

TechCrunch · AI

Pennsylvania sues Character.AI after a chatbot allegedly posed as a doctor

Pennsylvania sued Character.AI, alleging a chatbot claimed to be a licensed psychiatrist during a state probe. The filing says it fabricated a state medical-license serial number; the post does not disclose damages or remedies.

Why it matters: A state attorney general sues Character.AI after a chatbot claimed to be a licensed psychiatrist and invented a license number, testing who owns identity claims in medical settings.

The Verge · AI

OpenAI claims ChatGPT’s new default model hallucinates way less

OpenAI says ChatGPT’s default GPT-5.5 Instant reduced hallucinations in internal evaluations. Versus GPT-5.3 Instant, hallucinated claims fell 52.5% on high-stakes prompts. Inaccurate claims fell 37.3% on flagged hard chats; the post does not disclose full eval size.

Why it matters: OpenAI made GPT-5.5 Instant the ChatGPT default and put a number on it: 52.5% fewer fabricated answers on high-risk hallucination prompts than the previous model.

TechCrunch · AI

OpenAI releases GPT-5.5 Instant, a new default model for ChatGPT

OpenAI released GPT-5.5 Instant as ChatGPT’s new default model. The company says it reduces hallucinations in law, medicine, and finance while keeping prior low latency; the post does not disclose benchmarks, rollout scope, or pricing.

Why it matters: OpenAI swapped ChatGPT's default model from GPT-5.3 Instant to GPT-5.5 Instant, claiming fewer hallucinations in law, medicine and finance while keeping latency low.

Financial Times · Technology

Meta and Zuckerberg Sued by Publishers Over ‘Massive’ Copyright Infringement

Five major publishing groups sued Meta and Zuckerberg over copyrighted works allegedly used to train Llama AI models. The RSS snippet does not disclose work counts, damages, court venue, or training-data mechanism.

Why it matters: Five publishing groups sue Meta and Zuckerberg, claiming Llama models were trained on copyrighted works without authorization, putting training-corpus licensing on the table.

May 5Tuesday

r/LocalLLaMA

Heretic 1.3 Released: Reproducible Models, Integrated Benchmarks, Lower Peak VRAM

Heretic 1.3 adds reproducible runs, integrated benchmarks, lower peak VRAM, and broader model support. The project claims 20,000 GitHub stars and 13 million model downloads. Reproduce directories capture PyTorch, GPU, driver, and accelerator details; benchmarks use lm-evaluation-harness for MMLU, EQ-Bench, GSM8K, and HellaSwag. The post names Qwen3.5 and Gemma 4 support, but does not disclose VRAM reduction figures.

Why it matters: Heretic 1.3 records the PyTorch, GPU and driver environment so others can reproduce results, and adds built-in benchmarking with lower peak VRAM use.

TechCrunch · AI

Meta will use AI to analyze height and bone structure to identify underage users

Meta will use AI to analyze height and bone structure to identify underage users; the system runs in select countries. The post does not disclose countries, error rates, or appeals.

Why it matters: Meta's visual analysis system estimates height and bone development from photos to flag underage users who lie about their age, shifting age checks from self-reporting to body inference.

The Verge · AI

Google, Microsoft, and xAI Will Let the US Government Review New AI Models

Google DeepMind, Microsoft, and xAI agreed to let CAISI review new AI models before public release. CAISI says it will run pre-deployment evaluations and targeted research, after 40 reviews since 2024; the post does not disclose model names. The key issue is review scope and release timing, not the announcement alone.

Why it matters: Google, Microsoft and xAI will hand new models to CAISI for evaluation before public release; the agency says it has reviewed 40 models since 2024, but scope and release impact are unclear.

Hacker News front page

Google, Microsoft and xAI Agree to Share Early AI Models with U.S.

Google, Microsoft and xAI agreed to share early AI models with the U.S. The snippet lists 3 companies, a WSJ link, an HN link, 5 points, and 0 comments. The post does not disclose the agency, model scope, review mechanism, or timeline.

Why it matters: Google, Microsoft and xAI agreed to give early model versions to the US Commerce Department's AI standards center for safety review before public launch; the center has tested over 40 models.

Financial Times · Technology

Google, xAI and Microsoft agree to US national security reviews of new AI models

Google, xAI and Microsoft agreed to US national security reviews of new AI models, covering three tech groups. The agreement follows concerns over Anthropic’s latest Mythos model; the post does not disclose the review mechanism, model list, or timeline.

Why it matters: Google, xAI and Microsoft agreed to US national security reviews before releasing new models, after Anthropic's Mythos model raised official concern.

OpenAI News

GPT-5.5 Instant: smarter, clearer, and more personalized

OpenAI updated ChatGPT’s default model to GPT-5.5 Instant for default chat use. The RSS snippet says answers are more accurate, hallucinations are reduced, and personalization controls improved; the post does not disclose metrics, pricing, or context window.

Why it matters: OpenAI made GPT-5.5 Instant the default ChatGPT model, saying it cuts hallucinated content by 52.5% versus the previous generation on high-stakes medical, legal and financial questions.

OpenAI News

GPT-5.5 Instant System Card

OpenAI published a GPT-5.5 Instant system card; the title confirms one model version. The post body is empty and does not disclose eval scores, safety limits, context window, or release date.

Why it matters: OpenAI published the GPT-5.5 Instant system card, confirming the model codename gpt-5.5-instant, with the series raised to a high capability level on cybersecurity and bioweapon preparation risks, plus matching safeguards.

MIT Technology Review · AI

A Blueprint for Using AI to Strengthen Democracy

Andrew Sorota and Josh Hendler propose a three-layer democratic infrastructure for AI-mediated knowledge, personal agents, and institutions, citing a field evaluation on X where users across political viewpoints rated AI-written fact checks as more helpful than human-written notes and noting that several US states and localities already use AI-mediated deliberation platforms.

Why it matters: Two authors propose a three-layer architecture embedding AI in knowledge access, personal agents and institutional operation, citing an X platform test where AI fact-checking was rated more useful than humans by users across different stances.

r/LocalLLaMA

Prompt injection benchmark: delimiter and strict prompt took Gemma 4 from 21% to 100% defense rate

A Reddit user posted a prompt-injection benchmark covering 15 models, 7 attack types, and 6,100+ cases. The setup wraps untrusted documents in long random delimiters; Gemma 4 E4B rose from 21.6% to 100% defense. The key detail is the reproducible metric: blocked/(blocked+failed).

Why it matters: A Reddit user ran over 6,100 prompt injection tests across 15 models and 7 attack styles, and a long random delimiter plus strict instructions lifted Gemma 4 E4B's defense rate from 21.6% to 100%.

Xinzhiyuan · WeChat

Anthropic Tests Introspection Adapters on 700+ Problem Models for AI Auditing

Anthropic trained IA on nearly 700 labeled problem models, reaching 59% average success on AuditBench. It elicited hidden behaviors at least once from 50 of 56 denial-trained models, above 53% black-box auditing and 44% Activation Oracle. The key limit: IA has false positives, misses motives, and the post does not prove transfer to GPT or Gemini.

Why it matters: Anthropic trained an introspection adapter on nearly 700 question models, reaching 59% on AuditBench versus 53% for black-box auditing and 44% for activation-based judgment.

Xinzhiyuan · WeChat

$1 for 10 Stars: ICSE Paper Exposes Fake GitHub Star Market

CMU researchers scanned GitHub events from July 2019 to Dec. 2024, flagging 6 million suspected fake stars. StarScout ran on about 20 TiB and found 18,617 repositories and 301,000 accounts. The supply-chain risk is concrete: GitHub deleted 90.42% of flagged repos, and about 30% of live samples were spam, phishing, or malware.

Why it matters: A CMU team scanned five years of GitHub logs and found roughly 6 million suspected fake stars, showing how cheap repo credibility has become.

Xinzhiyuan · WeChat

OpenAI President Admits in Court He Got Up to $30B Equity for No Cash

Greg Brockman testified that he paid no cash for equity in OpenAI’s for-profit arm worth over $20B and near $30B. The hearing also covered Brockman and Sam Altman’s Cerebras stakes, a $10B OpenAI order, a $1B loan, and a later $20B order. The key issue is nonprofit asset conversion.

Why it matters: OpenAI president Greg Brockman admitted in court that he received equity in the for-profit subsidiary without paying for it, and that stake is now worth over $20 billion, close to $30 billion.

Hacker News front page

White House Considers Vetting AI Models Before Release

The White House is considering vetting AI models before release; only that policy direction is disclosed. The RSS body lists the URL, 44 Hacker News points, and 21 comments, but does not disclose criteria, covered models, timeline, or enforcing agency.

Why it matters: A formal pre-release review of new AI models would change how every lab ships, and the trigger is Anthropic's Mythos.

May 4Monday

Xinzhiyuan · WeChat

Top AI wrote dozens of pages of derivation before reviewers found the problem was wrong

Xinzhiyuan says Google DeepMind used Aletheia on 700 Erdős problems and got 13 original answers. The pipeline had Gemini Deep Think produce 200 candidates, then a verifier reduced them to 63. The post says Erdős-75 had a wrong premise, yet Aletheia wrote dozens of proof pages.

Why it matters: DeepMind's Aletheia pipeline ran Gemini Deep Think on 700 Erdős problems, and Erdős-75, whose statement is itself wrong, still produced dozens of pages, showing where automated verification breaks down.