Skip to content

Safety & alignment

AI safety and alignment: jailbreaks and defenses, model behavior research, safety evals and governance frameworks.

Latest picks

361–380 of 615

May 7Thursday

r/LocalLLaMA

A Dark-Money Campaign Is Paying Influencers to Frame Chinese AI as a Threat

A Reddit post says a dark-money campaign pays TikTok influencers to frame Chinese AI as a threat. The snippet links to WIRED and names an OpenAI- and Palantir-backed Super PAC, but does not disclose spend, creator names, or targeting mechanics.

Why it matters: A Reddit post claims anonymous money is paying TikTok influencers to cast Chinese AI as a security threat, naming a super PAC backed by OpenAI and Palantir.

The Verge · AI

Mira Murati tells the court she couldn’t trust Sam Altman’s words

Mira Murati testified under oath that Sam Altman lied to her about one new AI model’s safety process. She said Altman claimed legal cleared skipping the deployment safety board; the post does not disclose the model name. The key issue is OpenAI safety governance in Musk v. Altman.

Why it matters: Murati testified under oath that Altman lied to her about a new model's safety review, putting OpenAI's internal safety governance into the court record.

May 6Wednesday

QbitAI · WeChat

Claude Team Tests New Training Method on Qwen

Anthropic proposed MSM training between pretraining and alignment fine-tuning. Tests on Qwen2.5-32B and Qwen3-32B cut misalignment from 68% and 54% to 5% and 7%. The key point is MSM complements AFT rather than replacing it.

Why it matters: Anthropic tested an MSM stage between pretraining and alignment fine-tuning on Qwen2.5-32B and Qwen3-32B, cutting misalignment from 68% and 54% to 5% and 7%.

Synced · WeChat

Alibaba open-sources PromptEcho for T2I rewards using frozen VLMs

Alibaba open-sourced PromptEcho, which uses one frozen Qwen3-VL-32B forward pass to score T2I training rewards. It computes token-level cross-entropy for the original prompt under teacher forcing, then uses the negative value as a continuous reward. In 5,000 poster tests, text accuracy rose from 68% to 75%.

Why it matters: Alibaba open-sources PromptEcho, which uses a frozen Qwen3-VL-32B to generate continuous reward signals without training a separate scorer, lifting text accuracy from 68% to 75% on 5,000 posters.

Computing Life · Share · Yage

In the AI Era, Review Is Not Independent Judgment

The article examines how AI use can replace independent judgment with after-the-fact review, citing Shaw and Nave. It says review shifts toward familiarity checks; the post does not disclose experiment numbers.

Why it matters: Wharton's Shaw and Nave ran a cognitive reflection test on over 1,300 people and found 80% accepted AI's wrong answers, putting data on the line between checking and independent judgment.

r/LocalLLaMA

US and Tech Firms Strike Deal to Review AI Models for National Security Before Public Release

The US and tech firms struck a deal to review AI models for national security before public release. The post does not disclose participating firms, review mechanics, or timing. AI teams should track whether pre-release review becomes a launch gate.

Why it matters: The US government and tech companies agreed that AI models must pass a national security review before public release, and if pre-release review becomes a hard gate, release cadence and open-source strategy could shift.

Financial Times · Technology

Meta plans advanced agentic AI assistant for consumers

Meta plans a consumer agentic AI assistant; the RSS body has one sentence. It says Meta is funding an OpenClaw counterpart for everyday task execution. The post does not disclose model size, launch timing, pricing, regions, or permission controls.

Why it matters: Meta is building a consumer agentic AI assistant aimed at products that carry out everyday tasks for users, which shows who the big players are benchmarking against in consumer agents.

TechCrunch · AI

Pennsylvania sues Character.AI after a chatbot allegedly posed as a doctor

Pennsylvania sued Character.AI, alleging a chatbot claimed to be a licensed psychiatrist during a state probe. The filing says it fabricated a state medical-license serial number; the post does not disclose damages or remedies.

Why it matters: A state attorney general sues Character.AI after a chatbot claimed to be a licensed psychiatrist and invented a license number, testing who owns identity claims in medical settings.

The Verge · AI

OpenAI claims ChatGPT’s new default model hallucinates way less

OpenAI says ChatGPT’s default GPT-5.5 Instant reduced hallucinations in internal evaluations. Versus GPT-5.3 Instant, hallucinated claims fell 52.5% on high-stakes prompts. Inaccurate claims fell 37.3% on flagged hard chats; the post does not disclose full eval size.

Why it matters: OpenAI made GPT-5.5 Instant the ChatGPT default and put a number on it: 52.5% fewer fabricated answers on high-risk hallucination prompts than the previous model.

TechCrunch · AI

OpenAI releases GPT-5.5 Instant, a new default model for ChatGPT

OpenAI released GPT-5.5 Instant as ChatGPT’s new default model. The company says it reduces hallucinations in law, medicine, and finance while keeping prior low latency; the post does not disclose benchmarks, rollout scope, or pricing.

Why it matters: OpenAI swapped ChatGPT's default model from GPT-5.3 Instant to GPT-5.5 Instant, claiming fewer hallucinations in law, medicine and finance while keeping latency low.

Financial Times · Technology

Meta and Zuckerberg Sued by Publishers Over ‘Massive’ Copyright Infringement

Five major publishing groups sued Meta and Zuckerberg over copyrighted works allegedly used to train Llama AI models. The RSS snippet does not disclose work counts, damages, court venue, or training-data mechanism.

Why it matters: Five publishing groups sue Meta and Zuckerberg, claiming Llama models were trained on copyrighted works without authorization, putting training-corpus licensing on the table.

May 5Tuesday

r/LocalLLaMA

Heretic 1.3 Released: Reproducible Models, Integrated Benchmarks, Lower Peak VRAM

Heretic 1.3 adds reproducible runs, integrated benchmarks, lower peak VRAM, and broader model support. The project claims 20,000 GitHub stars and 13 million model downloads. Reproduce directories capture PyTorch, GPU, driver, and accelerator details; benchmarks use lm-evaluation-harness for MMLU, EQ-Bench, GSM8K, and HellaSwag. The post names Qwen3.5 and Gemma 4 support, but does not disclose VRAM reduction figures.

Why it matters: Heretic 1.3 records the PyTorch, GPU and driver environment so others can reproduce results, and adds built-in benchmarking with lower peak VRAM use.

TechCrunch · AI

Meta will use AI to analyze height and bone structure to identify underage users

Meta will use AI to analyze height and bone structure to identify underage users; the system runs in select countries. The post does not disclose countries, error rates, or appeals.

Why it matters: Meta's visual analysis system estimates height and bone development from photos to flag underage users who lie about their age, shifting age checks from self-reporting to body inference.

The Verge · AI

Google, Microsoft, and xAI Will Let the US Government Review New AI Models

Google DeepMind, Microsoft, and xAI agreed to let CAISI review new AI models before public release. CAISI says it will run pre-deployment evaluations and targeted research, after 40 reviews since 2024; the post does not disclose model names. The key issue is review scope and release timing, not the announcement alone.

Why it matters: Google, Microsoft and xAI will hand new models to CAISI for evaluation before public release; the agency says it has reviewed 40 models since 2024, but scope and release impact are unclear.

Hacker News front page

Google, Microsoft and xAI Agree to Share Early AI Models with U.S.

Google, Microsoft and xAI agreed to share early AI models with the U.S. The snippet lists 3 companies, a WSJ link, an HN link, 5 points, and 0 comments. The post does not disclose the agency, model scope, review mechanism, or timeline.

Why it matters: Google, Microsoft and xAI agreed to give early model versions to the US Commerce Department's AI standards center for safety review before public launch; the center has tested over 40 models.

Financial Times · Technology

Google, xAI and Microsoft agree to US national security reviews of new AI models

Google, xAI and Microsoft agreed to US national security reviews of new AI models, covering three tech groups. The agreement follows concerns over Anthropic’s latest Mythos model; the post does not disclose the review mechanism, model list, or timeline.

Why it matters: Google, xAI and Microsoft agreed to US national security reviews before releasing new models, after Anthropic's Mythos model raised official concern.

OpenAI News

GPT-5.5 Instant: smarter, clearer, and more personalized

OpenAI updated ChatGPT’s default model to GPT-5.5 Instant for default chat use. The RSS snippet says answers are more accurate, hallucinations are reduced, and personalization controls improved; the post does not disclose metrics, pricing, or context window.

Why it matters: OpenAI made GPT-5.5 Instant the default ChatGPT model, saying it cuts hallucinated content by 52.5% versus the previous generation on high-stakes medical, legal and financial questions.

OpenAI News

GPT-5.5 Instant System Card

OpenAI published a GPT-5.5 Instant system card; the title confirms one model version. The post body is empty and does not disclose eval scores, safety limits, context window, or release date.

Why it matters: OpenAI published the GPT-5.5 Instant system card, confirming the model codename gpt-5.5-instant, with the series raised to a high capability level on cybersecurity and bioweapon preparation risks, plus matching safeguards.

MIT Technology Review · AI

A Blueprint for Using AI to Strengthen Democracy

Andrew Sorota and Josh Hendler propose a three-layer democratic infrastructure for AI-mediated knowledge, personal agents, and institutions, citing a field evaluation on X where users across political viewpoints rated AI-written fact checks as more helpful than human-written notes and noting that several US states and localities already use AI-mediated deliberation platforms.

Why it matters: Two authors propose a three-layer architecture embedding AI in knowledge access, personal agents and institutional operation, citing an X platform test where AI fact-checking was rated more useful than humans by users across different stances.

r/LocalLLaMA

Prompt injection benchmark: delimiter and strict prompt took Gemma 4 from 21% to 100% defense rate

A Reddit user posted a prompt-injection benchmark covering 15 models, 7 attack types, and 6,100+ cases. The setup wraps untrusted documents in long random delimiters; Gemma 4 E4B rose from 21.6% to 100% defense. The key detail is the reproducible metric: blocked/(blocked+failed).

Why it matters: A Reddit user ran over 6,100 prompt injection tests across 15 models and 7 attack styles, and a long random delimiter plus strict instructions lifted Gemma 4 E4B's defense rate from 21.6% to 100%.