Skip to content

#安全/对齐

5 today

Feb 27, 2025Thursday

OpenAI News

OpenAI GPT-4.5 System Card

OpenAI published the GPT-4.5 system card on Feb. 27, 2025 and set a deployment bar: post-mitigation risk must be no higher than Medium. The scorecard lists CBRN and persuasion as Medium, cybersecurity and model autonomy as Low; the post does not disclose benchmark scores, context window, or pricing. The key detail is the release condition, not the “largest model” claim: OpenAI says it found no significant safety-risk increase versus existing models.

Why it matters: The GPT-4.5 system card shows OpenAI scaling pretraining past GPT-4o with supervised fine-tuning and RLHF, betting on a general model over pure reasoning.

OpenAI News

Introducing GPT-4.5

OpenAI released GPT-4.5 as a research preview on February 27, 2025 for Pro users and developers worldwide. The post calls it the largest and strongest GPT model for chat, with lower hallucination and better steerability, but the excerpt does not disclose the SimpleQA scores or hallucination-rate values. The key detail is the training path: scaled unsupervised learning on Microsoft Azure AI supercomputers, plus new techniques using data derived from smaller models.

Why it matters: GPT-4.5 reaches Pro users and developers as a research preview, with OpenAI claiming more compute and data cut hallucinations and improve world understanding.

Feb 25, 2025Tuesday

OpenAI News

Deep research System Card

OpenAI published the Deep research System Card on Feb. 25, 2025 and said deployment is allowed only when post-mitigation risk scores are no higher than Medium. The card lists six risk areas and rates CBRN, cybersecurity, persuasion, and model autonomy as Medium. Deep research uses an early OpenAI o3 variant for web browsing, file reading, and Python execution, but the post does not disclose test set sizes or pass rates.

Why it matters: The Deep Research system card discloses the safety testing behind the browsing, file-reading, Python-running agent, rating all four high-risk areas medium.

Feb 12, 2025Wednesday

OpenAI News

Sharing the latest Model Spec

OpenAI published an updated Model Spec on Feb 12, 2025 and released it under a CC0 public-domain license for free reuse and adaptation. The update centers on chain of command, truth-seeking, boundaries, and style; OpenAI says adherence improved versus its best system from last May, but the post does not disclose scores, eval size, or model names. The key point is that OpenAI writes intellectual freedom into the spec while keeping platform-level refusal boundaries.

Why it matters: OpenAI published a new Model Spec under CC0, spelling out how platform rules, developer settings and user instructions rank against each other.

Feb 8, 2025Saturday

OpenAI News

OpenAI at the Paris AI Action Summit

OpenAI said ChatGPT has 300 million weekly active users globally and used the 2025 Paris AI Action Summit to update its safety commitments. The post says it has published system cards for five frontier models since Seoul—4o, o1, Sora, Operator, and o3-mini—and plans to update its Preparedness Framework later this year. The key signal for practitioners is procedural: OpenAI says deep research will get a system card before broader access expands.

Why it matters: OpenAI said at the Paris AI Action Summit that ChatGPT has 300 million weekly users and that system cards now ship with each frontier model.

Jan 31, 2025Friday

OpenAI News

OpenAI o3-mini System Card

OpenAI rates o3-mini's post-mitigation overall risk as Medium, with Medium in CBRN, persuasion, and model autonomy, and Low in cybersecurity. The post says o3-mini is the first model to hit Medium on model autonomy due to stronger coding and research-engineering performance, but it does not disclose benchmark scores and says its real-world ML self-improvement capability is still below High. The key policy gate is explicit: deployment requires Medium or below, and further development allows High or below.

Why it matters: OpenAI's o3-mini system card sets overall risk at medium and model autonomy at medium, the official basis for judging deployment and development risk boundaries.

Jan 30, 2025Thursday

OpenAI News

Strengthening America’s AI leadership with the U.S. National Laboratories

OpenAI said on January 30, 2025 it signed an agreement with the U.S. National Laboratories to deploy o1 or another o-series model on Venado, an NVIDIA supercomputer at Los Alamos, for a system that includes about 15,000 scientists. The resource will be shared across Los Alamos, Lawrence Livermore, and Sandia for science, cybersecurity, energy, and nuclear-security work; the key detail is that nuclear and broader CBRN use cases will receive selective review and safety consultation from OpenAI researchers with security clearances.

Why it matters: OpenAI deployed its o-series reasoning models on Los Alamos's Venado supercomputer, opening shared access to about 15,000 scientists across three national labs.

Jan 23, 2025Thursday

OpenAI News

Operator System Card

OpenAI published the Operator System Card on Jan 23, 2025 and said its Computer-Using Agent can be deployed only if its post-mitigation score is Medium or lower. The card rates CBRN, cybersecurity, and model autonomy as Low, and persuasion as Medium; it highlights harmful tasks, model mistakes, and prompt injection. The key mechanism is human confirmation plus task refusal: critical steps like financial transactions, emails, and calendar deletion need approval, while stock trading is fully restricted.

Why it matters: OpenAI published the Operator system card, capping deployment at post-mitigation Medium and requiring human confirmation for high-risk actions like financial transactions, sending email and deleting calendar events, while banning stock trades outright.

Jan 22, 2025Wednesday

OpenAI News

Trading Inference-Time Compute for Adversarial Robustness

OpenAI reports that o1-preview and o1-mini often drive adversarial attack success rates close to zero as inference-time compute increases. The paper tests math tasks, SimpleQA prompt injection, Attack Bard images, and StrongREJECT misuse prompts; it labels the result as preliminary, and the truncated post does not fully disclose all failure cases. The key point is that this gain comes from longer reasoning at inference, not adversarial training.

Why it matters: OpenAI tested o1-preview and o1-mini on math problems, prompt injection, adversarial images and jailbreak prompts, and attack success rates dropped sharply as inference-time compute rose.

Dec 27, 2024Friday

OpenAI News

Why OpenAI’s structure must evolve to advance our mission

OpenAI says its board is evaluating changes to its nonprofit/for-profit structure, after estimating in 2019 that AGI would require about $10B. The post cites ChatGPT’s 300M+ weekly users and $137M in 2015 donations, but the specific final structure under consideration is not fully disclosed in the provided text. The key signal is financing pressure: OpenAI says investors at this scale want more conventional equity.

Why it matters: OpenAI's board is weighing a structure change because the $10 billion projected in 2019 falls far short of funding AGI, and investors at that scale want conventional equity rather than profit-cap terms.

Dec 20, 2024Friday

OpenAI News

Deliberative alignment: reasoning enables safer language models

OpenAI published deliberative alignment on Dec 20, 2024, training o-series models to reason over written safety specs before answering. The post says o1 uses this method and needs no human-labeled CoT or answers; it says o1 beats GPT-4o on internal and external safety benchmarks, but the post does not disclose exact scores.

Why it matters: OpenAI has its o-series models reason over internal safety specs before answering, needs no human-labeled chains of thought in training, and says o1 beats GPT-4o on safety benchmarks.

Dec 9, 2024Monday

OpenAI News

Sora is here

OpenAI moved Sora out of research preview on December 9, 2024 and rolled it out to ChatGPT Plus and Pro users. Sora Turbo supports up to 1080p and 20-second videos; Plus includes up to 50 monthly 480p videos or fewer 720p generations. The key detail for practitioners is deployment scope: the UK, Switzerland, and the EEA are excluded, person uploads are limited, and OpenAI says physics and long complex actions remain weak.

Why it matters: OpenAI turned Sora from a research preview into a standalone product with Sora Turbo, and the free monthly 480p cap shows how video generation is being metered.

Dec 5, 2024Thursday

OpenAI News

OpenAI o1 System Card

OpenAI published the system card for o1 and o1-mini, with a deployment gate that requires post-mitigation risk scores of medium or lower. The listed Preparedness results are low for cybersecurity, medium for CBRN and persuasion, and low for model autonomy; testing covered o1-near-final-checkpoint and o1-dec5-release. The key point for practitioners is that OpenAI confirms large-scale RL for chain-of-thought reasoning, while the post does not disclose dataset mix or full benchmark scores.

Why it matters: OpenAI published safety test results for o1 and o1-mini, disclosing large-scale reinforcement learning that makes the model reason before answering and a launch bar requiring risk-mitigated scores of medium or lower.

Nov 21, 2024Thursday

OpenAI News

Advancing red teaming with people and AI

OpenAI published 2 papers on Nov 21, 2024, outlining its external human red teaming process and a new automated red teaming method. The post discloses 3 concrete design choices for external testing—threat-model-based team selection, versioned model access, and structured feedback via API or ChatGPT interfaces—but this excerpt does not fully disclose the automated method's metrics or results.

Why it matters: OpenAI published two red-teaming papers, one on picking external human red teamers by threat model, one on using AI to find flaws automatically, useful for how safety teams staff and feed back.

Oct 30, 2024Wednesday

OpenAI News

Introducing SimpleQA

OpenAI open-sourced SimpleQA, a 4,326-question benchmark for factual short-answer QA and model calibration. Two independent AI trainers verified each item; a 1,000-question audit showed 94.4% agreement and an estimated inherent error rate near 3%. The key signal: it is built to challenge frontier models, and the post says GPT-4o scores below 40%.

Why it matters: OpenAI released SimpleQA, a benchmark of 4,326 fact questions cross-verified by two trainers with 1,000 spot-checked and 94.4% agreement, for measuring how accurately models answer short factual questions.

Oct 24, 2024Thursday

OpenAI News

OpenAI’s approach to AI and national security

After the White House issued an AI National Security Memorandum on October 24, 2024, OpenAI published a framework for national security partnerships and said each use case goes through formal review by its Product Policy and National Security teams. The post names 3 existing examples: DARPA cyber defense work, USAID using ChatGPT to cut administrative burden, and bioscience collaboration with Los Alamos National Laboratory; it does not disclose pricing, model versions, or contract size. The key signal is the boundary: OpenAI says its policies ban uses that harm people, destroy property, or develop weapons, while it explores research, logistics, translation, summarization, and civilian-harm mitigation use cases with the U.S. and allies.

Why it matters: OpenAI lays out how it vets national security work, what uses it bans, and three named partnerships with DARPA, USAID and Los Alamos.

Oct 15, 2024Tuesday

OpenAI News

Evaluating fairness in ChatGPT

OpenAI analyzed millions of ChatGPT requests to test whether user names trigger harmful stereotypes, finding an overall rate of about 0.1%. The study used GPT-4o as a privacy-preserving evaluator; its gender-related judgments matched human raters over 90% of the time, while race and ethnicity agreement was lower. The key signal is model drift across versions: GPT-3.5 Turbo showed the highest task-level bias.

Why it matters: OpenAI audits ChatGPT fairness across millions of real conversations, finding answer quality holds across name-implied gender and race, with about 0.1% harmful stereotypes.

Oct 9, 2024Wednesday

OpenAI News

An update on disrupting deceptive uses of AI

OpenAI says it has disrupted more than 20 operations and deceptive networks that tried to abuse its models since the start of 2024. The post ties this to election-related influence campaigns, social-media manipulation, and state-linked actors, and links an October 2024 threat report; the post does not disclose model-level breakdowns or exact enforcement mechanics.

Why it matters: OpenAI says it disrupted over 20 networks this year using its models for election interference and social media manipulation, some state-linked.

Sep 26, 2024Thursday

OpenAI News

Upgrading the Moderation API with OpenAI's new multimodal moderation model

OpenAI released omni-moderation-latest on September 26, 2024, a GPT-4o-based Moderation API model for text and image inputs that is free for all developers. It adds illicit and illicit/violent text categories, supports image moderation in 6 subcategories, and improves 42% on an internal 40-language eval, with gains in 98% of languages tested.

Why it matters: Extends free moderation to images and broadens what text categories get flagged.

Sep 16, 2024Monday

OpenAI News

An update on OpenAI's safety and security practices

OpenAI said on September 16, 2024 that its Safety and Security Committee will become an independent board oversight committee, chaired by Zico Kolter, for critical safeguards in model development and deployment. The committee can review major model safety evaluations and delay launches until concerns are addressed; the post also cites a 90-day review, evaluation of an AI-sector ISAC, and work with Los Alamos National Laboratory.

Why it matters: Gives a board committee the power to review safety evaluations and delay risky releases.

Sep 12, 2024Thursday

OpenAI News

Introducing OpenAI o1

OpenAI released o1-preview and o1-mini on Sept. 12, 2024, with access for ChatGPT Plus, Team, and tier-5 API developers. The post cites 83% vs 13% on an IMO qualifier, 84 vs 22 on a jailbreak test, and says o1-mini is 80% cheaper than o1-preview. The tradeoff is clear: the API lacks function calling, streaming, and system messages, and the models do not yet support browsing or file and image uploads.

Why it matters: OpenAI's o1 models trade speed for deliberation, lifting math olympiad accuracy from GPT-4o's 13% to 83% and reaching Codeforces top 11%.

OpenAI News

OpenAI o1-mini

OpenAI released o1-mini on Sept. 12, 2024 for Tier 5 API users at 80% lower cost than o1-preview. The post reports 70.0% on AIME and 1650 Codeforces Elo, close to o1 at 74.4% and 1673, with about 3-5x faster answers than o1-preview in one word-reasoning example. The key tradeoff is explicit: it targets STEM reasoning, while non-STEM factual knowledge is only comparable to small models like GPT-4o mini.

Why it matters: OpenAI's o1-mini costs a fifth of o1-preview per API call, opening first to paying ChatGPT and Tier 5 API users, so the price-capability tradeoff is the key detail.

Aug 16, 2024Friday

OpenAI News

Disrupting a covert Iranian influence operation

OpenAI said it banned ChatGPT accounts tied to the Iranian influence operation Storm-2035 in August 2024 after they generated election and geopolitics content for X, Instagram, and five websites. The company identified 12 X accounts and one Instagram account; on Brookings' Breakout Scale, the operation ranked at the low end of Category 2, with most posts getting few or no likes, shares, or comments. What matters is the workflow: the models were used for long articles, comment rewrites, and English-Spanish posting, not for meaningful audience reach.

Why it matters: OpenAI banned ChatGPT accounts tied to a covert Iranian influence operation that used the models to write long articles and short comments for fake news sites and social platforms.

Aug 13, 2024Tuesday

OpenAI News

Introducing SWE-bench Verified

OpenAI released SWE-bench Verified, a human-validated subset built with the benchmark’s authors to assess real software issue resolution more reliably. The post names 3 failure modes in SWE-bench: overly narrow tests, underspecified issue statements, and unreliable environment setup; as of Aug. 5, 2024, top agents scored about 20% on SWE-bench and 43% on SWE-bench Lite. The key point is that the original benchmark can systematically underestimate coding-agent ability.

Why it matters: OpenAI and the original SWE-bench authors built a human-verified subset, arguing the original systematically underestimates real coding ability due to rigid unit tests, vague descriptions and environment issues.

Aug 8, 2024Thursday

OpenAI News

Zico Kolter Joins OpenAI's Board of Directors

OpenAI appointed Carnegie Mellon professor Zico Kolter to its board on August 8, 2024, and added him to the Safety and Security Committee. The post says he will advise on critical safety and security decisions across all OpenAI projects alongside Bret Taylor, Sam Altman, and other members. The signal here is governance adding AI safety and robustness expertise, not a product launch.

Why it matters: Carnegie Mellon professor Zico Kolter joins OpenAI's board and its safety and security committee, bringing a model alignment and robustness research background to technical governance.

OpenAI News

GPT-4o System Card

OpenAI published the GPT-4o System Card on August 8, 2024, reporting 3 of 4 Preparedness categories as low risk and persuasion as borderline medium. The post says GPT-4o accepts text, audio, image, and video inputs, responds to audio in as little as 232 ms with a 320 ms average, and is 50% cheaper than GPT-4 Turbo in the API. The key issue for practitioners is voice safety: the card names unauthorized voice generation, speaker identification, and sensitive trait attribution, and says only models with post-mitigation scores at medium or below can be deployed.

Why it matters: OpenAI's official system card gives the measured voice latency and names the audio safety risks, so teams can judge the multimodal limits from primary data.

Jul 24, 2024Wednesday

OpenAI News

Improving Model Safety Behavior with Rule-Based Rewards

OpenAI said on July 24, 2024 it uses Rule-Based Rewards in the RLHF pipeline to reduce repeated human feedback for safety alignment. The post defines three response types—hard refusal, soft refusal, and comply—and says the method has been part of OpenAI’s safety stack since GPT-4, including GPT-4o mini. The key point is maintainability when policies change; the post excerpt does not disclose quantitative gains.

Why it matters: Replacing part of human feedback with rule-based scores means safety policy changes become a rule edit instead of a fresh labeling round, which changes the cost of iterating on alignment.

Jul 18, 2024Thursday

OpenAI News

GPT-4o mini: advancing cost-efficient intelligence

OpenAI released GPT-4o mini on July 18, 2024 at $0.15 per 1M input tokens and $0.60 per 1M output tokens, replacing GPT-3.5 in ChatGPT. It supports text and vision, offers a 128K context window and 16K max output, scores 82.0% on MMLU and 87.2% on HumanEval. The key detail for builders is that its API version is the first to use instruction hierarchy against jailbreaks and prompt injection.

Why it matters: GPT-4o mini's price per million input tokens and its replacement of GPT-3.5 in ChatGPT set a new floor for cheap inference, and the instruction hierarchy shows how OpenAI handles jailbreaks.

Jul 17, 2024Wednesday

OpenAI News

Prover-Verifier Games improve legibility of language model outputs

OpenAI trained GPT-4-family prover-verifier games so stronger models write solutions weaker models can verify; under time-limited human review, correctness-only optimization led to nearly 2x more evaluation errors. The post says the large and small models differ by about 3 orders of magnitude in pretraining compute, and checkability training recovers about half the performance gain of correctness-only optimization; the full experimental numbers are not fully disclosed in the provided text.

Why it matters: Training a large prover against a small verifier cut human evaluation error from double to half, offering a training pattern that transfers to other scalable oversight problems.

May 28, 2024Tuesday

OpenAI News

OpenAI Board Forms Safety and Security Committee

OpenAI's board formed a Safety and Security Committee; that action is the only confirmed fact so far. The source provides only a title, and the post does not disclose members, authority, reporting lines, or timing. Watch governance power, not the committee name.

Why it matters: OpenAI's board creates a Safety and Security Committee led by Bret Taylor with Adam D'Angelo, Nicole Seligman and Sam Altman, and sets a 90-day deadline to lay out safety rules for all projects.