Skip to content

Reasoning

Progress in model reasoning: chain of thought, reasoning models, math and logic benchmarks and the debates around them.

Latest picks

581–585 of 585

Oct 31, 2024Thursday

OpenAI News

Introducing ChatGPT search

OpenAI launched ChatGPT search on Oct. 31, 2024 for Plus, Team, and SearchGPT waitlist users, adding web answers with source links inside ChatGPT. It can trigger web search automatically or manually, shows a Sources sidebar, and uses a fine-tuned GPT-4o post-trained with distilled outputs from o1-preview. The shift to watch is distribution: search is folded into chat, not a separate search engine hop.

Why it matters: This is a same-day OpenAI product launch, not a minor feature tweak; search is merged into the chat UI, so HKR-H/K/R all pass. The post confirms source-linked web answers and launch conditions, and the move hits search distribution directly, which pushes it to P1.

Sep 12, 2024Thursday

OpenAI News

Introducing OpenAI o1

OpenAI released o1-preview and o1-mini on Sept. 12, 2024, with access for ChatGPT Plus, Team, and tier-5 API developers. The post cites 83% vs 13% on an IMO qualifier, 84 vs 22 on a jailbreak test, and says o1-mini is 80% cheaper than o1-preview. The tradeoff is clear: the API lacks function calling, streaming, and system messages, and the models do not yet support browsing or file and image uploads.

Why it matters: A major OpenAI reasoning-model launch with all three HKR signals: HKR-H from the new “think before answering” hook, HKR-K from concrete benchmark, safety, and pricing numbers, and HKR-R from the tradeoff practitioners must manage between stronger reasoning and missing API basics.

OpenAI News

Learning to reason with LLMs

OpenAI released o1-preview and reported 74% single-sample accuracy on AIME 2024, versus 12% for GPT-4o. The post says o1 reached the 89th percentile on Codeforces and exceeded human PhD experts on GPQA Diamond; it attributes this to large-scale RL and gains from both train-time and test-time compute. The key signal is scaling reasoning with compute, not just pretraining a larger base model.

Why it matters: This is a substantive OpenAI research release with product implications. HKR-H lands on the new reasoning line, HKR-K on the disclosed benchmark jumps and compute-scaling mechanism, and HKR-R on the direct impact to model strategy and inference economics; strong 90s, not 95+.

OpenAI News

OpenAI o1-mini

OpenAI released o1-mini on Sept. 12, 2024 for Tier 5 API users at 80% lower cost than o1-preview. The post reports 70.0% on AIME and 1650 Codeforces Elo, close to o1 at 74.4% and 1673, with about 3-5x faster answers than o1-preview in one word-reasoning example. The key tradeoff is explicit: it targets STEM reasoning, while non-STEM factual knowledge is only comparable to small models like GPT-4o mini.

Why it matters: OpenAI shipped a substantive model release, so this lands in the must-write band. HKR-H comes from the 80%-cheaper/nearly-o1 tradeoff; HKR-K from AIME 70.0 and Codeforces 1650; HKR-R from immediate developer cost/performance implications.

Jul 17, 2024Wednesday

OpenAI News

Prover-Verifier Games improve legibility of language model outputs

OpenAI trained GPT-4-family prover-verifier games so stronger models write solutions weaker models can verify; under time-limited human review, correctness-only optimization led to nearly 2x more evaluation errors. The post says the large and small models differ by about 3 orders of magnitude in pretraining compute, and checkability training recovers about half the performance gain of correctness-only optimization; the full experimental numbers are not fully disclosed in the provided text.

Why it matters: This is a substantive OpenAI research release with HKR-H/K/R all present: novel setup, clear mechanism, and strong relevance to scalable oversight. The excerpt confirms the method and the human-evaluation effect, but not the full experimental tables, so it fits the 78–84 band, نه