Skip to content
Hacker News front page

Anthropic introduces the Conceptual Reasoning Index to benchmark philosophical argumentation

Anthropic: Introducing The Conceptual Reasoning Index

Anthropic and Redwood Research built three benchmarks to measure how well models reason when empirical feedback is absent—what they call conceptual reasoning. LMCA contains 560 position texts and 1,461 expert-rated counter-arguments; ACCoRD uses 567 human-vetted consistency constraints to check logical coherence; DTBench offers 407 handcrafted decision-theory multiple-choice questions. The three are combined into the Conceptual Reasoning Index (CRI), weighted 60/20/20. As of August 10, 2026, Anthropic's own models score highest, though the post does not disclose exact numbers or a full leaderboard. The LMCA dataset is available by request, and CRI results are updated at conceptualreasoning.ai.

Why it matters: Anthropic and Redwood Research released a conceptual reasoning index with three new benchmarks for problems where models must argue without experimental feedback; the first has 560 position essays and 1,461 expert-scored rebuttals.

Read the original ↗Export Markdown