Skip to content

#评测/基准

1 today

Nov 1Sunday

Hacker News front page

GPT-5.6 Sol can beat Factorio on peaceful mode

Ryan Madden 实测让 GPT-5.6 Sol 在 Codex 中以低思考级别运行,用 13.5 小时游戏时间通关无虫族敌人的 Factorio 和平模式。模型通过 Python 库操作游戏,并遵守玩家操作距离、战争迷雾等与 GUI 玩家相同的限制。相比 GPT-5.5 不使用传送带的问题,本次表现有所改善,但后期生产系统衔接失败拖慢了进度,Sol 最终依靠反复搬运物资完成火箭发射。

Hacker News front page

Jepsen: ParticleDB 1.1.4

Jepsen 测试面向 AI 工作负载的数据库 ParticleDB 1.1.4 至 1.1.18.11.rc-22,发现 96 项问题,多数无需故障注入即可在单服务器、单客户端下触发。

Oct 30Friday

AI HOT · Tips & opinions

OpenRouter tests Claude Opus, Sonnet and Haiku 5.5 and picks a default

OpenRouter ran the three Claude models on the same coding tasks and recommends Sonnet 5.5 as the default. Haiku 5.5 suits high-volume requests and sub-agents, while Opus 5.5 is kept for long, ambiguous tasks where errors are costly.

Why it matters: All three models passed the tests, yet cost per task still differed by 19 to 30 times.

Hacker News front page

Claude Duet is the first personal-assistant-type harness that feels useful

Claude Duet 在作者实测中凭借快速响应、频道自动整理和分层权限,正在替代其大部分 AI 工作流。它可按频道限制 GitHub 仓库访问,不存储凭据,优先使用 Claude 现有连接器;顶层统览工作流,频道与子线程逐层收紧上下文和访问范围。产品仍处于 beta,存在运行线程不可见、委派仓库权限未正确附加的问题,作者也认为 Dot 的语音交互仍更出色。

Oct 29Thursday

Hacker News front page

Google's Gemini 4 Carbon tops Artificial Analysis Intelligence Index v5

Google's Gemini 4 Carbon scored 59 on the Artificial Analysis Intelligence Index v5, taking first place. It ranked second on the coding agent index. Artificial Analysis compared it with similar models and found its cost per task lower than Claude Opus 5.5.

Why it matters: The benchmark puts Gemini 4 Carbon first on general intelligence and cheaper per task than Claude Opus 5.5.

Hacker News front page

I Used AI to Fake a Scientific Paper. Here's What Happened

Benjamin Skuse 经 IEEE Access 同意,分别用免费大语言模型和自动化科研系统 FARS 生成论文,两轮测试均未通过期刊检查。第一轮论文由 ChatGPT 和 Gemini 生成,STM Integrity Hub 检出 12 条虚构引用,MolecularConnections 无法验证其中 24 条引用。

Oct 28Wednesday

AI HOT · Tips & opinions

Pydantic AI benchmark: code mode cuts long-task cost 58% and time 51%

Pydantic AI tested 8 models across 5,276 scored runs in an internal benchmark. Code mode cut long-task cost by 58%, time by 51%, and raised success rate by 16.0 percentage points.

Why it matters: The same tool set was run two ways, and code mode saves on long tasks while adding overhead on short ones.

Hacker News front page

Artificial Analysis Intelligence Index v5

Artificial Analysis 推出 Intelligence Index v5,新增两项评测、升级三项评测,60% 的指数基于私有测试集或保留答案的评测。

Hacker News front page

Artificial Analysis Intelligence Index v5.0

Artificial Analysis 推出 Intelligence Index v5.0,由 9 项评测组成,覆盖知识工作、编程、科学、通用四类,各占 25% 权重。

Oct 27Tuesday

AI HOT · Models

Artificial Analysis tests GPT-6.1 Terra: 6-point intelligence gain at under a third of the cost per task

Artificial Analysis found GPT-6.1 Terra scores 6 points higher than GPT-5.6 Terra on its intelligence index at the highest reasoning effort, while cost per task falls from $1.40 to $0.40. At low and medium reasoning effort the model sits on the intelligence-cost Pareto frontier.

Why it matters: A 6-point intelligence gain with cost per task down to $0.40 changes the price of running hard reasoning work.

Oct 26Monday

TechCrunch · AI

Grok releases the first seven minutes of its AI-generated ‘Odyssey’

Grok 发布 AI 生成版《奥德赛》的前七分钟,将其称为《奥德赛》系列第一集。三个月前,Elon Musk 曾宣布将推出一部“历史准确”的长片版本;此次作品由 Grok 与 Wonder Studios、导演 Dev Walker 合作制作。TechCrunch 作者 Anthony Ha 观看后认为,部分镜头接近真人实拍,但冗长旁白、生硬对白和缺乏推进感的叙事削弱了观看吸引力。

Oct 25Sunday

Hacker News front page

Five Qwen3.8-27B models match Fable 5 Using GVS5H Harness

一种名为 ledger-based zero-shot self-orchestration 的免训练方法,让同一模型的新实例通过共享文件系统分解问题并协作,在 100 道最新高难度 LiveCodeBench 题上最高提升 25.6 个百分点。

Oct 24Saturday

Hacker News front page

Show HN: Searching for optimal strategies in a puzzle game with chance (Drop7)

一项针对 Drop7 的自主策略研究显示,基于 n-tuple 查表的评估器在深度 3 搜索下、512 局未见过的公开开发局中平均得分 487,066,比同一搜索内手工评估函数(326,717)高出约 16 万分。该结果于 2026-09-05 首次在 screen 层级击败冻结的公平评估叶,次日以 SHA-256 校验的同一查表复现,但仍远低于百万分目标,尚未进入资格赛。

Oct 23Friday

Oct 22Thursday

The Verge · AI

AI agents are ready to shop. Some retailers are wary

Mia Sato 实测 Meta 的购物智能体 Muse,发现零售商访问限制、反机器人检查和配送条件会妨碍购物任务完成。Amazon 已要求 Meta 将 Amazon 从 Muse 中移除,Target 的检查则使访问时而成功、时而受阻,限制了比价覆盖范围。在 The RealReal 上,Muse 能反复将 10 件商品加入购物车、续上每次 20 分钟的保留期,阻止其他顾客购买这些商品。

Oct 21Wednesday

Hacker News front page

Eight Bits of Trust; eMudhra's CA keys cracked in milliseconds

eMudhra 的 8 个中间 CA 私钥仅有单字节取值,文章作者从公开证书中用不到 1 秒恢复了全部私钥。Chrome 和 Firefox 已封禁相关中间证书,但作者本地实测发现,默认配置的 curl、Python 和 Node 接受了为 api.stripe.com 伪造的证书,Windows CryptoAPI 在未开启吊销检查时也接受该证书链。

Hacker News front page

C# now beats Rust on WASM size

NetWasm 0.8.0 编译同一 print-42 工作负载产出 47,798 字节 Wasm,比 Rust 1.95.0 的 53,635 字节小 10.9%,Brotli-11 压缩后小 20.4%。

OpenAI News

Terminal X matches senior investor judgment 88.6% with GPT-6.1

Terminal X 的 Eval Agent 基于 GPT-6.1 Sol,在 500 条评分判断中与资深人类评审的一致率达 88.6%,比次优模型高 220 个基点。该 Agent 按 54 项标准评分,对偏离基金投资授权范围的回答检出率提升约 25%,单次查询成本降低约 30%。一家私募股权客户借助自动化的尽职调查与投资备忘录流程,实现年化 240 万美元节省。

Computing Life · Share · Yage

开放模型任务成本解析:为什么 Muse 比 DeepSeek 更省 token 却更贵?

开放模型的单任务成本同时受 token 消耗和单价影响,Muse Spark Open 输出量不足 DeepSeek 的一半,任务费用却高出三倍多。Artificial Analysis 的评测中,两者综合成绩均为 39 分,平均每任务费用分别为 0.84 美元和 0.27 美元,计算涵盖输入、缓存、推理思维链及最终答案的 token 费用。

Oct 20Tuesday

AI HOT · Models

DeepSeek-V4.1-Flash-20261017 (Max) 登顶 Agent Arena 开源模型第一

DeepSeek-V4.1-Flash-20261017 (Max) 成为 Agent Arena 排名第一的开源模型,总榜第 11,基于超 7.5K 真实会话净提升 +5.1%。其中位任务成本 $0.20,约为 GPT-6.1 Sol 提升幅度的 37%、成本的 27%,Confirmed Success 提升 +11.5%,且无 Tool Hallucination 问题。

Oct 19Monday

Oct 18Sunday

The Decoder

Four Gemini 4 Argon agents beat Minecraft without any human help

Google DeepMind 团队的四个 Gemini 4 Argon 智能体在无人干预下击败末影龙,完成 Minecraft 通关。团队将它们置于随机生成的世界,未指定角色、策略或游戏专用工具;智能体自行分工、互相警告威胁、回收死亡队友的物品,并在共享记忆中记录失误。

Hacker News front page

Show HN: 500 clocks in one 6.3 MB HTML file by Opus 5.5

开发者用 Claude Opus 5.5 在 Claude 聊天界面分多次会话生成了 500 个时钟,全部打包进一个 6.3 MB 的自包含 HTML 文件。作者称这是其个人版的「自行车上的鹈鹕」测试,用于检验前沿模型生成自包含网页的能力,并特别称赞 Opus 5.5 所造时钟的多样性与质量。

Oct 17Saturday

AI HOT · Industry

404 Media tests how ChatGPT ads target users by prompt, phone number and email

404 Media found in testing that OpenAI lets advertisers target specific groups of ChatGPT users by email, phone number or a phone's unique identifier. Targeting also uses the prompts users type, and some users may not know that personal data uploaded by advertisers is used to match ads. The original article says ChatGPT has 1 billion monthly active users, and notes that personal data targeting is common in advertising while prompt-based targeting is newer.

Why it matters: 404 Media's tests show ChatGPT ads can be targeted by prompt and personal identifiers, with both user input and advertiser-uploaded personal data feeding ad matching.

Oct 16Friday

The Decoder

Google Gemini 4 Argon benchmark results land closer to the frontier than recent Google models

Google DeepMind's Gemini 4 Argon scores 165 on Epoch AI's composite capability index ECI, close to GPT-6 Astra at 166 and Opus 5.5 at 167.

Why it matters: Argon's scores differ sharply between the two test setups, and vendor API features such as retained reasoning and context compression were part of the benchmark comparison.