Arjun questions AI's creativity and judgment in self-directed research
What happened
On November 6, Hacker News' front page featured Arjun Panickssery's doubts about AI's ability to do research on its own. Drawing on Scrabble, Ultimate Tic-Tac-Toe and philosophy research projects, he says models miss important literature, run experiments that yield little information and chase trivial research directions. Fast-feedback tasks may be the exception: by repeatedly prompting Astra, he raised the state of the art on listing all legal Scrabble moves tenfold. Two of the larger gains came from his own suggestions on evaluation method and algorithm partitioning, so the results include human guidance.
Written by AI from the coverage · updated 3 hours ago
Coverage
Follow the reports to see the story from different sides.
- Hacker News front pageI struggle to believe that AI R&D is ingenious
Arjun Panickssery 根据 Scrabble、Ultimate Tic-Tac-Toe 和哲学研究项目的观察，质疑 AI 自主研发的判断力。模型会遗漏重要文献、开展缺乏信息价值的实验，并追逐琐碎的研究方向。快速反馈任务可能是例外，他通过反复提示 Astra 将列举所有合法 Scrabble 落子的 SOTA 提升了 10x，但其中两次较大进展来自他提出的评测方式和算法划分建议。
Heat over time
Not enough continuous observations to draw a trend yet.