Skip to content
Trending storyWatching

PostHog uses decision models to cut LLM calls

1 report1 sourceupdated 3 days ago

What happened

AI digest

On November 18, Hacker News featured PostHog's production use of decision models. PostHog uses decision models such as Jev for request screening, model routing and search reranking. The models return structured answers with probabilities for simple judgments, leaving complex reasoning to LLMs. In Replay Vision, two questions screen for personal information, and if either probability reaches 0.4, the case goes to Gemini for rewriting. In production, 98.7% of summaries skipped the LLM call. Search reranking improved ranking quality by about 25% across 20 test queries.

Written by AI from the coverage · updated 3 days ago

Coverage

Follow the reports to see the story from different sides.

Nov 18
  1. Hacker News front page
    Not Everything Needs an LLM

    PostHog 将 Jev 等决策模型用于请求筛选、模型路由和搜索重排,以带概率的结构化答案处理简单判断,将复杂推理留给 LLM。Replay Vision 用两道问题筛查个人信息,任一概率达到 0.4 就交给 Gemini 改写,生产环境中 98.7% 的摘要跳过了 LLM 调用;搜索重排在 20 个测试查询中使排名质量提升约 25%。

Heat over time

Heat now 3·Comparable peak 9(Nov 18 07:00)·Comparable change over 24 hours -50%

02.557.510Nov 1807:00Nov 1821:00Nov 1912:00Nov 2002:00

The trend only compares accounts observed without gaps, so its range may be smaller than the current heat. Hover or tap the chart for each hour; the left and right arrow keys step through it.