Self-built LLM proxy gives teams call logs, token timing and data-flow visibility
What happened
Jared Siegel argued on Hacker News on October 19 for running your own LLM proxy. A proxy captures model call logs, token usage over time and where data goes. It also puts multiple models and providers behind one endpoint, with virtual API keys for managing calls and fast switching between models and providers. Siegel modified his own proxy to add more observability and a custom MCP server endpoint. He says that during coding this let him manually trace a cache miss in 20 seconds.
Written by AI from the coverage · updated 1 hour ago
Coverage
Follow the reports to see the story from different sides.
- Hacker News front pageIf you don't run your own LLM proxy, how do you know whats going on?
Jared Siegel 主张自建 LLM 代理，以掌握模型调用日志、token 使用时序数据及数据流向。代理可统一接入多个提供商和模型，通过虚拟 API 密钥管理调用，并支持快速切换模型与提供商。他修改了自己的代理以增强观测并添加自定义 MCP 服务端端点，称编码时能在 20 秒内手动定位缓存未命中的原因。
Heat over time
Not enough continuous observations to draw a trend yet.