Skip to content
TechCrunch · AI

OpenAI's Jalapeño chip targets fast inference at scale, first benchmarks show

OpenAI’s Jalapeño chip is built for fast inference at scale, benchmarks show

OpenAI shared the first benchmarks for its in-house inference chip, Jalapeño, at Hot Chips. On SemiAnalysis' InferenceX test, it delivered more tokens per user and higher throughput per kilowatt than the current state-of-the-art. The post doesn't name the competitor or disclose latency figures. I'd hold off until third-party numbers land.

Why it matters: OpenAI presented first test results for its in-house inference chip Jalapeño at Hot Chips, and SemiAnalysis's InferenceX benchmark shows it beating the strongest current competitor on tokens per user and throughput per watt.

Read the original ↗Export Markdown