Skip to content
Hacker News front page

Jev's calibrated probabilities break when your data distribution differs

Jev Can't Be Calibrated

TypeSafe's Jev model promises calibrated probabilities for structured decisions, but the author argues calibration depends on the data distribution. A model calibrated on training data won't stay calibrated on your production data. Worse, Jev reportedly assigns 0.92 probability to a fair coin landing heads, and probability semantics shift across different primitives. Treat Jev's outputs as ranking scores, not true probabilities. If you need real calibration, fit Platt scaling on a few hundred of your own labeled examples.

Read the original ↗Export Markdown