Unsolved Problems in MLOps

This ACM Queue piece lays out why classical ops practices break down for ML: non-deterministic outputs and data as a system driver make canary deploys, health checks, and alerting nearly useless. Azure validates new models by having LLMs judge LLM output—the SRECon audience was audibly surprised. The authors argue the field must either find a better paradigm or fix the ones we have.

Why it matters: This ACM Queue piece lays out MLOps' core tension: traditional ops relies on deterministic responses for health checks and canary releases, but ML systems are non-deterministic and data-driven. The Microsoft Azure example—using LLMs as judges with employee thumbs-up as fallbac...