10-hour AI collab puts MobileNetV4 on an ESP32-CAM to check if a garage door is open
10 Hours of AI Collaboration, 30 Mins of Dev Time: Running a Neural Network on a Thumb-Sized Microcontroller for Garage Door Recognition
The author used GPT-5.6 SOL to go from raw security footage to on-chip inference in 10 hours, with roughly 30 minutes of human steering. The AI first annotated a small seed set with a local vision LLM, then ran an initial ViT across hundreds of thousands of frames to mine rare positive examples of the open door, retraining iteratively until the dataset balanced. It fine-tuned MobileNetV4 (~900k params) for the task; inference takes ~200 ms. Direct int8 quantization caused over 30% accuracy loss, so it switched to LSQ-based quantization-aware training, cutting the drop to 8 percentage points. The final firmware captures a photo every 10 minutes, runs inference, and pushes the result over Wi-Fi while the chip mostly stays in Deep Sleep. The post does not disclose long-term accuracy numbers—only that real-world data collection is ongoing.
Why it matters: The author used GPT-5.6 SOL to turn surveillance footage into an inference program on an ESP32-CAM, covering data mining, ViT filtering and MobileNetV4 fine-tuning, with about 200ms per inference.