A char-level transformer trained from scratch on an $8 ESP32-S3
An SLM trained on $8 ESP32-S3
Carloscodix open-sourced qapla, a char-level transformer trained entirely on an $8 ESP32-S3 chip. The chip runs the full training loop, not just inference, with backprop hand-written in C. The repo doesn't disclose parameter count, dataset size, or training time, but the code is public with 6 commits. Treat this as an extreme engineering demo rather than a practical tool—fitting training onto a microcontroller is the wild part.
Why it matters: A developer ran a full character-level Transformer training loop, backprop included, on an $8 ESP32-S3 in hand-written C, a reference for end-to-end training on very low-resource hardware.