Skip to content
Hacker News front page

A char-level transformer trained from scratch on an $8 ESP32-S3

An SLM trained on $8 ESP32-S3

Carloscodix open-sourced qapla, a char-level transformer trained entirely on an $8 ESP32-S3 chip. The chip runs the full training loop, not just inference, with backprop hand-written in C. The repo doesn't disclose parameter count, dataset size, or training time, but the code is public with 6 commits. Treat this as an extreme engineering demo rather than a practical tool—fitting training onto a microcontroller is the wild part.

Why it matters: A developer ran a full character-level Transformer training loop, backprop included, on an $8 ESP32-S3 in hand-written C, a reference for end-to-end training on very low-resource hardware.

Read the original ↗Export Markdown