Training LoRA adapters for Apple's on-device 3B model on a free Colab T4 and a Mac
The author built a QLoRA pipeline for Apple’s on-device 3B model, cutting training needs from about 24GB to about 1GB RAM and 5GB GPU, enough for a free Colab T4 or a 24GB Mac. The post says A100 LoRA, T4 QLoRA, and Mac QLoRA adapters perform about the same, raising accuracy from about 40% to 75%, or 86% with retrieval; it also reports a confirmed Apple bug that writes a hidden ~160MB cache copy per CLI call, reaching 269GB over ~300 runs.
Why it matters: A walkthrough of fine-tuning Apple's on-device 3B model with QLoRA, cutting memory needs to 1GB RAM plus 5GB VRAM so a free Colab T4 or a 24GB Mac can run it.