Skip to content
Hacker News front page

TokenTown: A live, step-by-step isometric city that visualizes Transformer inference in your browser

TokenTown: A visual way to understand how LLMs work

An interactive browser demo that maps a Transformer into an isometric city. Each token is split at the docks, embedded at the foundry, stamped with position, then driven through attention, residual, and feed-forward stages per layer before sampling. The vector bars and softmax beams are genuinely computed live—just scaled down to 12 dims and a few hundred vocab entries. Weights are random, but output is blended with a bigram prior for readability. The first tour pauses at each district to explain; afterward you can speed up, step through, or roam freely.

Why it matters: A browser demo maps each layer of a language model onto an isometric city, so you can watch tokens travel from tokenizer to sampling.

Read the original ↗Export Markdown