ttok 1.0 ships with default tokenizer switched to GPT-5/GPT-6

On October 9, 2026, Simon Willison released ttok 1.0. The change in this release is the default tokenizer: it previously used GPT-4, and version 1.0 switches it to GPT-5/GPT-6. The report confirms the new release and the default tokenizer change, but does not explain how the GPT-5/GPT-6 tokenizer was chosen, what the change means for users, or what else changed in the version.

Written by AI from the coverage · updated 58 minutes ago

Not enough continuous observations to draw a trend yet.