Meet the New Biologists Treating LLMs Like Aliens
Meet the new biologists treating LLMs like aliens
MIT Technology Review reports that Anthropic, OpenAI, and Google DeepMind are using mechanistic interpretability to study LLMs; as a scale reference, a 200B-parameter model in 14-point print would cover 46 square miles. The post says Anthropic uses sparse autoencoders to mimic target models, linked a Claude 3 Sonnet region to the Golden Gate Bridge in 2024, and in a July experiment found Claude used different internal paths for “bananas are yellow” versus “bananas are red.” The key point for practitioners is that weak internal coherence constrains alignment and predictability.
Why it matters: Anthropic, OpenAI and Google DeepMind are using mechanistic interpretability to take apart how large models work inside; the report compares a 200-billion-parameter model spread across San Francisco to convey the scale this research faces.