Posts tagged "circuits"
4 posts
The Redundancy Trap: Why Single-Head Ablation Lies
# The Redundancy Trap *I deleted the two attention heads with the largest positive direct effects in GPT-2 small. The model got **better**. Then I found the same failure across seven models — and in
Circuit Tracing: How We Draw the Wiring Diagram of an AI's Thoughts
 In a companion post we toured *the biology of a language model* — watching Claude plan rh
The Biology of a Large Language Model: Tracing the Circuits Inside Claude
 We don't really *build* large language models — we **grow** them. We set up a trainin
Zoom In: How to Read the Circuits Inside a Neural Network
window.MathJax = { tex: { inlineMath: [['$','$'], ['\\(','\\)']], displayMath: [['$$','$$'], ['\\[','\\]']] }, svg: { fontCache: 'global' } }; ![Curve detectors — a family of neurons