Posts tagged "interactive"
3 posts
The Redundancy Trap: Why Single-Head Ablation Lies
# The Redundancy Trap *I deleted the two attention heads with the largest positive direct effects in GPT-2 small. The model got **better**. Then I found the same failure across seven models — and in
Where Does a Language Model Think? Finding and Removing the 'Workspace' Layers of Llama-3.1-8B
*A hands-on interpretability walkthrough. We watch concepts form layer-by-layer inside Llama-3.1-8B-Instruct, measure precisely which layers carry meaning, then **delete layers** — one at a time and
World Models from Scratch: Two Toy Experiments (with an in-browser demo)
A world model has three pieces: an **encoder**, a **state-space model**, and a **decoder**. This post builds two of them from scratch — one for CartPole, one for a little visual "counting + mass" wor