Blog
74 posts.
Category
Where Does a Language Model Think? Finding and Removing the 'Workspace' Layers of Llama-3.1-8B
A hands-on interpretability walkthrough. We watch concepts form layer-by-layer inside Llama-3.1-8B-Instruct, measure precisely which layers carry meaning, then delete layers — one at a time and in whole bands — and…
World Models from Scratch: Two Toy Experiments (with an in-browser demo)
A world model has three pieces: an encoder , a state-space model , and a decoder . This post builds two of them from scratch — one for CartPole, one for a little visual counting + mass world — and at the end you can…
T
The Steerability Spectrum: When Reading a Feature Isn't Enough to Steer It
You can read an attribute off a vision model perfectly — and still fail to steer it with a single vector. A number called $k^*$ says why, and prescribes the fix. Sumit Yadav · June 2026 · DINOv2 · CLIP · SigLIP ·…
SafeConstellations: Mitigating Over-Refusals in LLMs Through Task-Aware Representation Steering
LLMs refuse benign requests because the text looks dangerous. Our ACL 2026 paper finds that each NLP task traces a stable constellation in the residual stream, with refusal and non-refusal forming distinct sub-trajectories — so over-refusal can be fixed per task at inference time. Up to 73% less over-refusal, no fine-tuning, no utility loss, ~0.2s overhead.
Double Descent Explained: Why More Data Can Make a Model Worse
One of the strangest facts in deep learning: adding more data or more capacity can make a model worse before it gets better . Test error falls, then rises into a bump, then falls again — the famous double descent curve…
Why Does Attention Attend There? Tracing the Other Half of the Circuit
Attribution graphs gave us a way to see what information a model moves between tokens. But they froze one crucial thing: the attention pattern itself. They could tell you a head copied information from an earlier token…
How Does an LLM Know When to Start a New Line? The Geometry of Counting
Here's a deceptively simple puzzle. When a language model wraps text to a fixed width, it has to know how many characters still fit on the current line. But it has no eyes and no ruler — only a stream of tokens. So…
Towards Monosemanticity: Pulling Clean Features Out of a Messy Neuron
Inside a neural network, a single neuron fires for many unrelated things at once — it's polysemantic , which makes it almost impossible to read. So how do you pull out clean, single-meaning concepts? The 2023 paper…
Reading the Mind of Claude: Millions of Features Inside a Frontier AI
In 2023, Anthropic learned to pull clean, single-meaning features out of a tiny one-layer model. The obvious question was whether the trick scales. In Scaling Monosemanticity, they answered it on Claude 3 Sonnet — a…
Circuit Tracing: How We Draw the Wiring Diagram of an AI's Thoughts
In a companion post we toured the biology of a language model — watching Claude plan rhymes, chain facts, and even scheme. But every one of those discoveries rests on a single technique. The goal sounds impossible: take…
The Biology of a Large Language Model: Tracing the Circuits Inside Claude
We don't really build large language models — we grow them. We set up a training process, pour in data, and a mind emerges that no one explicitly designed. So to understand it, we can't read a blueprint; we have…
Can an AI Look Inward? Emergent Introspection in Language Models
Ask a language model what it's thinking and it will happily tell you — it describes its reasoning, intentions, and inner states. But there's a problem: these models were trained on mountains of human text where…
Do AI Models Have Emotions? Inside Claude's Emotion Concepts
Ask Claude for help and it sounds enthusiastic. Get it stuck on a hard bug and it sounds frustrated. Share bad news and it sounds concerned. Is any of that real — or just surface mimicry? In a 2026 paper, Anthropic…
An Image Is Worth 16x16 Words: How Vision Transformers Work
By 2020, the Transformer had conquered natural language processing. But computer vision was still ruled by convolutional neural networks (CNNs) , which had dominated for nearly a decade. Then a team at Google Brain…
Zoom In: How to Read the Circuits Inside a Neural Network
For most of deep learning's history we've treated neural networks as black boxes: we judge them by what they output, not by what they do inside. But what if you could put a network under a microscope and read it…
BirdCLEF+ 2026: A Per-Block kNN Probe, a Tiny Sequence Head, and a Bronze Medal
A two-week sprint on BirdCLEF+ 2026 — Cornell Lab's flagship bird-sound competition — wrapped up with an official Kaggle Competition Bronze Medal : rank 354 / 4084 (top 8.7%) , public LB 0.950 , private LB 0.942…
Boltzmann Machines & Energy-Based Models: The Nobel-Prize-Winning Physics Behind Neural Networks
In 2024, the Nobel Prize in Physics went to John Hopfield and Geoffrey Hinton — for foundational discoveries and inventions that enable machine learning with artificial neural networks. A physics prize, for AI? It turns…
A Mathematical Framework for Transformer Circuits: How LLMs Actually Work
What if you could reverse-engineer a transformer the way you'd decompile a program — turning billions of weights back into human-readable algorithms? That's exactly what Anthropic's 2021 paper A Mathematical…
Mechanistic Interpretability & Sparse Autoencoders: Giving AI an X-Ray
Imagine you're a physician a hundred years ago. You can observe a patient's symptoms, listen to their heartbeat, and make educated guesses about what's happening inside. Then someone hands you an X-ray…
Natural Language Autoencoders: Reading a Model's Mind in Plain English
Inside a large language model, every thought is just a vector of numbers — an activation at some layer. These vectors drive everything the model does, yet to us they're opaque. There's no dictionary that turns…
G
Git Govind — Jayadeva's Gita Govinda
A chapter-by-chapter retelling of Jayadeva's Gita Govinda — the divine love of Radha and Govinda across all twelve cantos.
The Lunar Wheel of Devotion
Jai Shree Ganesh. Open any panchanga and you will find the lunar month divided into thirty fragments — fifteen in the bright fortnight (shukla paksha) and fifteen in the dark (krishna paksha). Each fragment is a tithi…
S
SafeConstellations: Mitigating Over-Refusals in LLMs Through Task-Aware Representation Steering (ACL 2026)
A deep-dive companion blog for SafeConstellations , accepted at ACL 2026 Main . An inference-time method that reduces LLM over-refusal by up to 73% by steering task-specific trajectories in the residual stream — no…
K
King Salhesh, Malini & the Unfulfilled Loves – The Magical Bloom of Salhesh Fulbari and the Cool Touch of Jur Shital
नमस्कार! Jur Shital Mubarak ho! Naya Barsha 2083 BS ko hardik shubhakamana! 🌸 As the sun rises on Baisakh 1, 2083 BS (April 14, 2026), Maithili communities across Nepal’s Madhesh Province and the broader Mithila region…
E
Electric Circuit Theory (EE 501) — Complete Study Material
A comprehensive, interactive study guide for Electric Circuit Theory (EE 501) — Tribhuvan University, Institute of Engineering syllabus. Open the Study Material Seven full chapters covering the entire EE 501 syllabus…
Internet Penetration and RSP's Success in Nepal's 2082 Election — A Quick Correlation Look
Exploring whether districts with higher internet penetration showed stronger support for Rastriya Swatantra Party (RSP) in Nepal's recent election. Visual quick analysis.
C
Can You Predict a Network's Performance from Its Geometry Alone?
We show that two unsupervised geometric metrics, effective dimension and total compression, predict neural network accuracy across 92+ models spanning vision, NLP, and language modeling, without requiring labels. What…
A
Attention Is All You Need: A Visual Journey Through Semantic Space
Exploring the landmark Transformer paper through interactive 3D embedding visualizations The 2017 paper Attention Is All You Need by Vaswani et al. revolutionized natural language processing and laid the foundation for…
S
SAFE-MCP: A Comprehensive Security Framework for Model Context Protocol
An in-depth exploration of the Security Analysis Framework for Evaluation of Model Context Protocol - adapting MITRE ATT&CK methodology for the AI era As organizations rapidly integrate AI capabilities into their…
M
Mastering Hexagonal Architecture: A Complete Guide with Python Implementation
The business logic should not depend on the frameworks; instead, the frameworks should depend on the business logic. - Alistair Cockburn Introduction What is Hexagonal Architecture? Core Concepts Architecture Layers…
N
NPL- Nepal Premier League
Below is a compact, readable flowchart that lays out each match in chronological order with score & result. flowchart TD M1["1. Janakpur Bolts 130/6 → Kathmandu 131/5 — Kathmandu won"] M2["2. Karnali 166/3 → Chitwan…
N
Neural Modularity Principle
In much of the current model-design practice, we see input, processing and output all bundled together into one monolithic block. But this conflation hides a core architectural opportunity. I propose that Input…
A
Advance MCP Scanner
The Advance MCP Scanner orchestrates a complex multi-component system that: Uploads and processes MCP configuration files Discovers and executes MCP tools dynamically Uses AI for sequence generation and success analysis…
layers-travel
Interactive 3D visualization of how neural network embeddings evolve across layers
The Legend of Jimutavahana: A Detailed Story of Sacrifice and Compassion
The legend of Jimutavahana, also known as Jimutvahana or Jitvahan in various regional retellings, is an ancient tale rooted in Buddhist and Hindu traditions. It originates from the *Katha-Sarit-Sagara* (The Ocean of the…
I
Implement your own MCP SSE-to-SSE proxy
A deep dive into implementing secure, scalable Server-Sent Events proxying with intelligent tool access control In today's distributed systems landscape, API proxying has evolved far beyond simple request…
embeddinggemma-300m
Text embeddings are one of the most fundamental components of modern NLP systems. But how do we know if our embeddings are actually good? In this deep dive, we'll explore various techniques to evaluate embedding…
Ten Mahavidyas
# Exploring the Ten Mahavidyas: The Profound Wisdom Goddesses of Tantra Jai Shree Ganesh! Welcome to this super detailed blog post on the Ten Mahavidyas, the powerful and enigmatic group of Hindu Tantric goddesses who…
G
GPT-OSS: OpenAI's Open-Weight Mixture-of-Experts Language Model - A Technical Deep Dive
OpenAI has released GPT-OSS (GPT Open Source System), marking a significant milestone in open-weight language models. This technical deep dive explores the innovative architecture, implementation details, and key…
Kakbhushundi and Thoth: The Timeless Scribes of East and West
Image: Kakbhushundi depicted as a wise crow (Credit: Wikimedia Commons) Across cultures, we find mythological figures who act as divine scribes, wise narrators, and guardians of cosmic knowledge. Two such figures from…
Lagpachai Festival of Nāgas in Hindu Mythology, Mithila
Nag Panchami, also known as Lagpachai in Nepali, is a Hindu festival celebrated in honor of the serpent deities, Nagas. It falls on the fifth day (Panchami) of the bright half (Shukla Paksha) of the Hindu month of…
The Churia Forests and Rainfall in Nepal’s Terai: Impacts of Deforestation on Precipitation and Ecosystem Balance
Explore how Nepal’s Churia forests regulate rainfall in the Terai and how their deforestation disrupts precipitation patterns, increases flooding in the Himalayas and hilly regions, and unbalances Nepal’s ecosystem.
T
The Brain's Thinking Cycle: From Thought to Reply
Unraveling the Neural Symphony Behind Every Idea and Action flowchart TD A[Thought Initiation\nPrefrontal Cortex] --> B[Sensory Input\nThalamus] B --> C[Memory Retrieval\nHippocampus] C --> D[Action Selection\nBasal…
C
Clock
9196 days 1314 weeks 302 months 25 years
The Cost of a Wandering Mind: Why Being Present Matters for Happiness
Have you ever found yourself daydreaming during a meeting, thinking about weekend plans while stuck in traffic, or mentally planning tomorrow's tasks while having dinner? If so, you're not alone. Human beings…
Love
Love remains one of humanity's most enduring enigmas, a force that has inspired countless songs, poems, and philosophical treatises while continuing to baffle scientists and philosophers alike. It operates by its…
G
Go: The Silent Revolution in AI/ML Backend Development
Why an AI/ML developer chose Go over Python and C++ for production APIs As an AI/ML developer, I've spent countless hours in Python's warm embrace - training models, crunching data, and building prototypes…
T
The Tatva Journey: Building a Custom Jekyll-like Static Site Generator
A technical memoir of creating a personal blog from scratch Five days ago, with a simple commit message initial , I embarked on what would become an intense journey of building Tatva - not just another blog, but a…
A
AgentsAI
Ever wondered how to create a system where AI agents automatically choose the right tools for any task? Welcome to the future of intelligent automation! 🚀 In the rapidly evolving landscape of AI, we're witnessing a…
T
Tatva
In Indian philosophical traditions, Tattva refers to the fundamental realities or principles that constitute existence. The term comes from Sanskrit “Tat” (that) and “Tvam” (thou), implying “thatness” or “principle of…
A
Agents
1. portfolio.llm — Your Living, Chatty Portfolio This isn’t your run‑of‑the‑mill GitHub page—it’s a conversational resume powered by LLMs . Chat with it, ask about projects, and watch it spin up summaries on the fly. A…
Line
A road has distance, A path has direction, A journey has purpose, A life has connection. Lines drawn in sand, Fading with the tide, But the road remains, A constant guide. From here to there, From now to then, The road…
L
Life
Life is a beautiful, complex journey that we all navigate in our own unique way. It's filled with moments of joy, challenges, growth, and discovery. Life isn't just about existing—it's about living. It's…
F
Family Tree
लालजी यादव बरोहन यादव गढई यादव राजदेव यादव जीरेखन यादव बिन्देश्वर यादव अमित यादव सुमित यादव चन्देश्वर यादव प्रतिक यादव
M
Mahadev
In a village, Shiva ruled as the king and held sway over his kingdom. However, over time, two influential individuals arrived from another land. They were known as Vishnu and Brahma, each having their own kingdom in…
Jur Shital: A Celebration of Water and Life in Mithila
Jur Shital is a vibrant and colorful festival celebrated by the people of the Mithila region, which comprises parts of India and Nepal. This festival is celebrated on the 2nd of Baisakh, the second month of the Hindu…
F
Functional Dependency
In the context of relational databases, a functional dependency is a relationship between two or more attributes (or columns) in a table where the value of one attribute determines the value of another. For example, if…
O
Operating System
An operating system (OS) is a software system that manages computer hardware and software resources and provides common services for computer programs. The main functions of an operating system include: Process…
Q
Query Optimization Techniques
Query processing and optimization are two important aspects of database management systems that deal with the execution of user queries on a database. Query processing involves transforming a user query written in a…
Maithili Script Recognition — IOE Bachelor's AI Project (Pulchowk Campus)
TRIBHUVAN UNIVERSITY — INSTITUTE OF ENGINEERING PULCHOWK CAMPUS Machine Learning Analysis of Tirhuta Lipi By: Sumit Yadav (076/BCT/088), Raju Kumar Yadav (076/BCT/100), Prashant Bhandari (076/BCT/049) AN AI PROJECT…
M
Mithila Festivals
Here in this post you will find different festivals that are celebrated in the region of Mithila (side of Nepal — Janakpur, Tirhut, and parts of Bihar). Mithila festivals are very much hidden from the rest of the world…
Tensor Processing Units
Sumit Yadav (PUL076BCT088) Prof. Dr. Subarna Shakya Director, Information Technology Innovation Center, Tribhuvan University Computer Organization and Architecture — 09 June 2022 Full paper PDF: TPUs-2.pdf Tensor…
Multi-Class Text Classification
This post and code are used in a challenge organized by LOCUS called DataRush. Hi! Today in this post I am going to share our effort which was applied by us in a competition organized by LOCUS. Lets start. For this…
Dashain
Day 1 - Maa Shailaputri 🌸 ॐ देवी शैलपुत्र्यै नमः॥ Om Devi Shailaputryai Namah । वन्दे वाञ्छितलाभाय चन्द्रार्ध कृतशेखराम् ॥ वृषारूढाम् शूलधराम् शैलपुत्रीम् यशस्विनीम् । या देवी सर्वभूतेषु माँ शैलपुत्री रूपेण संस्थिता ।…
VIM
IT IS MODAL EDITOR (DIFFERENT OPERATING MODES) Normal mode (navigating around file) i — Insert mode (press esc to return) r — Replace mode (press esc to return) v — Visual mode (press esc to return) Line mode (keyboard…
S
Syllabus of BCT (Bachelor in Computer Engineering)
Course Structure for Bachelor's Degree in Computer Engineering Here I presented the TU, IOE Syllabus of BCT . BCT PDF
Nepali OCR with Python
In this post I am going to present Nepali Optical Character Recognition (OCR) that extracts Nepali text from images and scanned documents so that it can be edited, formatted, indexed, searched, or translated. Below…
Julia Set Fractal
The Julia set is named after the French mathematician Gaston Julia who investigated their properties circa 1915 and culminated in his famous paper in 1918. While the Julia set is now associated with a simpler…
AutoEncoder
In this notebook you will find how autoencoders are trained in TensorFlow (Keras). This notebook covers two different types of encoder: a normal Auto Encoder and a Denoising Auto Encoder . LIBRARY ARE LOADED FOR PLOTING…
S
Simple Math
NUMPY AND PYTHON The following code computes the derivative of a polynomial function using NumPy. Enter coefficients of the equation in the given pattern: constant, x, x^2, x^3, ... Result:
Unsupervised Learning
This is like a simple generative algorithm in which one model is used to reduce the data (like simple compression) and another is used to expand it back (like a decompressor). This model contains different neural…
Nepali Devanagari Letter Classifier
In the search of deep data analytics we come to deep machine learning, which is the most famous way of finding patterns in data and predicting the future state of data patterns. In this Python-based machine learning…
S
Sentences That Define Me
In this post I will put some quotes and some historic lines which defined me and suggested what to do at very critical stages. Tweet 1535582031299477505 Tweet 1426834591793369098 Tweet 1426326159176867845 Tweet…
C
Capture Your Moment
A digital space where memories are made, one selfie at a time. In this age of fleeting moments, sometimes we need to pause and capture the essence of now. 📹 Start Camera 📸 Capture 💾 Download 🔄 Retake Every moment is…