Self-Attention Heatmap Visualizer
Interactive Scaled Dot-Product Attention matrix. Select any token to view real-time attention weights across the entire sentence.
Concept Breakdown: Self-Attention Scoring & Contextual Weighting
When you read "The animal did not cross the street because it was tired", your brain instantly highlights "animal" when looking at "it". Attention is the AI's digital highlighter linking words that explain each other.
The word "bank" means money in "river bank" vs "savings bank". By computing attention weights between all words at once, the AI lets every word borrow context from its neighbors.
Click any word in the black bar below. Watch the orange and green heat bars jump to show which other words the AI connects to that word!
Attention Distribution for Query: "it"
How Attention Computes This Result:
1. Query Vector (q): Generated for token "it".
2. Key Vectors (k): Compared with q via dot product (q · k).
3. Scaled Softmax: Dot products are scaled by 1 / √d_k and normalized with Softmax, producing the percentage bars above.
4. Value Weighted Sum (v): Attention weights blend value vectors into the final contextual embedding.
Interactive Challenge: Co-Reference Resolution
In Example #1, click on the pronoun "it" in the sentence strip above to see what the model thinks "it" refers to.
Quick Reference & Instructions
Simple steps, pro tips, and execution details
Provide Inputs
Type, paste, or select your values in the form fields below.
Instant Live Analysis
Calculations and formatting happen automatically with zero delay as you type.
Copy or Use Output
Copy results or apply the clean output directly to your projects.
How It Works
Calculates query-key dot products for each token in an educational sentence, applies scaling and softmax, and animates attention weight flow.
Formula & Logic
Frequently Asked Questions
Common questions about calculations, assumptions, and edge cases.
Yes, Self-Attention Heatmap Visualizer is 100% free with unlimited calculations and zero paywalls or subscriptions.
Related Tools
Explore all llm tools →Query-Key-Value (QKV) Numerical Playground
Interactive step-by-step matrix laboratory: inspect Q × Kᵀ dot products, division by √d_k, Softmax normalization, and weighted V sums.
Transformer Layer Architecture Visualizer
Interactive pipeline of a complete Transformer block: Embeddings, RoPE, RMSNorm, Multi-Head Attention, Residual Add, and SwiGLU FFN.
2D Vector Embedding Visualizer
Visualize semantic embeddings in 2D vector space. Compare Cosine Similarity, Euclidean Distance, and semantic clustering.