Cross-Entropy Loss Laboratory
Explore Cross-Entropy Loss for next-token prediction. Adjust predicted target probability to see how loss scales logarithmically.
Concept Breakdown: Logits to Normalized Probability Distribution
When an AI calculates candidate next words, it gives each word raw points (like +3.8 or -1.2). Softmax is the referee that converts raw points into percentage odds that always add up to 100%.
Negative scores confuse percentage math. Raising e to the power of each score turns all numbers strictly positive and exaggerates winners so the best word clearly stands out.
Slide Temperature down to 0.2: the #1 winner dominates at 99%. Slide it up to 2.0: all options become equally likely (maximum creativity and chaos).
Detailed Step-by-Step Numerical Breakdown
| Token | Logit (z) | Scaled (z / T) | Exponent (e^z) | Probability (P) |
|---|---|---|---|---|
| east | 3.80 | 3.80 | 1.000 | 66.6% |
| morning | 2.20 | 2.20 | 0.202 | 13.4% |
| sky | 1.90 | 1.90 | 0.150 | 10.0% |
| west | 1.50 | 1.50 | 0.100 | 6.7% |
| horizon | 0.80 | 0.80 | 0.050 | 3.3% |
Quick Reference & Instructions
Simple steps, pro tips, and execution details
Provide Inputs
Type, paste, or select your values in the form fields below.
Instant Live Analysis
Calculations and formatting happen automatically with zero delay as you type.
Copy or Use Output
Copy results or apply the clean output directly to your projects.
How It Works
Plots the negative log curve and shows how high-confidence incorrect predictions produce extreme penalties.
Formula & Logic
Frequently Asked Questions
Common questions about calculations, assumptions, and edge cases.
Yes, Cross-Entropy Loss Laboratory is 100% free with unlimited calculations and zero paywalls or subscriptions.
Related Tools
Explore all llm tools →Backpropagation & Gradient Flow Lab
Interactive calculus chain rule simulator: adjust weights and inputs to trace forward outputs, loss, and reverse gradients.
LLM Training Loop Simulator
Simulate training an LLM across multiple epochs: batches, forward passes, cross-entropy loss, and AdamW weight updates with live loss curves.
Gradient Descent Optimization Game
Interactive optimization game on 2D cost surfaces. Control learning rate to guide parameters to the global minimum without overshooting.