Step 01 / 02

Before probabilities

Module map

Before you see neat probabilities like 0.78, the model usually produces raw scores called logits — relative preference numbers for each candidate token.

Delhi   8.2
Mumbai  5.1
London  2.3

A normalization step such as softmax turns those scores into a probability distribution that sums to 1. You do not need to compute softmax by hand. You need the concept: logits are “how much the model currently prefers each option” before they become probabilities.

Why this matters: temperature and other sampling controls act on this distribution. Change the shape of the probabilities, and you change how adventurous generation becomes.