What are AI tokens? ↗
A token is a unit selected by a model's tokenizer. It might be a word, part of a word, punctuation, whitespace or something less intuitive. Models process token IDs, not English words as such.
The ideas behind AI, one at a time.
Start with the words you know. Then investigate the calculations, actual model values, and the scientific sources behind them. Each guide connects to a lab you can try.
What are AI embeddings? ↗
An embedding maps a token ID to a learned numerical vector. These coordinates let the network operate on discrete text, but individual dimensions do not have reliable human-readable labels.
How transformer attention works ↗
Attention lets a position calculate weighted combinations of information from other allowed positions. The weights depend on learned query, key and value projections, plus the model's masking rules.
How neural networks learn ↗
A neural network learns by repeatedly measuring an error, calculating how parameters affect it, and updating weights. Training changes the model; inference uses the weights that already exist.