Chapter 2

Foundations · The math + ML survival kit

📖 2 min read · 5 sections · May 2026

Why this matters

You don't need a PhD to ship AI products. You do need enough math to read a paper, debug a similarity-search bug, and know when an evaluation result is statistical noise. This chapter is the minimum.

Linear algebra: vectors, dot products, cosine

An embedding is a vector — a list of floats, typically 384 to 3072 dimensions. Two embeddings are "similar" if their cosine similarity is high. Cosine similarity is the cosine of the angle between two vectors:

cos(a, b) = (a · b) / (‖a‖ × ‖b‖)
          = sum(a_i × b_i) / (sqrt(sum(a_i²)) × sqrt(sum(b_i²)))

If both vectors are normalised (length 1), cosine similarity equals the dot product, and you can use much faster ANN indexes that operate on dot products. Most modern embedding models return normalised vectors — you can assert it.

Mental model: think of each dimension as a separate "concept the model picked up". Two pieces of text with high cosine similarity light up the same concepts at the same intensities. It says nothing about which concepts — that's why semantic search has weird false positives.

Probability — the three numbers you need

Independent events: P(A and B) = P(A) × P(B). Used everywhere in retrieval calibration ("if I retrieve 5 chunks each with 0.6 precision, P(all 5 wrong) = 0.4⁵ ≈ 1%").
Bayes' rule: P(A|B) = P(B|A) × P(A) / P(B). Why "spam filter says spam" doesn't mean "is spam".
Entropy: how uncertain is a distribution? Lower entropy = more confident. LLM logit distributions are the practical place this matters.

ML fundamentals that still matter

Even if you'll never train a model from scratch, two concepts keep mattering:

Train/validation/test split. Don't tune prompts on your final eval set — you'll fool yourself. Build the eval set first, freeze it, then iterate prompts on a separate dev set.
Overfitting. A prompt that scores 95% on 20 hand-picked examples will score 60% on a new sample. Same disease, prompt-engineering edition.

War story: a friend's team shipped a "RAG with 92% accuracy" feature based on 50 hand-curated questions. Real users asked different questions. Accuracy in production: 47%. The fix was building a 500-question eval set sampled from real traffic — and then re-running every prompt change against it.

Gradients — just enough

A gradient is "how does loss change if I nudge this parameter?". Backprop propagates that signal backwards through the network. You will not implement this — but you'll occasionally read about gradient checkpointing, gradient accumulation, gradient clipping. They're operational knobs in fine-tuning recipes. Read about them when you need them.

Further reading: Karpathy — The unreasonable effectiveness of RNNs · fast.ai practical deep learning · Goodfellow Deep Learning book (chapters 2-5 only)