Linear algebra is the language ML is written in because data and model parameters are almost always represented as vectors and matrices, and “run the model” almost always reduces to matrix multiplication. You don’t need a full linear algebra course to get productive — you need to be fluent in the four notes below, and more importantly, what each operation represents, not just how to compute it.

Reading order

  1. vectors — [Beginner] — a list of numbers representing a data point, an embedding, or a direction
  2. matrices — [Beginner] — a table of data, or a transformation — both readings matter
  3. dot-product — [Intermediate] — the operation that measures how aligned two vectors are; the basis of similarity/search
  4. matrix-multiplication — [Intermediate] — dot products, batched — what a neural network’s forward pass actually is

What you don’t need to memorize upfront

Eigenvalues, singular value decomposition, and most of a formal linear algebra course are used inside specific algorithms (PCA for dimensionality reduction, for instance) but aren’t needed to understand how a neural network works day-to-day. The four notes above cover the vast majority of what shows up when reading about how models work; treat anything beyond them as “look it up when a specific technique needs it,” not a prerequisite to start.

  • calculus — the other half of how training actually works
  • what-is-a-model
  • llms — embeddings and attention are built directly on these operations
↑ Back to Mathematics