Machine Learning
Part I: Mathematical Foundations
Every machine learning algorithm rests on three mathematical pillars: linear algebra provides the geometry of data and transformations; probability theory formalises uncertainty and inference; optimisation theory explains how models improve. This part builds each pillar from first principles, with full derivations and Python simulations.
1
Chapter 1: Linear Algebra for ML
Vectors, matrices, eigendecomposition, SVD ā the geometric and algebraic backbone of every machine learning algorithm.
Matrix multiplication & transposeEigendecomposition A = QĪQā»Ā¹SVD: A = UĪ£Vįµ (full derivation)Rank, null space, positive definiteness
2
Chapter 2: Probability & Statistics for ML
Probability axioms, Bayesā theorem, distributions, MLE and MAP estimation ā the language of uncertainty in learning.
Bayes theorem from joint probabilityGaussian, Bernoulli, Categorical, PoissonMLE: derive normal equationsMAP estimation & conjugate priors
3
Chapter 3: Optimization Theory
Gradient descent, convexity, Adam optimizer, Lagrange multipliers and KKT conditions ā how ML models actually learn.
Gradient, Hessian, Jacobian, Taylor expansionConvexity: definition & second-order conditionGD, Momentum, Adam derivationsLagrange multipliers & full KKT conditions
What you will learn
āRepresent data as vectors and matrices and reason geometrically
āDecompose matrices with eigendecomposition and SVD for compression and analysis
āModel uncertainty with probability distributions and derive MLE/MAP estimators
āApply Bayesā theorem to update beliefs as data arrives
āProve gradient descent converges on convex objectives
āDerive the Adam optimiser from first principles
āFormulate constrained optimisation with Lagrange multipliers and KKT conditions
āUnderstand every Part IIāVII algorithm through these three lenses