Mathematics for Machine Learning

Jul 22, 2024

Mathematics for Machine Learning

Importance of Mathematics in Machine Learning

  • Essential for aspiring machine learning engineers.
  • Computers perform calculations but understanding & applying math is crucial.
  • Math helps in data analysis and model creation.
  • Core components: Linear algebra, multivariate calculus, probability, statistics.

Topics Covered

  1. Why Math in ML
  2. Linear Algebra
  3. Multivariate Calculus
  4. Probability
  5. Statistics

Linear Algebra

Scalars

  • Scalars: Single value entities.
  • Operations: Basic arithmetic (addition, subtraction, multiplication, division).

Vectors

  • Represented differently in computer science, physics, mathematics.
  • ML uses vectors as list of numbers or pixel data.
  • Operations: Addition, scalar multiplication, projection.
    • Addition: Point-to-point sum, also called dot product.
    • Scalar Multiplication: Multiplies vector by scalar value.
    • Projection: Projects one vector onto another.

Matrices

  • Mixture of numbers, symbols, expressions in rectangular array.
  • Converts equations into arrays for computational operations.
  • Matrix Operations: Addition, subtraction, multiplication, transpose, determinant, inverse.
    • Transpose: Converts rows to columns.
    • Determinant: Scalar measure of matrix; crucial for eigenvalues.
    • Inverse: Reverse-direction matrix; helps in reverting the effect of matrix multiplication.

Eigenvectors and Eigenvalues

  • Eigenvectors: Vectors that do not change direction after transformations.
  • Eigenvalues: Scalars associated with eigenvectors.
  • Applications: Principal Component Analysis (PCA), encoding, optimization.

Coding PCA (Principal Component Analysis)

  • Reduces dimensionality of data.
  • Identifies main components that explain data variance.

Multivariate Calculus

Differentiation

  • Breaking down functions to understand changes in variables.
  • First-order and second-order differentiation.
  • Rules: Power rule, sum rule, product rule, chain rule.
  • Partial Differentiation: Differentiates multiple variables with respect to one while others are constant.
    • Applications: Jacobian, Hessian, optimizations in ML models.

Gradient Descent

  • Optimization algorithm using differentiation.
  • Adjusts weights to minimize error.

Probability

Definitions and Basics

  • Probability: Measure of likelihood of an event.
  • Terminologies: Random experiment, sample space, events.
  • Events: Joint (common outcomes) vs. disjoint (no common outcomes).
  • Always sums to one.

Distributions

  • Probability Density Function (PDF): Continuous probability function.
  • Normal Distribution: Bell-curve, dependent on mean and standard deviation.
  • Central Limit Theorem: Sampling distribution of means becomes normal with large sample sizes.

Types of Probability

  • Marginal Probability: Single event probability.
  • Joint Probability: Probability of two events happening simultaneously.
  • Conditional Probability: Probability of an event given another has occurred.

Bayes Theorem

  • Relationship between conditional probabilities.
  • Equation: P(A|B) = P(B|A) * P(A) / P(B)
  • Applications: Naive Bayes Classifier.

Statistics

Definitions and Basics

  • Statistics: Data collection, analysis, interpretation, presentation.
  • Terminologies: Population (entire set) vs. sample (subset for analysis).

Types of Statistics

  • Descriptive Statistics: Describes main characteristics; graphical summaries.
    • Measures of Center: Mean, median, mode.
    • Measures of Spread: Range, interquartile range, variance, standard deviation.
  • Inferential Statistics: Makes inferences and predictions based on sample data.
    • Techniques: Point estimation, interval estimation.

Hypothesis Testing

  • Testing the validity of a hypothesis through sample data analysis.
  • Null Hypothesis: No effect or difference.
  • Alternate Hypothesis: Indicates some effect or difference.
  • p-value: Probability value indicating the result’s significance.
  • T-value: Used for hypothesis testing comparisons.
  • Example: Assessing if a student has cheated based on test scores.

Confusion Matrix

  • Describes performance of classification models.
  • Components: True positives, true negatives, false positives, false negatives.
  • Accuracy Calculated as (TP + TN) / (TP + TN + FP + FN)

Example Applications

  • Naive Bayes Classifier: Uses probability for classification accuracy.
  • PCA: Dimensionality reduction to simplify data while retaining variance.

Summary

  • Mathematics is crucial in machine learning for model creation, data analysis, and understanding transformations.
  • Core Math Areas: Linear algebra, multivariate calculus, probability, statistics.
  • Practical applications through PCA, covariance matrices, classifiers, and optimization techniques.

Conclusion

  • Mastery of math fundamentals is essential for advancing in machine learning.
  • Practical coding examples enhance understanding and application.
  • Encouraged continuous practice and exploration of mathematical concepts for machine learning.

Back to Top