Essential Mathematics for Machine Learning: Lecture 1 - Vectors

Introduction

• Vectors are fundamental in machine learning algorithms.
• A vector is a mathematical object representing both length and direction.
• Elements of a vector space, closed under addition and scalar multiplication.

Vector Representation

• Vectors are represented as one-dimensional arrays: column vectors (vertical) or row vectors (horizontal).
• In n-dimensional space, vectors represent coordinates, where n is the number of components.
• Example: Vector V in R^n (real vector space) consists of components V_1, V_2, ..., V_n.

Vector Spaces

• Real vectors belong to vector space R^n.
• Example in R^2: Vector V = (1, 2) with components in x and y-directions.

Vector Operations

Addition and Subtraction

• Vectors can be added/subtracted if they have the same dimensions.
• Example: V1 + V2 = (x1 + y1, x2 + y2, ..., xn + yn).

Dot Product

• Dot product of V1 and V2 in R^n is a scalar: V1 • V2 = Σ(xi * yi).
• Example: V1 = (1, 0, -1), V2 = (2, 3, 1), V1 • V2 = 12 + 03 - 1*1 = 1.

Length (Magnitude)

• Length of vector V: ||V|| = sqrt(V • V).
• Example: V = (1, -1, 2), ||V|| = sqrt(1^2 + (-1)^2 + 2^2) = sqrt(6).

Angle Between Vectors

• Angle θ between vectors V1 and V2: cos(θ) = (V1 • V2) / (||V1|| * ||V2||).

Linear Combination

• New vector V from vector space is a linear combination: V = α1V1 + α2V2 + ... + αkVk.

Linear Independence and Dependence

• Vectors V1, V2,..., Vn are linearly independent if α1V1 + α2V2 + ... + αnVn = 0 only when all αs are 0.
• Linearly dependent if this holds with non-zero αs.

Orthogonal and Orthonormal Vectors

• Orthogonal: Vectors V1 and V2 are orthogonal if V1 • V2 = 0.
• Orthonormal: Orthogonal and each vector has length 1.

Applications in Machine Learning

• Vectors can represent data, e.g., feature vectors of employee data.

Implementation in Python

• Use of numpy for vector operations.
• Examples of vector addition, subtraction, scalar multiplication, dot product, and length calculation in Python.

Conclusion

• Introduction to vectors and basic vector algebra.
• Next lecture will cover matrix algebra.