Linear Algebra for Data Science: Lecture Notes

Hyperplanes

• Definition: A hyperplane is a geometric entity whose dimension is one less than that of its ambient space.

  • Example: In 3D space, a hyperplane is a 2D plane.
  • Example: In 2D space, a hyperplane is a line.

• Equation:

  • For n variables: X1 * n1 + X2 * n2 + ... + Xn * nn + B = 0
  • For 2D: X1 * n1 + X2 * n2 + B = 0

• Subspaces: Hyperplanes generally are not subspaces. However, if a hyperplane passes through the origin, it can be considered a subspace.*

Half-Spaces

• Definition: A half-space divides the space into two parts with respect to a hyperplane.

• Example: In 2D, if a line (hyperplane) is drawn, the space is divided into two half-spaces.

• Classification Problem:

  • Binary Classification Problem: Involves distinguishing between two classes based on their characteristics.
  • Example: Class 1 (likes South Indian food) vs Class 2 (doesn’t like South Indian food).
  • New data points are classified based on their proximity to the hyperplane.

Discriminating Function

• To classify points, define a discriminating function that helps determine which side of the hyperplane a point lies.

Evaluating Points Relative to a Hyperplane

• To determine which side of the hyperplane a point lies, evaluate the function X^T * n + B:
  • If the result is positive, the point is on one side of the hyperplane (half-space).
  • If the result is negative, the point is on the other side.
  • If the result is zero, the point is on the hyperplane.*

Eigenvalues and Eigenvectors

• Linear Equations: Exploring Ax = B, where A is an n by n matrix, x is an n by 1 vector, and B is also an n by 1 vector.
• Interpretation: When a operates on x, it generates a new vector B.
• Eigenvalue Definition:
  • Directions (eigenvectors) that when multiplied by A do not change orientation, represented as Ax = λx.
• Finding Eigenvalues:
  • Rearranging gives: (A - λI)x = 0.
  • Non-trivial solutions exist if the determinant |A - λI| = 0.
• Calculating Eigenvectors:
  • For each eigenvalue, substitute back to find the corresponding eigenvector.

Example Calculation

1. Consider matrix A = [[8, 7], [2, 3]].
2. Compute determinant (A - λI).
3. Solve to find eigenvalues (λ1 = 10, λ2 = 1).
4. For each eigenvalue, find corresponding eigenvector ensuring unit magnitude.

Summary

• Eigenvectors are associated with eigenvalues and can be computed through the determinant of the matrix.
• The next lecture will connect eigenvalues and eigenvectors to column space and null space in more detail.