📐

Understanding Hermitian Matrices in Linear Algebra

Nov 8, 2024

Machine Learning Foundations: Week 5 - Lecture on Linear Algebra

Introduction

  • Focus on linear algebra continuation.
  • Goal: Understand the spectral theorem in the context of complex vector spaces.
  • Importance of Hermitian matrices (generalize real symmetric matrices).
  • Main result: Every Hermitian matrix is orthogonally diagonalizable.

Complex Matrices and Vectors

  • Complex Vector Space:

    • Denoted by (\mathbb{C}^n), analogue of Euclidean space (\mathbb{R}^n).
    • Elements are complex numbers (x_i) (e.g., (x_1, x_2, \ldots, x_n)).
  • Operations with Complex Numbers:

    • Addition: ((a + ib) + (c + id) = (a+c) + i(b+d)).
    • Multiplication: ((a+ib)(c+id) = (ac-bd) + i(bc+ad)).
  • Complex Conjugate:

    • Of (a + ib) is (a - ib).
    • Geometric representation using polar coordinates.

Inner Product in Complex Spaces

  • Inner Product Definition:

    • Different from Euclidean space due to non-real nature.
    • Defined using conjugates: (x \cdot y = x^* y) (conjugate transpose of (x) times (y)).
    • Key property: (x^* y \neq y^* x) for complex numbers.
  • Length of a Vector:

    • Defined as (||x|| = \sqrt{x^* x}).
    • Example: Length of ((1, i)) is (\sqrt{2}), not zero.

Conjugate Transpose and Hermitian Matrices

  • Conjugate Transpose (A^*):

    • Combination of transposition and conjugation.
    • For real matrices, (A^* = A^T).
  • Hermitian Matrices:

    • Equivalent to symmetric matrices in complex spaces.
    • Defined as (A^* = A).
    • Significant because they are orthogonally diagonalizable.*

Recap

  • Key Concepts:
    • Defined inner product for complex vectors (involves conjugation).
    • Introduced conjugate transpose for matrices.
    • Identified Hermitian matrices, a crucial class of matrices.
  • Significance:
    • Hermitian matrices simplify diagonalization, essential for spectral theorem.
  • Upcoming lectures will further explore Hermitian matrices and their properties.