Lecture 18: Advanced Linear Algebra

Introduction to Inner Products

• Previous lecture:
  • Introduction to inner products
  • Standard inner products on vector spaces like ( \mathbb{R}^n ) and ( \mathbb{C}^n )
  • Vector space of continuous functions and its inner product (integral)
  • Various non-standard inner products

Today's Focus: Inner Products on Matrices

• Goal: Introduce the concept of inner products on matrix spaces
• Key concept: Trace of a Matrix

Trace of a Matrix

• Definition:
  • Simplest function on square matrices
  • ( \text{tr}(A) = \sum ) of diagonal entries ( a_{11}, a_{22}, ..., a_{nn} )
• Examples:
  • Matrix ([1, 2; 3, 4]), Trace = 1 + 4 = 5
  • Matrix ([1, 2, 6; 0, -4, 8; 3, 1, 2]), Trace = 1 - 4 + 2 = -1_

Importance of the Trace

• Theorem:
  • Matrix multiplication is not commutative: ( AB \neq BA )
  • Trace property: ( \text{tr}(AB) = \text{tr}(BA) )
  • Allows treating matrix multiplication as commutative in some contexts

Proof of Trace Theorem

• Compute diagonal entries of AB and BA
• Diagonal entries themselves may differ but their sums (traces) are equal
• Proof involves swapping sums and products, showing equality

Properties of the Trace

• ( \text{tr}(A + B) = \text{tr}(A) + \text{tr}(B) )
• ( \text{tr}(cA) = c \cdot \text{tr}(A) )
• Trace as a linear transformation
  • Provides commutativity property
  • Nicest linear transformation from matrix to number
• ( \text{tr}(A^T) = \text{tr}(A) )

Inner Product on Matrix Spaces

• Definition:
  • Standard inner product between matrices A and B: ( \text{tr}(A^*B) )
  • ( A^* ) is the conjugate transpose of A

Properties and Interpretation

• Trace of ( A^*B ) as dot product of coordinate vectors of A and B
• Equivalent to dot product on vector space of matrices
• Known as the Frobenius or Hilbert-Schmidt inner product*

Conclusion

• Trace and inner products on matrices provide powerful tools in linear algebra
• Next lecture: Continuation of matrix-based inner products and applications