Gram-Schmidt Process (Orthogonalization)

Overview

• Goal: Given a subspace of (\mathbb{R}^n), find an orthogonal or orthonormal basis.
• Process: Start with any basis for (W) and construct a new orthogonal or orthonormal basis.

Steps

1. Initial Basis: (X_1, X_2, \ldots, X_p)
2. Construct New Vectors: (V_1, V_2, \ldots, V_p)

Construction of (V_1, V_2, \ldots, V_p)

• Step 1: (V_1 = X_1)
• Step 2: (V_2 = X_2 - \frac{X_2 \cdot V_1}{V_1 \cdot V_1} V_1)
  • Note: This represents the projection of (X_2) onto (V_1). The result is orthogonal to (V_1).
• Step 3: (V_3 = X_3 - (\text{proj}{V_1} X_3 + \text{proj}{V_2} X_3))
  • Note: Subtracts components of (X_3) in the directions of (V_1) and (V_2), the result is orthogonal to both.

Theorem

• The constructs (V_1, V_2, \ldots, V_p) form an orthogonal basis for (W).

Orthonormal Basis

• Process: Use Gram-Schmidt to get orthogonal basis and normalize each vector.
• Normalization: Divide each vector by its length, (\text{length}(V_i)).
  • (\text{u}_i = \frac{V_i}{|V_i|})

Example

• Given: 3 vectors in (\mathbb{R}^4), (X_1, X_2, X_3), spanning (W).
• Step-by-step:
  • Confirm linear independence of (X_1, X_2, X_3) via row reduction.
  • Compute Vectors: Use formulas to find (V_1, V_2, V_3).
    • (V_1 = X_1)
    • (V_2 = X_2 - (-\frac{2}{6})V_1)
    • (V_3 = X_3 - (\frac{4}{6}V_1 - \frac{8}{3(\frac{7}{3})}V_2))
• Normalize:
  • (|V_1| = \sqrt{6})
  • (|V_2| = \sqrt{\frac{7}{3}})
  • (|V_3| = \frac{4}{\sqrt{7}})
  • Result: (u_1, u_2, u_3) as orthonormal basis.

Practical Application

• In practice, use software for computation (e.g., Mathematica).
• Mathematica Command: Orthogonalize
  • Input: List of vectors [(X_1, X_2, X_3)].
  • Output: Orthonormal basis.

Conclusion

• The Gram-Schmidt process transforms any basis into an orthogonal or orthonormal basis, ensuring the new basis spans the same subspace.