Hey folks, my name is Nathan Johnston. Welcome to Lecture 24 of Advanced Linear Algebra. Today we're going to talk about unitary linear transformations and unitary matrices, and the idea is now that we're comfortable with inner products and norms induced by inner products, we would like to investigate a family of linear transformations that leave those quantities alone.
In other words, we want to look at which linear transformations leave the inner product alone, and also which linear transformations leave the norm induced by the inner product alone. Okay, so our starting point is actually dealing with norms. Okay, what linear transformations leave the norm induced by an inner product alone? So that's our first definition. It's a unitary linear transformation.
Okay, so our setup is the same as usual when we're working with things based on inner products. Suppose you've got some inner product spaces V and W and you've got a linear transformation that sends one of them to the other. Well, we say that that linear transformation is unitary if it doesn't change the norm induced by the inner product. If the norm of T of V always equals...
the norm of v no matter what v from the input space you pick. Okay so unitary transformations they don't alter the norm. Let's just do a quick example here and just sort of as a brief note on notation here we also say that a matrix is unitary if the same thing if it doesn't alter the norm induced by the inner product which is just the dot product right if you're working on fn you're thinking of a matrix as a linear transformation that sends fn to fm. Okay so let's show that the matrix u equals 1 over root 2 1 minus 1 1, 1. Let's show that that's unitary.
In other words, what we want to show is that if I compute this vector u times v and then compute its length, I want to see that I get the same thing as just the length of v itself, no matter what vector v I plug in there. And just for the sake of making the calculation a little easier to go through, I'm going to square all these lengths just so that I don't have square roots to deal with at every single step in the calculation. But then at the very end of the calculation, you can square root both sides to see that the norms themselves are the same as well. All right, so here's how it works.
We want to compute the norm of u times v. So let's just plug in what u is and what v is. Okay. And then do that matrix multiplication. Okay. So to go from here over to here, I just did matrix multiplication and I got this column vector and I want to compute its norm.
And remember the norm when I'm working just in Rn or Cn is just the usual Euclidean norm, right? The two norm. You sum up the squares of the absolute values of the entries and then square root at the end of the day. Here, the whole thing's squared. So I don't have the square root on top of everything.
All right. So it's this squared plus this squared. And I get that guy there.
And now I just expand this out and do some algebra on it. These are all real numbers. It's just a matrix acting on Rn. And then what happens is when you expand this bracket out, you get a negative cross term. And when you expand this one out, you get a positive cross term.
And wouldn't you know it, they just cancel out. And all you're left with is you get a half v1 squared plus a half v1 squared gives you a whole v1 squared. And similarly for the v2 squared term, So that is exactly just the norm of v squared. So the norm of uv squared equals the norm of v squared, no matter what v you pick, okay?
So yes, our conclusion is that matrix really is unitary. It leaves norms of vectors alone, okay? It does not change them, okay? And if you think about this matrix geometrically, this makes sense because this u here, this matrix is actually a rotation matrix, right? Remember rotation matrices, they have a cos theta and a cos theta along the diagonal.
and they have a minus sine theta and a sine theta in the off-diagonal entries. And this matrix here is exactly of that form with theta equal to pi over 4, 45 degrees. Okay, so we can draw a little picture of that just to remind you of how this works if we just start off with a copy of R2 there.
If we apply this unitary to it, then E1, the first standard basis vector, gets moved over to 1 over root 2, 1, 1, so just sort of up to the right there. And E2 gets sent over here to minus 1, 1 over root 2, so off to the left here. And this angle here... is 45 degrees. It just took our entire coordinate system and rotated it counterclockwise by 45 degrees.
So of course it doesn't change the lengths of vectors, right? It's just rotating them. It's not stretching them at all.
Okay, our first big goal for today's lecture is to characterize unitary matrices. Unitary matrices, they're one of these really nice families of matrices where, yeah, they have this defining property, which happens to be in terms of the norms of vectors in this particular case, but it's also equivalent to a whole bunch of different things, okay? So let's see how it works.
In particular, one of these characterizations is going to make it a lot easier to actually check whether or not a matrix is unitary or not, okay? So Here's how it works. Okay, suppose that you're working with real or complex numbers as your ground field so that we have inner products to work with in the first place.
And then you've got some matrix, n by n matrix. It has to be square for these things to make sense, over that ground field. In other words, with entries from that field. Then the following are all equivalent.
Property A, A is unitary. And again, remember what that means is U, as a linear transformation, preserves the norms of vectors. Alright, but it turns out that's equivalent to, well, if you take u and multiply on the left by u star, the conjugate transpose or adjoint of that matrix, then you get the identity matrix. In other words, the inverse of u is just its adjoint, its conjugate transpose. Okay, that's equivalent to the matrix being unitary.
That's equivalent to it preserving norms. Property C is just the same thing with multiplication on the other side, and again, that follows fairly straightforward. just from things that we know about inverses, right? If u star is the inverse, well, you know, it doesn't matter which side you multiply on, you're going to get the identity. This is also equivalent to property d here.
Property d says that, well, if your matrix is unitary for preserves norms, then actually it preserves inner products, okay? It says that the dot product of v with w always has to equal the dot product of uv with uw, okay? So a unitary matrix is also, it's exactly one that preserves the dot product, it doesn't change it.
Property E says that, well, your matrix is unitary if and only if its columns form an orthonormal basis of the vector space that acts on as a linear transformation, right? It's an n by n matrix, so it's a linear transformation acting on fn, okay? Property F says, well, the same thing except with rows of the unitary matrix rather than columns, okay? So those are all equivalent to each other, okay?
If one of them happens, all of them happens. And before we prove this theorem, we're actually not going to prove the whole theorem, we're just going to prove pieces of it. But before we even prove any of it, it's worth thinking about these properties a little bit more and how some of them compare to properties of invertible matrices.
Because the way you should be thinking about unitary matrices is they're like invertible matrices, except even nicer. So let's think about this. So what I'm going to do is I'm going to draw a table here where we've got invertible matrices on the left.
I'm just going to call them P and then unitary matrices. I'm going to call U on the right here. One property, well, sort of the defining property of invertible matrices was that, you know, they had an inverse. So P inverse existed. Well, for unitary matrices, you can say something more.
Not only does U inverse exist, but U inverse is just the adjoint of that matrix or that linear transformation. It's conjugate transpose. All right. And one of the really nice properties, again, you can take this as a definition if you like.
It's equivalent to a matrix being invertible. One really nice property is that, well. Invertible matrices, they send non-zero vectors to non-zero vectors, right? The only vector that gets mapped to zero by an invertible matrix is the zero vector.
So another way of phrasing that is, well, if the length of a vector is non-zero, then P keeps that length non-zero, okay? So it doesn't necessarily preserve the lengths of vectors, but preserves non-zeroness of the lengths of vectors. Well, unitary matrices just take that one step further, and they really preserve the lengths or norms of vectors.
All right. Another nice property of invertible matrices, again you could take this as a definition if you want, it's equivalent to a matrix being invertible, is that a matrix is invertible if and only if its columns form a basis of fn. Not an orthonormal basis, this is what's different between invertible matrices and unitary matrices.
For just regular old invertible matrices, columns form a basis, but for unitary matrices, the columns form an orthonormal basis. So again the idea here is you know it's just like invertibility except even nicer it's a stronger property. Okay and one more thing that I should note is yeah we really should dwell a little bit on this property d here okay we defined unitary matrices as ones that preserve the norm induced by an inner product but property d says we could have equivalently defined them as matrices that preserve the dot product as a whole right if you apply the unitary matrix on both half of both halves of a dot product.
you leave that dot product alone. Okay, so you can sort of think of unitary matrices as the sort of the inner product space version of just invertible linear transformations. Invertible linear transformations, they preserve vector addition and scalar multiplication, they preserve your vector space operations. Well in inner product spaces you have one more operation, the inner product, and hey, unitaries preserve that extra operation as well. So there's sort of the natural objects, the natural linear transformations to work with when you have an inner product floating around.
Let's look at how we can prove some of the equivalences of this theorem. We're not going to go through all of them, but we'll at least go through some of them to give you the general idea of where this theorem comes from. Let's start off with properties B and C.
Let's see why they're equivalent. And there's not much to this one. We proved pretty much everything we need for this back in linear algebra 1, introductory linear algebra.
We saw in that class that if you have any two square matrices whose product is the identity, well, then the product in the other order must be the identity as well, okay? And this is the identity on the right-hand side is very special here. Usually matrix multiplication is not commutative, but when the right-hand side is the identity, it is, okay?
Another way we phrase this typically is saying that, you know, one-sided inverses really are two-sided inverses for matrices at least, right? If you get, you know, this product in one way, then B must be the... inverse of A so the product works the other way as well.
All right so that's just a linear algebra one fact. All right next implication that we're going to show is that hey property D here so this inner product preservation property implies property B here so we're going to see why D implies B. All right, and the way that we're going to show this is we're going to look at this equality here and we're just going to rearrange it a little bit.
Okay, so I'm going to use properties of adjoints. We learned about this last lecture. I can move this u over to the right hand side as long as I change it to a u star, as long as I change it to an adjoint.
So I end up getting this equality here v dotted with u star u star u w equals v dot w. So I didn't do anything on the right hand side. And now I'm just going to manipulate this a little bit, okay?
I'm going to take this right-hand side and subtract it from both sides so that it's on the left, right? I'm just going to get this minus this equals zero. And then when I use linearity in the second entry of the dot product, I can group that together like this. This is v dot, well, u star uw minus v v dot just w itself.
And that's got equal zero, no matter what v and w I pick. Okay, and this sort of has the flavor of what I want, right? Remember, I'm trying to show property b holds.
And property B was the statement that u star u has to equal the identity matrix. And certainly this is getting very close there. I sort of feel like that's implying that u star u has to equal i. It's kind of looking like this matrix here had better be the zero matrix.
So the way we pin that down is, well, we make use of the fact that this holds for all v and w. So I'm going to make a clever choice of v. I'm going to choose v to be this junk that's over here on the right-hand side. And then plug that in and see what happens.
If I choose v to be that particular vector. then what I find is okay zero equals this junk I just replaced v by that cleverly chosen vector but that junk here the point is that junk this dot product of a vector with itself that's the norm induced by the vector squared and remember the only way a norm induced by an inner product can equal zero is if the vector itself equals zero so this junk's got to be the zero vector okay but if this is the zero vector no matter what w I pick Well, that means matrix times vector always equals zero. It shouldn't be too hard to convince yourself.
That means this matrix has to be the zero matrix. The zero matrix is the only one with the property that when you multiply it by a vector, you always get zero, okay? So u star u minus i is zero. So in other words, u star u equals i, which is what we wanted to show. That was d implies b.
All right, let's show that b implies a. So what is that? Let's just go back up here. b implies a. So we're gonna show that if u star u equals i, then u must be unitary.
In other words, you must preserve the norm induced by the inner product. So where does this come from? Well, if u star u equals i, let's just start off, let's try to compute the norm of uv. And again, I'm just going to square it so I don't have square roots throughout my entire calculation. Well, the norm of uv squared is just uv dotted with uv.
Okay, and now I'm going to use properties of adjoints. Again, I can move this u over to the right-hand side as long as I throw a star on it, as long as I change it into the u adjoint. And then I get this expression here.
But now, well, here I've got a u star u. Ah, well, I'm assuming that u star u equals i. It's the identity matrix.
So this guy here is just v dotted with v. And of course, v dotted with v is just the length of v squared. It's the norm induced by the inner product squared. So yeah, u preserves. Norm induced by the inner product.
So U is unitary. We're happy with that one. All right. And then the last one that we're actually going to go through here, there are more implications to show, but this will maybe be enough. Let's go through.
Let's see that property B is equivalent to property E. So property B, again, U star U equals I. We're going to show that that's equivalent to the columns of U forming an orthonormal basis of the vector space of that matrix axon. Okay.
So let's see why those are the same as each other. Alright, so what we're going to do, we want to prove something about the columns of U. So let's start off by writing U in terms of its columns.
And now let's just compute this product U star U. Again, we want to show that that being equal to I is equivalent to these columns here forming an orthonormal basis. So let's just compute this product and we're going to use block matrix multiplication to actually evaluate it. So U star is this first thing here and U1 star, that's a row vector now. U2 star, that's a row vector.
and so on. So all of these, these are the rows of u star and then here these are the columns of u itself. And when you do that matrix multiplication, all that happens is I'm gonna get u1 star times u1 as my top left entry and then u1 star times u2 as my next entry and all the way over to u1 star times un as my top right entry and I just go down one row.
u2 star times u1 is my first entry in the second row, u2 star times u2 is, you know, the 2, 2 entry, and so on until you fill out the entire matrix. So the matrix, you know, every entry, the i, j entry is just u, i star times u, j. All right. And now, well, we want to show that this equals the identity matrix if and only if these u's form an orthonormal basis. Well, how could this equal the identity matrix?
It equals the identity matrix if two things happen. The first thing is that all of these diagonal entries have to equal one, right? Because, well, identity matrix has ones on the dot. But, I mean, how can those diagonal entries equal 1? u1 star times u1, that's exactly u1 dot u1, right?
All of these entries are actually dot products just written in matrix multiplication notation, right? So this, this is u1 dot u1, and we want that equal to 1. In other words, we want the length or the norm of u1 to equal 1. And similarly here we want the second diagonal entry to equal one so we want u2 dotted with u2 to equal one so we want the norm induced by the inner product to equal one and that happens all the way down the diagonal. So the diagonal entry is equal one if and only if every vector in this set every column of u has length one has norm induced by the inner product one.
Okay and now look at the off diagonal entries again if this is the identity matrix now we need all of the off diagonal entries to equal zero so we want u1 dotted with u2 t equals 0. We want u1 dotted with un t equals 0 and so on. We'd want ui dotted with uj t equals 0 whenever i doesn't equal to j. In other words, this is equivalent to these columns here being orthogonal to each other.
And these two properties taken together are exactly the defining properties of an orthonormal basis, right? They're mutually orthogonal and they have length equal to one. That's an orthonormal basis because there's n of them in an n-dimensional space. Great, and that's basically the proof. I mean the other implications they're all more or less similar to those ones there.
So we're not going to check any more of them. Alright so one really nice thing about this theorem, this big beast of a theorem that we went through, is now it's really really easy to check whether or not a matrix is unitary because you can just use property B or property C. Whichever one you like. You just take the matrix multiply it by its conjugate transpose. If you get the identity, great, you're unitary.
If you get something else, too bad you're not unitary. Okay, so let's go back to this 2 by 2 matrix that we saw earlier, this rotation matrix, and let's show that that's unitary now using this theorem. So all we have to do is we have to compute u star u. Okay, so just do that matrix multiplication. Here's u star, here's u, you multiply them together.
Well, I would get 1 1 plus 1 1, that gives me a 2 there, then I get 1 1 dotted with minus 1 1, that gives me a 0 there, and so on. I get, you know, these guys multiply together to 1 half, so I get 1 half of 2 0 0 2, which of course, That's the identity matrix. So yeah, U is unitary. That's all there is to it now.
You just do that product, identity matrix, great. It's unitary. And again, of course it's unitary.
It's a rotation matrix, okay? Well, let's sort of ramp up a little bit now, now that we have this easy method of checking unitarity. And let's show that yes, every rotation matrix is unitary, no matter what the angle is. We just showed it for the angle pi over four, 45 degrees. Let's show it for every angle now.
All right, so. How do we do this? Well, again, first off, by geometry this is obvious. All this is asking us to do is show that every matrix that just rotates vectors does not change their length, does not change their norm induced by the inner product. And, well, yeah, I mean, you just said that, right?
I mean, it rotates the vector. It doesn't stretch it or anything like that. Okay, well, let's prove it algebraically though, okay, just to get ourselves comfortable with this new method.
Okay, well, Algebraically, a rotation matrix, a 2x2 rotation matrix, what it looks like is cos minus sin sin cos. This is the rotation matrix that rotates R2 counterclockwise by an angle of theta. So to check that this really is a unitary matrix, just multiply together u star u, the conjugate transpose of u with u itself.
Here's u star, here's u. You do that product, and it looks like a big hideous ugly mess at first when you do the matrix multiplication. But in the top left corner here, you've got cos2 plus sin2.
Didn't you remember it back? Oh, great. That equals 1. I know a trig identity, right?
Similarly, in the bottom right entry, sin2 plus cos2. Oh, cool. That equals 1 as well. And these two off-diagonal entries, well, they're just something minus that same something. They both just equal 0. OK?
So you really do get the identity matrix here. You get 1's on the diagonal, 0's off-diagonal. So yeah, u is a unitary matrix.
All right. Well, let's do a similar sort of thing. with reflection matrices now. A reflection matrix, reflection matrices first off you can consider them in acting on n-dimensional space, and by a reflection matrix here I'm going to mean a reflection across a line, just because we've seen a formula for these matrices. The n by n matrix that reflects space across a particular line in that space is, well it has this formula here.
u equals two times uu star minus the identity matrix, where u is just some unit vector pointing in the same direction as the line that you want to reflect space across. So we showed this in the previous course. We showed that this is the formula for the standard matrix of a reflection through the line in the direction of u. All right, so to show that this is a unitary matrix, again, obvious by geometry, if you reflect a vector through a line, you're not changing how long it is.
But to show it algebraically, we just multiply this by u star and we see what we get. Okay, so u star times u just equals, well, here's u star, here's u. And now we're just going to do all sorts of properties of matrix multiplication to expand this out. Okay, so first off, when we do the star... Sorry, when we do the star on this first matrix here, nothing actually happens, right?
The star distributes inside the brackets, but the star doesn't do anything to this because it just reverses the order of multiplication and puts a star on both of them. You're going to get the exact same thing back. You're going to get 2 times u, u star. And the star of i, the identity matrix, is just identity matrix itself.
So doing the star here doesn't really do anything. All right, so now we can just expand out the brackets. It's this times this.
Okay, so we're going to get 2. uu star times 2 uu star that gets us this first term here this is 4 uu star uu star okay and i've just inserted some parentheses here to sort of indicate what i'm going to do next okay the parentheses don't affect anything right i mean matrix multiplication is associative so i can throw those in wherever i want and i want them there all right next up let's deal with the cross terms i've got minus i times 2 uu star and here i've got minus i times 2 uu star so altogether i get minus 4 uu star And then over here I get minus i times minus i, which is just a plus i. The minuses cancel out, right? And now let's simplify a little bit. And the key observation to make here is this middle term here that I've parenthesized, u star times u, well that's u dot u, right?
That middle piece of this product here is just the dot product of u with itself. But, I mean, u is a unit vector. It has length 1. It has norm induced by the inner product 1. So this u dotted with itself is going to equal 1. Forget about it, it goes away. So you're just left with 4u and then the u star from the right hand side. So you get 4u u star minus 4u u star from, you know, this cross term, and then plus identity.
And isn't that beautiful? I mean, these first two terms just cancel out, you're left with the identity matrix. So yes, again, u is unitary just because this product here happens to equal the identity matrix. Okay, and these previous two examples, rotation matrices and reflection matrices, are the things you should really keep in mind when you think about unitary matrices.
The idea behind unitary matrices is just, it's a way of defining an arbitrary dimensional version of rotations and reflections. They're linear transformations, but they're very rigid linear transformations. They can't deform space at all. Maybe they reflect it, maybe they rotate it around, but that's kind of all they can do. They can't stretch space at all, so they're sort of again, they're very rigid, they're very structured, they're like invertible matrices, but even more restricted than that.
Okay, so that'll do it for today's class. I will see you in the next lecture for week seven's notes.