[Music] okay welcome to week five of machine learning foundations we will be continuing the discussion of linear algebra all through this week and one more week after this the goal of this week the lectures in this week is to finally get to the most important theorem in linear algebra which is the spectral theorem and we will get to this in a more general form not necessarily real vector spaces but we look at this theorem in the context of a complex vector space and the important class of matrices that you know that generalize real symmetric matrices is hermitian matrices so we will show we will prove spectral theorem in the context of a hermitian matrix to show that it's the main result very simple term says that every hermitian matrix is orthogonally diagonalizable so as a gentle start in this lecture we look at complex matrices complex vectors and complex matrices and try to understand how we can define inner product between two complex vectors in the more natural form and then we will also define what it means to take the conjugate transpose of a complex matrix and use this notion of conjugate transpose to define a hermitian matrix so we will denote this by c n this is the complex counterpart of the usual euclidean space in dimensional euclidean space rn so when i say x1 up till xn belongs to this then what i am saying is each x i is a complex number for i equals 1 2 now i mean this is something that you may already know for your addition of complex numbers if i have e a plus i b plus c plus i d then i add the real and imaginary parts and multiplication also this will be ac minus b t plus i into bc plus e d this is something uh that i hope you are all familiar with so i'm not going to spend too much time on this now the important thing to understand is this complex conjugate complex conjugate of a complex number a plus ib is a minus i b we to understand this is if you have a complex number you can have use the polar coordinates to represent a plus i b as r e power i theta and now this is the real axis and this is the imaginary axis and now the conjugate would be written as a minus i b this is the conjugate denoted like this and that can be represented as r e power minus i theta where r square is this is nothing but the length of this vector now you can take linear combinations as before linear combinations with a slight tweak so if i write c 1 b 1 plus let me use x one c n x n equals zero let's say k if i take k vectors x k equals zero then the difference from before is these scalars could be complex numbers okay and these belong to here now the next bit of background to cover is inner product and also the length the concept of length of a vector in a comple in the complex case so recall in euclidean space this was your dot product this was the inner product and that was used to define the length of a vector okay now we cannot use the same definition in the complex case why because it does not make intuitions take an example for example using star what would be the length of 1 i this would just be 1 square plus i square which will just be 0 okay so in the case of a complex vector space define the inner product a little differently using what are known as conjugates as if i want x dot product with y that will be x bar transpose y which is like i take the conjugate of the first one and multiply with the second one the important thing is this relation very important note using this we can easily see that x bar transpose y is definitely not equal to y bar transpose x i mean if you want take for example x is 2 minus i 1 plus i and y s i mean if you take real numbers if you take real x and y then obviously this relation holds so you need to take complex numbers now check x bar transpose y is not equal to y bar transpose now how do i length define the length of a complex vector the way i add voltage x belongs to this define the length as x bar transpose x now with this you know the length of one comma i what would this be this won't be 0 when you take the conjugates but you can check that this turns out to be square root 2 and it's not a 0 length vector which is which coincides with our intuition so 1 comma i is not a 0 length vector and [Music] so our definition if that vector is 0 otherwise we can otherwise we won't get the zero length vector now you can check this i am going to just check these 1 this can be done by definition if i take x dot y is y dot x conjugate and if i do x dot some constant times y then this is going to be just that constant times x dot y simply because you are taking the conjugate of the first component in the inner product and but the same is not true of this one when you do it like this the constant comes out as a conjugate okay so check this this can be verified by definitions yeah check this [Music] using definition of dot product in our product that is x dot y equals x bar transpose y summation x bar i do y equals 1 now given this we define what is called conjugate transpose everybody knows transpose so here you add the conjugate so so a star is that matrix which is the conjugate transpose of a and it is defined as a star equals you can do this in either order and they lead to the same solution as an example suppose a is this 1 plus i 3 minus 2 i 2 minus 4 i and i if you have this then what this a bar a bar is just 1 minus i 3 plus 2 i 2 plus 4 i minus a and a bar transpose a star equals a bar transpose equals we would get the same result by transposing first and then taking the conjugate get the same result that won't change important remark for a real matrix a star is nothing but a transpose because for a real number it is its own conjugate okay now some claims again this can be checked check these are just the equivalent of transpose that we have if i do a star and do another star what i recover is this and a v star is p star a star now the real in the real case what you would get is the equivalent is equivalence is a transpose transpose equals a and a b transposes b transpose a transpose just reiterating the inner product so not to have defined in one place x dot y is x bar transpose y so this is x 1 bar y 1 blah blah blah x n bar y n this can be written as x 1 bar until x n bar y 1 2 y n but this is the same as x star into y okay so x dot y equals this the last bit of definition before we close this lecture is what is called as a hermitian matrix matrix a is hermetian this is the equivalent of symmetric matrices [Music] if a star equals a hermitian matrices are the equivalent of [Music] symmetric matrices in a complex vector space for a real symmetric matrix you have you know a transpose equals a and for hermitian matrix if i take the conjugate transpose let's see it is the same so quick recap before i end this lecture uh what did we look at the you know the what are the contents of this lecture at a very high level uh we understood how we can define the inner product between two complex vectors and this involved some change in particular we had to do a conjugate operation instead of just the transpose we had the conjugate operation in addition and the second thing that we looked at was conjugate transpose the notion of conjugate transpose which when applied to matrices you know we used we applied it to a matrix and if a matrix and its conjugate transpose coincide that's the so-called hermitian matrix and it's very it's a very important subclass of complex matrices which will be concentrating on because these are matrices that will be orthogonal diagonalizable which will become apparent as we go on in this week in the forthcoming lectures