Hello friends, so welcome to the first lecture of this course essential mathematics for machine learning. In this lecture we will learn or we will recall in fact, the concept of vectors. So, you know that we have done vectors in school level mathematics. So, from there we will recall some concepts from there and then we will try to relate those in context of machine learning.
To be very frank. is very basic entity of any machine learning algorithm. So, if I define it a vector, so a vector is a mathematical object that encodes a length and direction.
If I talk as a mathematician they are elements of a vector space, where we will put infinite vectors those share some common properties into a. we will put them together. So, it is a collection of objects that is close under an addition rule means if you add two vectors of a vector space the resulting vector will also belong to the same vector space and a rule for multiplication by scalar means if you multiply by a scalar to a vector the resulting vector will also.
in the same vector space. What we do in terms of representation? We represent vectors by one dimensional array.
This may be a vertical array means a column vector or a horizontal array that is a row vector. Geometrically vectors typically represent coordinates within a n dimensional space. where n is the number of components in that particular vector.
A simplified representation of a vector might be a row in a vector space with an origin, direction and length that is also called magnitude of the vector. So, let us try to understand all these concepts. So, let me take a vector V equals to V 1. So, it is a vector in a n dimensional space, if these V 1 are real numbers means all these V 1, V 2, V n belongs to set of real numbers, then I will say it is a real vector of dimension n.
And, I will say in that case it is a vector in vector space R n, we will learn this the concept of vector space more formally in third lecture. So, we can represent it by a one dimensional array which can be a row or I can write it in form of a column also. belongs to R n here each V 1, V 2, V n are real numbers.
So, for example, take n equals to 2 it means I am talking about vector space R 2 it is nothing just R by R. So, a real number means an order pair of real numbers. So, let us represent it. So, I am saying x and y axis.
Let me take a vector in this R 2, V equals to 1 and 2. So, here what you see this first component which is the component in the direction of x axis. The second component is nothing just corresponding to y direction. So, I am having a point here 1 2 and this is my vector v. The length of this is the magnitude of v and this is the angle the vector v. is making with x axis means to represent the direction of vector v. Similarly, in three dimensional space let me take x, y and z, we will be having a vector having three components, with in x direction, y direction and z direction.
z direction. So, for example, you take V equals to 1, 2, 3 here in R n a vector will be having n components. So, I cannot plot n dimensional vector here like I can do 2 d in 2 d or using some software I can plot 3 d.
But I cannot plot vectors those are having dimension more than 3 easily, but this is very abstract setting of a vector. Now, let us see some vector algebra, so first addition and subtraction. So, we can add or subtract 2 vectors if they are having the same dimension.
So, for example, if I take this in R 2, so let me take a vector V 1 which is 1 3 and V 2, let me take 1 minus 1, then what is V 1 plus V 2 or let me take 1 1, because it will be more easy to. So, V 1 plus V 2 you will add x component of V 1 with the x component of V 2. So, 1 plus 1 2 the y component of V 1 with the y component of V 2. So, 3 plus 1 is 4. Geometrically if you see let me say x and y. V 1 is 1 3. So, let me this is my vector V 1 which is 1 and 3 and V 2 is 1 and 1 1. So, let me take this is my vector 1 1. So, now sum of these two vectors will be so 1 1 will be somewhere here. So, 1 1 1 3 and 1 1. Now, the sum of these two vectors will be like this which is 2 4. So, this is the sum of V 1 and V 2. Similarly, V 1 minus V 2 will be you subtract first component of V 1 from the first component of V 2. So, 1 minus 1 0 and second component of V 1 from the second component of V 2 that is 3 minus 1 is 2. So, this is addition and subtraction similarly, we can have in R 3 or in general R n.
So, in R n if you are having two vectors let us say V 1 is x 1 x 2 x n and V 2 is y 1 y 2 y 1 then V 1 plus V 2 will be. add first component of both of these vector and write it as the first component of the sum of these two vector. So, x 1 plus y 1 x 2 plus y 2 and so on and then you will be having x n plus y n. So, this is v 1 plus v 2 similarly you will be having v 1 minus v 2 in this way. x 1 minus y 1, x 2 minus y 2, x n minus y n.
Now, dot product of 2 vectors. So, let me take 2 vectors v 1 equals to x 1, x 2, y 1, y n. x n and v 2 is again y 1 y 2 y n. So, both belongs to R n then dot products of v 1 with v 2 will be a scalar which is nothing just component wise multiplication.
1 y 1 plus x 2 y 2 plus x n y n or in short I can write it summation i equals to 1 2 n x i y i. So, for example, you take in R 3 a vector v 1 equals to x 1 y 1 plus x 2 y 1 1 minus 1 and another vector v 2 is 2 3 1. Then v 1 dot product of v 1 and v 2 is nothing just 1 into 2 plus 1 into 2 3 3 minus 1 into 1 minus 1. So, it comes out to be 4. So, this is the dot product between two vectors. Later we will see the concept of inner product which is generalized one version of this dot product. The third one is length or magnitude of a vector.
So, let me take a vector V in R n having component. x 1, x 2, x n. Then length of V is denoted by like this and again it will be a scalar.
So I am writing simply V. It is nothing just. Square root of dot product of V with itself. So, let me take V dot V will become x 1 square plus x 2 square plus x n square and square root of this. So, this in this way we can calculate length or magnitude of a vector V. So, for example, take v equals to 1 minus 1 2 then length of this vector v will become square root 1 square plus minus 1 square plus 2 square. This comes out to be square root 6. If the length of a vector means the length of a vector is 0, then the vector is 0 vector.
For a non-zero vector, the length or magnitude will be greater than 0. Another concept angle between two vectors. So, let me take two vectors. So, V 1 and V 2 belongs to R n then the angle between these two vectors is given by theta equals to cosine inverse V 1 dot V 2 upon 1 into V 2. So, what I am having in numerator, I am having the dot product of V 1 and V 2 and in denominator, I am having product of their lengths. This will give me the length between two vectors V 1 and V 2. So, here I am making use of dot product. Now, come to the concept of linear combination of vectors.
So, consider a set S of let me say k vectors v 1, v 2, v k. So, v 1, v 2, v k are some k vectors from some vector space. Then a new vector of the same dimension or in fact, in the same vector space V which is nothing just alpha 1 V 1 plus alpha 2 V 2 plus alpha n V n is called. Linear combination of V 1, V 2, V n where alpha 1, alpha 2, I am having k.
So, here alpha 1, alpha 2, alpha k are scalar. So, they are coming from the field on which vector space is defined we will learn very soon they are scalars. So, we can assume that these are real numbers at this moment.
So, for example, you take three vectors V 1 equals to 1 2 minus 1, V 2 equals to 1 1 0 and V 3 equals to 0 1 minus 1. Then I will be having linear combination as alpha 1 V 1 alpha 2 V 2 plus alpha 3 V 3. So, alpha 1 alpha 2 V 2 plus 1 2 minus 1 plus alpha 2 1 1 0 plus alpha 3 0 1 minus 1. In other way I can write it a new vector alpha 1 plus alpha 2 which is the first component of this linear combination. alpha 1 plus alpha 2 as the second component plus alpha 3 as the second component and minus alpha 1 minus alpha 3 as the third component. So, if you vary alpha 1 alpha 2 and alpha 3 or the set of real numbers then you will get different vectors from.
r 3 and those vectors can be formed by the linear combinations of v 1, v 2 and v 3. Next concept is linear independent and linear dependent vectors. So a set of vectors, let me take again S. which is having v 1, v 2, v n, n vectors is linearly independent if the equation or let me write vector equation. alpha 1 V 1 plus alpha 2 V 2 plus alpha n V n.
So, it is linear combination and this equals to 0 vector. So, here the 0 in the right hand side is a vector 0 vector of the same dimension which is the means of the dimension. to dimension of V 1 V 2 or V 1 or V 2 or V n.
So, if the vector equation alpha 1 V 1 plus alpha 2 V 2 and so on equals to 0 holds only when alpha 1 equals to alpha 2 equals to alpha n equals to 0. So, they are linearly independent if this vector equation equals to 0 holds only when these scalars alpha 1 alpha 2 alpha n are 0. So, what I want to say that you cannot write any of the vector. V 1, V 2 or V n in terms of other vectors. You cannot write V 1 in terms of vectors from the subset V 2 to V n and similarly true for other vectors V 2 up to V n. If this is not true means if this vector equation is 0. holds and some or all alpha is are non-zero, then we say that the set of vectors is linearly dependent. So, else the set has is linearly dependent.
So, for linearly dependent I will use L D in short, whereas for linearly independent I will use L I. So, let me take some example. So, first example I am taking from R 2. So, I am taking a set S which is having vector 1 0 and 1 1. So, let me take linear combination alpha 1 1 0 plus alpha 2 1 1 and this equals to 0 vector from R 2 that is 0 0 it gives me alpha 1 plus alpha 2 equals to 0 and second equation is giving me alpha 2 equals to 0. So, when alpha 2 is 0. put it in first equation you will get alpha 1 is also 0. So, this vector equation holds only when alpha 1 and alpha 2 both are 0, it means S contains linearly independent vectors. On the other hand, if I take another set S dash which is having vector 1 1 and 3 3. So, here if I take 3 times 1 1 plus or minus 3 times 1 1 plus 3 3. out to be 0 0, it means if you take alpha 1 equals to minus 3 and alpha 2 equals to 1, then the vector equation holds and hence S dash contains linearly dependent vectors. Believe me, we will make lot of use of this concept of linearly independent. dependent in subsequent lectures and in machine learning also.
Similarly, we can see an example in R 3, so you take a set let me take 1 minus 1 0 1 0 0. 0 1 1. So, what you can say and this belongs to R 3 each of these vector. So, what you can say about these vectors? So, if I take alpha 1 equals to 1 alpha 2 equals to minus 1 and alpha 3 equals to 1 just check what I will get alpha 1 into 1 minus 1 0 plus alpha 2 1 0 1 plus alpha 3 0 1 1 this equals to I am taking alpha 1 is 1. So, 1 minus 1 0 alpha 2 is minus 1 1 0 1 plus alpha 3 is 1. So, 0 1 1 this comes out to be 0, 0 and finally 0. Hence, I am getting alpha 1 V 1 plus alpha 2 V 2 plus alpha 3 V 3 equals to 0 vector. When alpha 1, alpha 2 and alpha 3 are non-zero, it means S is LD set of vectors or these set of vectors are linearly dependent.
Means I can write if I take it as V 1, V 2, V 3, so what I can write? I can write V 2 H. V1 plus V3. There is a linear relationship between these vectors and this you can verify V1 is 1 minus 1 0 plus V3 0 1 1. So, what you will be having 1 0 1 which is nothing but V2.
If this alpha 1 V1 plus alpha 2 V2 plus alpha 3 V3 holds. if alpha is R 0 then we will say these are a lie. One of the example in R 3 you take 1 0 0, 0 1 0, 0 0 1 that is standard basis in R 3. Similarly, we can extend this concept in R n. Now, come to next concept that is orthogonal vectors.
So, before going to orthogonal vectors I am having some remark about L i and L d vectors. So, first remark is in R n, we have A set of more than n vectors is L d, like in R 2 if you take a set having 3 vectors, is L d 3 or more vectors. In R 3 if you are having a set having 4 or more vectors that will be L d. Second any set of vectors containing 0 vector is L d.
And this you can easily see from the definition. Now, come to the concept of orthogonal vectors. So, we say that a set of vectors V 1, V 2, V n is orthogonal, V n are mutually or pair wise.
If V i is orthogonal, then V dot v j equals to 0 for all i not equals to j. So, for example, in R 3 you take a set of vector 1 0 minus 1 and then you take 1 root 2 1. 1 and then you take 1 minus root 2 1. Then you can check 1 0 minus 1 dot product with 1 root 2 1 equals to 0 1 0 minus 1 dot product with 1 minus root 2 1 equals to 0. and the dot product of second and third vector 1 root 2 1 with 1 minus root 2 1 equals to 0. So, they are pair wise orthogonal. We are having another concept which is orthonormal vectors.
So, this is a set. of orthogonal vectors is orthonormal if each vector has length 1, length or magnitude 1. So, a set of orthogonal vectors is orthonormal if each vector in this set has length 1. So, for example, you take vectors in R 2 1 by root 2 1 by root 2 and 1 by root 2 minus 1 by root 2. So, this is the set of orthogonal vectors. You can verify they are each of these vector is having length 1 and they are orthogonal also.
Here one remark I want to tell you that a set of orthogonal vector is a lie. means, orthogonality implies linearly independence, but converse is not true. In later part of this course, we will see a process by which we can make a set of a live vectors, we can convert a set of a live vectors into a set of orthogonal vectors. So, an example of feature vectors means. How we can see vectors in machine learning?
So, take a very simple data set, I am having data of the employee in an office, I am having their height and weight and then I am having employee id let us say e 1, e 2, e k and then I am having some number. alpha 1, beta 1, alpha 2, beta 2 and so on alpha k, beta k. So, for this data set E 1, E 2, E k are observations or samples, height and weight are features or attributes. Now, if I take any vector that is row corresponding to any sample, let us say alpha 20 beta 20. So, it is a feature vector of the 20th employee of employee E 20. So, in that way for each data set we will make the feature vectors. Let us see a brief implementation of these concepts which we have seen so far in Python.
So, for Python I will use Google Colab, one can use Jupyter notebook also or any other editor. So, for opening Google Colab you can type Google Colab in Google. So, first of all I will import a very important package from Python numpy. So, numpy is used for all array types of operations in Python. So, it is having lot of functionality related to multi-dimensional array, related to linear algebra and many more.
So, how to define a vector? So, I am defining a vector let us say v So, V will be a one dimensional array vectors are one dimensional array. So, I am taking a vector 1 minus 1 2, I am taking another vector W N P dot array and then I am having let us say. So, both of these vectors V and W are from R3.
So, print V plus W will give you the addition of these two vectors. So, you can see 3 4 4. So, 1 plus 2 3 minus 1 plus 5 4 and 2 plus 2 equals to 4. Similarly, you can print V minus W, you can see minus 1 minus 6 0, you can see scalar multiplication. So, I will print 3 star V means 3 times V, you can see 3 minus 3, 6, 3 into 1, 3, 3 into minus 1, minus 3, 3 into 2, 6. I can find out the length of this vector, let us say let me find the length of V. So, simply I can use a command for finding the length that is np dot lin else. So, lin else stands for linear algebra which is a sub package in numpy. And Then norm is for getting the length and of which vector?
Let me find out the length of vector v. So, you can see it is 2.44, it is nothing just square root 6. Let me also print the dot product. So, let me write s. So, dot product you can simply get N P dot and then vectors V and W let me take. So, it will give you dot product which is a scalar and store it assign to S and then you can print S to see the result. So, 1 and you can 1 plus into 2 minus 5 plus 4. So, this is So, 6 minus 5 which will be 1. So, similarly you can explore more operations related to vectors and we will do it in subsequent lectures.
In next lecture we will take some basic matrix algebra with this let me close this lecture I hope you have enjoyed it. Thank you very much.