hello everyone and welcome to this lecture in this lecture i wanted to show you guys a tool known as tensorflow playground where we can visually see how we build an artificial neural network and how do we actually train it as well how to change the number of layers and how to also change the number of neurons activation layer and so on okay please note that this tool is independent of the teachable machines but it's again provided by google as well okay all right so let's go ahead and get started so this is simply the type of artificial neural network that we are going to build together using tensorflow playground just to clarify so tensorflow is google's framework to build train and deploy ai and ml models at scale and please note that tensorflow specifically is beyond the scope of this course so we are not going to learn i would say how to code in python or leverage you know tensorflow and keras here i just wanted to show you from a very high level how can we visually build our newer network and how we can train it essentially live as well without writing any lines of code okay so please go ahead and and go to that link which is playground.tensorflow.org all right and i have it open in here okay so as you guys can see here essentially i have the inputs okay here i have the first hidden layer and what i could do is i can change the number of neurons and you guys would notice that here we have what we call it a dense or fully connected artificial neural network meaning that all the neurons in one layer are fully connected to all the neurons in the subsequent layer okay so if you guys can see here this input here is fully connected to all the neurons in all the subsequent hidden and if i wanted to make or create a deep neural network well i can add a hidden layer so if you press on the positive here i'm adding a new hidden layer and if you click on that adding neurons you should be able to see again additional neurons that are being added and the more that you add neurons the more that you're increasing the complexity of the network and again you guys will notice that here also i have a dense fully connected artificial neural network as well and a very important point to notice here is that this network hasn't been trained yet if you guys recall when we cover the training and testing phases of artificial neural networks is that we build our artificial neural network first we train it with the training data and that's when we actually go and change the values of these weights if you guys remember and then afterwards we go ahead and test it and assess its performance using the testing data set so you guys will notice here that all these values of weights like minus 0.29 minus 0.19 point you know like 1 4 all these values are randomly initialized because we haven't trained it yet we haven't done anything yet here i'm just building the network i'm randomly initializing the weights and i haven't even trained any the network yet okay and what you guys can see here is well you can see the learning rate we are actually now familiar with the learning rate and this is simply how aggressive you would like to update your network or update the weights so let's set it to 0.03 as is also here you should be able to see that we have different activation functions if you guys remember we only covered the basic step activation function you guys can go ahead and change the activation function again we're not going to go into the details of what's really relu stands for rectified linear units but don't worry about it let's just you know let's take let's maybe pick like relu for example for now and we can also change the problem type to either classification or regression and let's stick with classification because that's what we're doing right now and you can also change what we call it the regularization and this is simply just a term to ensure that the network network is able to generalize and not memorize it's more of a penalizing term that try to improve the generalization ability of the network again don't worry about it for now what i could do right now is i can well select the training data that i need here so i can select for example a very simple two classes and now the network is not trained yet i haven't trained it yet but now i can go ahead and actually train it so if you click play when i say play that means i'm going to start off the training that means now i'm going to feed in the training data to my network and i'm going to try to update the values of weights so if you click play here we go you guys will see that the loss here starts very high and then it decayed afterwards and now that boundary has been drawn between the two classes so now i have that green i'm sorry that blue area here and the orange area indicating that now the model my ai model has been trained and as you guys notice here the values of weights have actually changed right so here for example the value of weight becomes 1.1 for example or 1.2 and that's it that means now the model was able to train to learn the difference between these two classes and you guys will see here this is the epochs counter if you guys remember we also covered the definition of epochs before and that's every time we feed in all the training data to my model and update the weights once okay so maybe if i try a little bit harder problem like this problem for example and i click play again as you guys can see the loss start is very high or the error starts high and then it decays afterwards and i was able to draw that boundary between the blue class and the orange class okay pretty incredible all right and what you guys can see here as well is that you can increase or vary the noise you can also change the ratio of the training to testing data so maybe you can change it make it 80 to 20 you can increase the noise for example in the input as you guys can see as you increase the noise as the problem becomes much harder because now the two classes are are not linearly separable and you can also change the batch size as well by changing the number of samples or data points that you feed at once to the network if you guys remember and you can again you can click play as well and you should be able to see that the network also was able to trade right now and was able to draw the boundary to classify the two classes all right okay so now it's time for a quick mini challenge actually it might be a little bit i would say uh hard but that's the point i want you guys to experiment with the um with the tool so i want you first to choose the most challenging data set which is the spiral data set so you guys will see that here we had four options before let me go back so here we had four options i want you guys to start with this one because this one the spiral one is the most difficult of all and i wanted to choose it first i wanted to train the model okay and then because it's a difficult problem we have to do something about it maybe we need to change the architecture of the model for example maybe we need to add new hidden layers or maybe add additional neurons or maybe change activation functions i want you guys to play with it to try to improve its performance and next step number three i wanted to tune the model hyper parameters maybe play with a learning rate maybe play with the regularization and try to achieve better model performance and then finally i want you guys to perform some feature engineering to improve model performance so when i say feature engineering that means instead of feeding in the inputs as is maybe you can feed in the input squared or maybe you can sum up the two inputs together and feed them as one input or maybe add let's say x squared plus y squared for example again there are so many options so that's it please go ahead pause the video attempt to solve the mini challenge and i will be back with a solution in a couple of seconds best of luck and i will see you guys after the challenge all right hope you guys were able to figure out the challenge let's go ahead and give it a shot so what i'm going to do is i'm going to first select the spiral okay and then next what i'm going to do is i'm going to give it a shot maybe click play without changing anything and you guys would notice here that the network is actually really trying really hard to try to come up with that boundary but because the problem is quite difficult you guys will see that now the loss training loss is at 0.3 and the test loss is at 0.4 right it's going down it's improving over time which is good and the network here you guys will see that now i was able to actually like draw the boundary which is pretty great and the loss settled at around 0.17 approximately okay and again it's trying it's trying which is not bad actually the performance is pretty good so far and what i could do as well is well i can maybe change the number of neurons maybe try additional neurons maybe try additional neurons too maybe add an additional hidden layer okay and maybe let's add an additional hidden layer as well and give it a shot and let's click play and here we go you guys will see that again the network is learning that's good sign here we're going to ultimately of course we would like the loss to drop to almost zero that means the error has each almost zero and that's a great sign they mean the network was able to um to generalize and here we go so now the loss actually reduced to almost 0.06 which is again pretty pretty great you guys will see that here i have all the spiral here it's pretty good and now i'm standing at 0.05 of error okay so that's again another way of solving the problem another way of solving the problem too is maybe we can perform some sort of feature engineering so instead of only feeding x1 and x2 which is these two classes in here maybe i can feed in let's say x1 squared and maybe x2 squared and maybe x1 times x2 and maybe sine x and maybe sine x1 and sine x2 again you guys can there's a lot of trial and error in here and let's go ahead and run it and as you guys notice here the network is as well training and now we are not only feeding in the original data but we're also performing again feature engineering while increasing the number of inputs that are being fed to the network and you guys will see that the loss now achieved 0.03 which is again pretty pretty good i think it's better than the previous run and that's it the network is pretty stable the error is actually very very low it's almost 0.034 and you guys will see that the the actual blue here has been completely separated from the other class from the orange class which is pretty amazing that's exactly what we're looking for what you guys as well could do is maybe you can increase the noise level here to try to make it a little bit harder so now it's much harder problem and maybe you can give it a shot as well and see if the network is able to do it and please note that you can do a lot of stuff here maybe you can change the activation function for example maybe change or tweak the learning rate for instance or maybe add regularization to try to improve the generalization ability of the network and here we go the loss is standing at 0.256 and please note that this is a much harder problem okay and still the network was actually able to get it right so we are standing at .24 of loss and again it's really trying okay again the loss dropped to 0.06 0.05 again pretty incredible and here we go so now the model after around 350 epochs was able to actually draw that boundary here and separate the two classes together the blue from the orange class all right okay and that's it that's all i have for this lecture and this is the mini challenge solution as well if you guys want to check it out and that's it i hope you guys enjoy this lecture in the next lecture we are going to learn how to export our model how to save and deploy our trained ai model so please stay tuned and i'll see you guys in the next lecture