[Music] thank you hello everyone welcome to exploring Technologies in this video I will explain about the Adaptive neurophysi inference system and face in this video I will explain its basic Theory and its implementation in the Matlab so let me show you the contents first I will explain what is neurophysic modeling then I will explain adapted neurophysian friend system that is an phase then I will talk about its architecture then I will explain its hybrid learning algorithm then I will talk about some applications of and phase and then I will show you that how n phase can be used as universal approximator and then in the last uh I will explain the Matlab implementation of n phase for that I will take the example of the universal approximator so I will explain all the Matlab codes and I will show you the execution of the programs so that will help you to understand the concept so now let me go ahead but before going ahead I recommend you to watch my following previous video that is about the fuzzy logic controller where I have explained its Theory and I have shown its Matlab implementation so the link is given in the description so if you go through this video it will help you to understand this current video so now let me explain what is the neurophysi modeling uh before understanding the neurophysic modeling let me explain the fuzzy modeling and the neural network based modeling separately so that you can compare so first I'm taking the fuzzy modeling or making fuzzy inference system uh actually in many situation it is not possible to obtain the precise mathematical model of system due to its complexity so in such case of weakness ill-define and complex system models the one important solution is a fuzzy logic modeling so due to the rule structure of a z inference system it is easy to incorporate the human expertise about the target system directly into the modeling process so building FSC system requires both prior knowledge of fuzzy rules fuzzy sets Etc and the manual tuning that is the drawback actually so which is really a time consuming and error prone so FIS are not able to learn so now let's see what is the neural network based modeling so in this approach the artificial neural networks that are low level computational structures are used for system modeling so here neural networks work well when lots of input output data for Target Model is available for training so obviously they have ability to learn they can change themselves but a n cannot handle FIS like capability of dealing with the uncertainties weakness and with the human-like knowledge representation and explanation so that is a drawback of neural networks so obviously fuzzy has some disadvantages and neural Nets also have some disadvantage so we are combining them together to exploit their advantages so to get advantages of both FIS and en they are combined together and that gives birth to the neurophysic systems so we can say the neurophysi systems are the fuzzy systems with the learning capability or we can say that they are neural Nets that use linguistic information from the domain experts with more transparency I mean both the uh both the modeling approaches are taking advantages of both the approaches so we have several combination schemes I mean how to combine FIS and a n together uh the first one is a Cooperative neural fuzzy systems uh concurrent neurophysics systems and the hybrid approach in Cooperative neurophysic systems actually neural Nets are used to fine-tune The Fuzzy systems uh the neural Nets are only used for tuning purpose okay after that they are removed and only the fuzzy systems then works but in concurrent neurophysi systems both work together and hybrid is also a similar approach so there are many proposed combination schemes with different architectures such as Falcon and Phase 1 nefcon Etc so in this video I am concentrating only on and phase so let's see what is this anaphase and phase is known as the Adaptive neurophysi inference system or adaptive network based fuzzy inference system so n phase is actually the neurophysi system that uses five layer network with supervised learning it was first proposed by the Roger Chang and sun in 1994. the link of the research paper is given in the description you can refer that or you can search on the Google so this anaphase is based on the hybridization scheme of NN nfis that is just I have explained in the previous Slide the third approach so it has become very popular and has been used in number of applications since then so it is simple and very efficient when number of inputs are less usually lesser than five but It suffers with the curse of dimensionality it means when the number of inputs all the number of membership functions per input increase then the number of fuzzy rules also increase by this relationship so uh fuzzy rules number of fuzzy rules are equal to the number of membership functions Power number of inputs so if you have let's say two membership functions for each input and the number of inputs let's say uh are 10 then there will be 1024 fuzzy rules okay so it Demands a huge computational efforts so the computational complexity is the main drawback of the N phase when the number of inputs are large okay and now let me show you the structure of the N phase uh that was proposed by the Zhang so this is the network structure which has the five layers layer one layer two three and four uh in the layer one we have the actually the membership functions so for the two input this uh Network shown here is for two inputs uh if the inputs are different I mean more than two it's a structure will change and we have two membership functions for each input I mean a one and A2 for X B1 and B2 for y so if I increase the number of memory functions also also ah it's uh appearance will change so this is the case of two inputs and two membership functions for each input so the layer one is actually adaptive layer that means the values of a one a two b one V two will change during the training in the layer 2 it is a fixed layer the nodes are fixed so they have the multiplication operation so the output W one of this node is obtained by multiplying these two incoming signals and layer 3 is a normalization layer that's why represented by the N so here weights are normalized later I will show you that how normalization is obtained and the layer 4 is again the Adaptive layer uh here uh during the training the values are updated and uh in the layer 5 we have a single node ah which is a summation node so output F depends on the simply the summation of all incoming signals to this node so now let me explain uh the function of each layer in detail one by one so first the layer one so this is the part of the layer which we have just ah seen in the previous slide so as I said for the two inputs X and Y I have a two membership functions A1 and A2 for X B1 and B2 for y so these are the membership functions a one a two b one and B 2 for X and Y respectively so these are the Bell functions so the as I said the node uh every node in this layer is adaptive and the output is given by these two equations so for I equal to 1 to 2 I get mu a1x and mu A2 X like this mu a 1 X and mu A2 X and for I equal to 3 and 4 I will get mu B1 y and mu B2 y so I get four values for two inputs X and Y so that is actually the fuzification so for two crisp values input values X and Y I get these four fuzzy values so this is the operation you can clearly understand from this figure that for a particular given value of x I get these two fuzified values you can say that the X cuts the membership function at ah at this point so I get mu 1X and it cuts the membership function A2 at this point so I get mu A2 similarly I get mu B1 and mu b 2 Y for the given value of y so this is how the fuzification ah takes place and now as I said that membership functions are actually the Bell functions okay so that is the expression which is governing the shape of the membership functions so here the parameters are a i b i and c i so these parameters are known as the premise parameter so these parameters are updated during the training process later we will see that how they are updated now I'll let ah let's come to the layer 2. so in this layer every node is a fixed node as I said uh and the level adds pi the output of the node here is a multiplication of all the incoming values so obviously the output of the second layer node is w i represented by the wi where uh output is simply the multiplication of these two fuzzy values mu a i x and mu b i y okay so actually the node output represents the firing strength and this operation is equivalent to The Fuzzy end operation so it means I will perform the minimum operation so later I will tell you that ah what is equivalent to this mu a i x into mu b y okay and in layer 3 this is the normalization layer marked by n so the normalization is done by this equation so output of the node at in the third layer is represented by w i bar so that is equal to w i upon W one plus W2 so the current value of uh output of ith node divided by output of all the nodes okay so that is a normalization of the firing strength and in layer 4 which is again the Adaptive layer the output of the node in this layer is given by this equation that is w i uh bar which is obtained from the layer 3 multiplied with f i so what is f i this is f i so f i is given by p i x plus q y y plus RI here X and Y are input while p i q i r i are the parameters so these parameters are known as the consequent parameters and they are updated during the training process okay and now layer 5 so this is a final single node ah which is labeled as a sigma and its function is obviously the sum summation of all incoming values so this is how I get the total output and this is how my neural networks as exactly the second of type for the influence system so this structure is not unique we can combine the layer 3 and layer 4 also the weight normalization can be done at the output stages so there may be some change uh in the network and now let me summarize all those mathematical operations here so for input X and input y I get the four fuzzy values as I said so from this figure for the particular given value of x I get mu a1x and mu a 2x so here I get mu a 1 x and here I get mu A2 X at this point I get mu B 1 Y and here I get mu B to Y okay so that is a specification and now in the layer 2 I get the W one so w 1 is the output of the node and that is simply multiplication of the two incoming fuzzy values so mu A1 X multiplied with a mu B1 y mu b 1 y okay and similarly W 2 equal to W2 that is Mu A2 X mu a2x multiplied with mu B2 y okay so now uh what I should write here as equivalent to W1 as I said in the previous slide that this operation is the end operation okay so I will take the minimum out of these two so just go to this figure and compare mu A1 X and mu B one y so mu A1 access this and mu B one y is this so which one is minimum obviously mu B one y so my output is Mu B1 y okay for w 2 I have to compare mu a to X mu A2 X and mu b 2 y so which one is minimum obviously mu A2 X so my output is Mu A2 X so that is the expression of W1 and W 2 now the normalization so w 1 bar is W 1 upon W1 plus W 2 so w 1 is a mu B1 y upon mu a to X Plus mu B1 y okay similarly I can write for w 2 bar so that is W 2 upon W 1 plus W 2 so that is Mu a 2 x upon mu A2 X Plus mu b 1 y okay so these are the expressions for w 1 bar and W 2 bar now I can write expression for f one that is the output of the layer 4 so f 1 is simply W 1 bar into uh P 1 X plus Q 1 y plus R1 and F2 is W 2 bar into P2 X plus Q 2 y plus R2 okay and the output is submission of these two so that is simply F1 plus F2 so from this equation and this equation if you sum these outputs F1 and F2 you will get output so this is the actually overall summary of the mathematical operations uh in this uh neural net and now let's see the architecture with the more complexity where the number of membership functions are increased so this is one example uh where uh the complexity has increased so here we have the two inputs again but the number of membership functions are increased so instead of 2 now I have three membership functions okay so when the membership functions are increased you can see the complexity has increased so how many rules are there so fuzzy rules are membership functions Power number of inputs I mean 3 power 2 so that is 9. so obviously we have the nine rules one two three four five six seven eight and nine fuzzy rules so this is uh just to show you the complexity uh of the network uh here is still the inputs are two if I increase the number of inputs also then you just can imagine the complexity of this network so that is a drawback of n phase so now let me explain the hybrid learning so what happens during the training actually so here both site parameters the premise parameters and the concept the consequent parameters are modified as I said so premise parameters are at the input side I mean those a i b i n c i which are actually uh governing the shape of the membership functions while the consequent parameters were p i q i and RI okay so informal pass the premise parameters are fixed okay so these parameters are fixed while the consequent parameters are updated using lse least Square estimate and in backward pass uh the promise parameters are updated while the consequent parameters are fixed and this updation is using the gradient descent that is known as a membership function fine tuning as I said because aibi CI are governing the shape of the membership functions so during the backboard pass they are fine-tuned so this is the actually the hybrid learning approach so followings are some applications of anaphase one is the universal approximator predictive modeling there is a Time series product prediction hopefully I will explain this in the next video the controller design pattern recognition data mining decision making Etc okay so now let's come to the Matlab implementation so I will show you the universal approximation example of an phase that how you can utilize and phase to approximate any mathematical expression both in 1D and 2D so I will take first problem like this so I have to approximate this equation I mean I will make an Phase 2 behave like a sine function and in the second example I will implement this uh two dimensional sync function Sig function okay so my n phase model will behave like this sync function so what are the steps in Matlab I have to use the two functions gen phase and and phase gen phase is used to create the anthis model while anphis is utilized to train that or tune that okay so now let me come to the code so this is the first file which is for 1D approximation I mean sine function so first I am generating the data so this is my T and this is the sign so T is uh the x-axis value and X is y axis value so this is my sign one cycle and in the data I have arranged this t n x like this so as a column Vector I have arrange so this is T and this is x in data right so that will be used for training uh now generating uh the initial FIS so with help of the Gen phase I will create my anthis model so first I have to Define some parameters so we have the function gen phase of options okay so I'm using the grid partition method uh there are other partition methods also so if you go to the theory as proposed by the junk you will have the better idea about what is grid partition so uh with help of the great partition uh I have created this option variable optg so now I will Define the some parameters like how many membership functions I need default is 2 so but here I have taken three so optg dot this is actually the Matlab syntax uh you have to write as it is without any change so number of membership functions three then opt g dot input membership function type I mean which type of membership functions you uh you are taking so you can take any there is a list in the Matlab so you can take help of the Matlab documentation here I am taking the generalized Bell membership functions you can take a gaussian MF also okay so now with help of genfish I am creating my model my neural land which has the name in face here you have to give the input so this is the first column that is input and this is the output so it will be treated as input and this will be treated as output for the network so I have input output data set available for training okay so with help of these options my network will be created with help of gen phase and this is my input and corresponding output so now I will train this created model so again you have some options that you can Define during the training for example Epoch numbers I mean how many epochs you want to train so here I am giving 100 the default is 30. and the initial FIS is in FIS so this is the model which is already created using janvis so these are anphis options so these are this is the variable which is carrying all these information opt so now this is the uh function and phase which will train my model okay so n phase training is there so with help of the options defined here with the help of the data which is carrying both input and output the training is started and after the training the complete model goes into this variable FIS so now FIS training is complete I mean NFS training is complete then now you can evaluate it so with eval face function you can find its output for any input so I am giving the same input which is uh taken during the training uh data I mean the second First Column that is a t column in this FIS so you will get the output okay so now I am plotting uh the output so first I am plotting the actual uh signal T versus X and then I am plotting this output that is approximation done by the end phase so we can compare and here I am finding the error I mean I'm finding difference between the original sine wave and the approximated sine wave by the N phase so we can see the difference so let me uh jump to the Matlab where I can show you the output of this program so this is the Matlab and this program is already written here so let me run this program Okay so this is the output so here you have actually the two curves red and blue so actual function is a blue and the end phase approximation is given by the red so here you are seeing a single one but actually they are overlapping with each other so here I am taking the 100 epochs and three membership functions so I am getting very good approximation here okay so both are coinciding with each other I mean actual function sign and the sign which is approximated by the end phase uh while just looking this you can say that both are same but exactly they they are not same there is some error if you subtract these two curves and that is the error so error is obviously in Milli you can see that 2 into 10 power minus 3 so some ah 10 power minus 3 values so you can see that both curves are very very close to each other so that is a very good example of that how close uh is approximation but you can see the difference so instead of uh let's say membership function 3 I can make it two and uh the epochs let me take it 30. so I have reduced the number of training epochs to 30 and the membership functions per input is 2. so let me run this program and see what is the difference okay so this is the error and now here you can see the difference so this blue one is actual sine wave while this red one is approximated sine wave I mean the approximation done by the N phase so n phase has strained itself you understand and phase has trained itself so for any input T it will give the corresponding red output although it should get the blue it should give give me the blue output but it will give me the red output with the difference so at every instant you can see there is some difference and difference is used now so with help of this error you can see at the different instant of time we are getting error up to the 0.08 although it is very small but as compared to the previous case uh where the error was in the Milli 10 power minus 3 this is not good so just by increasing the membership function to the 3 and the epox number to the 100 I'm getting very good approximation so that is the output so here the both the curves are coinciding with each other so I can say that my n phase is now behaving as a sine function so whatever input I will give to the end phase it will simply take sign of that and give you the output so my n phase box you can say that model or that system is behaving like a sine function so it means uh uh with by appropriately choosing number of membership functions and the number of number of training epos I can make my n phase to behave like any one dimensional mathematical function so that is that's why I call it as a universal approximator right so now let me uh jump to my presentation back and here I will take the example of the two dimensional approximation so in the previous case I have taken uh the one dimensional sine function now I will show you you can approximate a two dimensional function also that is really complex uh I have already shown you that I have taken the case of the two dimensional sync function so I am defining the variable X and Y then Mash grid both X and Y are ranging from minus 10 to Plus 10. and then square root of x square and Y square and this is my sync function that is sine R upon R here EPS is added just to about 0 upon 0 because when R is 0 then there will be sine 0 upon 0 0.0 so that will be undefined so the EPS will ah avoid this error so I'm plotting this actual function with help of the serve function then I am preparing the data so I have just uh because uh these X and Y are two dimensional so I I have converted them into the columns okay so X is converted into L1 Y is converted into L to L1 L2 and uh uh output Z is converted into L3 so Z is converted into L3 then l 1 L 2 L 3 all are uh combined together here ah to make the data okay so uh data variable is created which is carrying all the inputs and output so in this Matrix the First Column is L1 which is carrying the X the second column L2 which is carrying the Y X Y input and then Z the third column is set so these two first two columns are input while the last column is the output so all input outputs are arranged in this data Matrix and now again the same type of syntax which we have already seen in the previous example so first with help of the Jan fish we will create the structure and first but before that we have to Define its parameters so with help of the Gen phase options ah we will Define the number of membership functions we are taking 6 per input and then we are defining the input membership function type so here we are taking gaussian membership functions then with help of gen phase we are creating this model fis1 so here this is input I mean the first two columns X and Y and this is output Z and then options defined here so with this gen phase option the fis1 model is created now uh the N phase options we will train the model so this FIS one is here initial FIS and epoque number is 100 so iterations will be done in the 100 epochs you can change it as per your requirement and now with n phase we are starting the training with all inputs and outputs and then options uh given in this line so once the training is finished it will take time because it is more complex because we are taking six membership functions and 100 epochs right so once the training is finished this fis2 is my train model so it should behave like that two-dimensional sync function so for that we have to evaluate it so with eval face uh we have given the input X and Y uh to this model and the corresponding output is this so NFS output is my approximated output all right so since it is in column Matrix so I have to convert back into the two Dimension so with the help of the reshape I am converting back into the two dimensions and then I am plotting it with a surf and here I am finding the error between uh the actual and approximated plots so now I will show this into the Matlab and uh and then we can have idea how it is working so this file is is already written here so let me run this it will take some time because uh the system is now more complex and we have defined the 100 epochs so depending on the speed of your computer it will take time so let me see what is happening in okay it uh yeah it is still busy Matlab is still busy yes it is done so this is uh actually error between the actual and approximation so you can see this is the error and it is going up to the 0.01 that is also in Milli 10 power minus 3. so I should expect a good approximation in this case yeah I get this this is actual function and this is the estimated function you can see so both look very similar but of course there is a difference as we had just seen in the error plot but they look very similar okay so this is your actual uh sync function and this is approximated sync function so now you can say that my n phase model is trained and it is behaving like a two dimensional sync function so if you give any input of X and Y it will create the Z output okay and that is equivalent to the finding sink of input values X and Y so this is one good example now let me show you the some differences in the input and output this is the best uh and I will not say best but this is a quite good approximation because I have taken the six membership functions and 100 Epoch so let me reduce the number of epochs let's say epoxy 30 and uh the number of membership functions are let me let let it remain six so run this program and see what happens okay so this is the error and this is the estimated output and this is input so it is uh not good actually from upside it is looking almost similar but if you see the downside there is a difference here you can see A Perfect Circle in the bottom but here uh it is not a perfect circle I am getting some bounces here right some ups and downs like this so there is a uh a huge error actually as compared to the previous case when I reduce the epoch numbers so now let me take one more example where I will reduce the number of memory functions so I keep membership functions 3. and epox to be let's say 50. and let's see what will happen okay so this is the error and this is the estimated and this is actual function so you can see that there is a huge difference so this is my actual function and this is the estimated function or approximation so this is not matching at all with this input function as you can see that there is a huge error so obviously by reducing the number of membership function it is working very badly so let me go back to the six and approx 200. so it is giving me the quite good approximation okay so this is uh the case where I have taken six membership functions and the 100 epochs so this is actual function and this is the estimated function you can see okay so that is giving me quite good approximation with very small error like 0.01 so this is uh how an and phase model can be trained to approximate any mathematical expression so that's why we say it as universal approximator so in similar way you can train any and phase two work like uh any mathematical functions I mean you can design some controllers also because controllers are also some mathematical equations right so okay that's it for this video I hope you have liked it and you have learned something new today so thank you very much for giving me your precious time I hope you will uh like this video and you will share this video to the maximum so thank you once again have a nice time