hello and welcome to this session on insights into explainable AI my name is Akash I am the CEO and Lead instructor at jogin with that let's begin so the first question you might have is what is explainable AI well we're going to answer that and we're going to look at why it is important and we're going to look at some ways in which you can use explainable AI so we'll start by what and why of explainable AI then we'll talk about the different types of explainable AI that you have then we look at some case studies three specifically line shap and PDP and then we are going to also talk about how you can learn more because this is a emerging and vast area of research so I'm going to point to some beginner friendly tutorials that you can start following and applying to your AI models immediately so first what is explainable AI and why is it important and explainable AI is also sometimes referred to as X AI so if you see the term xai you know that it basically refers to explainable AI now let's say you have a machine learning model and you're using it to make predictions maybe it is for predicting whether or not you should accept an insurance application maybe it is for predicting whether somebody is going to click on an ad maybe it is for predicting if somebody's gonna potentially suffer from a heartstroke in the next 10 years so AI is used for all kinds of simple use cases as well as some fairly Advanced ones and you might want to know why the AI model has given a specific result but if you ask the AI model directly it simply gives you a result but doesn't exactly explain why some do but most models don't especially some of the larger deep learning computer vision and natural language processing models that we are building these days so explainable AI is the process of taking the AI model that you have created and often this model is created by first capturing some data and then training that data using some form of a machine learning model let's say deep learning linear regression random Forest Etc and then applying some interpretability methods so that we can not only get some predictions from a model but we can actually understand why the model gave a particular result and there are various approaches into understanding why the model gave a particular result and we are going to look at a few approaches today briefly at a high level now why is explainable AI important well the first reason is simply because you want to understand the model's Behavior as humans we have this innate curiosity of not being able to trust something unless we know how it works so we want the models that we built to be predictable in at least some form and we want to understand why they are giving the results that they're giving and if you're going out as a data scientist and offering a model to other stakeholders the first question you're going to get is why of why it returns a particular result so you better have an answer about it you cannot say okay that is how machine learning models work okay so that is the basic reason why we want explainable AI but then a second more important reason is because often the data sets that we are working with can contain some bias so data sets can contain bias and that can cause a problem so it's important for you to identify bias within the model by looking at whether a particular feature like race or gender is causing the model to give a particular result and if it is then is that biased in some fashion do you need to devise the data set another thing that you may need to look at is whether there is some leakage within the data so often when you have a data set you might find that there is some hidden correlation between a specific feature and the target variable and this hidden correlation doesn't exist in the real world it exists maybe just because of how this particular data set was corrected all right so in that case the the data leakage may cause your model to give a good result when you're training it but in the real world it may give wrong or maybe even harmful results and for you to be able to find that out you need to look into the model in terms of how it is actually coming up with a prediction okay so overcoming data leakage is an important reason as well the third is regulatory reasons in many areas for example Healthcare Finance insurance you have to provide reasons for why you are coming up with a particular business decision so if your insurance application is rejected then the company is bound to give you a reason for why your insurance application respect was rejected nowadays you're starting to see this in advertisements as well if you see a particular advertisement you can now click on it to identify why a particular advertisement was shown to you and that is where you again need explainable AI to demystify why a certain prediction is being given next you also want explainable AI simply to improve Black Box models because we are building these large deep neural networks and we are spending so much money so much compute training these models putting in so much data sometimes you may not understand why a model is not doing better you've tried everything you've tuned our hyper parameters you have come up with all kinds of techniques to regularize the model but it is still not giving you a good result and it takes a very long time to train so understanding which features are having an impact on the outcome without necessarily looking at the model internals itself that could be a useful way to understand maybe you need to change something about the features that could give you ideas for feature engineering how to combine feature teachers how to get new features and things like that so all of these are reasons why explainable AI is imported okay so I hope you're convinced at this point that you do need to have explainable ai so with that let us understand the different types of explainable AI so there are two types of explainable AI one is model based so model based explainable AI techniques depend on the structure of the model itself so for somewhat simpler models like linear regression logistic regression decision trees you can use the structure of the model itself to understand why the model gave a particular result they are interpretable by Design but in many cases models are not interpretable by Design so then the other form of explainable AI which is what is gaining popularity nowadays which is where most of the research is happening is post hoc explainable AI or model interpretability so you've trained a model you have obtained some results on a test data set so you have the input data you have predictions and you have the targets and now you want to identify why a model is giving a particular result and again within post hoc explainable AI techniques we have a couple of ways one is a black box technique the Black Box technique does not look at the model in any way whatsoever it simply looks at the inputs it looks at the predictions and then you do something with them with them to figure out which features have impact on which predictions or have how much of an impact on the prediction and so on the other is the white box approach which is somewhere between the Black Box post hoc technique and a model based technique where you do not use the model's structure inherently but you do analyze some parts of the model maybe in a deep neural network you try to identify layer by layer which features are important instead of looking at the entire model as an input as an output so this is where you could look at things like in convolutional networks identifying which particular pixels were led to a particular result in the outcome and things like that okay so your model based approaches and then you have post hog black box and white box approaches now the model based approach is really simple let's say you've trained a decision tree the decision tree if you just try to visualize it it is trained in such kind of a format so let's say you are predicting whether somebody is going to have a heart attack based on their health information you could have something like this that if their age is greater than 50 then you check their BMI and if their BMI is greater than 25 then they have the chance of having a stroke in the next five or ten years but if their BMI is less than 25 then they do not have a chance of a stroke okay so this is a decision tree that has been trained on some data set and it is interpretable by default now as your models become more and more complex the interpretability goes down so this chart gives you a basic idea of the the spectrum of interpretability versus accuracy now the models that are high on interpretability things like linear regression decision trees have a very limited number of features or weights or biases within them so a very limited number of parameters and that is why they are easy to interpret but on the other hand as you go to bigger models Ensemble models or neural networks which can have potentially millions of parameters inside them and multiple layers then the accuracy goes up because their models are more powerful but the interpretability of these models goes down all right often some of the more interpretable models are either linear or tree based they have a certain very regular kind of structure but the non-interpretable models can capture a complex non-linear relationships and often have a long computation or training time as well okay so based on this you can decide whether you need a model based xai or a post hoc xai so another way to look at classifications in xai is by scope now you could want to maybe explain for the entire data set which features are important in the predictions that the model has given you so just looking at the entire data set or in many cases you could simply talk about a single local prediction let's say you have an insurance application which has a bunch of Health Data and then you have an AI model that says whether or not you should approve this insurance application now in that case you need to look at that particular specific prediction and identify which of the particular features led to a particular prediction all right and that is the two forms of xai dividing by scope finally another way of xai another way of looking at explainable AI techniques is by the kind of models that they can handle a lot of explainable AI techniques especially the post hoc Black Box techniques are model agnostic which means you they do not require knowing anything about the model but then you also have a bunch of explainable air techniques that are model specific even if they might read the model as a black box they may give you a better result let's say when you're working with deep learning models or you're working with the random forests okay so that's another way to look at the divisions Within xai so I hope with that you have gotten some understanding of the types of explainable AI and the different ways in which you can separate them out so let us look at some case studies of explainable AI right now so for this case study we are going to look at this problem of heart stroke prediction so let's say you have this tabular data set and in this tabular data set you have information about various patients and then you have information about way whether they have had a stroke or not and what you want to do is you want to build a model given some information about the patient you want to build a model that can predict whether they're going to have a stroke or not so we have information like the person's gender age whether they've had heart disease or not their body mass index their smoking status so bunch of information Health Data basic health data and then here is the target column we want to predict whether this person is going to have a stroke or not you predict one if the person is going to have a stroke you predict zero if the person is not going to have a stroke in this case you can see that a large number of people in this data set are healthy and will not have a stroke in a small number are going to have a stroke okay so you may have to deal with things like data imbalance and all as well but now let's say you have trained some machine learning model let's say you've trained maybe a decision tree maybe you've trained a random Forest how do you then interpret or maybe you've trained a deep learning model you have a huge data set how do you then interpret why your model is giving a certain prediction after training so we are going to look at three techniques on how you can interpret models using post hoc explainable AI methods so the first technique that we are going to look at is called the lime technique which stands for local interpretable model agnostic explainability okay so as the name suggests what it does is it zooms into a specific area so let's say you have a specific prediction from the model it zooms into a specific area of that model's prediction and then it looks at other inputs around that particular prediction that you're trying to make and it fits a linear model so your model may be non-linear overall in the entire space of predictions but if you really zoom into a particular example and look at the other examples around it you can fit a small linear model to it and then from there you can compute feature importance using the linear model itself right so here's what I mean by that I'm going to show you visually what that means let's say you have this feature you let's say you have this model here okay so it's a complex model which creates a decision boundary between a bunch of input data points so here you have a bunch of input data points where there you you're not going to predict that there is going to be a stroke and here you have a bunch of input data points where you predict that there is going to be a stroke okay so you have a bunch of data points based on a couple of features we're not looking at all the features because we can only plot two dimensions on a graph so some points indicate there's a stroke some point and indicate there is not going to be a stroke and let's say this is the decision boundary of your model okay and non-linear models like decision trees random forests or neural networks can have such complex decision boundaries now let's say you want to explain why the model predicts for a particular point that this person is going to have a stroke okay for a particular customer why it's predicting that it's going to have a stroke so what you can do is you can just take a snapshot of just this area of a bunch of words around this particular data point and in this area you can see that the model has a linear decision boundary and then you can fit a linear model over here and now after fitting that linear model you can provide feature importance for each of the features so you can then say okay this is the weight of feature one and this is the weight of feature two so if you know how linear models work they basically say you have a certain weight multiplied by feature one plus a certain weight multiplied by feature two plus a bias and that is the prediction of the model so you've simplified in that small space you've simplified your entire model to look as look like a linear model and then you are predicting the importance based on that linear model all right so that is local uh that is linear or model agnostic interpretability so that's line all right and the way you do this essentially is this is sort of the loss function or this is sort of the approximation that you do here you don't necessarily need to understand what what each part of these means you have some pretty good libraries that can allow you to use the lime technique to perform model interpretability but the idea here is you try you take a particular complex model and you fit a simple interpretable model in the proximity of the particular data point that you are looking at and that gives you an approximation for your main model and then you can interpret it easily okay so that is the lime technique next up we have the shap technique so sharp stands for sharply adaptive explanation now sharply adaptive explanation is a technique that can be used locally for a particular specific prediction or it can be average out over predictions and it can be used globally to identify which features are important in the overall prediction given by the model okay now unlike lime it does not zoom into a specific area but it looks at the impact of each feature on the model's prediction so once again let's say you have a trained model for predicting whether a person is going to suffer from hard from a heartstroke or not given their medical data so what shop is going to do is it applies a technique from game three Theory where it computes various feature contributions so what it's going to do is let's say you have 10 different input features it's going to ignore three of the features and it's going to look at the prediction that is made based on the values from Seven features from the remaining seven features and see how far away that prediction is from the prediction when you take all 10 features okay so it looks at it every time so it does this randomization process where every time it skips a few features and then it makes a prediction using the remaining features and then it sees how this skipped prediction is far away from the original prediction and then you can average it out over a bunch of different predictions to get the overall feature importance okay so I'm just giving you the intuitive understanding here and then I'll share some resources where you can see dig deeper into the math and you can also dig deeper into examples of how to use shop so broadly this is how it works let's say you have an input a particular input where the age is 56 the gender is female and then the body mass index is 30 heart diseases yes and so on so what it's going to do is it's going to pick a random subset of the features and then it's going to insert a random value from the data set for each of the other features then it's going to feed that into the model and get a prediction that prediction is going to be compared with the prediction that was obtained by putting in all the features and that difference is going to be computed and this is going to be repeated many many times over a bunch of different predictions and then the difference the feature wise differences are going to get averaged out okay So eventually what this gives you is that based on this process of picking subsets and putting in random data for some of the for the features not picked in the subset and then looking at the differences you are going to see that what is the impact of each of the features on the final output both at a local level and you can average these out at a global level so for example here there is a prediction of 0.4 and the prediction of 0.4 comes the impact of each feature is such that a BMI adds 0.1 and BP adds 0.1 sex adds 0.3 now subtracts point three whereas age adds 0.4 okay so that is how sharply can give you a feature by feature explanation into why you're getting a particular result again there's a way in which you can just import a library and then give it a set of inputs and outputs and all the shapley values are going to get computed but I want you to take away the intuition here which is that we are going to look at a subset of features and then apply those and put pick random values for the other features then look at the difference so that is the sharply method or the sharp method the third technique that I want to talk about is the partial dependence plot now sometimes you may want to study how the target variable varies as a particular feature varies over the entire possible Spectrum for example you may want to Simply study for the entire model for the entire data set what is the dependence of the prediction of the model on the age as the age varies from maybe 10 15 to 65 70 how does the model's Target prediction change that is what you use a partial dependence plot for so the key idea here is this computes variation with feature and the way it does that is by varying a feature from 0 to Infinity so what you do is you have you've trained a model already and the model given an input is going to come up with a prediction so what you do is you take your training data set and in your training data set let's say yeah so let's look at that here so in your training data set you have the age column what you do is you put 0 for the age column entirely so you replace 0 in the entire training or testing data set for the age column then you make predictions for the value 0 and then you take the average of all the predictions and that gives you a certain average what is the average prediction for this data set from your trained model for the age zero okay then you change the age to 1 and then you make the predictions again and then you average the predictions then you make the then you change the age to 2 and then you make predictions again and again and again so at every age keeping everything else constant you just vary the age from 0 to infinity or 0 to 50 let's say and you make predictions on this training or test set and then you take the average of those predictions and maybe also look at the standard deviation okay so it varies a feature from 0 to infinity and then it plots the standard average and standard deviation of the predictions and that is why it's called a partial dependency plot that how does the Target value depend on the variation of a particular feature while keeping everything else constant okay so there's one key thing to remember partial dependency plot that this assumes feature Independence if features are dependent on each other then wearing a particular feature from 0 to Infinity may not make sense but here is essentially what a partial dependency plot looks like on a particular model prediction so let's say we are looking at the prediction of a particular model it could be a neural network it could be a random forest all we do is here we are looking at this particular column called glucose is first in the entire data set we set the glucose value to zero and then we pass the data through the model and as we pass the data through the model we get some predictions we average out those predictions then we slowly start increasing the glucose value and as we increase the glucose value and make predictions for the entire data set we're going to track the average here so you can see here that as the glucose value increases may be up to the point 150 up to the point where glucose value reaches 150 the average prediction is generally negative okay whereas once the glucose value crosses 150 the average prediction tends to become positive so that means now you can now you can understand that maybe under 150 maybe a glucose level is actually healthy somewhere in between uh but whereas once you cross 150 the glucose levels become unhealthy no matter which model you are doing this with all you need is the inputs and the predictions and then you simply vary the glucose feature now apart from the average you're also we have also plotted the standard deviation here so this is one standard deviation bill before and this is one standard deviation after so this can give you the amount of variability based on the value of the glucose feature so you can say here with a fairly good confidence that if the glucose feed if the glucose value is about 200 then I don't even need to make a prediction I can simply tell with a very high degree of confidence that this person is going to have a stroke at some point in the near future whereas over here you might need to actually make a prediction because the standard deviation crosses over into negative and positive okay so that is what a partial dependency plot looks like so those are the three techniques that I just talked about and this was just a quick insight into three techniques there are dozens more and there are dozens of libraries that you can use to implement these techniques no matter which kind of machine learning model you're training so I'm going to share a few references with you that you can check out now the first one is the interpretable ml book so there is this book called The we'll share these resources in the chat let me drop that in the chat chat right now in fact let's see yeah so the first is the interpretable ml book so most of the examples that we picked out in this session come from this particular book and you can see here it talks about the different kinds of inter interpretability that you have it talks of it has a bunch of different data sets so it first looks at a bunch of interpretable models then it looks at maybe some model agnostic methods and it looks at partial dependency plots it looks at lime it looks at uh I believe it looks sharply as well so you have a bunch of different uh techniques described in a lot of details so you can check that out the second one that I would point you to is this collection of explainable AI resources so this is a GitHub repository called awesome machine learning interpretability and here you can find all kinds of tutorials all kinds of libraries there is this big chart showing the different ways depending on the kind of use case you have which kind of explainable AI technique you need to have it's an active area of research but there's a lot of research done already so you can check out here A bunch of examples once of tutorials a bunch of libraries there's almost everything you need related to explainable AI is going to be covered here so this would be a good place to start when you want to start implementing it then I want to share with you a tutorial a series of tutorials so this is on this YouTube channel called Deep finder where you have both the theory of explainable AI specifically these three or four techniques that we're talking about and you also have coding examples so I encourage you to check out this particular playlist it would be about a couple of hours long not too long so just check out this playlist here you can see here it talks about explainable AI models that are interpretable by default it looks at maybe the lime technique the shapley technique and a bunch of different techniques so you have both code examples here so for you can see here that there are examples of how to use the code and there is a pretty good explanation of some of the theory here as well so this is the place that I would go to if I want to understand explainable AI maybe in a two or three hours and follow along with the tutorial and then the final thing that I will point you to is an interpretable machine learning kaggle tutorial so this is a tutorial built on top of this very data set that we're talking about right now so this is by a notebook by parul Pandey and she is I believe she's using the uh yeah she's using a bunch of data sets here but she's explaining a bunch of things here so partial dependence plots shapley values lime and again there are code examples here for you to look at as well along with some explanation so yeah let me drop in all these resources I've dropped them in the chat uh there's a question is this book using r no the book is not using R the book is using I believe it's using python but I think there are some examples in r as well so that was a quick overview of explainable AI and I will be happy to take a look at some questions here uh if you do have questions please post them in the chat and we'll take maybe five or so questions and by the way if you are looking to learn more about data science machine learning then and also maybe explainable AI then do check out our data science bootcamp we have a new batch of our bootcamp starting next month so if you just head over to www.jobin.com uh you can check out our bootcamp on data science this is a six month program where we cover a bunch of different courses starting from basic data science even basic programming to data science and machine learning and we have one-on-one guidance and mentorship and we are also including a three month internship now uh so this is sort of roughly the curriculum where we talk about programming with python SQL and business intelligence data analysis visualization statistics for data science machine learning foundations and you build three projects as well so you build three real world projects and then there is an internship that you can get as part of this program as well and you learn all the standard tools and techniques that you will need to use in data science and machine learning in companies around the world so we have a new badge starting soon and I encourage you to check it out if that is something of interest to you okay so let's see if we have any questions all right I'm going to start from the top because there are a bunch of different uh techniques mentioned of the bunch of different questions that have been asked here let's see okay any resources for making your own custom xai model okay that's a good question so there is this interpretable [Music] interpret.ml I believe is a library which is which provides you a way to create machine learning models that have interpretability built in so you're probably used to something like scikit-learn and scikit-learn contains a bunch of machine learning models linear regression logistic regression Etc but uh random Forest Etc but interpret ml is a library which contains similar which contains basically wrappers over all these machine learning models and it gives you explainability by default so here for example once you fit let's say you have an explainable boosting classifier once you fit it it's going to give you this nice way of just selecting a particular feature and understanding why you got a particular outcome on a particular prediction or maybe on the entire data set okay so if you want to train your own interpretable models check out interpret interpret.ml all right next question here um there's a question here what is the difference between AI machine learning and deep learning well AI is artificial intelligence it's sort of a very broad umbrella under which you have many kinds of different different techniques you can have rule-based systems you can have deep Learning Systems machine you can have machine Learning Systems you can have reinforcement learning systems so there are many techniques that are available under artificial intelligence AI systems are simply systems that can make decisions used by looking at certain inputs in a way that might appear difficult to program in a role-based fashion so that's the first thing machine learning is when you train a model using by looking at part previous data to make predictions on new data and deep learning is a specific class of machine learning models that is typically used with large data sets images and text okay next up um explainable AI libraries for time series models hmm well I will have to get back to you on that so how about you post that comment come back post that comment on the YouTube video and I'll have to get back on that because I have not used explainable AI with time series models but I would just check here on the awesome ml interpretability let's see I would just check Maybe no okay there doesn't seem to be any anything here but I would I would I will get back to you about that because I've not used any can you please explain the difference between model agnostic and model specific yes so if you look at the different types of shap models so there's something called kernel shop and there's like a deep shap and tree shop so there are a bunch of different shapley models or sharply shop approximation methods now the one called kernel shop can work with any type of model you can use it with linear regression you can use it with logistic regression random Forest deep neural networks but the one called Tree Shop is only going to work for tree based models like decision trees random forest and gradient boosting now the trade-off here is that for tree shop with Tree Shop you can get better feature importance or better interpretability for tree based models right so model specific techniques often look into the model a little bit even though they may not use the entire structure of the model or they may be developed for a particular kind of models and they may give you a better result but on the other hand these kernel shop or these General techniques are often better when you're training a very different variety of models and you want to compare the explainability across all of those different types of models so I hope that answers your question okay next question this is the boot camp beginner friendly yes the boot camp is beginner friendly so you can check out this we start from very scratch you don't even need to know programming and you can even go explore the curriculum or check out the brochure Etc I'm researching in this area but I'm finding a common pattern in research papers and online tutorials they don't really mention what these values actually mean in the context of the problem how do I understand this Okay so see you have to uh you have to really think about which kind of model you're training let me just go back here yeah so you have to uh look at the intuition behind each kind of model so what you need to know first is when you're looking at shop scores for example you need to understand intuitively what shop is doing what shap is doing is intuitively leaving out subsets of features and making prediction and studying how much difference they make to the final outcome and then using that to give you the explanation for a particular feature what these numbers actually amount to so for example what this explanation is saying is that let's say for this particular model if you pass in all zeros you get you're going to get the output of 0.1 but when you pass in the BMI of 40 that leads to a plus 0.1 addition in the output and then the blood pressure of 180 leads to another plus 0.1 prediction in the output and then the sex female rates to minus 0.3 prediction and then the uh plus 0.4 is based on the age so for this particular model you can say that BMI has the effect of adding 0.1 into our prediction so you have to understand the intuition and then use that intuition to interpret the numbers that are created by the model and of course the code and the library that you use the mathematics behind it is more to arrive at these specific numbers because ultimately this is all backed by the specific mathematics okay next question um okay can I ask a question on wise well we may not have the time for that today this is a slightly shorter event but I will but do post a question in the chat in feature engineering we can use both feature selection and feature extraction techniques we know we know features like extraction like PCS given more importance than let me just bring this here uh okay it says we can use both feature selection and feature extraction techniques we know that feature extraction is given more importance than feature selection so do shop and total skips uh shap skips features then keeps a few features when it explains about the dependency plot yeah I will I'm not sure I understand the question here but because you're combining a couple of techniques here you have the sharp and then you have the dependency uh plot but maybe I'll just try to answer it in a more General sense um so you use shop when you're trying to understand what is the contribution of each feature to a specific prediction and you want to quantify that contribution on the other hand you use the partial dependency plot when you're trying to understand what happens to the Target variable as you vary a particular feature keeping everything else constant okay so both of those are different in different contexts they may be required in different ways the I can't really say that one is more appropriate or the other for a particular use case right so we'll need to look at the particular use case please help okay I think we've answered the differences here do you have an idea of how xai can be used in AI verification but I'm not sure what you mean by that but if you mean you want to verify whether the results of an AI make sense from a domain perspective then yes so for example let's say you are predicting heartstroke and you get the importance or you get the predictions based on the live model okay you get the importance of each of the features based on the lime technique you can then go talk to a domain expert tell them that according to my model these these these features uh were given such such and such weightage do you feel this weightage makes sense so then that other person can actually tell you whether that the domain expert can give you um inputs on whether the weightage makes sense or not so if that is what you mean by AI verification then yes explainable AI can be used for it okay well there's a question about the data science boot camp well do apply to the program uh one of our academic counselors will talk to you so we'll we'll help you out there okay I think we have answered yeah where can I stay updated on different new algorithms and techniques like this well um come join our WhatsApp Community we also have a Weekly Newsletter we are sharing something every single week so you can go check out a newsletter as well we'll drop a link to our newsletter in the chat too it's newsletter.jogain.ai jobin.com I think should take you there okay then whenever new things come in AI uh okay again the same question okay I'm gonna probably wrap up here because I can see that there are a bunch of questions that are coming in but okay yeah in terms of the case studies well I will drop all the links in the chat and we'll also include it in the video description so you can go check out that and the WhatsApp link is simply jobin.com WhatsApp we are a little short on time today but I will see you in a future webinar or I'll see you if you join one of our communities or check out subscribe to our newsletter so please be in touch you can also go check out our YouTube channel we are posting new videos there almost every day so with that we will end this session thank you and have a good day or good night I'll see you next time