Transcript for:

this is a beginner-friendly introduction to common data structures and algorithms in python this course is taught by akash ns the co-founder and ceo of jovian data structures and algorithms in python is a practical beginner friendly and coding focused online course that will help you improve your programming skills solve coding challenges and ace technical interviews you can also earn a verified certificate of accomplishment by completing this course learn more and register at pythondsa.com this course runs over six weeks with two hour video lectures every week with live interactive coding using the python programming language you will get a chance to practice and improve your coding skills with weekly programming assignments consisting of real interview questions and you will also build a course project that you can showcase on your resume or linkedin profile this is a beginner-friendly course and some basic programming knowledge will help you follow along with the course don't worry if you're new to programming you can learn it as you work on this course with a little extra effort you will also get to access the course community forum where you can ask questions participate in discussions and share what you're working on during the course this course is created by jovian a platform for learning data science and machine learning with a global community of tens of thousands of learners from over 150 countries i'm your instructor akash co-founder and ceo of jovian and i'm really excited to kick off this course with you register now and invite your friends to join the course at pythondsa.com hello and welcome to data structures and algorithms in python this is an online certification course brought to you by jovian and today we are at lesson one binary search linked lists and complexity analysis my name is akash i am the ceo and co-founder of jovian and i will be your instructor you can find me on twitter at rcacheness this course runs over six weeks and over the six weeks if you enroll with for the course work on four programming assignments and build a course project you can earn a certificate of accomplishment along the process you will also learn about common data structures and algorithms in python and how to use these skills to ace coding interviews and technical assessments so let's get started then to begin we need to go to the course website pythondsa.com so if you open up pythondsa.com in your browser that will bring you to this page this is the course page and you can watch an introductory video about the course here you can enroll for the course for free you will need to sign in into jovian you can use your google github or email to sign in into jovian and once you're enrolled into the course you can also invite your friends to join the course the course is still open for enrollments so please invite your friends and colleagues this course is a beginner friendly introduction to common data structures and algorithms in python and this course will help you prepare for coding interviews we have coding focused hands-on video tutorials every week so you can either follow along with this video you can pause and run the code as we speak and you can practice coding on the cloud or you can watch the video right now and you can practice later in this course we will solve questions from real programming interviews and you can earn a verified certificate of accomplishment so let's go to lesson one binary search linked lists and complexity on the lesson one page you can see a recording of the lesson once it is completed and you will also be able to see a hindi version here and all the code used in this lesson is linked below so the first set of code that we will look at today is called linear and binary search so let's open it up so this is the first tutorial that we will work through in this lesson and you will be able to work through with it as well and this is part one and there are a total of 12 notebooks or 12 tutorials we will go through now this course assumes very little background in programming and mathematics but you still do need to know a little bit for instance you do need to know basic programming with python things like variables data types loops and functions and don't worry if you don't know them already you can click through and follow these links each of these is a separate tutorial the tutorial will take you about half an hour or so each of these and you can learn the basic programming with python in just a couple of hours you will also need to know some high school mathematics and if you want to brush up things like polynomials vectors matrices and probabilities you can click through and read these but no prior knowledge of data structures or algorithms is required you do not need to have an extensive coding background and we will cover any additional mathematical and theoretical concepts as we we need as we go along so how to run the code what you will see here is that some explanations and then you will also see some code so you can see here that there is some code written here and there is some func so the library is imported and a function from the library is used here now to run this code you have two options you can either run this code using free online resources which is what we recommend or you can run it on your computer locally and you can read these instructions i am going to use free online resources provided by jovin so we just scroll up here at the top of this page and click run and then click run on binder so this will take a second or two and what we're doing here essentially is setting up a machine for you on the cloud using a software called binder it's an open source software and now what you were looking at here this was actually not a blog post this is actually something called a jupiter notebook a jupiter notebook is something that can not only contain explanations but can also contain code and you can look at the code and its outputs right here in an interactive fashion so if i scroll down here you can see that we have all the same content that we were looking at except this time we can actually run this code so we can click the run button here and the run button will run the code and here we click the second run button that is going to run the second line of code now we will be using jupyter notebooks extensively throughout this course because jupyter notebooks are a great way to do interactive programming you can change the code for example instead of the math mat dot square root you can use mat dot seal and you can change the value here so jupyter notebooks are great for experimenting with code now just a couple of tips that you want to do as soon as you run a jupyter notebook you can click on kernel and click restart and clear output what this will do is this will remove all the pre-executed outputs from your code so you can now see that the output of the function is gone and you can see that the numbers here go away so now you can execute the code line by line yourself and see the output discover the output and then one other thing you can do if you want to hide the ui a little bit is to toggle the header and also toggle the toolbar now you might need the toolbar for the run button but there's a tip here instead of pressing the run button you can use shift plus enter so if you press shift plus enter that will execute a cell and that's a pretty handy shortcut so once again you go on the lesson page on the lesson page you will find a link to the notebook called linear and binary search on the linear and binary search you can read the explanations but you can't run them to run the code you need to click run and then select run on binder and clicking run on binder will set up a cloud machine for you and all the code that you see here will get executed on the cloud so you do not need to set up anything on your computer you do not need to download anything we've done all that for you so let's get started then this course takes a coding focused approach towards learning and in each notebook or each tutorial we will focus on solving one problem and then learn the techniques algorithms and data structures to device an efficient solution for that specific problem we will then generalize the technique and apply it to other problems so in this specific tutorial we will focus on solving this problem and here's the problem we're solving and this is a typical problem that you will come across in a coding challenge or a coding interview so here's how the problem goes alice has some cards with numbers written on them and then she arranges the cards in decreasing order and lays them out face down in a sequence on a table so this is what it looks like these are cards each of these cards has a number below it and the numbers are in decreasing order she challenges bob to pick out the card containing a given number for example she could say bob i want you to pick out the number 7 by turning over as few cards as possible so this is a puzzle that's given to us and we're not told how many cards alice has so you need to write a function to help bob locate the card so alice can put down any number of cards and the target number that bob has to pick out could be anything so we have to tell bob not us not the solution for a specific problem but a general strategy that he can use to turn over as few cards as possible so for instance look at these seven cards and maybe put some imaginary imaginary numbers before them below them and try to figure out a strategy try to start thinking about the problem and this may seem like a simple problem especially if you're familiar with the concept of binary search but the strategy and technique that we're learning here will be widely applicable and we will soon use it to solve harder problems now before while you think about the problem and before we start solving it i just want to talk about why you should learn data structures and algorithms and whether you're pursuing a career in software development or data science it's almost certain that you will be asked programming problems like reversing a linked list or balancing a binary tree in a technical interview or coding assessment now it's well known that you never face these problems in your job as a software developer so it's okay to wonder why such problems are asked in interviews and they're asked because they demonstrate the following traits and these are very important traits for a programmer number one is that you can think about a problem systematically and then solve it systematically step by step too and the number two is that you can envision the different inputs and outputs in edge cases for your problem because programs when you put them out in the wild as part of software can encounter any kind of inputs and as you have thousands or millions of users you will encounter any and every possible input and often this has many security implications it can take down the server it can take down your application or you can have a loss of data or loss of property you can communicate your ideas clearly to co-workers that's a very important part of problem solving and most importantly you can convert your thoughts and ideas into working code and the code should also be readable to other people so it's not really the knowledge of specific data structures or algorithms that's tested in an interview but it is your approach towards the problem so you may fail to solve the problem but you may still clear the interview or vice versa you may solve the problem and still not clear the interview so in this course we will focus on the skills to both solve the problem and to clear interviews successfully so that's why you need to learn data structures and algorithms so coming back to the problem at hand now you've read the problem and you may have been thinking about it and maybe you have some ideas on how to solve it and your first instinct might be to just start writing the code for it but that is not the optimal strategy and you may actually end up spending a longer time to solve the problem due to coding errors or you may not be able to solve the problem at all so what we are going to cover here is a systematic strategy that you should apply in interviews or in coding problems on encoding assessments or in general whenever you are faced with a problem like this so here's the strategy that we will apply step one state the problem clearly identify the input and output formats step two come up with some example inputs and outputs and try to cover all the edge cases step three come up with a correct solution for the problem it can be as simple as possible and state it in plain english step four and this is a step that is optional sometimes implement the solution and test it using example inputs and then fix any bugs in your in your first solution in step five analyze the algorithms complexity and identify any inefficiencies and finally step six apply the right technique to overcome the inefficiency and then go back to step three which is come up with a new correct solution which is also efficient then implement the solution and analyze the algorithms complexity so this is the technique that we will apply over and over for the course of six weeks to many different problems and applying the right technique is where the knowledge of common data structures and algorithms comes in handy so this is the method we'll be using so let's jump into the solution step one state the problem clearly now you will often encounter detailed word problems in coding challenges and interviews they will go on for paragraphs and paragraphs for instance here we are talking about alice having a deck of cards and then shuffling them putting them out on a table talking to bob etc etc etc the first step is to state the problem clearly and precisely in abstract terms because computers don't understand people computers don't understand cards computers understand numbers so for in this case we can represent the sequence of cards as a list of numbers so a list is a basic data structure in python and the turning over of a specific card is equivalent to the accessing of the value of the number at a certain position in the list for instance if we think of this set of cards being represented by this list you can see here that this list is sorted in decreasing order then turning over a certain card is equivalent to accessing that specific element from the list so turning over card number two or as we say in computer science card number one because this is card number zero and this is one thing that you might want to get into your head as well that whenever you're counting always start counting from zero otherwise you may run into many off by one errors so this is position zero and this is position one so if you turn over the card at position one it is as good as accessing an element from a given list which in this case will turn out to be 11. so these are the positions in the list starting from 0. and now what we have to figure out is how many elements do we need to access so we need to access the minimum number of elements to get to a particular element right so the problem can now be stated as follows we need to write a program to find the position of a given number in a list arranged in decreasing order we also need to minimize the number of times we access the elements from a list so we're finding the position of the given number 7 and the position in this case is 3 and we want to minimize the number of times we access elements from the list so if we go in this direction for example we would need to access 13 11 12 and finally we discover 7 we come from this end we may discover 7 6 5 4 and finally we may discover 7. so definitely coming from the left is better than coming from the right but is that the best that's what we're solving now once we've defined the problem and what you should do is you should try to write down the problem in your own words and primarily this is for you to make it clear to yourself uh either speak it out loud to the interviewer or write it down in your own words as short as make it as short as as long as possible so that you clearly understand what's in it and then come up with the inputs and the outputs so there are two inputs here there's the input cards which is a list of numbers sorted in decreasing order and then the second input is a query which is a number whose position in the array is to be determined and there is one output which is position and the position is simply the position of query in the list of cards for example seven is at position three counting from zero of course and as soon as you've written the input and output out you can now write what is called the signature of our function which is a structure of our function without any actual code inside it so now we can call it def locate card with cards and query and the single statement inside it called pass because a function in python cannot have an empty body you need to put in at least one statement so you always put in the pass statement first because it doesn't do anything there you go so now we have framed our problem in abstract terms and now we have a function signature to work with now a couple of tips here this is something that interviewers specifically will look for but also encoding assessments because your code is also shared with the company so you may want to name your functions properly and think carefully about the signature for example here you should not call your function f1 or func one or f or something like that it's better to call it locate card because that's what it is doing and the similar thing is true for variable names as well use descriptive variable names one because it's good for coding practice and second because as you work on the problem you may lose track of what a variable represents for example if you call this a and you call this b now 20 minutes down the line talking about the problem writing different lines of code you may forget what a and b represent so please call them what they represent even if it can get a little long and finally if you're unable to come up with a function signature if you're unable to come up with a simple description then discuss the problem with the interviewer if you're unsure how to frame it in abstract terms so keep that in mind and this is really the first and most important step which is stating the clarifying the problem statement and stating it clearly do not start coding before you have done this otherwise you may get halfway into the code and realize that you have not understood the problem at all so step two now we will come up with some examples take some example inputs and outputs and our goal will be to cover all the edge cases so before we start implementing a function we want to have some examples so that once we implement it the first thing we want to know is is it correct and in general the answer is no because coding especially when you get getting started is hard because you have to think about many different scenarios so and especially especially interviews or coding assessments are also stressful situations so you may not be able to focus and think about all the different things that you need to keep in mind so simplest way to reduce the risk of going wrong is to use a test cases so here's one test case that we came up with you know we what we've done is we've taken the information that we've listed above in the inputs and outputs and we've written it in as code so now we have a variable called cards which is a list of cards a list of numbers then we have a query which has the value 7 and then we have the output which has the value three so the expected output from the function is three and once you have a test case you can test your function at any point anything you want to test you can simply pass the input for example cards and query into the locate card function and get back the result and you can see here right now because there's nothing inside the function the result you get back is none but later you'll start getting back a proper result from your function and what you can then do is you can compare the result with the output of the test case so in this case when we compare them obviously the output is 3 the result is none we get back false now one thing we will do in this course to make testing easier because we will be testing our algorithms again and again as we keep improving them is that we will represent our test cases as dictionaries so here for example this this test case will be represented or every test case will be represented as a dictionary containing two keys input and output and the input will contain one key for each argument to the function so if your function arguments are called cards and query in the function signature and that's why we wrote down a function signature first so that we don't get confused here so if your function arguments are called cards and query then we can take one one key called cards and put the value of cards there one key called query put the value of query there and then in in the output we simply contain we simply put the output that we expect from the function and now you can test this function like this so how you might want to test it first is maybe by actually passing values like this so you have test input cards and then test input query but there's a trick here whenever you have a dictionary so here we have a dictionary with two keys and we want to pass these two keys as two arguments to a function so we want to pass cards as the cards argument to the function locate card and query as a query argument to locate cards what you can do is you can simply put the dictionary itself and just write star star now if you write star star what python does is it takes the keys from this dictionary and the values are then used as arguments for parameters with these names so there we are now calling locate card on test input and we can compare it with test output and you can see that we get back false so that's one test case for us but is that enough is that enough for you to now start writing code probably not because out in the wild your function should be able to handle any number or any set of valid inputs that we pass into it and here are some possible variations that we might encounter and it really helps to list them in fact while i was writing these variations i realized that there are many cases that i had not thought of so even after coding for 12 15 years almost i still find it really useful to list out all the scenarios that we can find our input in so the simplest scenario is that the query occurs somewhere in the middle on the list of cards this is what you imagine when you read the question this is what is called the general case but then there are some special scenarios as well what if the query is the first element in cards and what if the query is the last element in cards what if the list cards contains just one element which is the query itself or and this is something that i had not thought of what if the list cards does not even contain the number query what if alice is bluffing so what should be bob's strategy then to figure out that the number does not exist what if the list of cards is empty and what if the list contains repeating numbers this is again another interesting thing that may not come to mind because we said a list of numbers and we did not specify that the numbers are unique so the list can contain repeating numbers and finally what if the number the query itself occurs more in more than one position in cards so those are eight cases that i could think of and just see if you can think of any more variations and it's likely that when you first heard the problem you did not think of all these cases because you often tend to just focus on one generic case it's hard to hold too many cases in mind and that's why it helps to list them down actually write them down in a coding interview or in a coding assessment or an interview you may want to put this in comments if you have a page coding page you can just create a comments and list out all the test cases and some of these especially things like the empty array or query not occurring in cards are called edge cases because they represent rare or extreme examples and while edge cases may not occur very frequently your program should be able to handle edge cases otherwise they may fail in unexpected ways or somebody with the with male intentions can use the edge cases to hack your software so let's create some more test cases for the variations that we've listed and we'll store all our test cases in a list for easier testing so here we are creating a list called tests and this time we will create all our test cases in the format that we discussed which is a dictionary format and we will keep appending them to our list now if you do not understand lists and dictionaries and appending then you can go back and review some of the basic material on python which is linked at the top of this notebook so first we take the one test case that we already have we put that and we take maybe one more example of the query occurring somewhere in the middle so here you can see this is the cards list and then the query one occurs somewhere in the middle although it's closer to one end then here's one case where the query is the first element four and the output obviously the output expected is zero here's one case where the query is the last element minus 127 and this is another thing the numbers could be negative as well something you may want to keep in mind here's another one the weather card contains where cards contains just one element the query itself now the problem does not state what to do if the list cards does not contain the number query and you may often face these questions where it may not be clear what to do in a certain situation or if a certain situation can occur and when you have questions like this this is a process you should follow step one read the problem statement carefully or ask the interviewer to repeat the question so read the problem statement carefully and you me you will often find hints and sometimes these hints are just single number single words somewhere often you will also find some examples provided with the problem you will also find if you scroll down to the bottom you will find some conditions you will find limits on what the numbers can be whether they can be integers or can be decimals whether they can be negative or positive so it's important to read the problem carefully before you start coding and look through the examples and then ask the interviewer or maybe post a question on the platform for a clarification often it happens that interviewers because they take so many interviews they may forget to specify a certain detail and or they might expect you to ask the question because you should not be coding with an insufficient requirement so to clarify the specifications of the problem is very important so if you have any doubt ask the interviewer even if you are somewhat sure about it but just want to verify it's a good idea to ask then finally if you are done with all of these and you still do not have a solution then you just make a reasonable assumption state it and move forward so we will assume that our function will return -1 in case cards does not contain query so if cards does not contain query then we return we expect the function to return -1 now here's one other case where the card's array is empty and obviously then it does not contain the query as well and finally there's one last case which is the number itself can repeat in cards numbers can repeat in cards and then the query itself can repeat in cards so here the query does not repeat three does not repeat but the numbers on the in the cards that i do repeat and the last case is when the query itself repeats so you can see here in cards the query occurs many times once again it is not specified what to do here and sometimes it may be okay sometimes the problem statement may just say that return any one position but more likely than not what you will want to do is you may want to make it more deterministic and that will also make it easy for you to test the function so what we can say we can impose this additional restriction that we will expect our function to return the first occurrence of query and that will make it easier for us to test so that when we when we're testing our problem we we know that if we're getting a failure it's not because of multiple possible answers but it's because of some issue in our code right so you want to get good feedback from failures and that's why you want your tests to be deterministic so here is the final test and now we can see the full list of test cases so now we can see the list of test cases here so you have about eight or ten test cases here you may not need to create this many test cases in an interview or a coding assessment depending on how much time you have but you should create at least a few at least cover the three or four edge cases a good number to aim for would be five and this will not only help you in the coding interview help you solve the problem this will also be appreciated by the interviewer because it shows that you're thinking about the problem so definitely take a minute or two now we've spent 10 15 minutes talking about this but once you start applying this technique over and over you will see that you will start creating test cases in seconds so as soon as you read the problem and you state the problem find the find the input format find the output format write a function signature and write the test and then you will start working on tests the ideas will automatically start coming to you and within maybe two or three minutes you will be done with both all to both of these steps so great we now have a fairly exhaustive set of test cases and creating test cases beforehand allows you to identify different variations and edge cases and sometimes it may happen that you may have no clue how to work on the problem you may feel completely confused but if you simply start writing multiple test cases and start looking at them like literally list just staring at the test cases the question and the answer the solution will reveal itself to you so don't underestimate the power of writing things down and don't stress it don't stress out if you can't come up with an exhaustive list of test cases because this takes time it's a skill that you cultivate with time so what you can do is you can list out maybe the test cases that come to your mind right now and put them in a single place and keep coming back whenever a new test case comes to mind while coding or while discussing or while analyzing you can just come back to the same place and write down the test case the important thing is that you have a single place where you're listing all test cases so we've written our test cases now and now we can come up with a correct solution and how do you come up with the correct solution not by writing code but by first stating it in plain english so your first goal and by correct we do not mean the best or the most efficient solution first we want to solve the problem we want to figure out where the particular number lies in the list and not to minimize because that's solving two problems at once and sometimes that can get tricky so first aim for correctness then aim for efficiency and the simplest or the most obvious solution which almost always exists and is almost always very easy to see involves checking all the possible answers and this is also called the brute force solution so in this problem coming up with the brute force solution is quite easy bob can simply turn over the cards in order one by one till he finds the card with the given number on it so this is what this is how it might work if we want to implement it in code and this is where writing it in your own words becomes important so we create a variable called position inside a function with the value 0 then we check that the number at the index position in the card list equals query or not now if it does since we're starting from the beginning if it does then position is the answer and we can return it from our function but if it doesn't then we simply increment the value of position by one and then we repeat the steps so we go back to step two and then we check whether the number at the index position on in cards equals query and once again if it does we return position if not we increment the position once again and repeat and we repeat that till we reach the last position and if the number was not found we return -1 so it's a simple four-five-step description doesn't take very long you can either say it out loud to the interviewer they will also appreciate it that they will know you know you may know that you know the brute force solution and you may not say it because it seems too simple or obvious but the interview does the interviewer doesn't know that so it's important to state the brute force solution you may say that i the brute force solution is fairly straightforward and it goes like this steps one two three four just take 30 seconds but at the very least it informs the interviewer that you're able to think of some solution and it happens very often i've seen it in interviews where 30 40 minutes have passed out of 45 minutes and not a single solution has been proposed so far even though many lines of code have been written so it's important to state your solution and if you state your solution the interviewer will also help you and correct you as you go forward right so it is a collaborative experience it is a discussion so use that and if you are in a coding assessment you may just want to write out a few comments and what we've implemented here is congratulations is just our first algorithm and an algorithm is simply a list of statements a list of steps that can be converted into code and executed by a computer on different sets of inputs so this particular algorithm is called linear search because it involves searching through a list in a linear fashion element by element so now we're ready to implement the solution and just a quick tip as i've already said always try to express the algorithm in your own words and it can be as brief or as detailed as you like and don't underestimate the power of writing writing can be a great tool for thinking it's likely that you will find that some part of the solution is difficult for you to express and that simply suggests that you are probably unable to think about that part clearly so the most more clearly you are able to express your thoughts the easier it will be for you to turn it into code and you will not have to come up with a strategy while you're writing the code so you can focus on coding and focus on avoiding errors and that brings us to the next step implement the solution and then test it using the example input so now you can see how everything comes together we've already know what the function signature looks like what the inputs look like we already have some test cases and through the test cases we've also identified what are the different edge cases we need to handle and we've already written out a description a rough description of what the algorithm looks like and in fact what you can do is you can simply write out comments within your function as the english description then you simply need to fill out code for those comments so for instance here are the five steps that we have just written down create a variable position with the value zero set up a loop check if the element is matches the query if yes the answer is found if not increment the position and then go back and then check if you've reached the end of the array if we have then we return -1 so then the code now is pretty straightforward we create the position variable 0 we set while true so while true kicks off a loop and we just want to first set up a loop and then we can break out of it when we need to then we check if the element at the value position matches the query if it does we return the position if it doesn't so if it doesn't then this we come to this part if it does then we say the function exits and none of this code gets executed but if it doesn't then we increment the position and then we check if we have reached the end now if you have the hdn obviously we don't want to continue so we can simply return minus one and exit the loop and exit the function itself but if it if we have not reached the end then we go back to the top of the loop and now position starts out with value one so we check value zero one two three so on up to the end of the array simple enough great so now we have our first function and let's test our function with the first test case so here's our test case once again and we can simply call locate card with the test input and the test the cards in the query and this is the result we get and you can already see that the result matches the output and that's why when we compare them we get the value true so yeah the results match the output and because this is something that you should be doing very often in this course we have put together a small function for you within the jovian python library so the jovian platform also offers a python helper library that is that contains some utility functions so we've put together a small function for you called evaluate test case and you can write it on your own as well but you can use this library version so let's install the library we will install the jovian library using pip install jovian minus minus upgrade and then from jovian.python dsa so joven is the name of the library and then inside the joven library since we have many courses the python dsa course the utilities for this course are present inside the python dsa module from that module we import the function evaluate test case and finally we can call evaluate test case and then we can give it the function that we want to test so you know test the locate card function and the test case the test case needs to be defined in this format so all it is going to do is it is going to pick out the input pass it into the function get the output compare that output with the expected output and also print some information for you to see so here's what it does it prints out the input it prints out the expected output it prints out the actual output it prints the execution time and this is something that will become important later and it tells you whether the test has passed or not so it's nice to have this you know because so we don't have to look through the output and input and compare them especially when you are in a situation where you need to think fast it's helpful to create a small function that can just print pass or fail for a test case so now while it may seem like we have a working solution because our test case has passed we can't be sure about it until we test the function with all the test cases so for doing that we can use the evaluate test cases function so just as you have evaluated test case you have evaluate test cases also part of the jovian library and you can call evaluate test cases with the same function locate card and this time pass it a list of test cases each of the test cases is a dictionary again you don't have to use this function you can simply put things into a loop so you can always just do for test in tests and then simply call evaluate test with locate card and test or you can even just directly call locate card with the test inputs and the test out and compare the output with the test output right so you can do this as well and you can simply print that so here's a simple way to do this what we are doing here but what we'll do is we'll use the evaluate test cases function because it prints out a lot of useful information for us so now you can see that it prints out case case by case now test k0 we have input expected output actual output the case has test cases passed that's what we saw it in fact it's the same test we just did test case one passes as well test case two passes test is three four five six okay all of them are fine okay test case six seems to have caused an error so here is the error it says list index out of range so that's okay it's perfectly all right for your functions to encounter an error so the first thing the most important thing is not to panic in fact it's a good thing that we know exactly where the function is failing if you look back here you can see what the issue is and then we'll see how to fix the error but one one good strategy to approach this is to keep in mind that there will always be bugs in your code and approach writing code not with the assumption that your code will be correct but go with the default assumption that your code will be wrong that there will be issues what that lets you do is one you do not feel demotivated or you do not panic when you see an error and second you then tend to be a little more careful while actually writing the code so the way you should be writing code is every time you write a line of code you should be asking yourself how can this line of code go wrong or in this particular case how can card's position equals equals query in an if statement go wrong and throw an error and let's look at it one easy way to check this is to add what is called a logging or what is called printing the information inside a function so we'll just rewrite our function in in our locate card function we will put in cards and we will written query the exact same function that we have we'll set the position but before we create the value we'll simply print the cards in the query so just for our information just so that we can see what the function is working through we can get some visibility into the function we print out cards and query and then while true so this is the same loop at the beginning of the loop we will print out the position that we are tracking okay so let's do that we've simply added some print statements and this print statement will give us an insight into the inner working of the function now if you do not put in a print statement then you will have to work it out yourself by reading the code and executing it in your head it's always easier to just print all the all the information and then print it nicely just say cards and query you know we could also have done this without saying cards here but then that would make it a little harder to read then that would be more cognitive overload apart from already dealing with the stress of solving an error right so just add nice pretty print statements to make it very obvious what we are printing so let's see now let us get the test case out so let's get from test 6 get the input get the cards get the query as well and pass it into locate card and now we see that initially the cards array is empty and the query is seven and the position is zero and then we encounter an error we encounter the error list index out of range on the line cards position equals equals query and now at this point it should be fairly obvious what the issue is the issue obviously is that we have an empty list an empty list has no elements but we're trying to access the position 0 which is in normal human conversation the first element of a list but there is no first element to access and that is why we get the error list index out of range so this is very important whenever you get an error do not try to start looking at the code first just try to understand the error first and if you're unable to understand the error just add some print statements there are tools like debuggers that people use but i personally in 15 years haven't used a debugger i maybe used it a couple of times but i don't know how to use it print statements are really simple you just put them in chuck them into the function wherever you need them as many print statements as you need with nice clear messages make it very obvious and that will almost certainly solve the issue for you so the cards area is empty we cannot access position zero so what's the solution here the solution obviously is that before we access anything from a list we need to make sure that we can access that list and this is the way to do it so now we've rewritten our function slightly we once again start out with position 0 but this time instead of putting in a while true instead of assuming that we can access the zeroth element of the list we say that the position should be less than the length of cards now if you have a cards list of n elements the indices go from zero to n minus one or in the case of zero elements there are no indices to access so the position has to be less than the length of cards for you to be able to access it and in this case the length of the cards will be zero so zero is not less than zero so the while loop will not run at all and we will directly return -1 but if the card does have elements then we can check the element at the value position compare it to the query and return the position if it does not if it does not match the query we can increment the position so that was a fairly straightforward fix easy save so let's test the failing case again great so looks like the failing case is now passing because we have output minus one and the expected output matches the actual output of the function minus one because the query does not exist in the array which is empty of course now this is not enough it is every time you make a change to the code you want to go back and test all the test cases because what have what may happen is while fixing one error you may introduce another error and that is where having a good set of test cases is very important so let's run evaluate test cases once again you can see here this time that all the test cases are passing and it's just nice to it just makes you feel good as well makes you feel motivated as well to see that a bunch of test cases are passing now in a real coding assessment or a real interview you can probably skip the step of implementing and testing the brute force solution in the interest of time because it may take about five to ten minutes to implement the solution and then if you have errors in the solution it may take some more time to fix those errors so it's generally quite easy to figure out the complexity which we'll talk about in a second of the brute force solution from the plain english description and that is why you should first state it in plain english which only takes around 20 seconds or so and the computer doesn't throw errors at you for speaking so you can just state the plain english description and move on talk about the complexity and start optimizing it but while you're practicing always always implement the brute force solution too and there's an important reason why you should know how to implement the brute force solution because in case you're not able to figure out the optimal solution to the problem you can still go back and implement the brute force solution and in a lot of cases that's okay sometimes interviewers ask hard questions just to push your boundaries a little bit but if you're unable to figure out the optimal solution then they will allow you to implement a brute force solution so that is why you should state it and that is why you should know how to implement it okay so we are done with so we're done now with the implementation of our brute force or simplest solution and now we need to analyze it and this is where we'll now learn about what is called the complexity of an algorithm what does it mean now recall the statement from the original question alice challenges bob to pick out the card containing the given number by turning over as few cards as possible but right now what we're doing is we can say we're simply turning over cards one by one and before we talk about what does it mean to minimize the number of times we turn over cards or the number of times we access elements we need a way to measure it and let's think about it you know it's it's as simple as just thinking about it since we access the list element once in every iteration so here's the code our code is pretty straightforward and this is where we are accessing an element from the list so since we access the element since we access the element once in every iteration for a list of size n we access the elements from the list up to n times because we may have to access this element and then this element and this element and so on so bob may need to overturn up to n cards in the worst case to find the required card now let's introduce an additional condition that suppose bob is only allowed to overturn one card per minute so that means it may take him 30 minutes to find the required card in the worst case if 30 cards are laid out on the table now is this really the best he can do or is there a way for bob to arrive at the answer by turning over just five cards and save 25 minutes instead of turning over all 30 and this field of study and by the way bob in this case is represented of what our computer does and a computer takes some amount of finite time to perform each instruction so each array access actually takes some time although it's so fast that we do not see it especially for small inputs but this is something that will become increasingly important as we go week over week where we see that we will start to see the limits of how long it takes computers to solve certain problems so the field of study concerned with finding the amount of time or the amount of space or the amount of other resources required to complete the execution of a program is called the analysis of algorithms and the process of figuring out the best algorithm to solve a problem is called algorithm design and that is what we are doing here we are actually doing the analysis of algorithms right now and algorithm design next so there are a couple of terms we need to understand and then we will go back to writing code first thing is complexity and the second thing is the big o notation and both of these are terms that you will hear very frequently in when you're talking about data structures and algorithms when you're talking about coding interviews assessments so these are terms that you need to understand and they're fairly simple terms although the term itself is complexity but all it means is that the complexity of an algorithm is simply a measure some some measure of the amount of time or space required by an algorithm to process an input of a given size example if you have a list of size n then the complexity is the amount of time required or the amount of space required on the ram to process an input of that size now unless otherwise stated the term complexity always refers to worst case complexity so it's possible that the bob turns over the first card and that is the answer but we always talk about what is the longest or the highest possible time or space that may be taken by the program to process an input right so we need to design our programs keeping the worst case in mind now in case of a linear search which is what we've implemented just now the time complexity of the algorithm is some constant c times n assuming n is the size of the list n is the number of cards right so now this constant c obviously depends on the number of operations that we perform in each iteration so in each loop for example we have four to five statements and then the time taken to execute a statement on your specific hardware now if you have a two gigahertz computer that may be twice as fast as a one gigahertz computer if you're running it on a phone it may be different so the c captures all of these things so information about the number of specific operations that we perform in each iteration and information about the actual hardware that you're running on so cn is the time complexity and n is the size of the input so in some sense what we understand from this is that the time complexity is proportional to the size of the input and that's the important part here the constant you know it doesn't change as you change the input the constant doesn't really change now similarly the space complexity now since we are already given an array the additional space that our linear search requires is simply a single constant when we are calling it c prime or c dash and it is independent of n so no matter how many uh no matter how large a list is given to you and the list is already present in memory we just need to allocate one new variable called position and that variable is used to iterate through the array and it occupies a constant space in the computer memory because we keep go on updating the variable right so the space complexity is c or constant it is independent of n now what we do normally is to represent the worst case complexity we often use the big o notation and in the big o notation what we do is we drop any fixed constants and we lower the powers of the and we drop any fixed constants and we drop any lower powers of variables so the idea here is to capture just the trend just the trend of the relationship between the size of the input and the complexity of the algorithm for example if the time complexity of an algorithm is some constant times n cube plus some constant times n square plus some constant time n plus some constant where n is the size of the input in the big o notation we simply say that it is order of n cube which is that you know in the long run in the if you just study the trend it the trend will be some something which looks a little bit like the n cube function and it may be offset by a constant or such so putting it this way the time complexity of linear search is order n because we just drop the constant c and the space complexity is order one so we again drop the constant c prime and we'll see why it's okay to drop the constant sometimes you may find that okay we're not exactly doing n iterations but we're doing n minus one iteration so we drop the minus one sometimes you'll find that we are just doing n by two iterations and that's simply half of n so we drop the half and you might wonder that okay that that might take twice or three times the amount of time how why are we dropping that constant because that's probably an important thing to keep in mind but we'll see we'll see soon as we implement our efficient solution to the problem so before we move forward before we optimize the algorithm we are just going to save our work because this notebook as i mentioned to you is running on an online platform we've set up everything for you you've not had to install anything but because thousands of people are using this using this platform this will shut down this will not keep running forever and what you need to do is you need to save your work from time to time and here is how you can save your work and then pick it up everything happens on the jovian platform there's no need to download anything although you could download it if you want but you there's no need to download anything so all you need to do is use the jobin library once again we've got another helpful function for you so you say import jovian and then run jovian.commit so you run joven.commit and then give it a project name the project name by which you want to identify this specific notebook and then there are some other arguments it's not too important so you can even skip this and that should be perfectly fine so now when you run jobin dot commit we will capture a snapshot of your notebook from this online platform or wherever it is running even if you're running it on your own computer we will capture a snapshot of your notebook from your computer wherever it's running and we will upload it and give you a link where you can access it so let's open up this link here so now you will be able to see this page called python binary search and it will be on your profile and you can see you can scroll down and see that it contains all the explanations and it contains all the code so this is a read-only version of the jupyter notebook so the read-only version of the jupyter notebook obviously does not require us to keep servers running so that you can run this code and when when you need to run it you know your work is saved to whatever extent you have executed things and now when you need to run it you simply click run and then click run on binder once again okay so and that is how you resume your work so what this will do is this will set up a new machine for you and on the new machine it will post the jupyter notebook and it will start up the machine for you open up the jupyter notebook and you will be able to start running the code and not just you now you can make your notebooks public or you can keep them private you have multiple viewership options so you're public and private not just you but anybody else so you can take this link and tweet it out if there's an interesting problem that you worked on you want to tweet it out you can just share this link online and anybody will be able to read through your solution and they can run it as well right in fact the notebook that i have shared with you is hosted on my profile so jovin is not just a platform for you to learn it's also a platform for you to build a repository of projects now if you go back to your profile you click on your profile or click on the jovian logo and you can see here that you will find a notebooks tab and in the notebooks tab you will find all the notebooks that you have worked on in the past okay so anything that you have committed using juventus comment you will be able to resume working on it so that's uh that's how you save your work and keep saving your work from time to time all you need to do is run jobian.comit you do not even need to put in this project argument this is just something if you want to actually give your project a name otherwise the name will be picked automatically so just keep running jovian.com from time to time especially if you're leaving your computer for half an hour or so then and your computer get goes to sleep then this server will shut down and you may lose your work coming back to our problem we've just implemented linear search and we understood that it has the complexity of order n which is and that's why it's called linear it runs in a linear time is another expression that is used it is also called linear because we are going through the array step by step now the next step is to apply the right technique to overcome this efficiency now of course we've not learned any techniques yet but we can probably figure it out if you think about it and maybe this is something that occurred to you right at the beginning and the idea that occurred to you is something that we will now implement so at the moment we are simply going over the cards one by one and not even utilizing the fact that they are sorted and that's why our approach is pretty poor we're basically checking everything so it's not a great solution but it would be great if somehow this would be the this would be the best case if somehow bob realized somehow bob could guess the card at the first attempt that would be perfect then that would be an order one that would be a constant time solution but with all the cards turned over it's simply impossible to guess the right card now the next best idea is to maybe pick a random card so maybe let's say bob picks this card and this card turns out to be a nine now bob can use the fact that the cards are in sorted order so if this card turns out to be nine that means all of these cards have numbers greater than nine and the target card is seven so the target card cannot lie in this region so the target card has to lie in this region and just by picking a random card rather than picking the first card bob has eliminated four out of seven cards to be checked right so with one check bob has eliminated a total of five cards one two three four five and of course if this number turns out to be seven perfect great guess but even if it doesn't we've still eliminated quite a few if this number turns out to be less than seven we've still eliminated three cards so that's the basic idea here that we pick something not from the edges but somewhere in the middle now what is the best place to pick something in the middle now obviously when we are picking a card we do not know whether it is going to be less than or greater than the number that we want especially when everything is close so we it's best to just pick the middle card so that whichever case turns it turns out to be we're still left with ads at most three cards to process right so if you pick this card and it doesn't turn out to be seven you either need to look at these three or you need to look at these three so that is the strategy we'll follow and this technique is called binary search and why do it just once just keep repeating it so each time you pick the middle card and you can eliminate half of the array and this is what the strategy looks like so here we have the array and in the array we want to figure out the number six so the slightly different problem but still decreasing order we want to figure out the number six so we access the middle element okay we compare it with six now it is not six okay it was a bad guess no problem but we know that four is less than six so that means that six lies to the left of four so we've suddenly eliminated half of the array we've done one access and eliminated half of it and now we're left with three numbers we pick the middle number we get seven seven is greater than six that means the number lies on the right now we are left with just one card we overturned that last card or we checked that last number okay it is equal to six great if it is not well nothing more left to check all the numbers here are greater than 6 are less than 6 and all the numbers before this are greater than 6. so if this number isn't 6 then there's no 6. and just like that for an array of seven elements we have done just three checks and arrived at the answer and that was the worst case right it mean it will never take you can verify that it will never take more than three checks if six comes at this position we guess it immediately if six comes at this position or this position we guess it in two checks and then if six comes at any of the other positions we will guess it in three checks so that's pretty good and now the idea if you if you read this part it says apply the right technique to overcome the inefficiency and then repeat the steps three to six so now we're going to go back to step three which was come up with a correct solution for the problem and stated in plain english and we have come up with a solution already we just need to state it so here is how this technique called binary search is applied to the problem it's called binary because well we take a left and right decision so first we find the middle element of the list if it matches the query number then we return the middle position as the answer and if it is less than the queried number then we search the first half of the list and if it's greater than the query number then we search the second half of the list so the exact thing that we saw here we apply it here and finally if no more elements remain we simply return -1 so let's just save our work now let's from this point on we'll keep saving our work from time to time using jovian.commit so now we've come up with the algorithm and you can again it's important to write it in your own words whether you want to write a short description a paragraph or a step by step guide but write it in your own words and you'll do this in the assignment so let's implement the solution now and test it using the example inputs so here's the implementation so what we'll do is we will look at once again let's go back to this visual representation and we will keep a track of our search space so current initially our search space is the entire array so that means we have an array of seven numbers so our search space goes from position zero to position six and slowly we'll keep reducing our search space over time so to keep track of the search space we will create two variables low and high low will have the value 0 which is it will point to the first position in the array and high will have the value pointing to the last position last valid position in the array which is which is len cards minus one so while low and then the while loop becomes very simple because as long as we have at least one element in our search space we can go ahead now to have at least one element in a search space the low value which is the starting index should be less than or equal to the end value right so while low is less than equal to high because if the starting index is higher than the end index basically we've exhausted and we've covered the entire list and there's nothing more that we can search for so we should exit at this point okay so now once we have uh once this condition is satisfied and it is initially let's say you have seven cards lower zero cards is uh len cards minus one is six then you find the middle position and you can get the middle position by doing low plus i divided by two and now let's start applying that strategy here where we say that every time we write a line of code we should think about how it can go wrong now if you write it like this low plus high divided by two and think about how it can go wrong okay low plus high may not be divisible by 2. if low plus is not divisible by 2 you may end up with a decimal number now if you do end up with a decimal number in fact the division operator in python always retains a floating point number then you cannot use it as an array index because we want to use this as a position within the array so that's why we need the double slash which is the which is the integer division which simply returns the quotient so we get the middle position and then we get the number at the middle position so we also get cards made so we access that element from the array now this is where we it makes it easy for us to count the number of times we access because here is one axis happening inside the list and there are no other accesses then we get the mid number and remember last time we faced an error and we had to add print statements you might as well just add print statements right away so here's what you can do we can just print the value of low the value of high the value of mid and the value of mid number what this will do is this will help you check whether the number is working as expect whether the function is working as expected or not so now here comes the actual check and the meat of the problem if the middle number matches the query then we return the middle number great we found it well done now if the middle number is less than the query now remember the elements are in sorted array and we are looking for the number query now the middle number is less than the query so that means the query probably lies to the left of it because the query because the elements are in a decreasing order right so if the query lies to the left of it so then we need to search we decrease the search space from the beginning to the position just before the middle number right so what we can do is we can simply set high to mid minus one on the other hand if mid number is greater than query so that means because of the decreasing order of the array the query lies to the right now we need to move the starting of the search space to beyond the middle number so we simply said low to mid plus one and that's it and you can see that we've written a we've used if lf lf loop here so lf stands for elsif in python and here the last condition could might as well just have been else because there are only three possibilities either they're equal or mid number is less or it's greater but sometimes it's nice to list out all possibilities just to make it super clear and it makes it easy for you while debugging fixing issues as well okay so that's our binary search based algorithm and finally when we exit out of the loop if you have not returned the middle number if you have not exited the function yet then we return -1 that the number was not found so let's test it out using our test cases and we have our handy evaluate test cases function here but you can also test it manually if you want by passing individual test cases but i'll just do this from now on so great so now we have test case 0 this is the input and this is the query and it passed here we have test case 1 this is the input and this is the query and it passed and now because we have these print statements we can clearly look into our test cases and actually tell if the if this is tested correctly or not because now you can see here that we started out with low 0 high 7 and a mid mid value of three so zero one two three we we found the number seven the query is one so we need to check this half of the array and that's exactly what we did we moved low to four and high remained seven then mid number became three so that means once again we need to check this half of the array and then we check this number and then we found the output so now you can see exactly how the algorithm works and this is in general what you want as a programmer you want to have a full understanding of the code that you've written you don't want your code to work incidentally you don't want it to you don't want to be in a position where you are just fixing things trying out different things and somehow at once the code works you want to be in complete control you want to know that these this is exactly what the code is doing and if it is failing why it is failing so we go to test case two three four five six uh looks good looks like we may have solved everything ah probably not so test case eight seems to have failed so test case eight is this number this list and this list contains repeating numbers and not just repeating numbers but the query itself occurs multiple times and now if we look here and maybe let's go go down and evaluate just a test case separately so here we are now using the singular version of the evaluate function so if you look here you can see you have eight eight six six a bunch of sixes then three two two zero the query is six so we start out with a low of zero higher fourteen total of fifteen elements that gives you a middle position of seven and the mid number at that position so let's count one zero one two three four five six seven okay and the mid number at that position is six great uh six is also the query so that's why our function returns seven but remember that we had decided that our function should return the first position of the number within the array so a function is failing that condition and why is that happening because unlike linear search where we start from the left and so we'll always bump into the first position because of the decreasing order of elements so we'll hit we'll encounter this six before we encounter this six binary search does not access elements in an order it access elements sort of randomly there's still a strategy but it goes left and right and it also depends on the values of specific elements whether this element is accessed before this element can depend on the value of let's say this element right so as such it's kind of a pseudo random kind of order and so we need an additional condition condition to keep track of it right so how do we fix it so the way to fix it is actually quite simple when we find that the middle position in a particular range is equal to the query we simply need to check whether it is the first occurrence of the query in the list or not that is whether the number come that comes before it is it equal to query or not if the number that comes before the middle element is also equal to query then obviously the middle element is not the first occurrence so that simply means that we can go back and because it can occur multiple times before that simply means that we can go we can now search the left half on the other hand if the middle element if the number before the middle element is not equal to query and obviously because it is a sorted list it will be greater than query then all the numbers here are going to be greater than the query and you know the the and and so this must be the first or the only position okay so make sure you understand that this must be the first or the only position where the query occurs so once again to make it easier what we will do is because there is some logic involved here what we'll do is we'll define a helper function called test location and this is a very helpful thing that you can do every time you find that okay you have to you have to cover these special cases and your function may start to get slightly longer and slightly more complicated what you may want to do is create a helper function and a good rule of thumb is not to have functions that are more than 10 lines of code or so i try to keep my functions below seven lines of code because seven eight lines is approximately the amount of information that you can hold in your head at once so if a function is about seven eight lines you can probably take a quick glance and tell what it's doing identify issues but anywhere beyond that it's very hard and if you're writing functions that are going into hundreds of lines please stop doing that please start breaking your code into small functions there's a there's a code by i forget who it is by but he's a creator of i think it's eric meyer he created the rx library for reactive programming and he said that great programmers write baby code which is really small bits of code that anybody can understand with a single look so you should be writing as many functions as many small pieces of code small pieces of logic as possible so let's see our test location function its purpose is to take the query and then take just a specific position so forget about binary search for now just take a specific position and tell if that position is the answer and how do we do that we first get the mid number from the cards so we get a mid number from cards so we then we print out mid and we print out mid number and then we compare the mid number with the query so this is the special case that we need to handle this is where we had the error now what we need to check is if the element before the mid number is also equal to query so if the element before the mid number is also equal to query then we need to go left so just to make it super clear what we'll do is instead of setting high low etc we'll simply say that we need to go left so we'll return the the actual string left but one thing to keep in mind here because once again whenever you're accessing an array you need to make sure that the index is valid so we simply check that mid minus one should be greater than or equal to zero that we made is not this position and which can happen as your search space decreases for example if this is your search space your mid will actually be this position so if it is equal to if the number before the mid number is equal to query then we return left otherwise we return found once again making it very obvious that we have found the number so we return found else the other case is if the mid number is less than query that means that the query lies on the left because of the decreasing order of the list so once again we need to search on the left else it returns right so a test location simply tells us whether we found the solution or we need to look on the left or we need to look on the right now in sometimes you will see programs especially in c plus plus java return something like minus 1 0 and 1 and then use that to represent whether you should go left and right but python is a high level language and strings are a first class things are first class feature of the language so just use strings because they are really descriptive they make your code readable somebody else reading your code will be able to understand now if you're looking at minus one plus one etc that is going to be difficult for people to understand so now we can now simplify a locate card function once again we have our low high len cards minus one uh zero and len cart minus one the while loop is the same and we print low and high as well so we are planting row and high inside the locate card function and then we are printing mid and mid number inside the test location function wherever is the right place to print something you print it then we get the mid position and now we simply call test location so we are testing if mid is the answer and if it is not the answer should we go left or should we go right now that makes it really simple because now we get this result and we check this result and if it says found then we return mid that's the answer if it says left then we return mid minus one and then we simply move high to mid minus one and if it returns right then we simply set low to mid plus one so we are simply changing the start position of the search space to after the middle element and here we are changing the end position of the search space to before the middle element right so this makes it extremely obvious and it's really hard to go wrong when you write code like this especially so when you have and binary search problems are specially tricky because they always have certain these special cases that you need to handle and if you start handling them within this if loop so now you have a while loop inside which you have an if loop inside which you have another another if statement and it can get pretty tricky and difficult to debug so let's evaluate that test case and looks like that test case has passed this time perfectly you can go through the logs here to verify it let's evaluate the test case all the test cases as well we should do this every time we change the function and that is why it's helpful to have a function where you can every time you make a change you can just run the test and on a coding platform like elite code or hacker rank you will be given some test cases although those test cases will not be visible to you so you can submit your solution but you may not get an actual result you may not get to know what the test case was or where your answer was wrong and that's where you may want to create your own test cases if you're getting a lot of errors and in fact once you've written out the algorithm you may realize that okay maybe you need to add more test cases what if the number lies in the first half of the array what if the number lies in the second half of the rate so this was not an important factor when we were not thinking about binary search but now that we are thinking in this direction of splitting the array into half we may want to add some test cases where the number lies exactly in the middle in the left in the right and the simplest way to do that is now go back to the tests array so you can open you can create a new cell here by pressing the character b so if you click outside and press the character b you can create a new cell and then you can simply do tests dot append and then write your test case so here is the final code for the algorithm without without the print statements so we have test location and then we have locate card and try creating a few more test cases to test your algorithm more extensively and once again at every step we are going to save our work by running jobin.commit so now we are down to analyzing the algorithms complexity and identifying inefficiencies if there are any now you may have just read online you can actually look it up say just search for complexity of binary search and you will read and you will find an answer but and and you may even just say that in interviews but it's always nice to just come up with that answer from first principles it's always nice especially in an interview if you can talk through it if you can talk through why it is order why it is whatever it is and we'll see what that is so now let's once again try to count the number of iterations in the algorithm because we need to minimize the number of times we access elements from the array and to do that we know that in each iteration we are accessing the element just once and then we are comparing it so we're doing a bunch of other operations but in each iteration we're accessing one element so we need to count just the number of iterations the number of times the while loop was executed now if we start out with an array of n elements then each time each time the size of the array reduces to half for the next iteration now that's roughly true because when you come when you check the middle element and then you decide whether to go left or right it's actually probably n by 2 minus 1 if n is if n is even and if n is odd it is the floor of n by two but again with algorithms with complexities we are generally interested in studying the trend so we can ignore that small part in the calculation so let's say uh the important part is that even if it's okay to overestimate a little bit but try not to underestimate so after the first iteration we may be left with the search space of size n by two it may be slightly less than that but it's okay to overestimate so we have n so we after n we have after the first iteration we we are left with the search space of n by two then we split it into half again so next time we may be left with a search space of n by four which is n divided by 2 square and then we may be left by we may be left with n by 8 and it's possible that at any of these iterations we may just exit because we may have found the right number but what we always try to analyze is the worst case complexity of an algorithm what is the longest possible amount of time or the largest amount of space it can take so right now we are talking about time because we are counting iterations and each iteration takes some time so n by 8 after iteration 3 that's 2 to the power 3 and i think then you can start to see the trend here that after the kth iteration you will end up with n divided by 2 to the power k elements now when does the iteration stop so the final iteration is on an array of length 1 and that is when we access that last element and check whether after all this checking the last element is equal to the index or not so we can do n divided by 2 to the power k and if we set that to 1 we can rearrange the terms and we get back n equals 2 to the power k so after the kth iteration if you want to be left with one element then that means n divided by 2 to the k should be equal to 1 or n should be 2 to the k or in other words k should be equal to log log n remember logarithms and here obviously log refers to log to the base 2 but what i will argue is that you can change the base of the logarithm and that will simply add a constant so that will simply if you're taking the natural log then that will simply add a constant here and remember when we talk about time complexity we ignore constants so we can just generally say that our algorithm binary search has the time complexity of order of log n that means as the input grows the amount of time taken by binary search is proportional to the logarithm of the number of elements in the list passed to it or the amount of time taken is logarithm to the size of the initial search space and you can verify this you can verify that the space complex you can you can check this out by simply writing it out as well you can take some examples let's say you take a card list of size 10 and then work through it the worst case and count how many iterations you have and compare if that is close to login or not and then as an exercise you can verify that the space complexity of binary search is order one can you you can try posting in the youtube comments or in the youtube live chat how the space complexity of binary search is order one i'll let that steam so let's now compare linear search with binary search how are the two different and what we'll do is we will create a large test case because you start to see the benefits of the difference between the order n algorithm and the order login algorithm only when you have larger test cases because small test cases everything runs instantly so it's not really that much of a hassle so here we have a locate card linear and this is the linear version of the algorithm where we simply go through each of the cards one by one and then we have a really large test case here so we have the input and then we have the cards which goes in the range okay let's see so that's one two three so that's thousand another three that's million so we have 10 million elements here so we have 10 million elements and we are looking and so we are actually creating a range here so we are using a function in python so we are creating a list of numbers going down from 10 million all the way to one so a decreasing list going from 10 million to one and this is how you create it and you can check it out and in this list we are looking for the number two which occurs at the very end so we are sort of creating this is as we will see if you want to really analyze it this is going to be a worst case scenario both for linear search and for binary search approximately worst case so the query is two and then the output is this is the output that we expect obviously because 0 to nine nine nine nine nine nine nine is are the array indices and the last element is one so the element just before is two so this is the expected output great so now we have this large test let us call evaluate test case and let us pass a check the linear search pass in the last test and because this is a huge list we may want to turn off the display of the output we may not not want to actually see the input being displayed so we can simply turn off the display by passing display equals false and we can just get back the result from the evaluate test case function so the result will give the output the actual output of the function whether the test passed and the running time of the algorithm so it takes a second so it looks like the test did pass our algorithm is correct so that's great and it took one two two four point two nine one milliseconds or about 1.2 seconds to answer it and you can probably tell why because it because this is the result so it probably took nine nine nine nine nine nine eight iterations so it had to go through all the elements to get to the very end on the other hand when we talk about binary search so now we are passing in the binary search version once again turning display to false and we are displaying the output okay so this time the result is the same the test did pass but the execution time is 0.019 milliseconds so that's 55 000 times faster than the linear search version and in fact you can tell how many elements we actually had to access so if we just check log of so log of this number is about seven and maybe you know if you're checking log two we can maybe check something like this so not more than twenty elements had to be accessed so where we linear search needed to access about 10 million elements binary search was able to get to the answer with just about 20 checks so that's a lot of time saved and you can increase the size of the array by a factor of 10 and increase this by a factor of 10 as well and then you will see far bigger difference where for a 10 times larger array linear search would run for 10 times longer whereas binary search would only require three additional operations so the linear search would go from 10 million operations to 100 million operations binary search would go from 20 operations to 23 and that is the real difference between the complexities order n and order login and as the size of as the size of the arrays grows bigger another way to look at it is that if you just divide the complexities binary search runs n by login times faster than linear search for some fixed constant because there's always some constraints involved and as the size of the input grows larger the difference only gets bigger the difference in performance and that is what algorithm analysis of algorithms and optimization of algorithms is all about it's about overcoming the limitations of computers by devising clever techniques to solve problems and it's something that you can actually apply in real life as well in a lot of cases there are a lot of things that you may see a brute force solution to but if you just apply your mind you may find a more optimal solution a more easy way or a more lazy way to do it with less work so think about that and here is a graph showing how the comp how you can compare common functions how the how the running times of common functions vary so we will look at all kinds of functions we look at constant time functions order one for example accessing an element from an array is order one so even if you have an element of 10 a list of 10 million elements you can access the last element in constant time on the other hand we've looked at binary search which has which is order log n and we've also looked at linear search which is order n now in the future we will look at other techniques which have complexities of n square n cube n to the power n are far far higher and somewhere in between there is a very nice special type of complexity called n log n which is rather nice so we'll talk about that as well n login in fact a lot of questions in coding assessments and coding interviews tend to be taking algorithms which would be which would have n square complexity in in a brute force approach and optimizing them either to order n or to order n log n so we'll we'll discuss all of this so don't worry if this doesn't make sense just yet but i hope you see now why we ignore constants and lower order terms while expressing the complexity of the big-o notation so we've covered binary search but we've seen it in the context of a problem and now we can step away one more step and abstract it out further and identify the general strategy behind binary search and this general strategy is actually applicable to a wide variety of problems and this is what you want to keep doing as a programmer you need to abstract away peel away the layers of specific problem specific details and find the general technique find the general strategy and then encode that using your functions and programs so here's the general strategy come up with a condition to determine whether the answer lies before after or at a given position so we are assuming here that we have some kind of a range and we have to identify a position within a range or maybe an element within that range but we can access elements using the position so come up with a condition that that first tells you whether given a position the answer lies at or before or after that position once you have that condition first retrieve the midpoint and the middle element of the list now if the middle element or the midpoint is the answer then return the middle position that is the answer you're done if the answer lies before it repeat the search so repeat the process with the first half of the list or the first half of the search space and if the answer lies after it repeat the search with the second half of the search space so here is the generic algorithm for binary search implemented in python and you can see a classic detailed documentation here so while so here you have the binary search is going to take a search space low and high so low is going to be zero and high is going to be well we will pass in maybe the final we will pass it may be the final position the final index of the array but writing it this way rather than passing passing an array also allows you to use binary search for problems that are not based on arrays sometimes these could just be numbers for example if i ask you to find a number between 1 million and 10 million that is a perfect square then you can use binary search to do that then it takes a condition so what it does is it starts a loop so while low less than equals high we get the midpoint so low plus high divided by 2 that gives us a midpoint then remember earlier we had this condition test location so our condition simply is supposed to take the middle position and identify if the middle position is the answer or we need to go left or right so the condition should return either found left or right so if the condition returns found we return the midpoint as the answer if the condition returns left we return the high we we move to the left side so which is we take the end of the search space and set it to before the midpoint so we set high equal to mid minus one and if the condition returns right which is the else case here we set low to mid plus one so we take the start point of the search space and move it after the element then we return minus one so that's your binary generic binary search algorithm and if you start using this what will happen is now this is a tested piece of code and in fact we can see it here now now we can rewrite locate card and locate card can be we are passing in cards and we are passing in the query and we need to write a condition and here we're using a very interesting feature of python we are writing a function inside a function so this is called function closure and it's a very handy feature so now we can simply write condition inside locate card and what that does is binary search is going to pass the middle value the middle position but condition can also access cards and query so which is because it lies inside locate so what we do inside condition is okay we check them we get the middle element cards made if cards made is equal to query then here we have that check we check whether it is the first occurrence of query or can query occur before it if query occurs before it we return left else we return found and then these are the original conditions that we already had so you can verify this by going back and checking but the important part here is now the while loop has gone away now we can simply call binary search with zero len cards minus one so the start index the end index and the condition and we can evaluate the test cases and you can see that the test cases are correct and now you can use this binary search function because we have not tested it with one problem you can use this exact same function to solve other problems too in some sense it is a tested piece of logic so here's what we'll do we'll take a quick question and we will implement it now we've spent what one and a half hour talking about a particular problem but let's spend maybe two minutes talking about a new problem and solving it so now here's a slightly related question given an array of integers sorted in increasing order find the starting and ending position of a given number so once again you have a sorted array this time they are increasing the only difference is now apart from the fact that they are sorted in increasing order the other difference is that we are looking for both the start index and the end index so we're looking for both the start index and the end index of a particular number because the number can repeat like we saw one example and there's a very simple way to solve this a simple strategy is do binary search once to find the first position and that's what this function does i'll let you read through it the only changes here are this variable this has changed this order because now the now the elements are in increasing order and then the second change and there's no other change here so that there's just one change here and then there is another function called last position here instead of checking the left we are checking the right so instead of checking mid minus 1 we are checking mid plus 1 and if mid plus 1 equals the target we go to the right and of course we have the same change here in this code because instead of decreasing we have increasing order right so now we write two position now we write two functions first position last position and then first and last position is simply getting the first position once so that's one binary search and getting the last position once that's two binary searches and that's not bad now the complexity is still order login two times login or two times some constant times login when you express it in the big o notation is still login so that's okay and that was quick we were able to reuse most of the code that we have written and that's the benef benefit of making generic functions like binary search and in fact we can test a solution by making a submission here so let's go to leadcode.com let us here what i've done is i have already copied over the binary search function the first position function and the last position function so by the way lead code is a great platform for practicing so you can go to leadcode.com sign up with any account and you will find a lot of problems especially on the in the problems tab and here you can see that this is exactly the problem that we have been solving just now so we just post the code here binary search first position last position first and last position and lead code requires you to write this class called solution this is something that they give you beforehand and inside the solution you need to define a function called search range where we are simply calling our first and last position here i'll let you see and we simply we can test this code with a test case so you can pass a test case here and test it out great or we can simply submit it and here you can see that the problem was submitted successfully and it tells you things like how much runtime it used what was the memory it used and your solution was accepted right so check out leadcode.com go to the problem section and you can see all the different problems that they have you can also explore and you have different problems that come up every day it's a great place to practice so that's binary search for you but i just want to revisit the method once again so this is the systematic strategy that we've applied for solving the problem we state the problem clearly and we identify the input and the output formats this this shows that you've understood the problem you know what the solution will look like then come up with some example inputs and outputs and try to cover all the edge cases so this shows that you are envisioning what are the different inputs that can come in before you write code then you come up with the correct solution not necessarily the most efficient one and state it in plain english now when you try to state it you will have to clarify it and that will help you clarify your own thoughts and then you can analyze the algorithms complexity and you can implement the solution and test it using example inputs so this is the basic solution now in interviews and in coding assessments maybe you know where there is a time limit you may not want to implement the brute force brute force solution because then you may get stuck in fixing issues with brute force and you can directly jump ahead to step five but while you're practicing always implement brute force then step five analyze the algorithms complexity and most of the time it is simply a matter of counting the number of iterations how many times a while loop or maybe a loop within a loop is getting executed and identify inefficiencies and if it is a brute force solution it's generally quite easy to see the inefficiency for example in this case the inefficiency was that we know that the array is sorted that anything we do will be better than going line by line right we could pick a random element and that would help us eliminate a good chunk of the array so that is the inefficiency and then apply the right technique and we are learning the techniques so we've learned binary search today and then we are going to learn a lot more techniques that are asked in interviews so apply the right technique to overcome the inefficiency and repeat steps three to six which is go back and come up with the correct solution with the optimized technique implement the solution and test it using some example inputs and then analyze that algorithm's complexity and identify any inefficiencies so what we've done for you is we have created a template so you can see this python problem solving template and how you can use this template is to simply run it so you run the code you run this template and then when you run the template inside it you will see this question mark in a bunch of places so you can give it a nice project name and you can commit it to your profile one way you can save a copy over this template to your profile is by clicking the duplicate button now if you click the duplicate button you can copy it in your profile and you don't have to look for it you can just find it on your jovian profile but anyway once you have it copied you can click the run button and then click run on binder and run run the template then you go down once you run it and you can copy over a problem statement you can copy over a link to the problem so that when you need to make a submission you can go back and refer and then here the method is summarized for you and here we have created sections for you so you can simply start filling out this method step one step two step three step four step five so whenever you're faced with a difficult problem just use this template and i guarantee it one if you work through this course you will be able to solve a majority of the problems that you come across and specifically even if you are able to follow maybe about 30 to 40 percent of this course you will easily be able to solve most questions that are asked in interviews because questions asked in interviews are fairly simple in terms of the data structures or algorithms they test but the intention there is more to test your approach look at the quality of your code and see how clearly you're expressing yourself right and this is what this is exactly what this method teaches you to do now to encourage you to do this to encourage you to try it out and you can take problems from places like lead code uh code chef code forces there are a few links listed here you can see practice problems there are a bunch of links listed here so that was today's lesson for the next lesson common data structures in python so this is data structures and algorithms in python an online certification course brought to you by jovian thank you hello and welcome to data structures and algorithms in python this is an online certification course being offered by jovian today we're looking at assignment one binary search practice so let's get started first thing we'll do is go to the course website pythondsa.com on the course website you can enroll for the course and view all the previous lectures and assignments for assignment 1 you may want to review the video and notebook for lesson 1. let's open up assignment one it's called binary search practice now in this assignment you will apply and practice the concepts that we covered in the first lesson so you will understand and solve a system solve a problem systematically implement linear search and analyze it and optimize the solution using binary search and ask questions and help others on the forum let's open up the starter notebook for the assignment which contains that problem statement and other information now this is a notebook you're looking at hosted on jovian you can see some description here and if you scroll down below you can also see some code and you will need to execute this notebook modify the code with within it and record a new version which you can then submit to see your score so let's start reading through it as you go through the notebook you will find three question marks in certain places to complete the assignment you have to replace the question marks with appropriate values expressions or statements to ensure that the notebook runs properly end to end now keep in mind that you need to run all the cells otherwise you may get errors like name error or undefined variables you should not be changing any variable names or deleting any cells or disturb any existing code you can add new code cells or new statements but do not redefine or do not change some of the existing variables you will be using a temporary online service for code execution and we'll see how to use it in a moment so keep saving your work by running jobin.com at regular intervals and then the question marks optional will not be considered for evaluation although we recommend doing them they are for your learning but you can make a submission before you have solved the optional questions now you can make a submission back on the assignment notebook page and we'll see how to do that and if you're stuck you can ask for help on the community forum it's listed here and we'll see how to do that as well now one final thing i want to mention is you can get help with errors or ask for hints you can even share your code and errors that you are getting in the code but please don't ask or share the full working answer code on the forum this is so that everybody has the opportunity to work through the problem statement on their own make mistakes learn from their own mistakes and arrive at the right solution now how do you run this code the recommended way to run this code is by clicking the run button at the top of the page and selecting run on binder but you can also run it using some other options like google collab or kagger or you can run it on your computer locally so we're going to use the recommended method run on binder now we have the notebook running in front of us the first thing i like to do is go to kernel and click restart and clear output so that we can see all the outputs of the notebook from scratch and i'm also going to toggle the header and the toolbar so that we can zoom in a bit so now the same jupyter notebook is now running online on a platform called binder and before starting the assignment let's save a snapshot of the assignment to our jovian profile so that we can access it later and continue our work i'm going to run pip install jovian this is going to install the jovian library then run import jovian to import the library and set a project name here i'm just calling it binary search assignment and run jovian.commit now you have taken a starter notebook which was hosted on my profile and then you've run it on binder where as soon as you run jovian.comit a copy of the starter notebook gets saved to your profile so what you will see here is a link to a notebook hosted on your jovian profile let's open it up here and see so now this is your personal copy of the assignment notebook any changes that you make here and run jovian.comit will get added to your profile so if you want to come back and continue your work then you do not have to go back to the original starter notebook which contains all blanks rather you can come back to your profile and you can come to your profile simply by opening jobin.ai and on your profile you can go to the notebooks tab and on the notebooks tab you will be able to find as you can see here you will be able to find the binary search assignment here there you go this is the binary search assignment that we just created and you can open it and run it on binder to continue your work so moving along this is the problem we're looking at here you are given a list of numbers obtained by rotating a sorted list an unknown number of times okay so we have two new terms here rotating a sorted list and don't worry if you don't know what that means normally if you see any new terms in a problem they will be explained somewhere within the problem itself for instance here you can see that there's a definition we define rotating a list as removing the last element of the list and adding it before the first element for instance rotating the number the list three two four one leads to removal of the last number and then placing it at the very beginning so you end up with the list one three two four this is a new operation that we are defining this is not something standard but you will find that a lot of problems will define new terms or new operations so that they it becomes easier for you to understand the problem so that's rotating a list now rotating a list once produces one three two four now if you rotate that list again the resulting list one more time then you will end up with four one three two and so on and then the other term is sorted so sorted refers to a list where the elements are arranged in increasing order in this case we have numbers and the numbers 1 3 5 7 are increased arranged in increasing order so this is a sorted list but if this was 3 2 4 1 well that's not the numbers are not arranged in increasing order so that's not a sorted list so you are given a list of numbers update obtained by rotating a sorted list an unknown number of times for instance this sorted list 0 2 3 4 5 6 9 is rotated a certain number of times and you can verify that if you rotate this three times you end up with the list five six nine zero two three four right you can see that first nine comes to the beginning then six comes comes to the beginning and then five comes to the beginning so you need to write a function and you're given just this the list you're not given the original sorted list you're given the list obtained by rotating some sorted list and unknown number of times now you need to write a function to determine the minimum number of times the original sorted list was rotated to obtain the given list your function should have the worst case complexity of order log n where n is the length of the list and you can assume that all the numbers in the list are unique okay so three parts write a function to determine the minimum number of times you need to rotate the original sorted list in this in this case it is three the function should have the worst case complexity of log n so this determines correctness and this determines efficiency and then this is some additional information to help you that you can assume all the numbers in the list are unique if this was not mentioned you would also have to handle the case where your list does not contain unique numbers now we will apply the method that we have been applying all throughout this course for solving the problems number one state the problem clearly identify the input and output formats number two come up with some example inputs and outputs and try to cover all the edge cases number three come up with a correct solution for the problem and state it in plain english number four implement the solution and test it using some example inputs and having test cases and then implementing a solution allows you to test them using the example inputs and fix any bugs that's why it's very important to have some test cases number five analyze the algorithm's complexity and identify any inefficiencies and number six apply the right technique to overcome the inefficiency and then you go back and repeat steps three to six come up with the correct solution implement the solution and test it and analyze the algorithm's complexity and you can review lesson one for a detailed explanation of this method now let's apply it step by step the first step is to state the problem clearly and identify the input and output formats now while it is stated clearly enough it always helps to express it in your own words in the way that it makes it most clear for you and this is something that you can keep returning to rather than the original problem statement because this is something that you will understand better and it's okay if your problem overlaps with the original problem statement but do try to express it in your own words so in this case what i've just done is i have double clicked here once you double click you can now edit this text cell and now we can start writing a problem so let's say given a rotated list we need to find the number of times it was rotated and okay i think what i've probably missed here is that it is a sorted list so given a sorted list that was rotated some unknown number of times we need to find the number of times it was rotated right maybe i'm just going to say given a rotated sorted list because technically the input is not a sorted list it's a rotated sorted list so given a rotated short test list that was rotated an unknown number of times we need to find the number of times it was rotated now doing this exercise helps you determine if you've understood the problem correctly and you may often find that okay there's a certain detail in the problem that you missed okay but at this point i'm happy with my description and you will see that it is matching the description to a large extent but it's something that i understand better so i'll just refer to this from this point now i'm assuming here that i know what rotation and sorted means otherwise i could also include those then here's a question the function will you will write will take one input called nums what does it represent and give an example okay so once again we double click on this and one input is nums so this is a sorted rotated list and let's give an example here let's say we take the sorted list three five six seven nine and then we rotate it a few times let's say we rotate it a couple of times so we end up with this sorted rotated list so that's our input so we've answered the question here now the first question was to express the problem in your own words this is the solution the second question was what does the input norms represent give an example it represents a sorted rotated list 79356 the third question is the function you will write will return a single output called rotations what does that represent well you have to write a function that identifies how many times the list was rotated so this is the number of times the sorted list was rotated okay and in this case the example that we have is that this sorted list was rotated twice three five six seven nine was rotated two times so you mentioned two here now you can see these back quotes that i'm using here this is next to the number one on your keyboard or below the escape key what these backwards let you do is they let you express text as code within markdown you can see that they have a gray background and they have a different font this looks a lot more like code same is here true for nums so you can use markdown and its features to your advantage to organize your descriptions and your text better okay so now based on the above we can now create a signature of our functions we have a function called counts rotations it takes the num list of numbers and it returns well right now we're just putting pass in here but we know that it's going to return a single number rotations now after each step remember to save your notebook so we are going to just run joven dot commit and now if you leave your computer you do not have to be worried that your work may be lost so you can go in here and you can open up this notebook from your joven profile and press run at any point to run this notebook now step two is to come up with some example inputs and outputs and try to cover all the edge cases and our function should be able to handle any set of valid inputs so here is some variations that you can encounter a list of size 10 rotated three times a list of size eight rotated five times so these are two generic examples and then a list that wasn't rotated at all a list that was rotated just once a list that was rotated n minus 1 times where n is the size of the list a list that was rotated n times and what do you mean by rotating the list n times well let's see an empty list and a list containing just one element and if you can think of more test cases you should definitely add more test cases here and what we'll do is we will express our test cases as dictionaries so this will help us organize the test cases and test them all at once more easily using helper functions so you can see here that we've organized one test case here and we've expressed the task as a dictionary so here we have the input to the test case that is the input key and then we have the output to the test case now because a function can take many arguments the input itself is going to be a dictionary and then for each argument in this case there's just one so we just call it nums we have the input here and this is the size of the output okay so let's create this test case and let us then if you want to fetch the actual input and output out of it so here we can fetch test input nums that's going to give us the nums we can use test input the output should be test outputs this seems to be an error and the result is count rotations num0 okay so this is the actual result obtained by passing the test case into count rotations and you can see that the result we get back is none because right now we do not have any code we it just says pass inside and the result and the output are not equal because the output is the number three but the result is none so that's okay our test case is failing right now because we have not yet implemented the function but as soon as we implement it we expect to see the test case passing now to help you avoid all of this work we have given you a function called evaluate test case so from jovian.python dsa you can just import evaluate test case and then call evaluate test case with the function you want to test and the actual test case and you can see here it prints the out input that was passed in the expected output the actual output that was obtained and the test result in this case the test result was failed and the execution time is also printed here if you just want to evaluate if a certain implementation is faster than another so now your job is to create test cases for each of the scenarios listed above so here is test zero that is same as the original test case that we had created now here is test one a size a list of size eight rotated five times i will let you create this but it will look something like this you will open up you'll replace the three question marks with let's say a list of size eight so one two three four five six seven eight and you can imagine that if this was rotated five times then one two three four five four five of these numbers will then move to the first position and you get this as the input numbers and the output well it was rotated five times so i think you can guess that the output here should be five now here is a list that wasn't rotated at all what should be the output here i'm sure you can guess that the output here should be zero so i'll let you fill this out here is a list that was rotated just once so let's try let's fill out this one so this this list rotated once would give us seven three five there you go a list that was rotated n minus one times where n is the size of the list okay i'll let you do that a list that was rotated n times where n is the size of the list okay what does that look like so you take this list and then you first put 10 in the first position and then you put 9 in the first position so 9 10 comes to the beginning and 3 5 7 8 comes after it then you move 8 to the first position then you move 7 then you move five then you move three if you if you move all of these back to the first position you end up with the same list so you've rotated it n times now what should be the output in this case there are about six numbers here so is the output 6 i don't know i'm not so sure because remember the question the original question says write a function to determine the minimum number of times the original sorted list was rotated to obtain the given list so it has to be the minimum number of times we may want to just go back and change this we need to find not the number of times it was rotated but the minimum number of times it was rotated or it needs to be rotated right so coming back here the output should be not six but zero so keep that in mind then here's an empty list i'll let you figure out what should be the nums and the output here and here is a list containing just one element once again should be pretty straightforward can you rotate a list with one element i'll let you decide and then we're taking all the tests and putting them into a single list now since i have not defined all the tests i am not going to use this definition which contains all the tests but i'm just going to pick the number of tests that i have defined so we have defined here test 0 test 1 test 3 test 5 i'm just going to put in test 0 test 1 test 3 and test 5 and that's the full set of tests that we have you definitely need to fill out all the test cases and if you can think of some other cases that you should be testing then you should include those test cases here as well okay now to evaluate your function against all the test cases together you can use the evaluate test cases helper function from jovian so there are two functions evaluate test case works with a single test case and evaluate test cases works with a list of test cases so we have a list of test cases here i have four but you should have about eight at least and a few more if you have created them so we can import from juvenile python dsa evaluate test cases and then invoke evaluate test cases with the count rotations function still it we don't have any logic in the function so all the test cases should pass and the list of test cases we've created so you can see test cases zero fails one fails two fails three fails so out of the four test cases none of them have passed no problem we have completed step two which is to create some test cases and we'll know once we've defined a function whether the function definition is correct now the next step is to come up with a correct solution for the problem and state it in plain english and there's a hint here for you already coming up with the correct solution is quite easy and it's based on this simple insight if a list of sorted numbers is rotated k times so you keep rotating it step by step moving the last number to the first position then the smallest number in the list ends up at position key and you can verify this it's very simple to do this whenever you have a doubt just create a new cell by the way you can create a new cell by clicking on the left side of a cell and clicking insert cell below or if you're in a code cell just click here near the prompt and press the b character and that adds a new cell below so let's take the list 1 3 5 7 5 6 7 and let's rotate it k times let's try with k equals 2 so if you set k equal to 2 then you're going to take two of these numbers from the very end and move them to the beginning so that means zero comes at position six comes at position zero seven comes at position one and the starting element in the sorted list now comes at position two that's interesting let's move the third element as well okay so now we've moved three elements or rotated the list three times and the smallest element ends up at position three so it seems to hold true and you can verify this now with a larger list smaller list empty list and all the test cases that you have but if a list was sorted k times sorted list was rotated k times then the smallest number in the list ends up at position k counting from zero further it is the only number in the list which is smaller than the number before it and you can see this once again the smallest number is at position 3 and all of these numbers are higher than the numbers that come before them except the number 1 which is smaller than 7. so we simply need to check for each number in the list whether it is smaller than the number that comes before it if there is a number before it then our answer is simply which is the number of rotations is simply the position of this number right so if you can find the position of the number which is smaller than the number that comes before it the position of the number is also equal to the number of times the sorted list was rotated and if we cannot find such a number then the list wasn't rotated at all and that's it you can see here in this list now applying this logic 3 is the number the smallest number and not only that 3 is the only number which is lower than the number that precedes it the predecessor which is 29 and since 3 occurs at position 4 well actually three occurs at position three zero one two three the list was rotated exactly three times now we can use the linear search algorithm as a first attempt to solve this problem in the linear search simply involves working through this list walking through this list from the left to the right so now the task for use to describe the linear search solution in your own words and please write it in your own words but here's how i'm going to write it let's say create a variable position with values zero so this is the position for tracking this is for tracking the position then look at the number at the given position and not only look at it but compare the number at the let's say the current position to the number before it now if you're starting position with the value 0 maybe we may not there's no number before it so we may not be able to compare it with something we may even just start with the value one that's all right if the number is smaller than its predecessor then return position because position is the answer we found the number that is smaller than its predecessor there's only one such number otherwise increment position and repeat till we exhaust all the numbers okay simple now you can add more steps if your description of the algorithm requires more steps that's perfectly all right but at this point we have a very clear description of the solution now we're starting with the position 1 not 0 because we also want to track the previous position now we import jovian here and commit our project once again and keep saving your work after every step so that you can continue your work so now we're talking about implementing the solution and testing it so let's implement the solution we said that we want to start with position we want to start with position one and while when should the loop be terminated well while position is less than the length of nums i guess that's fair and then what is the success criteria so we have if position greater than zero and nums of position less than numbers of position minus one okay so that's the success criteria here now you can see that there's a condition if position greater than zero here so we don't really need to start position at zero we can start position at we don't really need to start position at one we can start position at zero as well and all that will happen is this condition will get skipped and position will get incremented and this is a good practice because whenever you iterate over a list you normally just want to start with 0 just to avoid any confusion later when you're reading the code that did you intend to write 0 here a 1 etc etc so just put in position equal 0 here and simply skip the check here or simply skip this comparison if position is not valid right so whenever you're accessing an element from inside a list or inside a dictionary you always want to make sure that that index or that key is valid okay here we are making sure that the key position minus 1 is valid by checking position greater than zero in any case we now have the logic and finally [Music] we are saying that if the number at position is less than the number that comes before it then we return that and that's just going to if it's not then it's going to increment the position and it's going to check again and again and again till we run out of numbers now if you've exa exhausted the entire list then it follows that there were no rotations or there were n rotations exactly in either case the number we returned should be zero okay so keep this in mind the sum you may have the doubt should you be returning -1 here or should you be returning 0 here well the question does specify clearly that you are given a sorted rotated list and you have to find the number of times it was rotated now obviously minus 1 rotations are not possible so minus 1 would not be a valid return value from your function and this is the reason we write test cases too now let's evaluate the test case so let's call evaluate test case for a single test case on count rotations linear and let's see what the test case is this is the test case here and this is the output we call evaluate test case with count rotations linear and test and that gives us a linear search result you can see here this was the number the list of numbers this was the expected output and this was the actual output so grade our function seems to have passed the test case now we can evaluate all the test cases by calling count rotations linear on all the test cases together and give that gives us a whole list of test results test case 0 and 1 and 2 and 3 all of them have passed now if you had put n minus 1 here you would see that one of the test cases would fail which is the case where the list wasn't rotated at all or was rotated n times so that should tell you that the answer here should be zero [Music] so that's our linear search algorithm and at this point you may face issues you may feel stuck you may not be able to figure out how to write the code and that's perfectly all right that's part of learning you may face errors you may face exceptions for instance if you did not have this check here position greater than zero or maybe you what you had here was some other condition like position less than equals position plus one and that's okay then you can go to the forum and post your issue so let's open up the forum here this is the forum discussion for assignment one and you can go into the original topic here which is a longer discussion so this is where everybody is posting small issues so you can see that there's about 321 messages that have been posted you can start looking through this forum you can start reading through some of the posts you can even search if you press ctrl f and you can even search for questions here now if you want to post your own question scroll down to the very end or you can just click this button here and click reply okay and mention your question here i have a [Music] an issue should i return -1 or 0 in the case the list has not been rotated okay maybe that's and if you want if you have code that's not working or there's an error you can also include a screenshot of your code or i'll show you another trick you can actually include let's say you commit your notebook so let me come up here i've committed my notebook and if you have a particular line of code that you want to share you can actually click copy cell link and paste it here so that will give a link to the entire cell and if somebody clicks on the link then they can view that specific cell of the notebook directly let's see you can see here that it brings us directly to this specific cell there's another option you can even click on embed cell okay for embed for secret notebooks we do not allow embedding but copying the cell link should work and then click reply and your question will be posted and somebody will reply to your question just come back to the forum in a few hours or maybe the next day and you should see an answer you will also receive an email so that's the discussion topic you can also go back to the topic here the category here and create a new question you can see that if you want to start your own thread if you think your question deserves a deeper discussion where multiple people can reply you can also create a new thread by clicking new topic okay so keep this in mind and do make use of the forum what we've seen is people who are active on the forum are at least four to five times more likely to complete the course and earn the certificate of accomplishment and continue working on these topics after the course as well okay so the next step is to analyze the algorithms complexity and the way to do this if you've seen lesson one is to simply count the number of iterations the number of executions of the while loop now if you have a list of numbers of size n then you can see here that this is the key loop here while position less than the num length of numbers so then there will be n loops or n iterations and then inside each each iteration we're performing certain comparisons and returning things so all of these are in effect constant time and based on this you can probably tell that the complexity of linear search is order of n so you can just put in a big o and in the big o notation this would be order n so that's the first part of the assignment linear search now the next step is to apply the right technique to overcome the inefficiency and that's where you can now you can now read through the rest of the assignment now the idea here is this binary search is the technique we'll apply and the key question we need to answer in binary search is given the middle element can you decide if it is the answer which means if it is let's say the smallest number in the list or whether the answer lies to the left or the right of it okay so given the middle given if the middle element is smaller than its predecessor then it is the answer we already know that because there's only one number in the list that is smaller than its predecessor so you can see here for example now if the middle element was 1 which it's not but suppose the middle element was 1 and you can see that 1 is smaller than 8 then we know that 1 is the answer so the position of the middle element is the answer however if it isn't then we need a way to determine whether the answer lies to the left of the middle element or to the right of it and consider these examples so here you can see that the middle element is 3 and the answer is the position 2 so in this case the answer or the smallest element lies to the left on the other hand in this case you can see the middle element is 4 and the smallest element minus 1 lies to the right of it so now you need to apply your mind and think of a check that will help you determine if the middle element given the middle element if the answer lies to the left or the right of it right and we're looking for the smallest element remember so the logic here if you just spend a couple of minutes you will come up with this quite easily if the middle element of the list is smaller than the last last element of the list okay or the last element of the range that we are currently looking at that means that all the numbers here are in increasing order so then the answer lies to the left of it on the other hand if the middle element of the list is larger than the last element of the range that means that because we know that the list is a rotated sorted list so that means that the numbers increase up to a point and then there's a decrease and then they continue increasing that's the only way in which the final element can be smaller so that means the answer lies to the right of it so that's the logic here for binary search and now what you have to do is describe the binary search solution in your own words so here once again you have these four five lines it's very important that you do this because if you cannot express it then coding it is also going to be difficult for you so always do this exercise of expressing the solution in your own words when you're practicing when you're solving a coding challenge or something even in an interview it's also very important because the first thing you need to do is to communicate to the interviewer your thought process and how you're thinking about the problem so the first thing you need to do is describe a simple solution in your in simple words and then they may or may not ask you to code that solution and then the next thing is to identify the complexity or identify the inefficiency then the next step for you is to describe the optimal solution or the binary search solution in your own words okay now if you don't describe the solution in your own words and you start writing the code they may not be able to follow your code so even if you've written mostly correct code maybe with one or two edge cases wrong they may still have a feeling that you don't know what you're writing but if you explain the solution clearly to them they will know that now you understand the solution and they will be able to follow the code as you write it and they will be able to pick up mistakes or errors and help you with the errors one secret is that interviews are always open to helping you unless you make them really confused so keep that in mind and describe the solution in your words once you do that you can commit now the next step is to implement the solution now the implement the binary search solution as described in the previous step let's run this again so you run count rotations define the function count rotations binary now you may want to review lesson one here on how to start it out you'll see that low starts out at zero and high starts out at length nums minus 1 and i will not solve the rest of this but there is a certain condition here between low and high so in binary search we are starting with the entire list as the range then we are looking at the mid number so we're getting the first the mid position and we look at the number at the mid position then we check if the middle position is the answer so if the middle person is the answer we return the middle position then we check if the answer lies in the left half so here's a condition where you decide if the answer lies in the left half and we once if the condition holds true all we do is we change the high so which we change the end point of the range to mid minus one and then we check if the answer lies in the right half in that case we change the starting point of the range to mid plus one and the while loop repeats okay so that's the general logic of binary search and one thing you have to keep in mind is if none of the elements satisfy the criteria that you have what is the answer and this is a very important condition this is where it is very easy to go wrong it is also called the edge case or the trivial case so you should handle and think about this carefully and then once you've done that you can evaluate the test case and you can a single test case you can evaluate multiple test cases now if your test cases are failing you may want to enable this print statement inside by uncommenting it but make sure to comment it out at the end once again and the print statement will help you see what the low high and mid points were now you may want to then take a pen and paper look at an example that is failing and see if the printed numbers match what you expect to see debugging your function is a very important skill so keep that in mind and use a debugging technique like this by adding print statements and working out the same problem side by side on paper to fix your issues otherwise you may feel lost if you're not able to look into the internal workings of the function next you have to analyze the algorithms complexity and identify inefficiencies this should be straightforward enough we've already looked at the complexity of binary search but all you need to do is make sure that what you're doing within the algorithm matches the analysis that we've done earlier so the problem size reduces by half each time and then we are doing constant work in each step before solving a problem of half the size so that should roughly give you an answer and keep committing your work now finally to make a submission you have two options now one option is to take this link so your notebook has been committed here and you can come to the assignment page let's open up the assignment page binary search practice come down here and paste this link here and click submit now once you click submit the assignment will be submitted and it will go into automated evaluation so in about a couple of minutes maybe up to an hour depending on the queue of submissions from different participants you will receive a grade over email let's just refresh the page and it seems like there was an issue here the issue was that count rotations binary was not defined so it's possible that this happened contradictions binary did not get defined because there are a bunch of question marks here so we may need to then fix the issue and then come back and make a submission once again okay so i have received failed grade i will go back and i will fix the issue and then come back okay now it's very important for you that's why to have good asset good set of test cases for you to test your function so that when you submit it or when you get an error you can maybe look at your functions performance on the test cases and fix anything that needs to be fixed and add new test cases if you need to now one other way you can submit is by simply running the code by joven dot submit assignment equals python dsa hyphen assignment 1 the code is mentioned here you can see here that the submission was made and you can verify your submission on this page ok so that's assignment1 so what should you do next review the lecture video if you need to and execute the jupyter notebook you may need to keep you may want to keep the jupyter notebook running side by side as you're working on the assignment then complete the assignment and even attempt the optional questions if you scroll down here on the assignment notebook you will find that there are some optional questions for you here's one bonus question use the generic binary search algorithm so inside the python dsa module in jovian there is a function called binary search you can use the generic binary search example then here's an optional bonus question two handle repeating numbers we did say that you can assume that there are no repeating numbers in the list but here's one list with repeating numbers can you modify your solution to handle the special case and then here's an optional bonus question three about searching in a rotated list so you're given a rotated list now instead of finding the number of times it was rotated you you're trying to find the position of a certain number for instance the position of six can you apply binary search and modify your previous solution slightly to search within the rotated list and find the position of a given number now here's a hint you can simply identify two sorted sub-arrays within the given array and perform a binary search on each sub-array using so to identify the two sorted sub-arrays you can use the count rotations binary function so that's one potential solution another way is to modify the counter rotations binary function to solve the problem directly so it's a very interesting problem to solve and if you found the assignment easy then you should definitely solve these bonus questions and if you can solve this question by yourself without taking additional help then you can solve pretty much any problem related to binary search that may be asked in an interview because most of the questions are some variations of something like this and this is pretty much the hardest problem you may get asked you can also test your solution by making a submission on lead code and this is only for the final optional question and there's a thread on the forum where you can discuss the bonus questions separately as well so do make use of the forum thread too here it is optional bonus questions discussion so that was assignment one of data structures and algorithms this is called binary search practice hello and welcome to data structures and algorithms in python this is an online certification course by jovian my name is akash and i am the ceo and co-founder of jovian you can earn a certificate of accomplishment for this course by completing four weekly assignments and doing a course project today we are on lesson two of six now if you open up pythondsa.com you will end up on this course website where you will be able to find all the information for the course you can view the previous lessons which is lesson one and uh you can also work on the previous assignment which is assignment one and you can also check out the course community forum where you can get help and have discussions so let's open up lesson two this is a lesson page here you will be able to see the video for this lesson you can watch live or you can watch a recording here and you can also see a version of this video lecture in hindi and in this lesson we'll explore the use cases of binary search trees and develop a step-by-step implementation from scratch solving many common interview questions along the way so here is the code that we are going to use in this lesson all the different notebooks containing the code are listed here and let's open up the first one so here you can see all the explanations and the code for this lesson this is binary search trees traversals and balancing in python and this is the second notebook in the course you can check out the first notebook in lesson one and if you're just joining us this is a beginner friendly course and you do not need a lot of background in programming with a little bit of understanding of python and a little bit of high school mathematics you should be able to follow along just fine if you do not know these then you can follow these tutorials to learn the prerequisites in just about an hour or two now the best way to learn the material that we are covering in this course is to actually run the code and experiment with it yourself so to run the code and you can see here if we scroll down you can see that there is some code here on this page as well now to run the code you have two options you can either run it using an online programming platform or you can run it on your computer locally so to run this code we will scroll up and click on the run button and then click run on binder and this is going to start executing the code that we were just looking at so once again you can go on the course page pythondsa.com open up lesson2 and you can watch the video there and on lesson 2 you can open up the link to the code where you can read the code and the explanations here and if you want to run the code just click the run button and that will execute the code for you so once you click the run button on binder you should be able to see an interface like this this is the jupyter notebook interface the same explanations that we were seeing on the lesson page you can see here the same explanations are now available here but the difference is you can now edit these explanations and you can go down and you can actually run some of the code in this tutorial you can see here that you have a run button and when you click the run button that is going to run the code in this particular cell and this is a jupyter notebook made up of cells now we'll do a couple of things here the first thing we'll do is we click on kernel and click on restart and clear output what this will do is this will clear all the outputs of the code cell so that we can execute them ourselves and then i'm just going to zoom in here and hide the interface so that we can look at the explanations and the code so finally we have some running code and in this notebook we will focus on solving this specific problem and this is a common question a question of this sort can be asked in interviews so this is an interview question but along the way we will also learn how to build binary trees and binary search trees and how to apply them to several other questions so here's the question as a senior back-end engineer at jovin you are tasked with developing a fast in-memory data structure to manage profile information which is username name and email for 100 million users it should allow the following operations to be performed efficiently you should be able to insert the profile information for a new user find the profile information for a user given their username and then update the profile information of a user once again given their username and list all the users of the platform sorted by username and you can assume here that usernames are unique so this is a very realistic problem that you might face if you're working at a company where you have a lot of users so let's see how we solve this problem now here's a systematic strategy that we'll apply for solving problems not just here but throughout this course this first step is state the problem clearly and in abstract terms and then identify the input and output formats then come up with some example inputs and outputs to test any future implementations and try to cover all the edge cases and then come up with a simple correct solution for the problem it doesn't have to be efficient it just has to be correct and stated in plain english and then implement the solution and test it using some example inputs fix bugs if you face any and finally analyze the algorithms complexity and identify inefficiencies if any now once you identify inefficiencies then we apply the right technique and that's where data structures and algorithms comes into picture so we apply the right technique to overcome the inefficiency and then we go back to step three so come up with a new correct solution which is also efficient state it in plain english implement it and then analyze the complexity now if you follow this process you should be able to solve any programming problem or interview question so step one we state the problem clearly and we identify the input and output formats now we can reduce the problem to a very simple single line statement we need to create a data structure which can efficiently store 100 million records and we should be able to perform insertion search update and list operations all of them as efficient as possible now the input the key input to our data structure the solution that we are building is going to be user profiles which contains username name and email of a user now before we come up with a solution we need a way to represent user profiles and a python class would be a great way to represent the information for a user so you may have heard of the term object-oriented programming and that is what we're going to look at now if you're not familiar with the class it's very simple a class is simply a blueprint for creating objects and what's an object well everything in python is an object whether you're looking at a number a dictionary a list anything and you can create your own custom objects with custom properties and custom methods by creating your own custom classes so here's the simplest possible class in python with nothing inside it we're creating a class user so this is how you declare a class and then when putting nothing inside it so whenever you put nothing inside a function or a class or anything you can put you need to put the pass statement because python cannot accept empty blocks of code so here we're creating a class which does not have anything inside it and we can create an object or it's often called instantiation which is take creating an instance of a class instantiate an object of the class by calling it like a function so we say user1 is user so this creates an object and the variable user1 points to that object now we can verify that the object is off the class user by simply printing it or by checking its type user1 and typeuser1 are both user now the object user one does not contain any useful information so let's add what's called a constructor method so constructor method is used to construct an object to store some attributes and properties so now we're defining the class user once again but inside it we're defining this function and you can see that this function is inside the class because there is some indentation here so we define this function underscore underscore init and it takes four arguments now the first argument is a special argument called self and we'll talk about this and then we have three arguments username name and email and inside in it what we're doing is we're setting self.username so we're setting a property on self to username we're setting a property on self to name and we're setting a property on self to email and finally we're printing user created so let's see let's create another user user2 and you can see that user 2 is also an object of the class user now here's what happening conceptually when we do this the first thing that happens is when you invoke this function when you invoke user as a function python first creates an empty object of the class user and then stores it in the variable user too and then python invokes the init function and to the init function it passes user to the object that was just created as self and then the other arguments that were passed while creating the object as the rest of the arguments so you can imagine that we are basically doing we are basically calling user dot underscore underscore init the function with user to an empty object and these arguments john johndoe and jondo.com and then inside the init function we simply set these properties on user two so now we get user two dot username is john user two dot name is john doe and user two dot email is johndoejohneddo.com so that's basically how classes work in python and that's why you always have this additional extra argument in all class methods which will refer to the object that finally gets created okay so once user two is created with the values john john doe and john doe dot com you can check that user two dot name is john doe and user two dot email is john doe dot com and user two dot username is john now you can also define some custom methods in inside a class so obviously we had the init method but here we are also defining another method called introduce yourself now introduce yourself takes again two arguments the first argument is self which will refer to the actual object that gets created later and then we have a guest name and we basically say hi guest name i am such and such contact me at such and such so these blanks are filled in using the guest name self.name and self.email okay so that's how you define a method in a class so here we have another user we're creating jane and jane doe at janetddo.com and you can see here that when we call introduce yourself with david so user3 which is jane becomes self and then david becomes guest name and that's why we get hi david i am jane doe contact me at geneto.com so that's a quick refresher on classes in python now there's a lot more to classes but the simplest thing you need to know is you how to define a class how to create a constructor which is underscore underscore init how to set some properties like we said the properties name email and username and finally how to define methods like we defined the method introduce yourself and that's all we will need today so we won't need much more than that and one final thing that we're doing with our class is we're defining two other special functions underscore underscore repr wrapper and underscore underscore str so now these two functions these two functions are used to create a string representation of the object and you can see here once we create an object user4 now and if we try to print user four you can see that 4 is now printed like this so user 3 was not printed i mean user 3 was printed just as a user but with user 4 we have all this information printed here as well so now here's an exercise for you which also brings us to the first quiz of the day now we are going to do three quizzes in this video and you can answer these quizzes on linkedin so go to our linkedin profile if you see the posts you will see a new post here which will give you a question and the question is what is the purpose of defining the functions str and wrapper within a class and how are these two functions different now leave a comment with your answer and we will pick the right answer one right answer and one lucky winner will get a swag back from us so that was the input we have we now have a way to represent users by creating classes and then the output that we want the final output that we want to create for our problem is a data structure so a data structure is once again something that we can define using a class so we can define we can expect our final output to be a class called user database which has four methods insert find update and list all and insert takes a user and inserts it into the database find takes a username and returns the user update takes a user and updates the data for that user and finally list all returns a list of the users so this is what the class will look like and we have not implemented it yet but we now have an interface so now the next step is to come up with some example inputs and outputs so let's create some sample user profiles that we can use to test our functions once we implement them so we're going to create these seven user profiles you can see that we're creating these seven user profiles with a username name and an email and storing them in these variables using the user class that we have just defined earlier and we're also going to store the list of users in this variable called users and as you can see we can access different fields within a user profile using the dot notation so you can check biraj dot username is bharaj and biraj dot email is biraj example dot com and biraj dot name is bhiraj now you can also view a string representation of the user as we have seen so if we print the user you can see some information about the user and here is the full list of users that we have created so it's always a good idea to set up some input data set up some test inputs that you can use to test with your implementation later on and since we haven't implemented our data structure yet it's not possible to list any sample outputs but you can try to come up with some different scenarios to test any future implementations so let's let's list some scenarios for testing the methods of our user database class so the methods are insert find update and list all and for inserting you may want to test that you're inserting a user into an empty database of users so that's what's called an edge case and then the general case is to insert a user into the database assuming that the user already does not exist then another edge case is trying to insert a user with a username that already exists right so these are all the different ways in which you can use the insert function and there can be some more so here's an exercise for you try coming up with all the different scenarios in which you would like to test the different functions insert find update and list so that completes step two now we have some sample inputs and then we have some scenarios in which you we are going to finally test our function so the next step is to come up with a simple correct solution and then state it in plain english now here's a simple and easy solution to the problem we simply store the user objects in a list sorted by usernames that's simple enough and suppose we do that so inside our data structure we have a list which simply contains a bunch of user objects then the various functions can be implemented like this so you have the insert function the insert function simply requires looping through the list and then adding the new user at a position that keeps the list sorted so for instance if you have the users akash and siddhant already and then you're inserting the user biraj then you can tell that bharat should go between akash and hemanth in alphabetical order so that's how you insert a new user and maintain the sorted property of the list then to find a user we simply loop through the list and then find the user object with the username matching the query so that's you if you're looking for he-man for instance you start from the beginning you go through akash biraj and finally hit him and then you can retrieve the user object associated with hemanth and then you have update now updating is very simple as well it's similar to find so you find the user object matching the query and then update the details of that user object and then finally because our internal representation is already a list of user objects sorted by usernames so we can simply return that list when we want to list the users so that's our plain english description and it's always a good idea to describe your solution in plain english so that you can clarify any doubts you have and even during interviews it's a good idea to have a conversation with the interviewer before you actually implement the solution and now one fact that we can use is that usernames which are strings can be compared using the less than greater than or equal to operators so we can compare strings just like numbers in python so that will make it easy for us to implement these functions and that brings us to the implementation and the code for implementing these is also fairly straightforward so now we have the user database class we are actually implementing this class and here you see that we have a constructor and the constructor does not take any additional arguments apart from self and all we do is inside self we set a property.users and that property.users is set to an empty list then we come to insertion so now assume that we already have some users in our user database so we start out with a pointer set to zero and we go through all the valid positions in the users list so which is from 0 to n minus 1 if there are n users and then we find the first username greater than the new user's username so for instance if you're inserting a month then you go through akash and bhiraj and then finally you realize that the next value is probably siddhant so you want to insert before siddhant right so you want the first username that's greater than the new user's username and you check this property and as soon as you find the next that the next user is greater than the user that needs to be inserted we break out and then we insert that user at that position okay so this is the insertion you can it's a just four or five lines of code so you can work through this code try to read the score line by line and see how it works now similarly you have the find function the update function and the list function they're all pretty straightforward there's really not much here so this is an exercise for you because this is also the brute force of the simple implementation so this is an exercise for you to go through each of these functions and try it out and use the interactive nature of jupiter to experiment and add print statements inside each of the functions if you need inside each of the loops if you need more visibility into what's happening okay but what we will do is we will try and test this implementation out and the first thing we do is instantiating a new database of users using the user database class so here we say user database and that gives us a database of users and now let's insert some entries into this database so we can now insert for instance we can insert the value hemanth akash and siddhant so here we have inserted three values into the database and now we can retrieve the data for a given user given their username using the find method so now we say database dot find siddhant that returns a user and we can check the value of user and you can see that now we have retrieved the data for siddhant which is username siddhant name siddhant sinha and email siddhant example.com now let's try changing the information for a user so to change the information we can call database dot update and then simply pass in a new user object so let's say we want to change the information from sadhan cena to siddhant u so this is how we do it we call database dot update and now if you find the information once again now if you can't find call database dot find once again we get back a user object and this time with the updated information so we have created the database we have inserted some values into it and then we have retrieved values out of it and we've also updated them and finally we can retrieve a list of the users in alphabetical order so now if we list it out you can see here that we have the username akash we have the username himant and we have siddhant these are the three values that were inserted and they are all in alphabetical order of username now if we insert a new user let's say let we insert barrage we can make sure that bhiraj is inserted into the right position okay so that's how we use the data structure that we just created and you can use the empty cells here to try out the various scenarios when you run the notebook so just to recap we created a simple class inside which we are storing a list of users in sorted order of usernames and then insertion is pretty easy we simply loop through find the right position and insert any new values finding values is very easy as well we simply loop through and keep comparing and updating values is simply a matter of finding them and then updating that specific value and listing is simple because we can simply return the internal list representation that we're already storing in the sorted order of usernames so that's the simplest solution or one of the simplest solutions there can be even simpler solutions maybe so the next step now is to analyze the algorithms complexity and identify any inefficiencies so typically in an interview setting you may not want to implement the simplest solution so you you can actually skip step four you know when you've described what the simplest solution is in english in plain english which was step three you can directly jump to analyzing its complexity and then move on to optimization and implementing the optimized version but when you're practicing or when you're learning it's always a good idea to implement even the brute force solutions so let's analyze the complexity the operations insert find update involve iterating over a list of users and in the worst case they may take up to n iterations to return a result where n is the total number of users now the list all function is slightly different because it simply returns an existing list so the list all function does not take linear time it takes constant time now based on this information it's very easy to check to guess the time complexities of the various operations insert find and update have a order n first case time complexity which means they can take up to n iterations however the list function has an order one complexity which means irrespective of how many users you have in your database it returns the list in the same amount of time now if you want to display the list or if you want to iterate over the list that may take you additional effort but getting the list itself is a constant time operation so that was the time complexity and an exercise for you is to verify that the space complexity of each operation is order one and if you're wondering what we mean by complexity then you can go back and watch lesson one where we talk about analysis of all algorithms complexities and the big o notation what we're calling order of n the big o notation all of these explained in a lot more detail so you can go back to lesson one and check it out now we've created a simple solution and our first question might be to wonder if this is good enough and to get a sense of how long each function might take if there are a hundred million number users on the platform let's create a while loop let's create a for loop and let's run it for let's see how many this is one two three four five six seven eight so let's run it for 10 million 100 million numbers so here we are creating a range of 100 million numbers and we're running a for loop which iterates over the entire range and we're simply performing a simple operation which we're not really using we just we're just multiplying the number by itself to simulate what might happen if we have a database of 100 million users and we're trying to access find a user now what is the worst case scenario here let's run this and you can already see that it is taking a while for 100 million users the loop takes about 10 seconds to complete here it took about 9.45 and a 10 second delay for fetching user profiles will definitely lead to a sub optimal user experience and that may cause users to stop using the platform altogether now imagine you came to joven.ai and it took 10 or 15 seconds to load your profile and then maybe even longer to load the other information and display it you would not be happy with the experience and then a 10 second processing time for each user for each request each profile request will also significantly limit the number of users that can access the platform at a time because if you're running the backend server on one computer which has eight cores then each core will be busy for 10 seconds each time a user tries to access the platform so you can only serve about 8 users in 10 seconds time now that's pretty bad that could significantly limit the number of users you will have a significant outage if a lot of users come to the platform or on the other hand you may have to increase the cloud infrastructure add more servers add bigger hardware more cores more ram and that could increase the cloud infrastructure cost for your company by millions of dollars so as a senior back-end engineer you must come up with a more efficient data structure and this is why choosing the right data structure for the requirements at hand is a very important skill now we can clearly see that using a sorted list of users may not be the best data structure to organize the profile information so let's see what better we can do here and before we do that let's save our work so remember that this notebook we were running it on an online platform called binder and binder can shut down at any moment because it is a free service so what you want to do is run pip install jovian and then import the jovian library and you can then run jovian.commit now when you run jovian.commit what this does is this captures a snapshot of your jupyter notebook whether you're running it on binder or you're running it on your own local computer and it saves the snapshot of this jupyter notebook on your jovian profile so here you can see now on my jovian profile i have this notebook and i can go back on my profile and view the other notebooks that i've created in the past so your jovian profile becomes a collection of all the jupyter notebooks that you're working on so always just it takes just a couple of lines import joven and run joven.com so always run your window commit inside your notebooks and if you want to resume any work that you were doing then all you need to do is click on the run button and then click run on binder once again and then you can start executing the code within the jupyter notebook once again right so remember that binder is a free service so it will shut down if after about 10 minutes of inactivity which is if your computer goes to sleep or you change your tab so keep running juvenile.com it from time to time so now we have a simple implementation and we have analyzed it and determined that it is not efficient it is inefficient so now we need to apply the right technique to overcome the inefficiency and we can limit the number of iterations required for common operations like find insert and update by ditching the linear structure that we had earlier and organizing our data in a more tree-like structure so this is a structure that we use for our data and we will call this a binary tree now this is called a tree because it vaguely resembles an inverted tree trunk with branches so you can think of this as the root so this has the root and then you can see each of these are like branches and then there are nodes where branches then split into multiple branches so these are called nodes and finally at the end you will have individual nodes which do not have any more branches and those are called leaves so these are some terms that are used the tree represents the entire structure the top node is called the root and each each element in the tree is called a node the top node is called a root and then the bottom most nodes which do not have any sub trees or what are called children which do not have any children are called leaves right so the root node has two children and then each node there with can have zero one or two children so it's not necessary to have exactly two children but up to two children is what determines a binary tree so that's a binary tree but the binary tree that we need will have some additional properties which was what will make it efficient for our purposes so you can see one thing you can observe here is that the root node seems also seems to be the central value if you sort the keys in increasing order so what you will notice is on the left we have keys which have which lie before jades and on the right we have keys which lie after jades so that's one thing and that is actually the second property listed here that the left subtree of any node consists only of nodes which have keys that are lexicographically smaller than the nodes key right so the key for this node is barrage and that is lexicographically smaller than jades and similarly hemant and akash are all smaller than jadish and then this property holds at every node so at every node if you check sunaks you can see that siddhant is less than sunaks and vishal which comes to the right is a more than sunaks and then so not siddhanta vishal all three are greater than jadeesh right so when a binary tree satisfies this property it is called a binary search tree so that's what we're looking at here this is a binary search tree so that's the first property but we need the second property is that our nodes will have both keys and values now sometimes you can create binary node binary trees with just keys each node will have a single number or string inside it and you can call it the key or value or element or whatever you wish but what we want is we want the keys to be user names so that we can compare the keys easily but along with each node we also want to associate a value which is the actual user object so if you're looking for hemath let's say we started the root node we see that jadeish is the root node and since it is a binary search tree we know that he meant lies to the left then we reach biraj we know that himans will lie to the right of biraj so we go right we reach him and then we access the values stored at hemanth which is the user details for hemath right so we need both keys and values in a binary tree and this is what is called a tree map or a map in many languages and then finally this tree that we will create this data structure that we will create it will be balanced so here what we're looking at is each node has two children left and right but it is also possible to have an unbalanced tree where you only have one child on each on on maybe one of the sides so we will require it to be balanced which means that it does not skew too heavily in one direction and we'll talk about what balancing means and we'll talk about how to check if a tree is balanced and how to keep it rebalanced okay so we'll go over all of these things step by step but these are some of the properties that we want our final data structure to have okay so one important property of a tree of a binary tree is the height of a tree in fact if you start counting you can say this is level zero where you have one node and this is level two or this is level one where you have two nodes the left and right the left and right child of the root node and then this is level three level two where you have four nodes the left and right child of the first node on level one and the left and right child of the second node on level one right so you can see that the number of nodes in each level in a balanced binary tree is double of the number of nodes of the previous level so if you have a tree of height k or which which means a tree which has exactly k levels then here's the list of the number of nodes at each level now level zero will have one node the root node level one will have two nodes its children level two will have four nodes their children so that's four nodes is two times two or two to the power two level three will have eight nodes two nodes for each of these four nodes so that's two to the power three and similarly if you keep going down level k minus 1 the final level will have 2 to the power of k minus 1 nodes so that if the total number of nodes in the tree is n then it follows that n is 1 plus 2 plus 2 square plus 2 cubed plus so on plus 2 to k minus 1 okay so what we're trying to determine here is what is the relationship between the height of the tree and the total number of nodes in the tree and this is the relationship and we can simplify it a bit if we add 1 to each side you can see here that this side we get n plus 1 and this side we get 1 plus 1 which gets simplified as 2 or 2 to the power 1 and then we can add 2 to the power 1 by 2 to the power 1 and that gets simplified as 2 to the power of 2 then we can add 2 to the power of 2 and 2 to the power of 2 and that gets simplified to 2 to the power of 3 and we can keep performing this reduction we can keep adding these together till we finally end with 2 to the power k minus 1 plus 2 to the power k minus 1 which is simply 2 to the power of k so what that gives us is that k the height of the tree is log of n plus 1 which is approximately or in almost in every case less than log n plus 1 so that's a a bit of an approximation are doing here but it is uh the height of the tree is less than log n plus one so to store n records we require a balanced binary search tree of height no larger than log n plus one now this is a very useful property in combination with the fact that nodes are arranged in a way that it makes it easy to find a specific key simply by following a path down from the root the binary search tree property and we'll see soon by the end of this lesson that the insert find and update operations in a balanced binary search tree have complexity order of login so in our original implementation a brute force implementation they had order n and this time we've reduced the complexity to order login and that is far better and we will see how that happens okay so that's a quick introduction to binary search trees we have we've had enough theory now let's get into some implementation but before that we have the second question now binary trees are very commonly used as data structures for a variety of different in a variety of different languages for instance java c plus plus python java and c plus plus have this concept of a map which is represented using a binary tree and it is also used in file system so binary trees are also used in file systems to store indexes of files so when you browse your file system or when you search for a specific file it is a binary tree that is used to look up the file and find the location of the file now that's where that brings us to our second question of today now you can you can find the second question on our linkedin profile so once again go to linkedin.com slash school slash jovian ai and you will find the second question here the second question is which tree based data structure is used to store the index in the windows file system and who invented this data structure so like this question follow us and comment with your answer and you can stand the chance to win a swag pack right so we repeat the question which tree based data structure is used to store the index in the windows file system also known as ntfs and who invented this data structure okay so let's get to the implementation of binary trees and here's a very common interview question that you might get implement a binary tree using python and then show its usage with some examples so what we'll do as we implement binary trees and binary search trees is to also cover many common interview questions in fact we'll cover exactly 15 so that's a quite a few the first one is to implement a binary tree and to begin we'll create a very simple binary tree so we will not have any of the special properties like key value pairs and binary search tree and balancing rather and we'll also use key numbers as keys within our nodes because are simpler to work with so here is an example binary tree so we have a root node and then we have a left child in right side and here's a simple class representing this representing a single node within the tree so we're calling this class tree node and it has a constructor function it simply takes a key and it sets self dot key to key it also has a couple of other properties self.left and self.write which are initially set to none so each node when it's created exists independently of other nodes and now let's create nodes representing each of these nodes so we have node 0 we're calling it we're calling tree node with the value three then we have node one and node two so there you go now we've created the nodes and we can verify that it is of the type tree node you can see here and if we check the key of node zero you can see that it has a value three and we can now connect the nodes by setting the dot left and dot right properties of the root node so if you go to node zero and set dot left to node one so now we've connected node zero to node one and similarly if we set node zero dot right to node two now we've connected node zero and node two and that's it we're done so now we have three nodes and then we've connected each of those nodes and we may also just want to track which is the root node so we can create a new variable called tree and simply point it to node zero so tree points to the root node of the tree and then the root node is connected to its children and the children will be connected to their children and so on so you can check here that if we check tree dot key we get three and if we check tree dot left dot key so three is the root node it has a value three tree dot left is this node so it should have the value four and tree dot right dot key should have the value of five okay so pretty straightforward and that's pretty much the answer to the question implement a binary tree in python now going forward we will use the term tree to refer the node root node to refer to the root node and the term node can be used to refer to any node in a tree not necessarily just the root okay so here's an exercise for you try to create this binary tree so now you have a root node here and then you have a left child and right child and then this left child has another left child but does not have a right child similarly here you have another right child and then it has a left child which does not have a left child but has a right child okay so there's a slightly more complicated tree structure and try to use these cells these empty cells that are given here to replicate this tree structure and then try to view the different levels of that tree manually okay now please do that because that's a great exercise in understanding how the structure works and how to connect the nodes but it's a bit inconvenient to create a tree by manually connecting all the nodes in fact here you may have to make a total of one two three four five six seven eight nine connections right so what we can do is we can write a helper function which can convert a tuple and the tuple will have this kind of a structure so a tuple is simply is kind of like a list except that it is represented with these round brackets of parenthesis so a tuple will have this kind of structure it will have three elements and then the middle element will represent the value or the key within the root node the first element will itself also be either a tuple if the left child is is an entire subtree or if it is a single number then it will be just a number and then the right element will represent the right subtree okay so here's an example here is one tree tuple now if you see this tree tuple it has three elements this is the first element this is the second element and then this is the third element so this first element two represents the root node and then this so the second element 2 represents the root node this first element or element at position 0 represents this subtree so you can see here that in this sub tree if you look at just that subtree of that tree 3 is the root in that subtree and then one is the left child and there is no right child so that's what this represent and then for this subtree where 5 is the root node and then you have 2 other subtrees that's represented here so 5 is the root node and then you have a subtree here and a subtree here so this is a very easy way it is a convenient way for us to represent a binary tree and what we can do is we can define a function parse tuple and this parse tuple function can take a tuple like this and then convert it into a tree like structure of linked nodes using the tree node data struct using the tree node class that we have defined above so we call the parse tuple function with some data for instance this tuple and the pass tuple first checks if data is of the type tuple and it has a length three if these two things hold true then first we create a node we create a node with data one so in this case we create a node with two as the key and then we set the left and the right subtrees of the node and then we're doing something very interesting here we are calling the pass tuple function once again so we call parse tuple this time so this is called recursion when a function calls itself inside it that's called recursion so we call parse tuple with the first element which itself is a tuple right so once again that it calls another invocation to pass tuple and for a moment let's assume that that returns the proper subtree the proper node so we set that node which which got created to node.left and similarly we create the right subtree using these values and then we set that node to node.right okay now you might wonder in the function we're calling itself so when will this stop can't it go on forever and that's where you have to track the actual function calls so when we call parse tuple with the entire tuple first it calls past tuple with this and when you call past tuple with this uh then you can see that three is used to create a node and then pass tuple is called with one so when past tuple is called with one this condition no longer holds true and we also check the next condition which is if the data if one is none when one is not none so this condition does not hold true so we fall into the else condition and we simply create a node right so we just create a node and this time we are not calling parse tuple once again right so this is called a terminating condition of the recursive function and similarly once we get back the result from one then we call patch triple with the value none once again this condition is not entered and this condition matches so we set node equal to none and then we return the node okay so when we reach either a leaf node which is either a single number or we reach the value none that is when we stop invoking the function recursively and then the function returns and that's how the entire tree gets con gets converted so this is a very powerful idea in programming the idea of recursion the idea of functions calling themselves and it can seem unintuitive and confusing at first so one thing you can do is you can add a print statement here inside this function to see how it works to see how the different calls are going so when you call past tuple with the entire tuple what are the internal calls that are made and and study how the result comes out maybe try it on pen and paper but it's a very important technique for you to learn you will be asked or you will find applications of recursions in many places throughout your programming or data science career so do learn it so let's now call parse tuple with this tuple as an input and let's see okay so that returned a tree and then that tree is of the type tree node that's great and now let's examine the tree to verify that it was constructed as expected so now we check tree two dot key so tree two dot key should be pointing to the root node which has the key two and then let's check the level one so that was level zero let's check level one so let's check three two dot left dot key and tree two dot right dot key you can see here we get the values three and five let's check the next level on this level we have three two dot left dot left and then we have three two dot left dot right but there's no value there so we can't really check for a key here then we have treated right dot left and tree two dot right dot right so you can see that tree two dot left dot left dot key is one but tree two dot left dot right is none because there is no child here no right child then we have left dot key and right dot key and that gives you three and a seven and similarly you can now check level four level three as well so here are all the levels of the tree so it looks like the tree was constructed properly and you can see the power of recursion at play here that the recursive function can now construct trees of any levels now you can create tuples within tuples within tuples and as long as they have the right structure as long as you have this three element structure where the left element represents a left subtree the right element represents a light right subtree in the middle element represents the current node you can construct a tree of any size so now here's an exercise for you we've defined a function to convert a tuple into a tree define a function now to convert a tree back to a tuple so if you have a binary tree convert return a tuple representing the same tree for instance for the tree created above tree 2 calling tree to tuple should return this original tuple which is used to create the tree and here's a hint on how to do this use recursion so do fill this out and see if you can figure out how to do this so now we have defined a class for a binary tree and we also have a way for creating a binary tree from a tuple so now let's create another helper function to display all the keys of the tree in a tree like structure for easier visualization so here we'll just use we'll call this function display keys and we'll not get into the code for this because it's once again it's a pretty straightforward but there are a few conditions we need to handle but here's what it will give us when we call display keys on a tree so then we'll then we'll get this kind of a representation of a tree and you can see that this is not exactly the same representation as this you will have to take this representation and then mentally rotate it by 90 degrees in the clockwise direction to get a representation like this but you can see roughly that the root node is 2 and then it has a left child 3 and it has a right child 5. then 3 again it has a left child 1 and there is no right child now 5 has a left child 3 and 3 has no left child and 3 has the right child 4 and so on so the exact same structure has been replicated here for us to view visually now this is a very useful thing we're spending all this time here or talking about how to create trees and how to visualize trees because the easier you make it for yourself to create trees the more likely you are to test the easier it is for you to test different scenarios out so always spend a little bit of time coming up with good string representations for any data structure you create something that helps you visualize them and an easy way to create these data structures okay so now we have a way to visualize the tree as well that's great now here's an exercise for you try to create some more trees and visualize them using display keys and you can use this tool excalidraw.com and that's where how that's how these diagrams were created as a digital whiteboard so you can create some trees you can create trees like this and then try to come up with tuples for those trees try to create those trees using the parse tuple function and finally try to display them okay so experiment with it and see explore what are all the different tree structures that you can create now the next one of the frequently asked questions in interviews is to traverse a binary tree binary tree traversals are very common so you may face one of these three questions write a function to perform the in order traversal of a binary tree or write a function to perform the pre-order traversal of a binary tree or write a function to perform the post-order traversal of a binary tree what do you mean by a traversal a traversal refers to the process of visiting each node of a tree exactly once now what do you mean by visiting by visiting it could mean any operation but generally it refers to either printing the key or the value at the node or adding the nodes key to a list and then there are three ways to traverse a binary tree and return a list of visited keys so the first one is called in-order traversal and the in-order traversal now traversal is defined recursively because binary trees have this recursive structure so you will see that almost all the functions that we write will have some sort of a recursive structure so in order traversal involves first traversing the left subtree recursively in order then traversing the current node and then traversing the right subtree recursively in order so what does that mean well we start out with this tree and we we're traversing it in doing an inorder traversal so we try we look at the root node and then we realize that there it has a left child so it has a left sub tree so we do not visit it yet which means we do not print it or we do not add it to our list yet rather we follow the we follow the path on the left side and then we come across three and then we realize that okay three also has a left child so we don't visit it yet so then we go down to one we go down to one and now it does not have a left child or a right child so we can visit one then we go to three and now we so we've visited the left subtree of three so now we can visit three and then the next step is to visit the right subtree of 3 but of course 3 does not have a right child so there is no right subtree to visit so we can move back up to 2. so now we have visited the left subtree of 2 so now we can visit 2 so we we print 1 three two and now once we've visited two we can now visit the right subtree of two so to visit the right subtree we go to five once again we realize that five has a left subtree so we go to three now three doesn't have a left subtree so we can visit three then we visit four then now since we visited the left sub tree of five we can now visit five and similarly we then visit six seven and eight okay so that's the in order traversal of the tree and then there is another traversal called pre-order traversal which is slightly different where you traverse the current node first so here we start out at 2 and we say that okay we're going to visit 2 first so we visit 2 or print it or add it to a list then we traverse the left subtree and then we traverse the right subtree so we go we visit three and one and then we come to the right side we visit five and three so you can compare these two diagrams and see how in order and pre-order traversal are different now these are very important for you for you to understand because they are great examples of dif different functions which have very similar implementations but there are just one or two things you will need to change uh and these are recursive as well so do understand the subtle difference between them and second they are very commonly asked in interviews you will most likely face some coding assignment or an interview where you will be asked to perform a traversal of a binary tree and then finally there's another order called another traversal called the post order traversal and i'll let you guess how it works you can also look it up and here's an implementation of in-order traversal now it may seem a little complicated but it's actually pretty straightforward so let's look at it here what we do is given a node we first traverse the nodes left subtree then we create so that should return a list a list of all the keys and then we create a list with just the nodes key so we get the list of keys from the left subtree in with the inorder traversal then we get add to it the current nodes key and then we call traverse in order with the right subtree and that recursively keeps adding these keys each one and the end condition so the terminating condition for the recursion is when we hit none so when we hit a node which does not exist so that means we come there from a parent which does not have a left or right child then we return the empty array okay so let's try it out with this tree so this is the tree we have and we just saw its traversal now if it travels the tree in order we get the values one three two three four five six seven eight and we can verify here we have one three two three four five six seven eight so that was the in-order traversal of a tree now the exercise for you is to print the pre-order and post order traversal of the binary tree and you can test your implementations by making submissions to these problems on leadcode.com okay so that was our discussion about traversals another thing that you may get asked commonly is writing functions to calculate the height or the depth of a binary tree and then writing a function to count the number of nodes in a binary tree once again these can be expressed recursively as well now the height of a tree given a node is simply 1 plus maximum of the height of the right subtree of the left subtree the height of retreat is defined it is defined as the longest path from a root node to a leaf so you can see that the longest path from root node of the to the leaf is of length four so two five three and four and the way to do get the longest length of the longest path is by checking the max of the left height right height and then adding one to it and of course the terminating condition here also is if you hit a node that does not exist you return zero so that's how you get the height of a tree and you can check that the height of a three is four then here's another function to count the number of nodes in a tree once again really simple all you do is this time instead of checking the maximum we simply get the size of the left subtree get the size of the right subtree add them and add one to it so here you can see that there are nine elements in the street three six and nine so we get tree size of tree as nine now here are a few more questions relating to the path lengths in a binary tree so you can just check there's a concept of maximum depth and minimum depth and then there's also the concept of a diameter so you can try out both of these now as a final step what we can do is we can compile all the functions we've written all the methods as methods within the tree node class itself and this technique is called encapsulation where we are encapsulating the data as well as the functionality related with the data of the data structure within the same class and this is real as really good programming practice so as you write more code try to think about how you can create these classes with not just the information inside them but also with the relevant methods inside them okay so we have now added the methods height size traverse and order display keys to tuple and we've also added these methods str and wrapper and remember quiz 1 or you can go on linkedin and post an answer to what these functions do and finally parse tuple as well so all of these functions are now added within the class and you can try it out here so for instance here we have a tree tuple and we can call tree node dot pass tuple to convert this tree tuple into a tree so you can see that now we are also representing the binary tree itself using this tuple like representation but we can also display it in this hierarchical structure using display keys then we can check the height using tree dot height we can check the size using tree dot size and we can traverse the tree in order using traverse in order and we can convert the tree to a tuple using 3.2 tuple so do create some more trees and try out the operations that we've just defined and try or you can also try adding more operations to the tree node class and before continuing we can just save our work so i'm just going to import jovane and run juventus commit so that concludes our discussion on binary trees next let's talk about binary search trees now a binary search tree or a bst is a binary tree that satisfies these two conditions the left subtree of any node should only contain nodes with keys less than the current nodes key and then the right subtree of any node should only contain nodes with keys greater than the current nodes key and we can see that this is let's just copy this over so we can see that this node this tree here is actually a binary search tree and you can verify that these two properties hold for each of these nodes and it should follow from these two conditions that every soft tree of a binary search tree must also be a binary search tree so i can let you verify that that if you pick up any subtree inside so you pick up any node and you see the tree under that node you will see that it is a binary search tree so here are some questions that are often asked relating to binary trees and binary search trees and we've lumped them together because we'll answer them with a single function so here's a function that you might be expected to write so write a function to check if a binary tree is a binary search tree which means ensure that these two conditions hold and second write a function to find the maximum key in a binary tree so this could be a generic question finding the maximum key and here's another question that you might face write a function to find the minimum key in a binary tree so what we will do is we'll answer we'll answer all of these questions together with a single function called is bst so isbst takes a node and then is bst returns three things so if you look at the return value it returns whether the node and the tree under that node is a bst so here so we this is going to be the value determining it's going to be either true or false telling us whether the tree under that node with that node as root is that a bst it also returns the minimum key from that entire tree and it also returns the maximum key from that entire tree now why are these two useful we'll see in just a moment so the way we calculate is bst node is by actually looking at the left subtree and the right subtree recursively so we call is bst on the left subtree of the node and we call isbst on the right subtree of the node so we get back three values which is is the left subtree a binary search tree is the right subtree binary search tree is the minimum key in the left subtree the minimum key in the right subtree and then the maximum key in the left subtree and the maximum key in the right subtree so now what we can do is we can say is is bst node so is the entire tree of binary search tree well if the left sub tree is a binary search tree and the right subtree is a binary search tree and then we verify these two properties which is the maximum key in the left subtree is either none which means that there is no left subtree or the current nodes key is greater than the maximum key and the minimum key in the right subtree the smallest key in the right subtree is either none which means that there is no right subtree or the minimum key in the right subtree is greater than the current nodes key so that this was condition one and condition two and that tells us whether this entire tree is now a binary search tree and then finally we can also calculate the minimum key and maximum key simply by computing the minimum of the left minimum node.key and right minimum and the maximum can be calculated by checking the maximum of the left maximum node dot key and right maximum okay so what we return from the sbst function is whether the node and the tree represented rooted at that node is a binary search tree and then the minimum and maximum key out of it so if we look at this tree right here let's verify whether this is a bst and well before we check we can probably tell that it's not because you can see that 3 appears as a left sub child of 2 but 3 the key is greater than 2 and that's a problem so this is a violation of the property elsewhere this property is satisfied you can check any other node here and you will find that the left subtree is always smaller than the right than the node and the right subtree is larger than the node so let's check is bst tree one it's not so it's false now on the other hand this tree is a bst this is the tree that we've been looking at all this while so once again we can create this using tree node.parse tuple and note that the keys can the way we've implemented tree node keys can not only be numbers but they can also be strings so we don't need to change anything here and that creates tree two and we can even display tree two so if we do tree two dot display keys you can see that it has this structure where jadeesh is at the center and then on the left you have biraj on the right you have sunaks biraj and sanaksh then you have akash siddhant and vishal and this is a bst so you get back true here and the smallest value here is akash and the highest value is vishal as you can verify in alphabetical order so that's pretty handy now we have a way to check if a binary tree is a binary search tree and this is again a very common interview question that you might face next remember that we need to store not just keys but also user objects within each key with each key within our bst so what we do is we will define a new class called bst node to represent the nodes of our binary search tree and bst node will not only have the key but in the constructor it can also accept a value and this is optional so we will set the key and we set the value we will also set the left and right apart from this we also set another property called parent and the parent will point to the parent node so for instance if this node is a left sub tree of this route then the parent of barrage will point to jadeish and this will be useful for upward traversal now if you're given a pointer to a node and you have to go back and find the root of the tree the parent will be helpful there so this is our bst node and let's try to recreate this bst right here with usernames as keys and user objects as values so first we create level 0. so level 0 we create bst node now the key is jaadesh or username which will be just the string jadeesh and then the value will be the jadeish user object so we've created that and we can check its key and value you can see that jadish is the key and then the user object is the value let's create let's create level one now level one is we set tree dot left to bst node barrage dot username and biraj now one other thing that we should do here is once we set it we should set tree dot left dot parent to tree and similarly we said tree dot write it's not tree dot write is sunak so we said bst node with sonar username as the key and sonax is the value and then we can set tree.write.parent as tree and now can you can view these values so now you can see that we have inserted barrage and the username barrage we've inserted sonaax and the user sonar shear as keys and values respectively now the exercise for you two is then try to add the next level of keys and values and then verify that they were inserted properly but you can see now that we now have a way to represent the data the both both the usernames and the user objects in a binary search tree so we're getting pretty close to the data structure that we want to create once again we can display the keys of the tree by calling the display keys function now this is also rather nice is a good thing about python that because python functions are dynamic because you do not need to specify the types of the objects while defining the function the same display keys function can be used both with tree node and bst node classes so all it requires is that the object of your class should have a property dot key for it to be able to display the keys in this visual setting and the same is true with most of the other functions that we've defined in fact any function we've defined for tree node will also work for bst node okay so moving right along now we have a way to construct a bst but it it's a bit inconvenient to insert values manually because what we're doing so far is we are manually checking whether we should insert a value in the left or the right rather there should be a way to do it automatically we should be able to call a function insert and here's this is a common question as well write a function to insert a new node into a binary search stream so we'll use the bst property to perform insertion efficiently once again let's grab a copy of this tree here so that we can think about it easily okay so now we have this tree and let's say we want to insert a new user with the username tanya into this tree so first we start at the root and then we compare the key to be inserted with the current node's t key so the current node is the root so we compare tanya with jades and we see that tanya is greater than jades because t comes after j so obviously tanya should not be inserted into the left sub tree rather than should be inserted into the right subtree so if the key is smaller we recursively insert it into the right left subtree and if the key is larger we recursively insert it into the right subtree so then we encounter sunak tanya is also greater than sonax t is greater than s t comes after s so once again we call recursively called insert on this subtree that subtree rooted at vishal this time we notice that tanya is smaller than vishal so t is less than v so then we need to recursively insert in the left subtree but there is no left subtree here and this is the point at which we can create a new node and attach it as the left child of vishal so you can see that the node tanya will get added here at this position in the tree so here is a recursive implementation of insert exactly what we just discussed first we check if the key is less than the current nodes key and if that is the case then we insert it into the left subtree then we check if the key is greater than the current nodes key and if that is the case we insert it into the right subtree and the ending condition is that if the node is none which means if we've hit a position where we do not have a left subtree and we need to go left or we do not have a right subtrain we need to go right then we create a new node so we create new node node equal to bst node and then we return the node so we return the node and this is an interesting thing that we're doing here we're returning the root node back from insert so when we called insert with node.left we get back the pointer to the left subtree so we can set it back to node.left and we can also set the parent of the left subtree to node okay so this is just updating the parent so just study this function carefully see how it works it does exactly what we just talked about and it finally returns a pointer to the to the tree once again so let's use this to recreate the tree that we had here now to create the first node we can call the insert function with none so initially we don't have a tree to begin with so we just called insert with none and remember that insert after performing an insertion returns the pointer to the tree so we call insert with none and we want to insert the value jaadesh.username and we want to insert the we want to insert the key short username with the value jadesh so that gives us a tree and now the tree has one element you can see tree dot key and tree dot value and now the remaining nodes can just be inserted into tree so now we call insert with tree and call it with barrage dot user name and biraj then we call it with sonar username and sunaks akashad username and akash and this way so we are adding barrage when we are adding sonax then we are adding akash siddhant vishal and see that we are not specifying exactly where these nodes need to be inserted but you can see that once these nodes are inserted then they are inserted in the right places so jadeesh you can see that the binary search tree property is preserved here and also we've exactly replicated the tree structure that we had here so the left subchild of jadeesh is biraj and the right child is sonak for biraj the left child is akash and the right child is hemanth and so on now note however that the order of insertion of nodes can change the structure of the resulting tree so for instance if we insert all the nodes in the increasing order of username so if your for example here we are inserting akash biraj siddhants vishal so this is the lexicographic increasing order and we try to display that tree this is what we end up with so we end up with an unbalanced or a very skewed tree and you can see why it was created as a skewed or unbalanced tree well let's look at it so we started with akash so we have a single node and then when we try to insert bharaj we realized that we need to go right so we insert barrage here then we try to insert hamad then we realize that we need to go right from akash and right from biraj and go to himant and then we keep going this way so how you set up the root node and how you set up each subtree and the order in which you insert the nodes is very important and that can create a huge skew within the tree now skewed or unbalanced trees are problematic because the height of sub such trees is no longer logarithmic compared to the number of nodes in the tree right so earlier we had deduced that in a balanced tree if containing n nodes the height is log n or log n plus one and that makes the operations like insert update and find very efficient but here where you have a very skewed tree the height can actually match the number of nodes for instance this tree has seven nodes and it has a height seven and in these q trees once again you may get back the fact that insertion finding and update can be order n because you may have to traverse the entire height of the tree which is equal to the number of nodes of the tree and that may once again defeat the purpose of using a binary search tree in the first place so maintaining the balance of a binary search tree is very important and we'll see how to do that so we've seen how to insert a node now the next thing is to find the value associated with a given key in a binary search tree so once again we can follow a recursive strategy here similar to insertion so we check we start from the top let's say we want to find the key heyman we start from the top and we compare it with the root node now here if it matches the root node we can simply return this node if it does not then we check whether we need to go left or right since hemanth comes before jadeish we need to go left then we encounter bharaj and here we realize that we need to go right and finally we encounter haiman and we return another option is that we have a value let's say tanya which does not exist here so if we try to search that we may go in this kind of a direction and we end up at an empty place so in that case we simply return none so you either find a node and return it or you return none so you can see here that if we called find tree with hemanth we get back the details for human and very interestingly because it's a balanced tree you we only had to take two steps and not go through the entire tree and in the worst case you can check that any path from the root to any leaf in a balanced tree will only be two steps long and that's what makes it so convenient now on the other hand if we try to find the key tanya you can see that it's not formed try creating larger bsts and try finding some more nodes it's important to experiment with these operations once they're defined because now it's simply a matter of calling the function we've written the code for it so experiment with it try creating larger trees with multiple levels and dozens or maybe hundreds of nodes try generating some fake data putting it into the trees and see how trees build up and that will give you a feel for how binary search trees work next let's talk about updating a value in a bst now updating a value is fairly simple we already have a way of finding a node so if you want to update a node let's say we want to update the node hema the key heimat and here we want to update it we want to update it to this value which is the new value of the user heman and we're changing the name and we're changing the email here so we first find the node and if the node is not none then we simply change the value at that node it's as simple as that and we what we're also seeing is we're reusing the find function here and this is a good practice to always incorporate into your programs into your functions whenever you find yourself copy pasting some code and maybe changing one or two things here and there think about whether you can extract that piece of code into a function and then reuse that function so always try to make your code more and more generic the less code you write the less there are the chances for errors the easier it is to understand and the smaller your functions become so write small reusable generic functions whenever you can and this is a principle called the dry principle or the dry principle which stands for don't repeat yourself whenever you're writing programs so in update we are not repeating ourselves by using the find function to find the right node and simply updating it by setting its value so let's update hemanth here to the new value and you can see that now we have the updated data here so we have payment j and human j at example.com now the value of the node was successfully updated and you can in it and you can easily check that the time complexity of update is same as that of find now finally we have the last operation that was required and this was to write a function to retrieve all the key value pairs stored in a binary search tree in the sorted order of keys is a question that you might face uh once again and this is simply the in order traversal it's a different way of stating the inorder traversal now what you will have to figure out or a reason about is why the inorder traversal of a binary search tree produces a sorted array of our sorted list of keys think about it so here's the list all function all we do here is we call list all on node.left and then we call listall on node.write and in between them then these give us two arrays so we assume that listall.node.left gives us the list of key value pairs from the left subtree in sorted order similarly here we get the list of key value pairs from the right subtree in sorted order and between them we simply insert this key value pair from the current node and recursively it automatically fills out the entire array and this is the end condition where we encounter an empty node we simply return the empty array you can see now when we pass in this tree we get back the list of users key value pairs arranged by the sorted order of keys now here's an exercise for you determine the time complexity and state compliance space complexity of the list all function now you can do this for a balanced tree or an unbalanced tree and here's a hint it will not make a difference but think about it so once again let's save our work and now we've talked about binary trees and operations on binary trees now the next thing is to look at balanced binary trees and this is once again a very common question that gets asked write a function to determine if a binary tree is balanced and here's a recursive strategy to do this in fact this is really the definition of balanced binary trees the left sub tree should be balanced the right subtree should be balanced and the difference between the heights of the left and right subtree should not be more than one okay so this is an important thing now when we're looking for balance we're not always looking for perfect balance because it may not always be possible to create a tree with perfect balance because to have a perfectly balanced tree where for every node the left subtree and the right sort you have the exact same height you will have to fill out all the nodes at all the levels and that can only have that can only happen for certain numbers for example you can have one node which satisfied this property or you can have a tree with three nodes which satisfy this property or you can have a tree with seven nodes which satisfies this property but you may not be able to get a tree with six nodes to satisfy that property for instance if you remove vishal here you will see that the left subtree and right sub tree of this node sonar will not be of equal height that's why for balancing we relax the criteria slightly we simply need to ensure that the difference between the heights of the left and the right subtrees is not more than one so here's the code for is balanced once again pretty straightforward but we will return two things here we will this is balance will not only return whether the tree node is balanced it will also return the height of the tree which is rooted at that node so the way we implement it is first calling is balanced on node.left and then calling is balanced on node.right and by the way this is exactly how we implement recursive functions as well sometimes we write the recursive functions signature then we immediately write the return value and then we assume that a recursive call is going to return these values so a recursive call is balanced node.left is going to return whether the left subtree was balanced and the height of the left sub tree and then we assume that is balanced for node.write is going to recall is going to return whether the right subtree is balanced and the height of the right subtree because that's what we return here then the entire tree is balanced if the left sub tree is balanced and the right subtree is balanced and the absolute value of the differences in their height is less than one which means the height l minus height r is either minus one zero or one and finally we calculate the height of the tree itself which is simply one plus the maximum of the height of the left subtree and the right subtree and we return it so that's how you implement a recursive function or think recursively and there's one last thing which is the end condition and the end condition although it's often the last thing you think about it's the first thing that you have to put in the end condition is to check whether a node is none because as we call node.left you may not have a left subtree so you may call is balanced with none and if the node is none we simply return true because an empty tree is balanced by default because there's no imbalance there and its height is zero so that's our is balanced function it's just four or five lines of code but if you are not able to reason about recursion easily you may get stuck with this and you may spend an entire 45 minutes trying to write this function and debug it so always try to think in recursive terms and that's why always it always helps to write down what you want to do in plain english so that you can determine what should be the inputs and outputs to your function maybe also have some test cases ready and then start implementing your function and it becomes really easy so this tree for instance is balanced here you can check is balanced you get back true but this tree here you're looking at this is not balanced so this was tree two and if you check is balanced here you get back false so here you also get the height of the tree which is three and here you get the height of the tree which is seven now here's another tree is this tree is this tree shown here balanced why or why not now create this tree and check if it's balanced using the is balanced function so there's another concept called complete binary trees which is slightly similar to balanced binary trees but it's a slightly stricter criteria so you can check out this problem here and you simply need to modify the is balance the code for is balanced slightly to get the code for complete binary so do check out this problem on leadcode.com all right so we've looked at binary search trees and we've looked at balanced binary trees now let's bring them both together into balanced binary search trees and here's one question that you will face at some point write a function to create a balanced binary search tree from a sorted list of key value pairs so you have a sorted list of key value pairs so the keys for example could be usernames the values could be the user objects and they are sorted by key and you have a list and you have to create a balanced binary search tree from it and here's the basic logic which is somewhat similar to binary search which is something that we've covered in lesson one do check it out what we can do is we look at the middle element for instance if you have a list of 15 elements then the element at position 7 counting from 0 the element at position 7 is the middle element now we can take the middle element and then create a new binary search tree with the middle element as the root node okay so you take you make the middle element the root node and then you take the left half of the list and use that to create a balanced bst and make it the left child of the middle element the root node and then you take the right half which both of the house will have seven elements each so if you take the right half and you create a balanced bst out of it and then make it the right child of the middle element so that's the idea here and how do you make a balance bst for the left or right child recursion right so once again here's a recursive solution make balance bst takes data which is a list of key value pairs it takes a low and high and it also takes a parent and we look at those now low is set to zero by default and high by default is set to the last index in the data so we use that to get the middle index so for instance if low is 0 and high is 14 the middle index is 7 then we get back the key and the value from the middle index so we calculate we find data made and that gives us the key and the value for exa since the username and the user object then we create the root node so we create the root node using bst node and then we call make balanced bst on data but this time from low to mid minus one so from the indices zero to six and make that the left child of the road and we called make balance bst on the right node so on the right half so from mid plus 1 so which is index 8 to 14 and we make this the right subtree and then we return the root and that's it that's pretty much it the only thing that we might need here is the terminating condition when low becomes less than high which means that we have no more elements to create trees out of we simply return none so the left or right subtree for those for the parents of those nodes get set to none so that's your makes balance bst function we also have this other thing called parent going around and this i will let you figure out what the parent does here but this is the basic idea so here is a list of key value pairs you have a key value pair sorted in increasing in the increasing or lexicographic order of keys and we're calling make balance bst with data and that gives us a tree and let's view the tree here so there you go now we have created the tree perfectly as we wanted it jadeish is at the center and we have viraj sunak on each side and then the appropriate nodes on each side as the children of those nodes now recall that the same list of users when inserted one by one resulted in a skewed tree here we are getting the list of users username and user from data and inserting them and you can see calling display keys on tree three returns a skewed tree okay so whenever you have a sorted array and you want to create a balanced bst the way to do it is to start from the middle out now finally one other question you may be asked is to balance an unbalanced binary search tree and this is pretty simple at this point and this is kind of a trick question because if you were given this question directly you may not be able to think about what to do how do you balance an unbalanced binary search tree but now that we have we have a way to create a balanced binary search tree from a sorted array of key value pairs and we have a way to get a sorted array of key value pairs so now it simply becomes calling the sorted array so calling list all on the node which is also the in-order traversal so doing an in-order traversal of the binary search tree which gives us a sorted area of key value pairs and then passing that into the make balanced bst function okay so that's the trick here it's a two part question and once again we see the benefit of reusing our functions here now we this now balancing and unbalanced bst now becomes a single line of code that's very nice so we create a tree here with the value none and now we insert into it the values one by one and you can see that that creates a skewed tree because we are inserting the values in increasing order electrical graphic order so we keep adding right children and we never add a right left child but then we call the balance bst function which internally takes this gets in order traversal so the inorder traversal lists all the keys and key value pairs in sorted order and then we call the make balance bst function which starts from the middle and then creates a bandwidth binary search tree out of it so there you see this is how you balance a binary search tree and what we can do now to maintain the balance as we grow our data structure is a simple thing that we can do is to ins to of insert to balance the tree after every insertion and that brings us to the complexities of the various operations in a balanced bst so if we are doing an ins if you're doing an insertion that takes order login because now if a tree is balanced its height is order login so for insertion you may have to traverse a path from the root down to a leaf and that path can be of length at maximum equal to the height which is order login but if we are also doing a balancing with every insertion then we also have an order n term added here and order n plus order log n because log n becomes much smaller than n as n grows so order n plus order login is the same as order n so that makes insertion order n finding a node becomes order login updating a node becomes order login and you can verify that listing getting a list of all the nodes is order n so what's the real improvement between order n and order login so let's think about it if you're looking at a 100 million records then log to the base 2 of 100 million is about 26 or 27 so it only takes 26 operations to find or update a node within a balanced bst as opposed to 100 million operations so you can see here a 26 or a loop of size of length 26 and we're doing some operation inside it only takes about 19.1 microseconds that is one microsecond is 10 to the power minus 6 seconds on the other hand order n involves looping through the entire list so looking through 100 million numbers rather than 26 and that obviously takes far far longer and we saw that it took about 10 seconds right about 9.98 seconds so to find and update finding and updating a node in a balanced binary search tree is 300 000 times faster than our original solution and all we have changed here is the data structure and that's the importance of data structures because now each user will be able to view their profile in just 19.1 microseconds at least that part of the request will take only this long so the user experience will be better and your cpu will be busy for a shorter time so you will be able to serve not eight but hundreds of thousands of users every second and finally your hardware cost will also be far lower because now your cpu is busy for a lesser time so you do not need to use a very large machine or you do not need to use too many machines to support hundreds of millions of users and that is the benefit of choosing the right data structure now there's one tip here how do you speed up insertions so what we may do is we may choose to perform the balancing periodically instead of at every insertion for example we can balance for every 100th insertion or every thousandth insertion or every 100 000 insertion whatever you know that and that's where we have to balance how often do we need to insert things versus how often do we need to restore the balance another idea is to do the balancing maybe periodically at the end of every hour so for a second or two there may be a slight dip in the performance because you may be performing the balancing but even that there's a way to do it so you can take a copy of the tree and then balance it and then simply replace the pointer to the original tree so there are many other tricks that you can apply and in fact there's also an algorithmic trick which brings insertion and balancing together into an order login operation which we look at right at the very end so stay till the end but before we do that let's come back and answer our original problem statement so remember now as a senior backend engineer you are tasked with developing a fast in-memory data structure to manage profile information username name and email for 100 million users and it should allow insertion find update and listing the users by username all as efficiently as possible and to answer this question instead of creating a user database class we we can create a generic class called tree map because we have been making things more and more generic as we have gone along so let's define a function called tree map which internally stores a binary search tree a balanced binary search tree inside it so when we initialize the tree map we set self dot root to none which means we have not created a tree so far and then instead of defining functions insert update and delete we are going to use some special functions in python classes so we are going to use the function set item we're going to use the function set item here and set item is just like insert except it is a combination of both insert and update so to set item we will pass a key and a value and of course self will refer to the tree map object itself so the first thing we do is we get the root which which is basically the binary search tree that we are storing internally here so we get the binary search tree and then we find we look for the key inside the binary search tree so if the key is found so if we find the node in our tree then we come into this else position and then we simply update its value and if we do not find the node so which is what happens initially because initially our self.root is none so when you call find with none and pass a key you get back none so then we first set self.root by inserting the key into the tree okay so if a key exists within our binary search tree then we update it and if the key does not exist within a binary search tree then we insert it into our binary search stream okay so we've combined insert and update into the single operation called set item and similarly we define another operation called get item this is the find operation all we do here is we find the node inside self.root using the find function we had defined earlier and if the node is present if it is found then we return the value of the node otherwise we return none so given a key we retrieve the value and then we have we defined one last function called iter and this is the replacement for our list all function so what we do is we simply say we call listall on self.root so that gives us a list of key value pairs and then we have the special syntax we say x for x in this list and we put these round brackets around it so what this round brackets around it does is that this creates a generator out of it so now this is no longer list but this is a generator and a generator is something that you can use within a for loop so the iter function will allow our class to be used directly within a for loop and we'll see the example in just a second and finally we have another function called underscore underscore len so remember there are double underscores here so there's double underscore set item double underscore double underscore get item double underscore similarly double underscore uh len double underscore here we simply return the size of the self.root so here we simply return the size of the binary tree and then we have this function called display this is going to simply display the keys okay so now we've defined the stream app structure and it has all of these funny looking methods like we know in it but what about all of these but we'll see what these do in just a moment and we know what the what the functionality is but you may be wondering why we've defined them like this so the reason is these are special methods that are treated especially in python so here's how you can use them let's first get a list of users that we'll later insert into a tree let's get a tree map so we instantiate the tree map function the tree mac class and that gives us a new tree map inside it there is no binary tree you can check if you check tree map dot root you will see that it is none there is no value here and if we try to display it you can see that this tree map is empty then to insert instead of calling treemap.insert or instead of calling premap dot underscore underscore set item we can use this indexing notation so we open these square brackets and we put in the key that we want to insert so if we want to against the key ins against the key akash which is the string if you want to insert the value akash then we simply say tree of akash is akash and similarly tree of a certain key with the indexing notation set to this right so this is going to first look for the key as we have defined in set item if it finds the key then it is going to update the value for the key if it does not find the key then it is going to insert that key value pair as a new node into our tree so let's check it out now and let's see here if we now check tree map dot root you will see that now it is a bst node and if you try to display it you can see that now it has a structure jadeish and akash also note that this is a balanced tree now if you go back here to set item you will notice that whenever we insert right after this we also balance the tree and now you can change the logic here so that we do the balancing not after every insertion but maybe after every 100 insertions so you may need to track somewhere what is the current number of what is the current insertion counter and when it gets 200 only then do the balancing and then set the counter back to zero so that's an exercise for you perform the insertion or perform the balancing it only at certain intervals and here's a way to retrieve an element so the retrieving element is also now really simple you just call tree map with jadesh as the index and that gives you the value if it is found and if it is not found it simply returns none now because we have defined the function underscore underscore len underscore underscore so you can see here that has the value 3 because that now we can use it with the len function which is used for lists and dictionaries and let's add a few more things and let's set the values and let's see here so you can see all this works exactly as expected now we are able to set values we are able to update values we are able to display the tree it is remaining balanced and remember i mentioned that you can use this in a for loop so you can now put the tree map directly into a for loop and what this will do is because we have defined the underscore underscore iter function and the iter function returns a generator so now you can use this in a for loop and you get back the key value pairs from the list all function that was used inside editor you can print the keys in the values and in fact if you want to convert it to a list all you need to do is pass it into the list and once again because this is a generator because this is an iterable this is now an iterable class and you have defined that the way to iterate over this class is to get elements out of the key value pair list so when you call list you get back this list of key value pairs okay so now we've made it a very python friendly class you know instantiating it is very easy we simply create a new tree map adding values is very easy we simply use the indexing notation removing elements is very easy well not removing a finding elements is very easy we simply use the indexing notation updating elements is the same as inserting we can also check the size of the tree quite easily using the len function and then we can also use iterate over the keys iterate over all the users in a for loop quite easily and we can also update values as you see here values have been updated now the the purpose of doing this is to make it easier for other people to use this data structure now as a senior backend engineer you may have designed this data structure and you may have implemented binary search trees inside it but it's not important for other people on the team or other people using your data structure to know what the internal implementation is what's important for them is to be able to use it easily so that's why always think carefully about the interface or the api of your functions or of your modules or of your classes try to make them as python friendly as possible this was something that will be appreciate that will be appreciated in interviews and by co-workers so make them python friendly so that when people want to use something you have created it is extremely intuitive you know and they do not need to really understand the underlying details for instance i could be using this class and i could have no idea that it is a binary search tree all i know is how to insert and how to get a value out of it and i know that it is super efficient because you have designed it and i don't don't need to worry about the internal details so encapsulation and good apis are very important skill to have to cultivate so do that as you work on programming problems now once again let's save our work before committing now i did tell you that there is a way to create self balancing binary trees and a self-balancing mandatory remains balanced after every insertion or deletion and in fact several decades of research has gone into creating self-balancing binary trees and not just binary trees but other trees as well which are not binary in nature and many approaches have been devised for instance red black trees avl trees and b trees so here's an example this is an avl tree so here whenever a node goes out of balance we rotate the tree and you can see visually what we're doing here now whenever you see that there is an imbalance in the tree we rotate it and how do you do this we do this by tracking the balance factor which is the difference between the height of the left subtree and the right subtree for each node and then rotating unbalanced subtrees along the path of insertion or deletion to balance them so you can see the balance factor is zero right now the balance factor becomes one and the balance factor becomes two then we rotate it to set the balance factor back and then the balance factor here becomes -2 so here we do a right rotation here the balance factor becomes minus 2 here and minus 1 here so here we do two rotations so there are four cases in total there's the left right case the right left case the left left case in the right right case all four cases were demonstrated here as well and then you may need to do this rotation not just once but you may need to do this multiple times along the path of insertion so when you insert a node and that node creates an imbalance then you need to work backwards so you need to keep going from parent to parent and keep rotating nodes whenever you need to rebalance them based on the updated balance factor of each node so it seems a little complicated but it's actually not it's just that there are multiple cases to handle so you will need to write a couple of helper functions you'll need to write a function left rotate which rotates node left while still preserving the binary search tree property you will need to write a function right rotate which rotates the function which rotates it to the right while preserving the bst property and then in the insertion you will also need to perform the rotation at the right places and you will need to track the balance factor inside each node so there are few things to work out here and don't worry you will normally not be asked to implement an avl tree within within an interview or within a coding assessment so you do not really need to learn the implementation but it's nevertheless a very interesting data structure to study and here are a couple of resources you can check out so you can check out this youtube video which explains it very wonderfully and you can check out this implementation on geeksforgeeks.org and the important thing for us to take away here and which is something that you may be asked if not the implementation but just the complexity the important thing is that each rotation takes constant time and at most log n rotations may be required because if you are starting with a balanced tree and you are inserting a new node then you may traverse a path of height at most of length at most log n so you may need to perform at most login rotations maybe twice of that so what that means is in order login time you will be able to insert and maintain the balanced property of a binary tree right so you do not need to recreate the entire tree again and that makes your tree very efficient because now when you're working with 100 million records inserting will also take 20 steps and finding will also take 20 steps and updating would also take 20 steps and all of these will work in microseconds so that makes your data structure very efficient and with that we conclude our discussion of binary search trees so here's a quick summary we we looked at this problem of creating a data structure which allows efficient storage retrieval and updation also efficient iteration in a sorted order we first started out with a list of sorted list of values sorted by the keys and we realized that that was probably not the right idea because we are working with really large number of records then we created this binary tree structure so we looked at binary trees we looked at how to create them we looked at easy ways to visualize them easy ways to create them from tuples we looked at how to calculate their heights their sizes how to traverse them in in order pre-order post order we then looked at binary search trees which have this property that the left subtree has keys that are smaller than the root nodes keys and the rights of trees keys are larger than the root nodes keys and that property holds at every sub tree and that makes it really easy to find to locate a specific element or find the position to insert an element so we created binary search trees we created the operations insert update find and list all in a binary search tree we also determined ways to check if a binary tree is a binary search tree or not then we talked about balancing and we saw how to create balanced binary search trees and binary search trees form the basis of many modern programming languages language features for instance maps in c plus plus and java are binary search trees and data storage systems like file system indexes or relational databases also use something called b trees which is an extension of binary search trees so it's very important to know about binary searches even if you may not ever need to implement them you may be asked about them and in many cases you may need to pick a binary search tree as a data structure for a problem like we did in this case now you may wonder if dictionaries in python are also binary search trees well they're not dictionaries and python are not binary search trees so we'll soon release an assignment that you can find on the lesson page and you will work on hash tables in the assignment and here are some more problems that you can try out so you can try to implement rotations and self-balancing insertion you can try to implement the deletion of a node in a binary search tree that's slightly more complicated because what you do if you have to delete a node that has both left and right subtree you can try deletion with balancing if you really are up for a challenge here a couple more finding the lowest common ancestor of two nodes in a tree so the common node which is a common parent of both nodes here you can use the parent property finding the next node in lexicographic order so given a node how do you find the next node what's its complexity or given a number k how do you find the kth node in a binary search tree so to do this you will have to employ some clever tricks and then there are a couple more resources here you can open up these and find more questions the important thing to take away is that almost all of these will involve some form of recursion so you will either work with the left sub tree or the right sub tree or both and some of them may also require you to store additional information within the node for instance for for this one the given the number k find the kth node this may require you to store the size of each balanced binary search tree in each node so what to do next you should review the lecture video and execute the jupyter notebook experiment with the code yourself then complete the assignment hopefully the next lesson is called divide and conquer and sorting algorithms this is data structures and algorithms in python and i will see you next time thank you and goodbye let's look at assignment 2 of data structures and algorithms in python the topic of the assignment is hash tables and python dictionaries let's get started the first thing we'll do is go to the course website pythondsa.com and on the course website you can find all the lessons and previous assignments we are looking at assignment two so you may want to open that up and assignment two is based on or inspired from some of the topics discussed in assign lesson two so you may also want to watch lesson two and complete the notebook before you work on assignment two let's open it up now in this assignment you will apply some of the concepts learned in the first two lessons to implement a hash table from scratch in python that's very interesting you will and hash tables are very important data structure they're present in pretty much every programming language and are a common topic discussed and asked in coding interviews so we'll see how to implement them from scratch and one of the central problems in hash tables is called collisions so we'll see how to handle hashing collisions using linear probing and we will also replicate the functionality of python dictionary so python dictionaries are actually implemented using hash tables so we see how to replicate the way python dictionaries are created and used and modified and the way we access keys and iterate over keys and set values and change values and so on so we'll pretty much re-implement the python dictionary now we have an assignment starter notebook here so we can click on view notebook to open up the notebook once again this is a jupyter notebook and as you work through the notebook you will find question marks in certain places to complete the assignment you have to replace all the question marks with appropriate values expressions or statements to ensure that your notebook runs properly end to end okay so make sure that you run all the code cells do not change any variable names and in some cases you may need to add code cells or new statements and since you'll be using a temporary online service for code execution keep saving your work by running jovian.commit at regular intervals there are some optional questions they are not considered for evaluation but they are for your learning okay so let's run the code now the recommended way to run the code is using free online resources binder specifically but you can also run it on your computer locally so we're going to click run and click run on binder once again this may take a few minutes sometimes depending on the current traffic on the platform there we have it now we have the jupyter notebook running the first thing i like to do is click kernel and restart and clear output so that we can execute all the code cells and see their outputs from scratch i'm also going to hide the header and the toolbar and zoom in here a little bit so we can see things a little better the first thing we will do is set a project name import the jovian library and run jovian.commit this will allow you to save a snapshot of your work to your trovian profile so now you have a copy of the assignment starter notebook any modifications that you make every time you run joven.com it will get saved to your personal copy and it is this personal copy that you will submit at the very end so let's talk about the problem statement in this assignment you will recreate python dictionaries from scratch using a data structure called hash tables and dictionaries in python are used to store key value pairs so keys are store used to store and retain values and here's an example here's a dictionary for storing and retrieving phone numbers using people's names so we have a dictionary called phone numbers and the way you create a dictionary is using this special character the brace or the curly bracket as it's called and then in a dictionary you have these key value pairs so this is one key value pair where you have a key the key in this case is a string akash and here you have a colon and then here you have a value the value in this case is a phone number so that's how you create a key value pair and comma separated key value pairs is what you need to create a dictionary you can see once the dictionary is created it is displayed in the exact same way and then you can access a person's phone number using their name so if we have the variable phone numbers and we use the indexing notation so this is the square bracket and we pass in a key here we get back their name and you may wonder what happens if the key is not present the great thing about jupiter is you can insert a new cell like you can just click insert cell below or you can use the keyboard shortcut b as i just did and check maybe let's check the key vishal okay and you get back a key error and you may also wonder what happens if is it case sensitive does that matter you can check it very easily so a lot of the questions that you might get a lot of the questions that you may want to even ask on the forum or look up online can be resolved simply by creating a new cell and typing out some code right what happens if questions can all be answered by writing some code so now let's add some new phone numbers so this is how you create an initial set of phone numbers this is how you access a phone number and this is how you add new values so adding new values is like accessing them but instead of accessing it you put an equal to and then you actually set the value here so we can add a new value here the phone number for vishal and we can also update an existing value in a dictionary simply by accessing that value and putting an equal to and putting in new value there you can see now that the dictionary is updated to contain the new phone number 7878 and not the original phone number 948948 you can also view all the names and phone numbers stored in the phone number dictionary using a loop so you can say for for name and phone numbers so when you put a dictionary into a for loop you get back a key within each loop you can see here that the name and the phone number here is displayed for you using the print statement so those are some things that you can do within a dictionary and dictionaries in python are implemented using a data structure called a hash table and a hash table uses a list or an array to store key value pairs and uses a hashing function to determine the index for storing or retrieving the data associated with a given key so here's what it looks like here you have the key john smith and you have a function called hash and the function hash takes any key and it returns an index within the list so why do we use a hashing function well one approach as we've discussed in lesson two is we can store our key value pairs in a list and we can simply search through the list each time we want to look up the value for a key but that is inefficient because that requires looking through potentially all the keys before we get to the key that we want or maybe half of the keys so that makes it an order n operation if n is the size of the list that's pretty inefficient we want something faster and a hash function actually operates in constant time so hash simply takes the key and it converts the key into a number so in that sense it gives you the index of the specific key value pair in constant time rather than ordering and that is what makes hash table so efficient right so hash function does not require looping through the list it simply takes a key gives you the index and you can simply then get the key value pair or the value from that index now your objective in this assignment is to implement a hash table class which supports these operations an insert operation a way to insert a new key value pair a find operation to find the value associated with a given key an update operation to update the value associated with the given key and then list operation to list all the keys stored in the hash table and here's where we are going to use python classes and there's a brief introduction to python classes in lesson two of this course so do check out lesson2 if you want a refreshing on python classes you have the class hash table and inside the class hash table you have a bunch of methods now the insert method apart from taking the self argument and remember that the self argument is refers to the object of the class that will be created so this is equivalent to this variable in java or c plus but these are the actual arguments of the method the actual arguments are key and value so the insert function or the insert method will take key and value then the find method will take a key the update method will take a key and value once again so the find method takes a key and your job is to return the value the insert method takes a key and value and you insert the key value pair into the hash table then you have the list all method which is used to list all the keys from the table so before we begin our implementation let's just save and commit our work so we're running joven.comit here let's just run that once again there we go the notebook has now been committed so what you can do is you can come back to this particular page and you can find this from your profile and then you can click run to continue your work based on the modifications that you've already made okay so we build a hash table class step by step and the first step is to create a python list which will hold all the key value pairs now remember that a hash table internally uses a list to store the key value pairs and we will create a list of a fixed size so we'll set this variable max hash table size of size 4096 initially and we're going to create a python list of this size how do you create a python list of the size and we want all the values to be set to none so this is the way to do it you can of course you can start typing none and that would take a long time or you can use a simple technique just put in none times 4096 and that's one of the great things about python it is such an expressive language that creating a list of 4000 elements simply requires this single expression here you can check that here you can even check the length of the data list now if the list was created successfully here are some test cases here is one check that the length of the list is 4096 here's another check and we're simply picking a random value from the list 99 and just checking if that is equal to none but if you really want to have a sure shot test here what you should be doing is you should be checking for item in data list item equals none okay and here's a trick you can do you can write a word called assert and what assert does is if this comparison is true then it does nothing it lets your code proceed as usual but if at any point this comparison becomes false then it throws an error so let's see here you can see here there was no error so that means it worked fine but if this comparison was wrong so let's say if item if we had here we wanted the items to be equal to seven if you put it and and the telus does not contain the item seven at a certain position then you will get an assertion error here okay this is how you can create your own test cases by putting in assert but the idea here is that whatever you try to do make sure that you're adding some more test cases and not just depending on the test cases that are given here these are simply to guide you in the right direction okay so next up we have a list now we need a way to store or insert key value pairs into a list that's where the hashing function comes into picture the hashing function is used to convert strings and other non-numeric data types into numbers which can then be used as list indices for example if a hashing function converts the string akash into the number 4 then the key value pair akash and the phone number seven eight seven eight seven eight seven eight seven eight will be stored at the position four within the data list and here's a simple algorithm for hashing which can convert strings into numeric list indices and a hashing algorithm does not have a single definition you can come up with a hashing algorithm and in fact coming up with a good hashing algorithm is an area of research in itself now of course python dictionaries use hashing that is inbuilt into python and that's a fairly optimized hashing algorithm that's probably the result of several years of research but here's one very simple technique we iterate over the string character by character and then we convert each character into a number using python's built-in ord function and you can see here that if you call ord on the character x you get back a number it already gives you a way of converting characters into numbers but not entire strings that's why we need to iterate over the string character by character then we simply add the numbers for each character to obtain the hash for the entire string it's a very simple technique we just keep if you have the number hello we take the odd for hello the odd for e the odd for l the odd for l and the odd for o and add them together and since we want that number the final result to be an index or a position within the list so we take the remainder of the result with the size of the data list right so it's possible that once you add the numbers together you may end up with a pretty big number but if you take the remainder with 4096 or the max hash table size variable you get back a number that is smaller than 4096 so you can use just that remain as the index so let's first define a function called get index all it does is it takes the data list and it takes a string and it returns it applies this hashing algorithm to return an index for that string for that key so for a character in a string we need to convert the character to a number so we convert the character from the string into a number by calling od on a character great then we update the result by adding the number so we say result plus equals a number pretty straightforward and that repeats for all the characters in the string and then we get back the final result now that result may be longer than the actual size of the list and this is where we then we may then want to check the size of the list okay now remember there's one no i could also have probably written max hash table size here but that would be wrong isn't it because we are passing in a data list here we are passing in a data list and although we have so far created a data list of size 4096 your function should ideally be you looking at the size of the data list that you have here and not any global variables so keep that in mind and the right thing you should check here is lend data list and what this will allow is now this will allow your function to work with data lists of different sizes and not just the standard size 4096 that we have defined above okay very important thing always make sure that your functions use the arguments that are passed into them that they are generic that they can work with any input and not just a particular input that have been that has been defined earlier okay so there you go now you have this is a function get index that has been defined and here are some tests now if you pass in the data list and you pass in the empty string because there are no characters the result is likely to be zero it's great here's another one the result here is 585 here's another one result here is 941 great now this is where you should be testing your function with some custom test cases i'm going to create a new data list 2 and this is going to have the size none times 48 so this is only going to have the size 48 and i should be testing get index with this data list as well so let's say we're looking at the key akash now we know that let's see we can actually test this out here what happens if you add ord of a plus or d of a plus k a s and h that number is 585 but since the size of the list is 48 what we should be getting back as the result is 48 divided by 585 so we should be getting oh sorry 585 and its remainder with 48. we should be getting back the number nine this should be equal to nine okay so let's check that if this is equal to nine and indeed this is equal to nine on the other hand if we had max hash table size you will see here that since we are not taking into consideration the actual size of the list that was passed into the function we are getting back the value 585 because we are taking the remainder with 4096 okay so remember to take the result remainder with the size of the data list that was passed in so this is one of the several gotchas in this assignment and they're there for a reason because this is something that you need to keep in mind a function which only uses its arguments and does not depend on any external global variables or constants and things like that is called a pure function of course a plot function also does not modify any external global variables so it simply takes some arguments and returns a result irrespective of anything else outside so now we can to insert a key value pair into a hash table we can simply get a hash of the key so here we have a key value pair and we simply get a hash of the key by calling get index for data list and key and we get back the index 585 and then inside the data list at the given index we can simply set the key value pair as the element stored at that index and the same operation can be expressed in a single line of code so here we're calling get index for data list and he month and that's going to give us an index and we're going to then invoke a set at that particular index within data list the the element hemanth comma hemans phone number now to retrieve or find the element associated with a pair we can simply get a hash of the element the value associated with the key we can simply get a hash of the key and look up that index within the data list so here we have the kiakash and we have the data list and we call get index so we get the index of the kiakash and that gives us the index here and we can then call datalist and pass in the position idx and that should give us a key value pair remember that we stored a key value pair at the given index so we should get back that value here so now we know how to store a value you get its hash for the key and you store the key value pair how to retrieve a value so you get a hash for the key and then you retrieve the key value pair and from there you can get the value you can also list the keys to list the keys here is some special code we are using so let's see this is called list comprehension and let's take a quick look at list comprehension so list compression works like this if you create a list y from a list x uh let's say let's call this list one and list two go to variable names always help so if you have a list one and you write this x for x in list one what does that do that for x in list one patches elements one by one from the list and then here you can specify what to do with the numbers that were fetched so right now i'm not doing anything i'm simply returning that number and then i'm putting the entire thing into a list what this does is this creates a new list so you can see this is a copy of the original list what i could do is i could write x times two for x in list one and now i would end up with a list which in which each element is the double of that particular element i could also do x times x if i wanted i could also call a function on it let's see what function we can call here let's maybe put in some numbers here 1.3 2.4 3.2 so we could put maybe the function math dot round x oh sorry math dot seal this is going to give us the ceiling the 1.3 becomes 2 2.4 becomes 3 so you can do any operation with each element of the list and once you put that in a bracket and you have this 4 here that's going to apply that same operation to the entire list and this is called list comprehension in python it's a very powerful way to express complex operations on lists and dictionaries and there's one final thing in list operations which is the if condition so for x in list 1 can be followed by an if condition and the if condition can once again apply on x so if x is greater than 3 let's say we put this condition then what happens is we choose only those numbers from list one which satisfy this condition x greater than three so that means we would skip one point three we would skip two point four we would get three point two we would get six we would get seven and we would apply math.cl to them and that's how we get back 467 so that's list comprehension in a nutshell so to get a list of keys all we can do is for key value pairs in data list if the key value pair is not none remember that we have a lot of non values and it's a huge list if the key value pair is not none then we simply return kv0 so remember if you have a key value pair if you have like a key value pair that's akash and a phone number and you can also put because these are tuples you can also put a round bracket here if you want but even without it it's the same thing that's a key value pair kv 0 is going to give you the key and kv 1 is going to give you the value so we simply get the key for those key value pairs in data list where the element at that position of the key value pair is not none and that should not be called pairs that should probably be called keys you can see that the keys are akash and hemath so that's how we can now use the get index function and the next step for you is to complete the hash table implementation here by following the instructions given in the comments so now you have this basic hash table class and in this class you have a constructor now the constructor takes the object self or this and the self is going to point to the actual object or the actual hash table that gets created using the class and then it takes a maximum size now what are we doing here we want to make our hash table configurable we don't always want to have 4096 elements in our internal list we may need a hash table that can store more values or we may need a hash table that can only store fewer values so we are going to set a a default value for it which is the max hash table size so if you do not provide this argument by default it will create a list of size 4096 but we also want the option to specify a maximum size now you need to create a list of size max size with all the values set to none now you may be tempted to do this but that would be wrong remember that always use the arguments to a function try not to depend on an external value or external constant so this would be wrong you may also be tempted to do this data list set dot data list equals data list that we've already created this would also be wrong not just because you're not using the max size but also because now you are tying this class implementation to a global variable and that global variable is a list which can be modified so if you all the objects of this class any number of hash tables that you create using this class we'll all use the same data list and that's not what you want each hash table that you create maybe you have a hash table for phone numbers you have a hash table for addresses you have a hash table for something else each of them should have their own internal data list and this is not going to create a copy of that original this is simply going to point to the original list so what you want to do is you want to do none and you want to multiply it with max size there you go this is the correct way to do this now we're looking at insert here now to insert we did see that to get the index all you need to do is you need to pass the key and remember here you need to pass not data list but self.datalist right because now we want to use the data list that is stored inside this specific object of the class we do not want to use the global data list and this is something that is an mistake that we often make initially i've still make this mistake where i have certain global variables defined and i'm using those global variables inside my class avoid doing that anything that you want to put inside a class object you need to put inside self like we've done here and then to access it you need to use self dot to access that specific property or element or even method so now we have self.datalist and we pass in the key and the data list into get index and that gives us the index now the get index function was defined earlier we've seen it already now we want to store the index inside the list so we call self.datalist idx and we want to store the key value pair there so we can simply put in key comma value here if you wish you can also put in the brackets but they are not necessary and that's going to insert the key value pair now how do we find the value associated with the given p key first we get the index for the key so we call get index on self.datalist and key then we retrieve the data stored at the index so this would be simply self dot data list of idx and then if the key value pair is not none if the key value pair is none well there's nothing in that index we can return none another option would be to also maybe raise an index error and with a message etc etc but return on is good enough for now then if not from the key value pair we get back the key and the value and then we return the value keep those in mind if you simply return this you would get an error you would get an exception that may go unexplained so whenever you are destructuring or you're trying to get two values out of a tuple make sure that the tuple is not none especially in this case because we're starting with a list of nones in a place where we're supposed to be storing key value pairs so that's fine now update is going to be pretty much identical to insert i don't see any difference here so we can simply say get index for self dot data list and key and then now we simply store the key value pair inside it so we can simply store oops the key comma value inside self.datalist idx then for list all again straightforward self dot fkb is not none so get all the key value pairs that are not empty and then we simply get kb 0 is going to give us the key from kv so that there it is here you can see already that we are creating a basic hash table of max size one zero two four so the first thing that we can verify is that the length of the basic of the data list is one zero two four there you go then you insert some values here so we insert the value akash we insert the key value pairs so we insert the value 9999 for akash so this is one key value pair we are inserting hemanth and 8088 and what this will do is when you call basic table dot insert it will call this insert function and self will now point to the basic table that we have just created because we're calling insert on that specific basic table so self will point to the basic table so self.datalist will become basic table.datalist and then the remaining arguments are cached in 9999 will get passed in as the key and the value so this code will execute we will get the index within self.datalist for the kiakash and then within self.datalist or basic table.datalist in this case add the given index that we just computed we will store the key value pair which is akash and the phone number and that's how it'll work so we're inserting some values and then we're finding a value so when once we insert the two values and then we find a value that should give us the value 888 you may want to then maybe modify this test case to also include the test for the other value that we inserted so feel free to modify the test cases or add new test cases so that we're checking not just one value but both the values next let's see how we if we can update a value so we call basic table dot update and we said seven seven seven seven now suppose you're not implemented update here that's for a moment return suppose you've not implemented update here then if you called update you would get false here because the value did not get updated and you can check that by simply checking basic table dot find akash you can see that it still has a value 9999 that's how test cases are helpful let's remove the return okay so now the value seems to have been updated just fine then let's get a list of all the keys and the list of keys should match true once again if we did not have this kv is not none then we would get back not just this key value pair but we would get back all the nuns and we don't want that so these were some test cases but you need to now create more test cases and test them out to make sure that your implementation is correct now once you've done that you would want to run jobin.comit now the next step and this is something that you may have thought about while working through the assignment is that how do you ensure that different keys do not point to the same index because we are doing all these things where we are converting each character into a number and then adding up the characters now obviously if you have words which have the same characters but in different orders now obviously there are different keys but they do not have they have the same hash listen and silent have exactly the same keys exactly the same hash so for instance instance you can check get index listen and get index silent okay we also need a data list let's put in a data list here both of them have the hash 655 that means if you insert a value at with the key listen and then you insert a value with the key silent the data at this position will get overwritten so when you try basic table dot find listen you will get the value associated with silent and that's bad and this is called collisions this is called a collision because here the two keys are colliding in some sense because they're leading to the same hash and any hash table that you implement is ultimately going to have collisions because the number of strings or the number of keys is possibly infinite but you have a limited number of positions or indices in your table so our hash table implementation is incomplete because there can be data loss and it does not handle collisions and there are multiple techniques to handle collisions and we the technique we will use in this assignment is called linear probing and here's how it works while inserting a new key value pair if the target index for a key is occupied by another key then we simply try the next index and if the next index is also occupied by another key we try the next and then if we try the next and then we try the next till we find the closest empty location and then while finding a key value pair we apply the same strategy but it's searching for an empty location this time we search for a location which contains the key value pair with the matching key we get the hash of the key that we want to find and then we check if that position is occupied by another key not the same key then we try the next index and then we try the next index and then we try the next index till we find a position which is occupied by a key value pair for the same key and if we find an empty position that means the key does not exist because if it did exist then it should have been somewhere in that string of searches that we just come that we just did now by updating the key value pair again we apply the same strategy but instead of searching for an empty location we look for a location which contains a key value pair with the matching key and update its value so that's how you handle collisions in a hash table and to handle collisions we will define a function called getvalidindex which first gets the hash using get index and then start searching the data list and returns the first index which is either empty or contains a key value pair matching the given key so we are now addressing two requirements in one shot with the get valid index function for insertion we are looking for an empty position for find an update we are looking for a position which is occupied by a by the given k value by the given key value pair okay or the given key specifically so here's the get valid index function and i will let you work through this so you will start with the index returned by get index then while true because we don't know how long we may need to iterate get the key value paired stored at the index this is where you may have to it's simply a question of putting the index into data list getting the key value pair now if the key value pair is none which means that there is nothing at that index it is empty that's great we are done we can simply return the index on the other hand if it does have values so then we get the key and value out of it if the key matches the key that we want to store great then we can return the index once again if neither of these hold true we move the index to the next position but as we move to the next position it's possible that we may run out of indices so the index may become equal to the length of the data list so then we wrap around and go back to the zeroth position so this is an important part where we go around so now our list is in some sense circular where we can keep looping around it so that if we have something that needs to be stored at the last position but the last position is occupied then we move back to the zeroth position and so on and then you can check if get valid index was defined correctly and if it was then these cells should output true once again these are just some sample test cases so you should include some more of your own test cases here and finally once you're done just save your work now the next step is to incorporate linear probing into your hash table so here's a new class called probing hash table here you need to use not get index but get valid index it has pretty similar code so i'll let you work this out be aware not to simply copy paste code and you will run into issues if you copy paste code so always make sure that you are writing the code yourself and carefully writing each word or each variable in each method and each argument of the code then there are some test cases here for you to test the probing hash table once again you can try it out with some examples and see if it works fine specifically here we are taking the same example listen and silent both of which in basic hash table would have the same key but in probing hash table would have different will have the same position but in probing hash table will have different positions and that's it you at this point you're done with the assignment so you can make a submission if you have run jobian.commit you can take this link and make a submission on the assignment page or the other option for you is to simply run jobin dot submit python dsa assignment 2 and once you make a submission it will be evaluated automatically so let's click through here so it will be evaluated automatically and if you scroll down here you will see that you will get a grade and not just grade but you will also get comments for each question so if you see here there are question numbers here you can see that there's question five question four and so on so it seems like we since we implemented the get index function since we implemented the data list correctly question 1 was a pass let's see what question 1 was very quickly question 1 create a python list of size max table hash size question 2 was a pass so question 2 was the get index function question 3 was a pass so question 3 was complete the hash table implementation question 4 was a field get valid index we've not defined it yet and question 5 led to an exception obviously because we have some code which will not execute because we have some blanks that need to be filled in so keep that use this as feedback you will know exactly what to fix and if you are stuck at any point you know what to do you can go to the forum so let's see the forum here so this is the forum sub category for assignment two you can create a new topic here if you want to have a longer discussion or you can simply go to the main topic assignment to hash tables and python dictionaries and you can ask a question here there are already a lot of discussions going on here so it's possible that your question may already have been answered and after this there are also some optional questions now here the optional question is for you to implement a python friendly interface for the hash table so instead of defining functions insert update and find you will define the functions get item set item and instead of list all you will define the function eter and also instead of using the hash function instead of using the custom hash function that we have defined you will define you will use a function that's inbuilt into python called hash and it takes any string or any object and it returns a number for it now since hash does not accept a list so you will have to take the remainder manually so in this case for example we've taken the remainder and gotten back the number three five six nine so you define a hash table here and once you have done that you will be able to use it just like a python dictionary you will be able to use it exactly like this you create a hash table and then to insert a value you use the indexing notation and insert the value to retrieve a value you use the indexing notation to get the value back and here you can compare it to the number to update a value you simply use the induction notation again and to get a list of values you simply call the list function or you can also use it within a for loop and we've also defined a function called wrapper and str what that will do is that will let python print a representation like this when you simply run a cell which just contains the name of this variable that's one and then there are a bunch of improvements that you can try to hash tables this is a great exercise if you want to improve your python programming skills and also understand how hash tables work if you can complete these four exercises there's pretty much no question related to hash tables that you cannot answer you will know everything about them and each of these exercises may take another 30 minutes to 45 minutes but it's completely worth the time maybe spend set aside a few hours on the weekend to work on these optional exercises now here's one how to track the size of the hash table instead of having to look through the entire table to get the number of key value pairs can you store the length somewhere so that you can track it in size order one here's one to implement deletion so to implement deletion you have a topic called technique called tombstones that are used so you can use this tombstone technique and implement it just a little more code can you implement dynamic resizing so instead of starting out with a hash table of a given size or requiring the user to specify a size can you or maybe start with a hash table of let's say 128 elements and then double it as soon as you reach 128 elements or maybe even before to avoid collisions you may want to double it as soon as you reach 64 elements like 50 of the capacity so dynamic resizing is the technique that allows you to automatically grow and shrink the data list internally and then here's another technique for collision resolution this is called separate chaining so instead of going to the next index what you do is you maintain a linked list at each position and for all the key for all the keys you still use that position but you look through the linked list while looking for a key or you add a new element to the linked list for that position if you're adding a new key there so here's separate chaining explained in a youtube video you can look through that and try to explain it on your own and one final thing here is also the complexity analysis and here's where you talk about average case time complexity because on average if you have a good hashing function and you've implemented some improvements like dynamic resizing then the average time complexity for insert update find and delete are order one and list of course is still order n on the other hand the worst case time complexity because there can be collisions are still order in so here's something for you to ponder upon what is average case complexity and how does it differ from worst case complexity this is also something that is discussed in lesson three of the course where we talk about quicksort and you see why insert find and update have an average case complexity of order one and a worst case complexity of order n if not it is something that you can look up online try to see if you can search a tutorial and learn why this happens then how is the complexity of hash tables different from that of binary search trees we've discussed binary search trees in a lot of detail in lesson two so now the question becomes when should you prefer using hash tables and when should you prefer using binary search trees or vice versa so all these very interesting questions and you may get asked some of these questions and interviews as well it will help you especially to ponder upon some of these questions even if you do not end up solving all of these optional questions do look at the complexity analysis and think about it and there's a forum thread where you can discuss your thoughts so what you do next review the lecture video review the assignment walkthrough video and execute the jupyter notebook complete the assignment and attempt the optional questions as well and do participate in forum discussions so this was a walkthrough of assignment 2 of data structures and algorithms in python hello and welcome to data structures and algorithms in python this is an online certification course conducted by jovian and today we're on lesson three my name is akash i'm the ceo of jovian and i'm your instructor for the course if you follow along with this course and complete four weekly assignments and a course project you can earn a certificate of accomplishment for this course so let's get started the first thing we do is visit the course website pythondsa.com so when you visit pythondsa.com this will bring you to the course website here you can find all the information and material for the course you can check out lessons 1 and 2 and assignments 1 and 2 both of which are still open for submission and let's open up lesson 3. so the topic today is sorting algorithms and divide and conquer and you can watch a video recording of the lesson here you can also catch a version in hindi now the code used for the lesson is provided here so let's open up this link sorting and divide and conquer this is where all the code is present so here we have it now we are looking at the tutorial and the code for this lesson if you scroll down you can see that there is some code here now to execute this code you have two options you can either execute this code online using free online resources which is what we recommend or you can download it and run it on your computer locally and the instructions for both of these are given here we are going to use the first one which is to click the run button at the top of this page and select run on binder so let us scroll up here and let us click the run button and then click run on binder now once you do this it will open up an interface like this and what you're looking at here is a jupiter notebook so a jupiter notebook is an interactive programming environment where you can write code look at the results and you can also write explanations and we've provided you with a cloud-based jupyter notebook setup so you don't have to install anything all the code that you execute here will be running on our cloud but you can also download it and run it on your own computer by following the instructions so the first thing we'll do is click on the kernel menu and click restart and clear output to remove any of the outputs from previous executions of the code so that we can execute the code and see the outputs fresh for ourselves now i'm also going to zoom in a little bit here so we can look at the code and let's get started so this is a coding focus and practical course and we're talking about different data structures and algorithms the topic today is sorting algorithms and divide and conquer algorithms in python so in every lecture we focus on a specific problem so in this notebook in this tutorial we will focus on this problem which you're looking at here so let's read the question you're working on a new feature on jovian called top notebooks of the week write a function to sort a list of notebooks in decreasing order of likes keep in mind that up to millions of notebooks can be created every week so your function needs to be as efficient as possible that is the key point here now this is a classical problem in computing the problem of sorting a list of objects and it comes up over and over in computer science and software development and it's important to understand common approaches for sorting how they work what the trade-offs are between them and how to use them so before we solve this problem we solve a simplified version of the problem it's quite simple to state write a program to sort a list of numbers and sorting usually refers to sorting in ascending order unless specified otherwise so that's a question for today write a program to sort a list of numbers and we'll expand upon it to answer this original question as well now this is the method that we've been following throughout the course and we will continue to follow a systematic strategy for solving programming problems step one state the problem clearly identify the input and output formats step two come up with some example inputs and outputs and try to cover all the edge cases and step three come up with a correct solution for the problem stated in plain english step four implement the solution and test it using example inputs so this is very important that you implement the simple solution so you just need a correct solution first not the efficient one and then you implement it and test it then you analyze its complexity identify inefficiencies and then you apply the right techniques to overcome the inefficiency and that is where the knowledge of the right data structures and algorithms comes into picture and once you apply the new technique then you once again state the solution implement it and analyze its complexity and repeat if necessary so this is the strategy we'll follow here today as well so step one state the problem clearly and identify input and output formats now the problem is stated clearly enough for us we need to write a function to sort a list of numbers in ascending or increasing order now here's the input the input is a single argument called nums and that is a list of numbers so for instance here's a list of numbers you can see that they're not in any specific order and then the output is the sorted version of the input so here is the same list of numbers in sorted order and based on these two we can now write a signature of our function so a function will be called sort or something else but it will accept just one input and right now we've not written any code here so we just put in pass now i'm running this code here using the shift plus enter shortcut but you can also use the run button on the toolbar so either run or shift plus enter and the great thing about jupyter notebooks is that you can add more code cells anywhere and test anything that you want for instance if you want to insert a code cell below just click the insert cell below menu option or click outside a cell on the left and press the b button and now you can write some code here and run it so please feel free to experiment with this notebook as you go along so step two come up with some example inputs and outputs now this is very important you need to think about all the different scenarios in which you may want to test out your function before you put it into production so that you catch errors early on and thinking about scenarios will help you identify what are the special cases you need to handle in code and it's easier to do it right now than while writing your code because that may lead to bugs so here are some scenarios that i was able to come up with and there may be more so you can continue and increase this list so the first one is some list of numbers in random order for some numbers in any random order and you can try slightly smaller lists and larger lists and so on second is a list that's already sorted we need to ensure that an already sorted list does not become unsorted a third is a list that's sorted in descending order we may want to check that see if we need to handle that case separately somehow then a list containing repeating elements this is something you may not have thought of but the question never said that the number should be unique so there could be repeating elements here an empty list interesting input the output is also an empty list or a list containing just one element or a list containing one element repeated many many times or even a really long list this is something that we may want to test because we want our algorithm to be efficient at the very end so a long list may help us just evaluate the efficiency empirically so these are the scenarios and what we now need to do is create some test cases for these scenarios so test cases involve creating an input and an output for instance here is an input num0 and this could be the list 431 and here's the expected output so let me call it output 0 and this would be 134 now this is a good way to put create a test case and you can use it later for testing but we will put our tests into this particular structure we'll create a dictionary and creating a dictionary like this will help us automate the testing of all our test cases with a single helper function so what we're gonna do is for each test case create a dictionary and then it'll have two keys first key is called input and the second key is called output and in the inputs for each of the arguments that go into the function and remember there's just one argument here we will have one key so we will have the key nums and the key numbers will have the input value for the test case and the output will simply contain the output returned by the function so that's how we'll set up our test cases so does our test 0 a list of numbers in random order then we have test one this is also another list of numbers in random order you can see here no specific order now we have a list that's already sorted and of the output obviously is the same now for the random ordered list the output is the same numbers in sorted order now we have a list that's sorted in descending order and the output is the same list in increasing order then we have a list containing repeating elements you can see that the numbers 1 2 6 and 7 and even minus 12 repeat here here we have the empty list here we have a list containing just one element and here we have a list containing one element repeated many many times and then the final test case which was to create a really long list that's where we can start with a sorted list created using the range function and then shuffle it to create the input otherwise you may spend a lot of time just creating a list and then writing the sorted version of it that's too much work so always use a computer always use helper functions whenever you can even to create test cases so we'll use the range function now the range function takes either a single number or two numbers so you can have something like this range 2 to 10 or just range 10 and if you just look at it this way it just prints range 0 to 10. now if you actually want to see what's in it there are a couple of ways you can do list range 10 and that gets converted into a list or you can use it in a for loop so you can put for x in range 10 print x so you can see that it contains the numbers zero to nine and that's important that the range does not include the end element of the range okay so just keep that in mind now what's the difference between a range in a list a list contains all the 10 numbers together at once but a range internally simply maintains a counter so when you use a range in a for loop it simply starts the counter from 0 and increments it up to the starts the counter from the starting value so if it's two to ten then it starts a counter from two and increases it up to the end value minus one so it does not use as much space as a list it simply uses a one single variable internally and that's why it's more efficient in any case right now we need lists so what we'll do is we will create a list of ten thousand numbers so zero to nine nine thousand nine hundred and ninety nine that is our in list and then our out list is also going to be 0 to 999 that's our outlist both of them are sorted now what we do is we shuffle the inner list so we import the random module from python and then we call random.shuffle and we call random dot shuffle on in list and that shuffles the the first list the n list so now we have that as the input nums and then the out list the sorted list is the output now once again we can even check that in list is actually shuffled maybe by looking at the first 10 elements you can see here that these are all shuffled numbers on the other hand if you check the out list you can see that these are all in order so those are our test cases and it's very important to create good test cases even in interviews before you start coding or before you even suggest a solution you should try and list out your test cases either verbally to an interviewer in a coding assessment you may create a block of comments at the top and start listing some test cases at the top or you can create proper test case dictionaries like this it takes a few minutes but it's totally worth it because you can then test your algorithms very easily and finally we'll take all our test cases test 0 to test 8 and put them into a single list called tests great so we've made some good progress so far next let's come up with a simple correct solution and stated in plain english and coming up with the correct solution is pretty straightforward we have a list of numbers so we iterate over the list let's grab a list of numbers so that we have something to look at here you go so we have a list of numbers so we iterate over the list of numbers starting from the left so we start from the very left and then we compare each number with the number that follows it so we compare 99 with 10. and if 99 is greater than 10 then we can say for sure that 99 should appear after 10 in the final sorted array and the sorted array by default it means the increasing order of numbers right so that's what we're solving first so what we can do is we can simply swap 99 and 10 because we know that 10 should appear before 99 and 99 should appear after 10. now once we continue the swap we move to the next position and then we compare 99 with the next element nine that turns out to be higher as well so we swap it and then we keep going so we iterate over the list and for each element compare the number with the number that follows it and if the number is greater than the one that follows it swap the two elements now you do that once and that alone is probably not enough to compile the entire list because the entire sorted list because 99 in this way will end up at the end if you follow the process but the rest of the list is still not sorted so we repeat these steps one two three so once again we start from the left and then we start comparing ten with nine and then ten with eight and so on and keep swapping elements as we go forward now i have a claim here that you may you will need to repeat the steps one two three at most n minus one times to ensure that the array is sorted can you guess why and here's a hint after one iteration of the process the largest number in the list will reach the very end so that means that each time you're putting one of the largest numbers at the very end so you need around n steps so here's an animation showing the same thing here we compare six and five and then we switch them then we compare six and three and we switch them then we compare six and one and we switch them now we compare six and eight and we don't switch them because they're in order next we compare eight and seven and we switch them next we compare eight and two and we switch them and finally we can compare eight and four and we switch them and in this way the largest number eight has reached the very end so now we can froze freeze its position and we can start again from the beginning and you can see that this time the next number seven will end up here and then the next time the number six will end up here and then next time the number five will end up here and so on so in n repetitions of this process of comparison left to right we will have sorted the array and this approach is called bubble sort because it causes the smaller elements to bubble to the top or to the beginning you can see that the numbers 1 3 slowly bubble up to the top and it causes the larger numbers like 8 and 7 to sink to the bottom and you can watch this entire animation to get a full sense of how bubble sort works what will also really help is if you can take an example on paper and work it out on your own step by step especially with sorting algorithms this really helps okay so now we've come up with a correct solution let's implement it and let's test it using an example now the implementation itself is also pretty straightforward so we have the bubble sort function here def bubble sort it takes a list of numbers now we may not want to modify the list of numbers in place because then our test cases will not be reusable so just to avoid modifying our test inputs we are going to create a copy of the list to avoid changing it and the way to create a copy simply call the list function with the list as input so now we are set replacing nums with a copy of nums now depending on your particular use case this may not be necessary so this is something that you can actually check while you're in a coding assessment or in an interviewer or talking to an interviewer just check with them do do they want an edit to be sorted in place or do they want a new array to be created if they want if they're okay with sorting it in place then you probably don't need this but you may still just want to keep it in because otherwise you may end up modifying some of your test cases unintentionally and that may lead to problems it's always good to create a copy of the input rather than modifying it in place okay so then let's come to steps one two and three and then we'll see step four which is the outermost step really so we iterate over the array so we go from we take i and we check the range len nums minus 1 so the number of elements in the array is n and n can be obtained using len nums then we want to go from indices 0 to n minus 2 so the total number of indices is 0 to n minus 1 but if you go to the n minus 1 or the last element there is no further element to compare it with okay so keep that in mind that you only want to run this iteration till your pointer comes to this point not till the last element and that is why we check if we put i in the range 0 to len nums minus 1. so the highest value that it can take is len nums minus 2. next we compare nums i with nums i plus 1. so we compare the number with the element that comes after it and if it is greater so that means these two are out of order so then we simply swap them so we set nums i comma nums i plus 1 equal to num i plus 1 comma num psi now this is a very interesting way of sorting in c or c plus plus or java you would have to write three or four steps to swap numbers but in python it's really simple first you say x y is let's say we're saying x comma y are two comma three so you can see they have the values two and three and then we simply write x y equals y x so what happens is the value of y gets placed into x and the value of x gets placed into y it's a single step for swapping two numbers there you go so we swap the two elements exactly what we're showing here swapping the two elements next we repeat this so now we're doing this from left to the penultimate element and in this way we have pushed the largest element to the end now we need to repeat this process n minus one times so that each time we are pushing one of the largest elements to the very end and in n minus one repetitions of these three steps we will end up with a sorted list and finally we return the sorted list and that's it so let's test it out with an example and by the way if some of this doesn't make sense so a simple way to debug it is to add print statements here so you can add a print statement and maybe just print this value so we've used underscore here because we don't actually use this value but let's say we wanted to use this value then we can print that this is iteration j and then inside it you can print that the value of i is i and you can also print the value of nums i and you can also print the value of nums i plus 1 and at the very top you can also print nums now if you add all of these print statements and then execute your algorithm now you will be able to see exactly what is happening inside each iteration so that's a great way to debug your code if you're facing any issues and also understand what the code does but in any case we won't need these so i'm just going to comment these so let's test it out so we get from test 0 we get the nums as input and then we get the output and we can print the input and the expected output and then finally calculate the result by passing num0 into bubble sort and then printing the actual output and finally whether the two match so you can see here now that the input was this unsorted list and then the expected output was the sorted version and that's what we got so in fact there was a perfect match and that's it so we've implemented our first sorting algorithm it was pretty straightforward a few lines of code as an exercise you can try to implement it once again from your memory just write it in plain english first and then try to implement it it's a good coding practice and we can also evaluate all the test cases that we have remember we created about nine test cases and to help you evaluate the test cases we've given you a helper function called evaluate test cases which is part of the jovian library so we installed the jobin library here pip install jovian and then from joven.python dsa so python dsa is the name of the course and that's also the module where we have helper functions for this course import evaluate test cases and evaluate test cases simply goes over the list of test cases that you have and it pulls out the inputs and passes them as arguments to the function provided here which is bubble sort and then gets the outputs and compares the outputs and also prints the information with like what was the input what was the expected output and the actual output and whether they match so let's check it out so you can see here this was test case 0 and that work which we just tested out here's a larger list including some negative numbers this worked as well you can see the test result is passed then you have another list here this seems to work fine too this is already sorted here you have one which is sorted in decreasing order that works here you have one with repeating numbers that works too the empty list works the single element works and this works too this is the same element repeated over and over and finally here is our final test case this had 10 000 elements remember so you can see that this was the expected output and this was the actual output so we have successfully sorted 10 000 elements and that's really the power of programming that without having to look at any of the numbers we've just written four or five lines of code and we've sorted 10 000 elements so all our test cases passed although do look here that it took about 15 seconds for the sorting of 10 000 elements now maybe that's not maybe that's not that bad but we we are looking at probably millions of notebooks every week at jovin so we want there to be a faster sorting algorithm okay so to before we improve the algorithm we need to understand the algorithms complexity and identify any inefficiencies now the core operation in bubble sort if you look at the code here once again is this operation of comparison so we're comparing a number with the next number and swapping now comparison almost always happens and swapping doesn't happen nearly as often so if you want to find the time complexity and we want an upper bound or the worst case time complexity we can assume that roughly every comparison also leads to a swap in the worst case so if we just count the number of comparisons as a function of the input size the size of the list that was given as an input that should give us an idea of the time complexity okay so here we can see that there are two loops and the length of each loop is n minus 1 and inside the inner loop there is a comparison so the total number of comparisons is n minus 1 times n minus 1 which is n minus 1 square or n square minus 2 n plus 1. now expressing this in the big o notation which is to get a rough idea of how the number of comparisons or the number of operations in the algorithm grows with time we can ignore the lower order terms like 2 n plus 1 so we can now conclude that the time complexity of bubble sort is order of n square and this is also known as quadratic complexity so we can now verify that bubble sort requires order one additional space that this is an exercise for you but here's a quick hint you can see that we are not allocating any new lists we did create a copy of the list but we didn't have to so let's not count that but apart from that there is no additional space that was required here we're not allocating any new variables we are creating this range but remember i mentioned that a range simple simply contains a single variable inside it which it keeps incrementing for a for loop so we have these two ranges so maybe we have two variables assigned so it's constant irrespective of the size of the input and that's how bubble sort requires order one additional space now you may be asked about space complexity and this is where it's a slightly tricky thing because sometimes strictly speaking space complexity also includes the size of the input because to store n numbers or n elements you need n spaces in memory so the space complexity of bubble sort in that sense is order n and this is something you can check with the interviewer if they're asking you what is the space complexity and you can ask them if they just want to know what is the additional space required so the overall space complexity is order n because we need to store the actual input list somewhere but on the other hand the amount of additional space required is order one which is a constant factor independent of the size of the list so that's that's how bubble sort works now analyzing this order n square complexity and keeping in mind that a list of 10 000 numbers takes about 12 seconds so if n is 10 000 and n square is multiplied by some constant is about 12 seconds then if you had a list that was of hundred thousand elements so that would be 10 n whole square or 100 times the same amount of time that it would take to sort it so that means it would take about 20 minutes to sort 100 000 numbers which is i would say is a bit inefficient now and a list of a million numbers would take close to two days to be sorted in python now if you do it in c plus plus maybe it might be uh four or five times faster but again the moment you go from a million to ten million well that will actually end up taking a year or so and that's bad and that is why n square or quadratic complexity is something that we would like to do away with because it grows very fast as soon as you hit maybe a ten thousand or a hundred thousand elements then it starts taking longer than a few seconds or a few minutes or a few days and at that point you can no longer use that particular algorithm so we need to optimize bubble sort and the inefficiency in bubble sort comes from the fact that we are shifting elements by at most one position at a time so each time we go through the list we capture some information about the list but we are simply moving one element from left to right so to speak and each time we're just moving it one at a time by doing swaps rather it would be nice to just place elements directly maybe a few positions ahead and that's where we will look at some optimized algorithms now another common algorithm that is used is called insertion sort and this is here is the code for insertion sort so you can look through the code for insertion sort here and here is an example you can see how it works and we will not look into insertion sort in a lot of detail but roughly this is how you arrange cards in your hand which is by starting to move cards around so that at the maybe on the left edge you have sorted cards on the right edge you have the unsorted cards and you keep moving the new cards into sorted positions that's how it works so here's an exercise for you go through this function read the source code and then describe the algorithm in plain english now reading source code is an essential skill for software development this is something that you'll have to do in your work whether you're doing software development or data science maybe because there are no comments in the code there is no documentation or the person who has written the code is not available or has left the company or this is some open source library so in all these cases you will have to read and understand code so read it and then describe insertion sort the algorithm in plain english then look it up online and see if it matches what you've written and then second is to also determine the time and space complexity of insertion sort and see if it is any better than bubble sort and explain why or why not so these are a couple of exercises for you so that's bubble sort and insertion sword now before we continue i just want to recall you that this is a jupiter notebook running on an online platform hub.binder.joven.ml and since this is free it will shut down after some time so you want to capture snapshot of your work at regular intervals and that's where you can use the jovian library so you install the joven library using pip install jovian import jovian and then run jovian.commit now when you run jovi.com it captures a snapshot of this jupyter notebook and puts it on your jovian profile so now this will be your profile when you run juvenile commit and you will be able to resume your work by clicking the run button on this page anytime and this notebook will go to your profile so you can just click on your jovian profile or just click home here and if you check either the overview or the notebooks tab you should be able to find your notebook here like here you go okay coming back now we're at step 6 where we want to apply the right technique to overcome the inefficiency in the algorithm now to perform sorting more efficiently we will apply a strategy called divide and conquer and divide and conquer is a very common strategy used across the board for many different kinds of algorithms and it has this general steps that is applied in different different ways across different problems so step one is to divide the inputs into two roughly equal parts okay they don't have to be exactly equal but two roughly equal parts and the idea here is that those two parts can themselves be used as inputs as sub problems so then we use recursion so we recursively solve the problem individually for each of the two parts so here you have a problem you have created two sub problems out of it and then you call recursion so the recursive solution itself will use divide and conquer and then keep going and so on uh but once it gives you the solution combine the results to solve the problem for the init for the original inputs okay so you have now results of the sub problems and you combine them you get back the final result and then the only last thing you need to know is because you're going to keep calling this keep doing this division recursively so if you have an input of size 100 you will call the same function on inputs of size 50 and 50 then you will call the same function for each of those 50 you will call the same function on inputs of size 25 and 25 so each half and as you keep going you will eventually end up with small or indivisible inputs and that is where you can solve the problem directly and include terminating conditions so that's where the recursion stops okay so you include terminating conditions for small or indivisible inputs so that's divide and conquer you take the problem divided into two sub problems recursively solve the sub problems get the solutions of the sub problems and then combine them so you can also call it divide conquer combine in some sense and merge sort is the algorithm that is the classic application of divide and conquer to the sorting problem so let's take a look at merge sort by looking at an example visually so here we have a list that needs to be sorted in increasing order so remember step one divide the problem into two sub problems so here we have half the list a little more than half here we have another half so we have split it into four elements and three elements then we take we call recursively we call the same sorting problem the same algorithm on these two so we split 38 and 27 into one half and 43 and three into another here 982 becomes one half and 10 becomes the other again we can split 38 in 27 we can split 43 and 3 980 to 10. so now we've ended up with single elements so with recursion we've ended up at this terminating condition we can no longer split the list so now we start combining the problems now if you're looking to sort a list with just one element 38 well that list is already sorted so you can return that and 27 is already sorted the single element so you return that now we have these two sub lists and we need to combine them each has one element so we can simply compare these two elements and we can tell that 27 comes first and 38 comes second so that's how you combine these two results to get 2738 then similarly with 43 3 you combine them to get 343 and you get 982 and 10. next you can combine these two results so this is where now the combination is important okay we need to look through and we can probably tell that 3 should come first and then 27 and then 38 and then 43 so we've combined them here and similarly here we've combined 9 10 and 82 and then we take the final results these two final lists and then we combine them back to get the fully sorted list okay and we'll talk about this combination or what is called the merge operation in a lot more detail soon but this is roughly the idea here you keep splitting it into half and then you combine the halves so let's now state it in plain english so first the terminating condition if the input list is empty or contains just one element then it is already sorted return it if it is not divide the list of numbers into two roughly equal parts then sort each part recursively using the merge sort algorithm and by the power of recursion you will get back two sorted lists then merge the two sorted lists to get a single sorted list and this is the key operation here and this is why it's called a merge sort because we are always merging sorted list and making bigger and bigger sorted lists out of them and the merge operation is something that you may be asked to write in an interview or a coding challenge apart from the whole merge sort operation itself so this is something that you can try to explain yourself so try to think about how the merge operation might work and explain it in your own words here is some space for you but let's jump into the implementation of merge sort then now we will implement merge sort assuming that we already have a helper function called merge and this is a very useful trick where your program may need some complicated piece of logic or some logic which you have not figured out yet so all you do is assume that you already have the function and write uh use it first and then implement it later so here's a merge sort algorithm so now we have the merge sort algorithm and we have numbers here given as an input to merge sort now here's the terminating condition if the length of numbers is less than equal to one which means if the list is empty or has just one element return the numbers then if not then get the midpoint so return length of numbers divided by two and remember using the double slash here because a single slash would return a decimal and we cannot use a decimal as an index or a position in the list so that's why we're using the double slash here so we take the length of numbers divided by 2. so if the size of the list is 10 so we get back five here then we split the list into two halves and here's some interesting syntax for you and let's look into what the syntax actually means so let's say you have a list so this is the list we have and let's say mid has the value well we can check it here one two three four five six so six elements by two mid has a value three now let's check x of mid what does that give us well that gives us one three five well actually x of colon mid means x of zero to mid and x of zero to mid means all the elements from position zero till before the position mid so that's very important once again it's like a range so you get the indices at position zero one and two not at position three okay so that gives us these three elements then let's check the other thing x of mid colon now what this gives you is this gives you the elements starting from the position mid all the way to the end so you can also write here minus 1 or we can also write here len of x minus 1 but or we can just skip it and python will automatically interpret that you want all the elements starting from mid to the end that is twelve five and one so positions three four and five and hence to split the list all we need is to invoke this one three five and twelve 1251 we get back two parts of the list so this is a nice thing about jupiter whenever you don't understand a line of code just create a cell above or below and try out a simple example so now we have the the left half num zero to mid and then the right half so numbs mid colon now here's where the magic happens we call the function recursively so we call the merge sort function itself so because we call merge sort on left and that gives us back a list a sorted list for the left half called left sorted and then we call merge sort function right and that will give us back a sorted list called write sorted and then we combine the results of the two halves by calling the merge operation so we now we are now saying that we want to merge left sorted and right sorted to get back the final sorted numbers and then we return the sorted numbers so that's merge sort so yeah it's almost seems like magic but it's pretty small pretty straightforward only about four or five lines of code if you combine some of these lines so then let's come to the merge operation because that seems to be the meat here right this is the only missing piece so to merge two sorted arrays what we can do is we can repeatedly compare the two least elements of each array and copy over the smaller one into a new array so here's what that process might look like let's say you have these two parts one four seven and zero two three and we want to get this sorted list and notice that these are both already sorted because these are the results of the recursive calls to merge sort so we keep a pointer on the left on each one so here we have the pointer at one here we have the pointer at zero we compare the two we take the smaller one and put it in the list how do we know we can put it because if this is smaller than this all these numbers are also greater than 0 and then since 1 is greater than 0 and all these numbers are greater than 0 are greater than 1 so that follows that all the other numbers to the right of 1 and to the right of 0 are greater than 0 and hence 0 should come in the first position so we put it there and advance the pointer now you can see here now we can compare one and two and this time one is smaller and you know that all the numbers here are greater than two so they're also greater than one and then all the numbers here are also greater than one hence we know that one is now the next largest number so we can now put in one and advance the pointer and keep going this time now you compare two and four so now you can put in two and advance the pointer now you put in three and then advance the pointer and at some point you will exhaust one of the lists and when you exhaust one of the list then you can stop comparing and you can simply copy over the remaining elements so we can now copy over four and seven and we've exhausted this list and we get back the sorted merged array zero one two three four seven so it's really simple it involves each step involves one comparison and incrementing one pointer so you're either incrementing this pointer or you're incrementing this pointer okay so let's now define the merge operation and you can see the benefit now of assuming that the function already existed now we do not have to worry about the actual sorting and recursion etc we simply have to worry about merging two sorted arrays so first we will create a list to store the results and we have nums 1 and num2 the two left and right list that we are going to combine then we are going to set up two indices or two numbers for iteration the two pointers on the two lists and we set up each of them at position zero so each of them are currently at position zero here and we loop over the two lists so we say while i less than len of nums one and while j less than len of nums 2. so if you have four elements in the left list then i can go from zero to three all four positions and if you have five elements in the right list j can grow from zero to five zero to four all five positions then we check and and we remember we want to make sure that both of these indices are valid if any of those have reached the end then you want to skip and we can simply copy over the remaining list right so as you see here as soon as we reach this point there's no more comparisons to be made so we can exit the loop so now we check which one is smaller so if we if norm's one i so the left list current element is smaller than nums to j then we append to the merged list num1i as we did here and we increment i so this is exactly what we've done here so we put in well let's say here so we put in 1 here and we increment the left pointer on the other hand if that's not true we append the element from the right so nums 2j and we increment the right pointer so in each case in each while loop we are incrementing one of the pointers and then when the while loop ends one of the lists would have been exhausted that's when the while loop ends so we can get the remaining parts of both the lists so we can get numbs1 i colon will get the remaining elements from the first list the left list nouns 2 j colon will get the remaining elements from the right list but remember since one of them is exhausted so one of these two is going to be empty right now we we can check which one is empty and simply add the remaining one but here's a simpler solution we just add both of them to the merged array so we append both the lists at the end and this automatically takes care of the empty case if the left side becomes empty then this adds nothing to the merged array and this adds the remaining numbers from the right side if the right side becomes empty then this adds the remaining numbers from the left and left side and this adds nothing so that's a small trick so that's the merge operation again not very difficult if you have any questions is take this out into specific cells and try it out with examples and you should see it working let's try out the merge operation now so here we have two sorted lists you can see here and there you go you can see that this is now arranged all these numbers are now arranged in a sorted order so now we have the merge operation and we have the merge sort operation so we can now test out the merge sort function so we get the first set of inputs and outputs from test 0 and you can see here that this is the input and this is the expected output and this was the actual output as well now let's test all the cases using the evaluate test cases function from jovian so here we're simply going to call evaluate test cases on the entire list of test cases and you can see all the test cases seem to be passing now if one of these test cases had failed what you should go do is you should go back and add some print statements inside your merge function or add some print statements inside your merge sort function the right places to add the print statements is right after the function definition right after in the body of the function it can be the first statement and then inside each loop so inside each loop whatever are the changing parameters you should print them inside the loop and then finally you can also print the return value of the function so in this way you can build a full picture of what the what your function is doing and that makes it much easier to solve issues so test cases and print functions make it easy to fix errors in code and don't worry if there are there are always errors in code what's important is you should be able to find a way to fix them easily and without test cases or without printing you may get stuck and you may just keep staring at the code and trying to figure out what exactly went wrong so please do that now one last thing i want you to notice is here the execution took only about 50 milliseconds on the other hand remember bubble sort took about 15 seconds to sort ten thousand numbers so that's mer sword is much much faster right a millisecond is point zero zero zero one ten to the power minus three seconds so in a second you can probably sort two hundred of two hundred list of size ten 10000 and that's what makes merge sort so much more powerful and because it is so much more efficient and as we analyze the complexity you will learn that merge sort is in fact more efficient in terms of the bigger notation as well so let's analyze the algorithms complexity and identify if there are any inefficiencies now analyzing recursive algorithms can get tricky and that's where it helps to track and follow the chain of recursive calls so what we'll do is we will add some print statements to our merge sort function and our merge function so we'll simply see what the merge sort function was invoked with okay so we'll add a print statement inside merge and we'll add a print statement inside merge sort both of them and we are also tracking something called a depth to track the chain or the depth of each recursive call and you'll see what i mean in just a second okay so this is what it looks like we called merge sort on this big list of elements unsorted and that merge sort internally led to two calls of merge sort so you can see this one here and this one here so you have two calls to merge sort one with the left half of the list and one with the right half of the list and they're unequal and these two merge sorts finally returned merged lists and we finally called a merge operation on the two of them you can see that this is the merge operation the final merge operation called here on the two merge sort lists and this merge operation is working with these two sorted lists okay so we can see that each merge sort invokes the itself invokes merge sort twice but this time with an area of half the size you can see merge sort was invoked with arrays of or lists of half the size and it also invokes the merge function once to merge the two resulting arrays the two sorted arrays now the two calls to merge or if you observe closely they themselves make two more calls to merge sort and one more call to merge and then those internal calls make two more calls to merge sort and one more call to merge and so on till we end up with single elements at which point merge sort simply returns that single element so the merge sort algorithm ultimately points down to a series of merge operations you can see here that each merge sort all it's doing is calling merge sort internally and then calling a merge operation so ultimately what we are doing is we are first merging 5 and -12 and then we are merging 2 and 6 and then we're merging minus 12 5 and 2 comma 6 and then we're merging 123 and we're merging 7 minus 12 and then we're merging 7 minus 12 7 and finally we're merging 123-1277 and then finally we are merging the big list right so it's ultimately just a whole bunch of merge operations and if you look inside the merge operation this is where a comparison is happening and this is where this append step is happening so we are comparing and appending so those are the two key operations here and with every comparison there is append so if we simply count the comparisons once again that's happening that should be enough to get the time complexity and what is the number of comparisons that's happening well that's straightforward too if you have two lists nums one and nums two each and the total length of the two list is n so because the size the number of iterations is equal to in the in the worst case it would be equal to the lengths of the two lists combined so you may have to first maybe increment i by one then increment j by one then once again increment i by one and j by one so the total number of iterations here is len of numbers 1 plus len of norms 2 right but remember the merge was called if merge sort was called with a list of size n then merge was called with a list of size n by two and n by two roughly so the total list the total length of nums one plus nums two is actually the overall length n so that's the real trick here that merge the merge operation is an order n operation where n is the number of elements the total number of elements okay so this merge operation takes four plus five nine comparisons and this merge operation takes five comparisons and this merge operation takes three comparisons and so on now this way now if we visualize a problem now as a tree where we're calling merge sort with nl with n elements and that ends up calling merge sort with n by two elements and that ends up calling merge sort with n by four elements all the way down and then we start merging so here when we get to individual elements we are calling merge with literally single elements and as we come up here we are calling merge at this point we are calling merge with elements of size n by eight and n by eight but we are calling merge eight times so now each of these sub problems makes a call to merge and each of these sub problems has the list size n by eight so you have eight calls to merge of size n by eight so the total number of comparisons done is n and at every stage you can check this at the top level uh you are calling merge with n total elements so the total number of comparisons is n at the second level you're calling merge here once with n by two elements and you're calling merge here once with n by two elements so the total number of comparisons is two times n by two that's n and here you're calling merge with n by four elements four times so that's n so if the height of the tree is h then the total number of comparisons is n times h right so on each level you'd require n comparisons for the merge and you call merges at every level for each of these sub problems so the height of the tree is so the total number of comparisons is n times h now how do we get the height of the tree if the height of the tree is h and you can see here that as we go down this is level zero and it has one element this is level one and it has two elements this is level two and it has four sub problems and this is level three and it has eight sub problems so level k has two to the power k sub problems so if you keep going down this is level h minus one so level h minus one should have two to the h minus one sub problems but remember at the last level we simply we simply have sub problems or merge merge calls with single elements so that means we have a total of n elements here or n leaf nodes here so it follows that 2 to the power of h minus 1 is n okay so i'll i'll let you think about that in reason with that now this is something that you may have to work out on pen and paper to get correctly that if the height of the tree is h then 2 to the power h minus 1 is equal to n because at the bottom most layer you have n leaves in the tree so it follows that h is log n plus one so since we said that there are n times h comparisons and h is log n plus one so it follows that the complexity of merge sort is n log n and that's a big improvement from n square it may not seem like much but it is so n square for ten thousand is ten thousand times ten thousand but n log n for ten thousand is ten thousand times 12 or 13 log to the base two so that's about a few hundred times faster now even for an array of a million elements it will only take a few seconds to be sorted and you can verify this by actually creating a list of a million elements okay so the complexity of merge sort is n log n and you get it by drawing this sub problem tree and realizing that there are you get a sub problem tree of height log n or log n plus one and at each step you perform a merge operation or multiple merge operations totaling to n comparisons so so n times log n is the complexity of merge sort now here's also discussion about space complexity and this is something that i believe is an exercise for you so do read through this and see if you can reason why the space complexity of merge sort is order n okay so time complexity is order n log n and the space complexity is order n but here's a hint why it's order n you can see that inside the merge operation we are creating a new list and then we are copying over elements from each of the two lists into the new lists so we are allocating a new list inside merge and now it's so now that's no longer constant that list will have the same size as the size of the problem itself and hence roughly that's why the space complexity is order n okay so with that we conclude our discussion of merge sort it's a divide and conquer algorithm you split the list into half recursively sort both of them then merge the two sorted lists and the initial condition is one or zero elements now there are several extensions and variations of merge sort called the k way merge sort where we split not into two parts but into k parts then we have the counting inversions problem where we modify merge sort a little bit to also find some other info information about the list and finally we have hybrid algorithms which combine merge sort and insertion sort so what they do is for smaller lists they use insertion sort because that's more efficient and then for bigger lists they use merge sort so as they're splitting the list when you get to a small enough problem let's say 10 or less elements they use insertion sort and that brings us to our next question where we make one level of optimization and then we stop but here we will go one step further what we do is we will apply an another technique to overcome the inefficiency in merge sort now the time complexity is pretty good you can actually sort millions or even tens of millions of elements with merge sort quite reliably but it's a space complexity that causes a problem now because merge sort requires allocating additional space and that is additional space is as large as the input itself that makes it somewhat slow in practice because memory allocation is more expensive than computations so doing a comparison is very easy you just tell the cpu to compare two things in the memory or swapping them is also easy because you're still working with memory that you already have but when you have to allocate new memory you often have to then request the operating system uh to allocate the new memory and then you have to get its address and do a whole bunch of operations so it's let's say an order of magnitude more expensive than simply doing some computations so you should try and avoid memory allocations as far as possible now one or two variables is fine but if you're dealing with a million elements so you're probably going to need maybe a few mb of additional space and that is what may slow down your algorithm a little bit it would still be n log n but the constant factor now the cost of each operation will be higher because it involves an allocation now to overcome the inefficiencies the space efficiency of merge sort we will study another divide and conquer based algorithm sorting algorithm and this is called quick sort and quick sort the array in place which means it does not create a copy of the array internally for sorting inside each operation inside each combination operation so let's see how it works it's a pretty interesting a pretty smart trick so here's how it works if the list is empty or has just one element return it it's already sorted straight forward then pick a random element if not pick a random element from the list now this element is called a pivot now there are many strategies for picking a pivot one is to pick a random element one is to maybe pick the first element the last element what we'll do is we will pick the last element but you can easily augment our implementation to pick a random element and then reorder the list and this is the key operation here reorder the list so that all the elements with values less than or equal to the pivot come before the pivot element while all the elements with values greater than the pivot come after the pivot element and this element is called partitioning you're partitioning the array around the pivot so here's an example you let's say we take three as the pivot element the final element now what we want to do is we want to reorder the elements and the way we reorder is by doing swapping and comparison in whatever way we can and that's what we will really focus on the partitioning algorithm and you reorder it in such a way that all the numbers to the left of the pivot are smaller than it and all the numbers to the right of the pivot are larger than it now here's the key observation here once you do that you can tell that all these all these numbers can now be sorted independently and none of the numbers from here will move to the right of pivot and similarly all these numbers can also be sorted independently and none of the numbers here will move to the left of the pivot so the pivot is in the correct position in the final sorted array so it's now in its correct final position and you can simply call quick sort on this half or less than half this portion of the array and this portion of the array and there's no real combination required anymore right so because we're doing it all in place we simply call quick sort on each side of the array and once this gets sorted and this gets sorted recursively then you will have end up with the entire sorted list right and that's what that's how we then continue doing the process recursively now on the left half you once again pick a pivot and then you arrange the elements around the pivot on the right half you once again pick a pivot and arrange the elements around the pivot and so on and so on okay so as i said the key observation here is that after the partition the pivot element is at its right place in the sorted array the two parts of the array can be sorted independently in place now maybe once again take pen and paper and try to work it out yourself again all of this makes a lot more sense when you actually put it down and solve a real problem as a real example so here's an implementation of quick sort and once again we will assume that we already have a helper function called partition which can pick a pivot partition the array and return the position of the pivot element for the next quick sort step okay so this entire process going from here to here this is where we'll assume that we have a function and write the quick sort algorithm and then implement the partition function so here's what quick sort might look like now quicksort takes a bunch of numbers and apart from the numbers it also takes a start index and an end index now why are we doing this remember we want to avoid creating copies of the list that's the whole pro that's the whole thinking here the line of thinking so we will call quick sort not with a sub list which is which is a copy of a portion but we will call quick sort simply by changing the by passing the same original list but by changing the start and end index okay now there is some code here if end is none then we are sent setting end to the length of the list minus one and here's one more thing that we're doing so the final invocation to quicksort that we'll make will be something like this we may call quicksort let's say there are a few numbers here so we may call quick sort on a list something like this and in this case automatically start will have the value 0 and end will have the value none now remember the quick sort is going to sort the array in place but we also said that we don't want to modify our test cases so here's one assumption we are making that if end is none which means if quicksort is called just with the just with the list then we'll create a copy of the list right so we'll just create one copy at the very beginning right when the list is passed for the first time and then we will not create any more copies and you can even skip this line entirely but the only trouble is that we'll start changing our test case input so that's why let's keep it and let's keep a copy but this is only done at the very top level right so only when we start we create a copy so that we're not modifying the input list but never again so that's what we're doing here creating a copy if uh quick sort what was called with the list and setting end to len nums minus 1 which is the final valid index in the list anyway putting this aside this is the real condition here so if start is less than end which means let's say here you have start and here you have end now if start is less than end that means you have two or more elements right if start and end are equal that means you have just one element and if start is greater than n that means you have zero elements really so if start is less than n that means if you have at least two elements then we call the partition function we call the partition function on nums and we say that we want to partition the region start to end so let's say this is the region start to end we want to partition it so we want to pick a pivot and then partition it in such a way that elements to the left of the pivot are smaller than it and elements to the right of the pivot are larger than it for example if you want 4 to be the partition element 4 to be the pivot element then we will partition the array as 3 comma 4 comma 5 comma 23 so that 3 is smaller than 4 and 523 are bigger than 4 and we will return the position of the pivot element okay so now you partition the array and return the position of the pivot element so this is the position we get back and then we can call quick sort on this region and on this region so we can now call quick sort on start to pivot minus one and we can call quick sort on pivot plus one to end okay so now we are passing actually explicitly passing in values for start and end so this will not kick in the next time so no more copies of the list will be created so all the recursive calls will keep modifying in place so all the even the partition call will modify in place and we'll see how partition works in just a moment so partition gets the slice of the original list and it returns the position of the pivot element then we call quick sort on the left slice which is before the partition the element smaller than the partition and then we call quick sort on the right slice which is elements that come after the partition okay now here is how the partition operation works it's pretty straightforward too not that difficult so what we'll do is we will pick the final element as the pivot element but if you don't want to pick the final element you want to pick a randomized element well just pick a random position and move that element to the final position and that's as good as picking the final element now so random pivot simply involves picking an element moving it to the final position but assuming the pivot is in the final position we then keep two pointers left and right now remember we want to create we want to push all the numbers smaller than the pivot to the left and we want to push all the numbers larger than the pivot to the right okay and what we'll do ultimately is we will arrange them in such a way that some of these are smaller than the pivot and some of these are larger than the pivot and then we'll move the pivot between them so we'll see how to do that so you have the left pointer and the right pointer now here's what we do inside partition while these two pointers are far away from each other first we check if the element at the left pointer is smaller than the pivot well if the element at the left pointer is smaller than the pivot which it is you simply advance the left pointer forward so this goes to 5 and then we go back to the next loop now this time once again we check if the element that the left pointer points to is smaller than the pivot 5 is not smaller than 3 5 is greater than 3. so if that is the case then we check if the right pointer is greater than the pivot now if the right pointer is greater than the pivot that means this number is in its right position it's greater than the pivot so we move the right pointer back one space okay so that's the operation we just did now once again we check is the left pointer smaller than the pivot no it's not is the right pointer greater than a pivot no it's not so that means these two numbers are out of place right we ideally would want this to be smaller than the pivot and this to be larger than the pivot so we swap these two elements so now zero comes here and phi comes here now once again we can check is 0 the left pointer smaller than the pivot yes so move the left pointer forward then we is the left pointer smaller than the pivot no 6 is now greater than 3 so we check is the right pointer larger than the pivot yes so we move the right pointer forward because five is still in its correct position it's you know on the on the right edge and everything is greater than three so now once again we end up in this position that the left element is smaller than the is larger than the pivot so we check the right element the right element is smaller than the pivot we want it to be larger so we swap these two because these two are once again out of order and now you can see that one zero two are all smaller than the pivot and six five eleven are all larger than the pivot so we do one final check is two smaller than the pivot yes so we advance the left pointer and now both of the pointers are at the same position so now we can tell at this point that here or from this point position onwards all of these numbers are larger than the pivot so we simply simply swap this element with the pivot so there you go so you end up with one zero two three five eleven and six okay so that's the partition operation so again to understand it yourself do it on pen and paper write out write down write out this array create the pivot create the left pointer right pointer and keep creating copies of the array for each step of the loop okay and that's how you understand these things it's not that difficult it's just it involves two pointers so it's a little tricky now this is the code for partition and i will let you follow this code we'll go over this briefly but at by this point since we are halfway into the course now you should be able to read the code and then there are also comments here and understand what we have just discussed in plain english understand that in terms of code okay so one exercise for you is to explain this visual approach in plain english step by step and then the second exercise for you is to read the code and understand it or maybe even try to write it from memory so just take the english description and try to write the partition function from your memory not memorize the code itself but convert the english text into code okay so once again here you know we have the nums the numbers that need to be partitioned the start and the end and if end is none we simply set end to the last index which is len nums minus one then we initialize the start and end pointers so we initialize the left and right pointers remember we want to use the end element so this is the end element so we want to use the end element as the pivot so the left point the left pointer is start and the right pointer is n minus 1 that's what we that's what we've set here and then white while the right pointer is greater than the left pointer we increment the left pointer if the number at the left pointer is less or equal to the pivot we decrement otherwise we decrement the right pointer if the number on the right pointer is greater than the pivot otherwise the two of them are out of place and they can be swapped so we swap them here and finally we place the pivot in place between the two parts and that's it that's exactly what's happening here so let's see here let's see this partition we are taking this list and we are calling partition on it and 3 is the number that was used as a pivot so now 3 ends up here in between so you have 1 0 2 and 5 11 6. and the partition function returns the position of the pivot so now you can see how it is used in quick sort the partition function returns the position of the pivot and then we call quick sort on the left partition before the pivot and on the right partition after the pivot so now we can test out quick sort okay and here's another exercise for you add print statements inside the partition function so there are already some print statements you can simply uncomment them uncomment the print statements to display the list the left pointer and the right pointer and the beginning at end of every loop to study how partitioning works and similarly you can also add print statements inside the quick sort function to study how the recursive calls are going on so study what we've done in merge and merge sort and add the same print statements in quick and quick sort and look at these recursive calls now what you want is to have a completely clear and perfect idea of what your code is doing you don't want to be lost about it and that's why adding print statements and looking at small examples and making sure that it's working perfectly really helps so let's look at quicksort in action so here's an input and here's the expected output and here's the actual output and they match great and we can now evaluate all the test cases using the evaluate test cases functions for function from jovian so we import from juvenile python dsa evaluate test cases and call evaluate test cases here and you can see that it passes all the test cases and not only that you will also notice that it is marginally faster than merge sort for sorted lists sometimes you may not see that but yeah you can see here that it's you will see that in most cases quicksort is marginally faster than merge sort for larger list and that's because it is not allocating new space okay so now coming to the time complexity for quick sort now assuming that we are able to have a good partition each time so each time we are dividing the list into roughly equal halves roughly equal parts like you start with a list of size n and you partition it into n by two and n by two so this is what the sub problem tree looks like so you call quick sort with two lists of n by two n by two then you call quick sort with four lists of size n by two n by four n by four and so on now what is the activity that we're doing inside quick sort in each quick sort the core operation is partition right and that's what puts one element the pivot element into its right place and then the element smaller than it to the left of it the elements larger than it to the right of it so the partition is where the actual work the comparison and swapping happens and how many comparisons do we perform in the partition i would say that the number of comparisons is equal to the size of the actual list and you can see that here you can see that we are going on comparing numbers like this we are comparing each number to the pivot so each number gets compared to the pivot exactly once roughly and that means that there are a total of n comparisons if n is the size of the list okay so we have n comparison and partition so partition performs n operations or partition is an order n function and what is the height of the tree once again the height of the tree is log n because to go from n to one it takes log n steps you keep going n by two n by four and by eight and so on n by two to the power log n becomes n by n one and so the time complexity of quick sort is n login if you're able to partition the array into roughly equal parts and that is what happens on average if you're picking random pivots each time then you do end up with roughly equal parts maybe it's 75 35 75 25 but that's still more or less in the same range so the quick sort complexity is about n log in and this is called the average case complexity on the other hand if you have a really bad partition and a really bad partition is maybe you picked the smallest element as the pivot now if you pick the smallest element as the pivot then all the elements will go to the right of the pivot and you will end up calling quick sort on a problem of size n minus one and then maybe once again if you pick the smallest element as pivot all the elements will go to the right of the pivot once again and you will end up calling quicksort with a problem size of n minus two now this is an unbalanced t or a skewed tree and what happens in a skewed tree is that the height this time is the same as n you can see n n minus one n minus two n minus three so going up to one the height of three is n but the amount of work involved in partitioning is the same because you have to run through the entire list to partition the list right so that in this case the time complexity is roughly n times n minus 1 by 2. so the time complexity is about order n square and that's bad because that's as bad as bubble sort but despite the quadratic worst case time complexity quicksort is still preferred in many situations now it really depends on what kind of algorithm you need to use and what kind of memory constraints you have because quicksort's complexity is closer to end login in practice especially with a good strategy for picking a pivot and a good strategy is picking a random pivot but there's another one called picking median of medians you can check that out as well so that's n log n is the average time complexity of quick sort and then n square is the worst case time complexity of quick sort now here's an exercise for you verify that quicksort requires order one additional space which means that it does not really need to copy the array we did create a copy because we did not want to affect our test cases but we could have removed that line and quick sort would work just fine so because you do not need to create a copy of the list or the array it requires order one additional space but because space complexity also includes often the size of the space required to store the input so you can say that quicksort has the space complexity of order n okay so if you get the question about space complexity you may want to ask are you talking about the additional space or do you also want to include the input in the space complexity so that's quick sort and those are the two sorting algorithms we've looked at so we've looked at we've looked at bubble sort and we've looked at insertion sort and then we optimized it using divide and conquer and got to merge sort which is order n log n but it also has a space complexity of or the additional space requirement of order n which can be avoided using quick sort which uses order one additional space but can have order n square complexity in the worst case time complexity but with the right choice of a pivot it is closer to n log n so that's sorting and you can see that python is such an expressive language that all these sorting algorithms which are often quite confusing to implement in c plus or java are actually pretty straight forward to implement in python all you need to do is follow the method which is to state it first in plain english have some test cases ready to test your function and then write your code carefully checking each line for errors and create small functions wherever you need to so try not to have too much logic in one function a good rule of thumb is about seven to eight lines of code per function no bigger than that and that's not just for toy problems but that's also even as a software developer something that you can try to follow just have seven eight lines of code in any function if you have more than that try to split it into two functions okay and and this way it's very difficult for you to go wrong so now let's return to our original problem statement and let's read it once again you're working on a new feature on jovian called top notebook of the week or top notebooks of the week and write a function to sort a list of notebooks in decreasing order of likes now keep in mind that up to millions of notebooks can be created every week you want to build this for scale so your function needs to be as efficient as possible so first we need to sort objects this time and not just numbers and second we also want to sort them in the decreasing order of likes for each notebook okay so all we need to do is to use our merge sorter quick sort techniques that we've already discussed is to define a com custom comparison function to compare two notebooks okay but before we do that we let's create a class that can capture some basic information about notebooks so here we have the class so we're still following the method so to speak right the step one was to come up with the input and the output format so here is the input format our input format would be using this class so we create creating a class notebook which is titled username and likes so you create the class and that gets stored as properties titles username and likes and then we also have a string representation here then create some test cases so now we are creating some test cases here so we are creating some test cases in nb0 to nb9 and let's put them all into a list and you can see here that we now have a list of notebooks nb 0 to nb9 and you can see that because we have a string representation we can see that the first notebook is this our caution slash pytorch basics and it has 373 likes and the second one is this and it has 532 likes and these are clearly out of order in terms of likes next we will define a custom comparison function for comparing the two notebooks what it will do is it will return the strings lesser equal or greater to establish the output order between the two objects okay so it should return lesser when nb1 should come at a position or a index lesser than the position of nb2 in a sorted list okay so in in in case of our problem what that means is we want to sort things in the decreasing order of likes so the first notebook should have the highest number of likes and then maybe the second notebook should have the second highest number of likes and the third notebook will have a lower number of likes and so on so if you have two notebooks nb1 and nb2 and if nb1.likes is greater than nb2.likes so then nb1 should come at a lesser index okay so we will return lesser because it should come at a lower position in the sorted list so we return lesser because we want a decreasing order and if nb1 dot likes is equal to nv two dot likes then we return equal and if nb one dot likes is less than nb two dot likes so that means this is not uh is nb two is the more like notebook and bit one is the less like notebook then nv1 should actually come at a greater position so we will return greater okay so this is this comparison function should return whether the first input to it should come up should show up at a lesser position in the sorted list compared to the second input now in languages like c plus plus and java normally the convention is to return a negative number zero or positive number but i find that python allows you to return strings strings are first-class citizens in python and it's a lot clearer when you are debugging things when you face issues to look at actual strings and it's also easier to write write code so i prefer using strings but you can also use you can also use numbers like negative zero or positive that's totally up to you so now here is an implementation of merge sort which accepts a custom comparison function so let's see the merge sort function so the merge sort function uses it takes a list of objects this time not a list of numbers and it also takes a compare function which by default we also provide a default comparison so that we can still use it with numbers now with numbers and default assumption is if you want sorting you want sorting in increasing order so this is what the default sorting looks like for numbers so that's pretty straightforward but you can also pass a custom comparison function so here we have the terminating condition if the length is less than 2 then we simply return the list then we get the mid index and then we call merge sort on the left half with the custom comparison function we call merge sort on the right half with the custom comparison function and we call merge with the custom comparison function now what happens inside merge inside merge earlier you know once again we have these two halves left and right and then we have a custom comparison function so we create pointers for the two of them and then we also create the final result list which is merged and then we iterate over the left list and the right list so while we are going through these we compare the left element and the right element so now we're calling compare now we're not doing the greater than less than comparison calling compare and if the result if the element on the left is lesser or equal to the element on the right then we append it to the result array and we increment the left counter otherwise so lesser or equal means that the element on the left the first element on the left should show up at a lower position in the sorted final sorted list so that's why we append it first otherwise we append the right child to the right element and we increment the right pointer and finally we attach any remaining elements here so this is something that you can review something we've covered in a lot of detail so now let's see let's call merge sort on our notebooks and let's check if the notebooks are sorted by likes and indeed they are you can see that at position 0 you have the notebook with the highest number of likes and then you have the next one and the next one and so on now since we have written a generic merge sort function that works with any compare function we can now very quickly use it to sort the notebooks by title as well or if we had maybe the number of views per notebook or the number of versions in each notebook or the number of comments on each notebook we could do that sorting as well so we could even use a hybrid of those so here the example we're taking is comparing by titles so here we have nb1 and nb2 and simple comparison strings can also be compared using the comparison operators so if nb1 dot title is less than nb2.title then we return lesser otherwise we return equal or greater and with this we should be able to sort them in the ascending order of titles you can see a n c i c i f e l i l o p y p y okay p y t h p y t h p y t o by torch okay so this is an order sorted in the order of titles an exercise for you is to sort in the order of username slash title which means you first compare the username and if the usernames are equal then compare the titles so you can compare you can probably write another comparison function compare username and titles and use that to do that two-level comparison and use that for sorting okay now another exercise for you going forward is to implement and test the generic versions of bubble sort insertion sort and quick sort using these empty cells that are given here right so at this point in the course you should start writing code you should be writing maybe solving one problem every day to really practice the concepts and internalize them and while you're doing that you can also any problem that you work on any notebook that you create you can save it to joven.commit and i'll show you also how to create new notebooks so one way to create new notebooks is to go to jovian joven.ei click the new button and click blank notebook and you can give it a title let's say you are doing quick sort generic and you can set up privacy and create a notebook and that creates a notebook for you and then you can click the run button and run it so that's one way to do it another way you can do it is we've given you a problem solving template so if you come back to the lesson page you will find a problem solving template here now you can click on the problem solving template and click duplicate to create a copy of this notebook in your profile so let's do that and now this is on your profile so you can now click run and then run it on binder or you can even run it locally on your computer and make some changes to it and come back and run jovian.commit and you will end up with a link that you can share so now you can now go on twitter and you can just share this link so write out a tweet and tag us and also use the hashtag 60 days of python okay and maybe say this is your quick sort algorithm for generic objects and tweet it out and we will retweet your tweet so we want to support everybody who's taking part in this course on the course page you will find a link to the course community forum which is where you can go and ask questions where if you have questions about any of these and you can even discuss some of the ideas that are discussed here some of the exercises that are shared so you can go into lesson three for instance and create a new topic maybe you want to talk about the generic implementation of quick sort so maybe you can create a new topic and post a query there if you're not able to make it work post your notebook there and ask a question have a discussion and if you are helping other people out if you are answering other people's questions and you've written some really great posts there are links to some more problems that have been shared here so you can check out these links on each of these links you can try out problems you can make submissions you can solve these problems some of these are interview questions as well you can check if your results are correct and you can use the solving problem solving template as a starting point as we've just shared so there is a starter notebook with each assignment and in the assignment all you need to do is run the notebook so you can run it on binder for instance and then there is a question mark in a bunch of places you will find like question marks here in the text and you will find question marks here in the code so you simply need to put in your code your answers into the question marks so replace that with your code you can see here there are some question marks here so you replace that and step by step there are instructions to guide you there is there are comments to guide you so step by step you can solve it and then finally you can also make a submission so right at the very end when you run the code you will also be able to submit directly and when you make a submission then the assignment will get automated will get evaluated in an automated fashion instantly and you will get a pass or a fail grade now if you get a pass grade that's great but if you get a fail grade then you will also get some comments about what went wrong in your solution so you can use those comments to fix the issues so it's a great way to get quick feedback and keep fixing your issues uh especially watch out for edge cases so that's assignment one and then assignment two is called hash tables and python dictionaries a very interesting assignment where you are going to implement hash tables which power python dictionaries from scratch in python and you will also replicate the interface of python dictionaries so do check it out a very interesting assignment again very similar format you will find question marks in certain places you need to replace them with appropriate values expressions or statements and in this way by working through each of these step by step you can see here by working through each of these you will implement hash functions and hash tables which again are very commonly asked in interviews as well so this is an important assignment for from an interview preparation or coding assessment preparation as well and it also teaches you a lot of really good practices in python programming in particular so do check out assignment 2 as well and we will send you an email as soon as assignment 3 is ready but you can check back in a couple of days and you should see it on the same page pythondsa.com so what you do next review the lecture video and execute the jupyter notebook use the interactive nature of jupiter to experiment with the code complete the assignment and attempt the optional questions as well so each assignment has some required questions and you can make a submission as soon as you're done with the required questions but there are some optional questions which are slightly harder but i highly recommend doing that because they will improve your understanding give you more practice help you internalize the concepts better and then participate in forum discussions and join or start a study group so this is a great way to learn get together with some friends maybe watch the lecture together over a zoom call pause the video have discussions wherever you have doubts discussion is a great way to solve the specific doubts that you may have and it will also help you to articulate your understanding better because you when you explain to others you also answer a lot of your own questions so please do that this is data structures and algorithms in python thank you and good day or good night hello and welcome to data structures and algorithms in python this is a live online certification course being organized by jovian today we are on lesson four recursion memoization and dynamic programming my name is akash and i am your instructor you can find me on twitter on at akashness if you follow along with this course and complete the weekly assignments you can also earn a certificate of accomplishment which you can add to your linkedin profile and you will find hosted on your jovian profile as well so let's get started now to the data structures and algorithms course this is pythondsa.com is the course website and on the course website you will be able to find all the information about the course so you can view the previous lessons lessons one two and three and you can also view the previous assignments assignments one and two today we're on lesson four so let's open up lesson four the topic is recursion and dynamic programming now you can find a recording of the lesson here and you can also watch a version in hindi if you would prefer that in this lecture we will cover recursion memoization and dynamic programming by looking at two common problems in dynamic programming the longest common subsequence problem and then knapsack problem and we'll do this by coding these problems live using the problem solving template that we have been using one in one way or another since lesson one so let's open up the problem solving template this is a template that you can use to solve any coding problem and we will illustrate this by solving two problems using this template today so the first thing we need to do is to run this template you can see that there is some explanation and then there is some code here as well now to run this code you have two options you can run it using free online resources or you can run it on your computer the simplest way to run it is click the run button here and select run on binder and with just one click this will set up a machine on the cloud for you start a jupyter notebook server and you will be able to then execute the code and modify the notebook and save a version of it to your own profile so that you can continue working on it so there we have it now we have a running jupiter hub server i'm just going to zoom in here a bit so that you can see things clearly okay so this is the problem solving template and i said we're working on two problems so i have some problem statements listed out here you can see the first problem longest common subsequence is listed here and this is a part of the lesson notebook lesson page as well so you will find link to this problem statement on the lesson page too so let's first modify the title of this notebook problem solving template let's change this title to dynamic programming longest common subsequence let's get rid of this i don't think we need this then i'm going to keep the section on how to run your code so that if i share this notebook with somebody else they have a way to run it and then before we start the assignment or the problem let's just save this to our own profile so i'm just going to give it a name longest common subsequence this is an appropriate name for it so i'm going to give it this a project name install the jovian python library and just run jovian.commit now what this will do is we started out with a template and now we are editing the template by running joven.com we've saved a copy of the template to our own profile you can see this is the link where you will be able to access this notebook and you can run it and continue your work if this jupyter notebook shuts down if you want to continue tomorrow for instance okay so now let's look at the problem statement now i'll just copy over the problem statement here as well so that we can see it directly within the notebook there we have it now you can paste the problem statement and if you are getting this problem statement from some other source then it's always a good idea to include the link to the original source as well okay now we have a problem statement in front of us so the question is write a function to find the length of the longest common subsequence so that's a new term we'll unpack that between two sequences now let's first learn what we mean by a sequence now a sequence is a group of items with a deterministic ordering for instance a list a tuple a range or even a string these are some common sequence types in python so here i have the string serendipitous this is a group of items and this also contains an order you can see that e comes after s and r comes after e and so on so this is a sequence a list would also be a sequence so that would be a list of numbers so that's a sequence then we're looking at subsequence what is the subsequence now a subsequence is a sequence that is obtained by deleting or removing zero or more elements from another sequence for instance if you look at serendipitous and if we remove the characters s r e n i i o u s then you will be left with e d p t so e d p t is a subsequence of serendipitous now two things to note here edpt does not have to occur continuously so these elements can occur anywhere within the sequence but the order should be the same so e d p t occur in this particular order here and e d p t should occur in the same order here so d should occur after e and p should occur after d and t should occur after p so those are the two requirements for edpt to be a subsequence of serendipitous and visually speaking what we can see is if you take a sequence and then you draw boxes around some of these characters or some of these elements of the sequence and if you just take the elements in the boxes then in the same order then you end up with a subsequence so now we understand what a sequence is and what are subsequences and once again if this is this question is asked in an interview and you're not sure what you mean by a longest common subsequence and or even what a sequence is then you should ask the interviewer what do you mean by a subsequence or what do you mean by a sequence and they'll be more than happy to tell you it's very important to communicate whatever you're thinking whatever questions you have contrary to what you might think asking questions is actually a good thing the more questions you ask the more it is appreciated okay so now we've talked about a sequence and a subsequence now what's a common subsequence so look at these two strings serendipitous and precipitation now if we pick just these elements that are in the boxes r e i p i t o now you can see that rei pito is a subsequence of serendipitous and r-e-i-p-i-t-o is also a subsequence of precipitation so a sub-sequence which is common which is a subsequence of both sequences is called a common subsequence so rei p-i-t-o is a common subsequence between serendipitous and precipitation now you can have many common subsequences for instance we could just look at re and re here and re would be a common subsequence too or you could just look at i t and i t and that would be a common subsequence as well or we've not picked n here but you could also pick r e n and r e n and that would also be a common subsequence between the two now the longest common subsequence as the name suggests is the subsequence which between the the common subsequence between the two sequences which has the maximum possible length and you can verify this you can try different subsequences and see that rei p i t o is the longest common subsequence between these two strings these two sequences and its length is seven one two three four five six seven so you have to write a function to find the length of the longest common subsequence between two sequences so that's a question and this is a visual example that tells you the answer okay so now that we have the question we've understood the question we can start applying the method that we have been learning throughout so this is the systematic strategy that we will apply and nothing about this method has changed since the first lesson even though we've covered a whole variety of topics like binary search and binary search trees and then sorting algorithms and divide and conquer this method has remained the same the first step is to state the problem clearly and identify the input and output formats then the second step is to come up with some example inputs and outputs and these will be used to test our solutions so we should try and cover all the edge cases and that will help us write code that is correct anticipating all the errors that we might face then we come up with a correct solution to the problem as stated in plain english very important for you to state the problem in plain english before you start coding so that you communicate your ideas and you also make it clear once you express yourself then you implement the solution and test it using example inputs and you fix bugs if you find any of them and you will be able to find bugs if you have written good test cases then you analyze the algorithms complexity and identify inefficiencies if you have any and most likely the first solution that you come up with it doesn't have to be optimal it just has to be correct so there will be some inefficiency but it's important to go through that process of first finding a brute force solution and then finding the inefficiency and then apply the right technique to overcome the inefficiency and repeat steps to three to six so you identify what's the right technique and in this case we will learn a couple of techniques called memoization and dynamic programming and then we go back and state the correct solution again then we implement the solution and test it and then we analyze it again and if there's further scope for improvement we do that otherwise we say that we've arrived at a optimal or good enough optimal enough solution okay i hope by this point this you've started to memorize this process and that's why we keep repeating it over and over that it should become second nature every time you see a problem so the first thing is to state the problem clearly and to identify the input and output formats now the problem is already stated clearly enough but let's just state it slightly more clearly so let's say we are given and just write it in your own words that's more important whatever is clear to you so we are given two sequences and we need to find the length of the longest common subsequence between them simple enough then we have two inputs now we decide the input and output formats we have sequence one a sequence example serendipitous sequence two another sequence example precipitation great and this these are the only two inputs that we require and the output would be the length of the longest common subsequence let's just abbreviate that as lcs which in this case is seven and we know what that subsequence looks like we've just seen it above so now based on this we can now create and you can see the problem is now created and before i talk about the next thing you if you double click on a text cell you can start editing it and here we are using a language called markdown so you can see this creates a block code this creates a bold font and this creates a code like font so let's see here no and the way to go back into the display mode is to press shift plus enter so now you can see here that now we have the problem we have the block code and then we have all the styling so markdown is a really useful and easy to learn language for formatting your text especially in jupyter notebook so do learn it but now based on this we can now create a signature of our function so our function len lcs will accept a sequence sequence 1 and sequence two and it will return something okay so that's the basic signature of our function and even though it's not doing much just establishing what the arguments are is the first step towards solving a problem and let's just save our work from time to time it's very important to keep saving your work on jovian because this is running on a free online service so this will shut down after some minutes of inactivity so just run jovian.commit and that will save the notebook to your profile and you can rerun it okay so now the next step is to come up with some example inputs and outputs and here we need to try and cover all the edge cases so i have written out a few test cases here already now the most common case is a general case of a string like we had serendipitous and precipitation that's a common case there is one of them both of them have some common elements and there's a subsequence common subsequence of length 7 but we may also want to test out another type of data and this is one of the nice things about python where you can write functions that operate not just on a particular class and its subclasses but on any kind of data as long as it satisfies certain criteria for instance strings and lists both allow indexing into them and picking out the ith element or the nth element from the sequence so they're both sequences so our function should be able to work with both strings and with lists then here is another case where we have two sequences and they have no common subsequence a function should not throw an error here it should gracefully return the number 0 because the empty sequence is a subsequence of every other sequence does that make sense think about it so in that case if you if there's no common subsequence then the empty sequence is the common subsequence so the answer is zero and here's one other extreme case where one is a subsequence of the other here's another case where one sequence is empty there's another case where both sequences are empty all of these are important otherwise you might miss out certain special cases and you will face an error when you code your solution finally you can also have this case where you have multiple subsequences with the same length for instance if you have a b c d e f and b a d c f e an a c e a c e is one long subsequence of length three and that's the longest you can verify and bdf is another subsequence which is common to the two and also has the same length those are some test cases now let's copy over these test cases here in an interview or a coding assessment what you might want to do is just write these as comments if you have just a single coding screen and try to list at least four or five if but go as far as you can because this will also help you streamline your own solution and it's always something that is appreciated by interviewers let's do that let's get let's copy over these test cases here and you can think of more so if you have some more ideas of things you should test come up with them there's no right number of tests whatever it takes for you to feel confident is what you need to do okay so now what we've done is we've taken these test cases and converted them into dictionaries so you can see here we have this first sequence sequence one and remember that's why we've written out that's why we've written out here the names of the inputs and the signature of the function now we can create test cases as dictionary so that we can test them all easily all at once so we have the sequence one and sequence two in the input sub dictionary inside the main test case dictionary and then we have the output which is the output of the function which should be seven and this you can verify so this is a general case then we have another case in this case we have two sequences these are both lists of numbers and in this case the output that we expect is 5 and we have another general case longest and stone in this case you can verify that o n e is the common subsequence it has the output three then here we have two sequences which do not have any common elements all these come from the left half of the keyboard all these come from the right half of the keyboard so that was a quick way to generate these two sequences then here we have dense and condensed and you can see that dense is actually a piece inside condensed so this is a special case where dense is a continuous sub string of the string but it even if we had d e s e that would still be a subsequence because d e s e occur in this order so that's one example and in this case the sequence one is itself the longest common subsequence and it has length five then we have this case where one of the sequences is empty and you can see in that case the output should be zero and both sequences are empty and here is the case where you can have multiple longest common subsequences and even in this case your function should be able to figure out the answer correctly so let's take this and let us copy over these test cases here so we have t 0 to p 7 so that's 8 test cases and you can add more test cases here please feel free coming up with good test cases is a skill that you should develop and what we'll do is we'll also put all these test cases into this function called lcs or longest common subsequent tests so that we have all of them easily available for testing at once okay okay now next step is to come up with a correct solution for the problem now we've seen the problem we have identified some scenarios now we need to come up with a simple correct solution stated in plain english it doesn't have to be efficient it just has to be correct so here's one idea here's one idea here you can see we have a couple of sequences let's create two counters idx1 and idx2 both starting at 0. so idx 1 will be a pointer which will start tracking elements on in the first sequence and idx 2 will be a pointer which will start tracking elements in the second sequence and what we'll do is we will write a recursive function so we'll write a recursive function which will compute the lcs of sequence one from idx to the idx1 to the end and sequence 2 from idx 2 to the end so what does that mean let's say idx 1 has the value 3 and idx 2 has the value 1. so you can see 0 one two three so sequence one idx one onwards is logy and sequence two idx uh idx2 onwards is lch m e m y so we are looking at this portion of the problem and this portion of the problem and our recursive function when invoked with idx 1 and idx2 should return the length of the longest common subsequence between these two portions so l o g y and l c h e m y now why are we doing this we need this longest common subsequence for the entire string don't we now here's the logic why why we're writing this recursive function which which can theoretically compute this subsequence for from any position onwards so here's how we do this if sequence 1 of idx1 so if idx1 was pointing to l and idx2 was pointing to l here as well if sequence 1 of idx 1 and sequence 2 of idx2 are equal then this character l belongs to the lcs of this portion and this portion okay why think about it it makes sense because these these elements are equal so if you pick the longest common subsequence of this and you pick the longest common subsequence of the remaining then you can always add l to bo to that subsequence and that will make the subsequence longer right and that way it follows that l will always occur in the longest common subsequence between l o g y and l c h e m y okay so we know now that this will occur l will occur in the longest common subsequence further the length of this longest the length of this longest common subsequence will be the length of the longest common subsequence between ogy and ch emy plus one okay and now you can see why a recursion is required because what we can now do is we can say that if sequence one of idx 1 and sequence 2 of idx 2 are equal then we simply call the recursive function on sequence 1 of idx 1 plus 1. so ogy and sequence 2 of idx 2 plus 1 ch emy and assume that recursion will give us the solution there and simply add 1 to it because this is equal okay so that's one case if sequence 1 of idx 1 and sequence 1 of idx 2 are equal great but if they're not equal right so for ins in this case for instance you can see that if i dx1 and idx 2 are both 0 so idx 1 points to a and idx 2 points to b so if they are not equal then one of the two things should hold either a does not occur in the longest common sub sequence between the two strings or b does not occur in the longest common subsequence between the two strings now we don't know which one but that's the power of recursion that we can just try both so we can simply ignore a and we can get the longest common subsequence between b s e and t and b est and check its length and then we or we can simply ignore b and we can get the longest common subsequence between absc and est and check the length now whichever is longer in length that becomes the solution for the two strings okay so this is what it looks like we start out with analogy and alchemy we compare a and a these two are equal so we know that the longest common subsequence is one the length is one plus lcs of analogy and alchemy okay now we compare n and l and now we see that they are not equal so either n does not come in the longest common subsequence or l does not come in the longest common subsequence so we try both we remove n here you see a l o g y and we remove l here we see c h e m now once again a and l are unequal so either a does not occur in the lcs of these two strings or l does not occur in the lcs of these two strings so if a doesn't occur in the lcs we can remove a and try again if l does not occur in the lcs we can remove l and try again and here once again we get a match so in this case we know that l occurs in the longest common subsequence of these two elements so now we can get the lcs of ogy and ch emy okay and then you know as these recursive calls complete you can see that this entire tree pans out you can see that each time you either get one child or you get two children and if you go all the way down and then you go back up and simply count the number of matches for each path you will kee and you take keep taking the maximum so here you get back an answer let's say you get back an answer of size two here you get back an answer of size one so the answer for this is simply the maximum of two and one which is two and then the answer for this is simply the maximum of two and let's say this is three then three and the answer for this is simply one plus three four okay so this is the way that we will build up the solution so we've now looked at the recursive solution expressed in text and we've looked at the recursive solution expressed as a tree now it's possible that it still may not make sense to you how exactly this is working and that is where you should start trying to create this tree yourself so pick up a pen and paper and then start drawing on pen and paper take an example and try to read each step here and try to work it out like a computer okay and just thinking about it that way will help you understand this algorithm now one last thing is that if either of the sequence one from idx onwards or sequence two from idx onwards is empty which means the index has reached the end point in after doing some recursion then their lcs is empty so the length is zero okay so that is the recursive solution here i will just copy over this recursive solution too along with the entire tree now obviously in an interview you do not need to write all of this in in a lot of detail or you do not need to so it helps to show a diagram sometimes but you don't really need to do all of this all you need to do is express yourself clearly that we will create two counters and the condition to check is whether these two elements at those counter positions are equal what do we do if they are equal what do we do if they are in equal and why are we using recursion here so we are using recursion we can because we can use reuse some of the sub problems to compute the final problem okay and understanding recursion is really important for solving data structures and algorithms problems because it's like a superpower pretty much pretty much every problem that you see one way or another can does boil down to recursion in one way okay so now let's save our work once again and now we're ready to implement the solution so we have the recursive solution in front of us and if you remember the four steps let's go let's go ahead and implement it so we see let's just call it lcs recursive and this will accept a sequence one and a sequence two and let's also initialize idx1 and idx2 because we will be calling this function recursively so we'll simply use these two counters idx 1 and idx 2 and set them to 0. now the first thing we need is if idx 1 is equal to the length of sequence 1 or idx2 is equal to the length of sequence 2. then we return zero again this is a common thing that happens that the base case or the end scenario is something when you're describing the algorithm you will describe at the very end as you're drawing the tree you will notice what the end case and scenario is but when you're coding the algorithm the end scenario or the base case comes at the very top because otherwise we'll try and access idx 1 from sequence 1 and that will throw an error so that's why you need to handle the base case at the very beginning okay next moving ahead if sequence one of idx one equals sequence 2 of idx2 great we found a match we simply return 1 plus now we can call lcs recursive on sequence one sequence two and we increment idx one by one and we also increment idx two by one both of these need to be incremented because we are going to use this element this common element as an element in the subsequence okay so there's just one recursive call here that was nice otherwise we have to either ignore the first element of or the current element from sequence 1 or the current element from sequence 2. so we have two options so we have option one which is we ignore the current element of sequence one so this becomes lcs recursive sequence one sequence two idx one plus one and idx2 and then we have option two this is lcs recursive once again with sequence one and sequence two and this time we increment idx2 okay so make sure you understand this piece because this is really the key here and then the length of the longest common subsequence is simply the maximum of option one and option two okay and that's it what may have seemed like a fairly tricky problem once you start thinking about it recursively okay what happens if we simply compare the first two and they're equal and they're unequal okay now we need to solve the problem for the remaining um either we add one or we take or we ignore one of the elements right once you get that thought the recursive thought then the solution and the code simply presents itself to you it's just about seven lines of code okay that's our lcs recursive solution now let's test it out let's look at a test case t0 okay so here we have serendipitous and precipitation as the inputs let's call lcs let's keep that around so that we can view it later let's cost call lcs recursive on t0 but of course we need to fetch from t0 the input and get sequence 1 out of the input and similarly we need to get the input and get sequence 2 out of the input you can see it it takes it returns the value 7 which is equal to the output by the way so if we simply put in here t0 output and i'm also going to put in this special command called percentage percentage time this is going to tell us how long the cell takes to execute yeah so now you can see here that we get back true and the cell takes 495 seconds or half a second to execute and that's it so now we have tested this test case one small thing i can tell you how to improve this slightly is because in t 0 of input is a dictionary and because the names of the elements of the dictionary are sequence 1 and sequence 2 which also match the argument names of lcs recursive you can see here we have sequence 1 and sequence 2. what you can do is you can simply say star star t0 input and python will automatically grab each key so sequence one will be passed as the argument sequence one and sequence two will be passed as the argument sequence two that's this is a small trick here that helps us speed up the reduce the amount of code we need to write okay now we've tested one test case but that's not enough we should be testing all the test cases so to test all the cases we can write a for loop for t in tests etc but we can do something else too we can use the evaluate test cases function from jovian so from jovian.python dsa the module we will import evaluate test cases it's a helper function that we've created for you but it's really simple to write you can just use a for loop as well and we call evaluate test cases on the function that we want to test which is lcs recursive and the test that we have which is lcs tests and when we do this it is going to try out each test case you can see it strike test k0 that was a pass it tried test case 1 and it's also printing out the input the expected output in the actual output the test case 1 was lists and lists work too because all we have used here is indexing and length and these are both things that are available in both strings and lists and this is something that's very nice about python the dynamic nature of the functions uh once again this worked perfectly fine then here we have another one longest in stone the expected output was three and the actual output was three as well here we have ads f e w ad and another string they have nothing in common so the expected and actual output are both zero here's one where one is the is already a subsequence of another so the smaller one becomes the longest common subsequence and then we have an empty string and then we have two empty strings and finally we have multiple longest common subsequences we still get back the right output now if any of these failed you would know exactly what went wrong for instance if you had an issue in this case where the two of these were empty then that would tell you that you've probably not handled that empty case properly and that is why having great test cases is very important okay and we can see the timings for these as well each of these took about well 480 milliseconds was the highest now that's still a bit high i would say 480 milliseconds because we are just looking at sequences serendipitous and precipitation which are of very short length if you're looking at a really long sequence for instance this technique is used for dna sequencing and we were looking at two dna strands or two in two dna strings and trying to get the common subsequence out of them and these can go into thousands or sometimes millions of elements that would make it rather slow okay so we do want to improve this algorithm further so let's do that and before that we can just commit our work once again but the first thing before we improve the algorithm is to analyze its complexity how long does it really take okay and identify any inefficiencies now to analyze the complexity let's look at an example and let's consider the worst case now when does the worst case occur here we've seen that if two elements match then we simply have one sub problem or one recursive call but if the two elements or two elements of the sequences don't match then we have two recursive calls so if you have two completely distinct sequences where none of the sequences none of the elements match then each time we will end up with two sub problems so that becomes the worst case so the worst case occurs each time we have two sub problems where the sequences have no common elements and here's an example this is a sequence of length six here's a sequence of length eight and this is what the tree will look like so now we have no longer put the actual sequences we've simply put what is the length of the string that we started with so here we start out with strings of length six and eight and then we say that we either ignore the first character of the first string or the first sequence or we ignore the first element of the second sequence and that gives us two sub problems and this time this the sequences have length five and eight in this case and six and seven in this case okay so we either reduce one from the left or we reduce one from the right and once again here we either reduce one from the left or we reduce one from the right so this way we create a tree and you can also see that a lot of common trees get created and that really is what is the inefficiency and we'll talk about that but what will happen here is five seven will then call four seven five six and five seven here will once again call four seven and five six and four seven and four seven will get repeated here and five six and five six will get repeated three times here so there's a lot of repeated calls that are going to occur and you can even see this here at the top you can see that alogy the problem was called repeatedly so that's really a source of inefficiency but now the question becomes that we know that all the leaf nodes will end at zero zero that's when the entire tree ends so can you count the number of leaf nodes okay can you count the num if you keep expanding this tree completely expand each of these don't skip any of them can you count the number of leaf nodes now if you count the number of leaf nodes we know that in a binary tree the number of leaf nodes if the number of leap nodes is l um then the height of the tree is if the number of leaf nodes is n then height of three is log n and based on that we can actually determine the actual size of the tree as well so we know that to count the number of unique paths from root to leaf will give us the number of leaves right so each time we have two choices we either reduce from the left or we reduce from the right so to get to 0 0 we would have to reduce all the elements from the left and we would have to reduce all the elements from the right that means if you have strings or if you have strings of length or sequences of length m and n then you would have to make m plus n choices in total and you so each time you have m you have to make m plus n choices and each time you have to choose whether you want to reduce from the left or from the right you have two choices and you have to make those two choices m plus n times that's the right way to put it really so that means each time you com you do two choices so you have two multiplied by two multiplied by two multiplied by two and you keep multiplying that and you will end up with two to the power of m plus n leaf nodes okay so here is an exercise for you draw this tree on a piece of paper mark out how the number of leaf nodes how the length of each part is m plus n figure that out and based on that can you conclude that it takes 2 to the power of m plus n leaves to complete this tree and if 2 to the power m of m plus n is the number of leaves then the total number of elements is in the tree simply double of that once again this is something that is very easy to verify you can check it here for instance if you just consider these two levels the if you have two leaves then the total number of elements in the tree is two plus one three actually it's double minus one so two into two four minus one three if you have three levels you can see here that if you have four leaves then the total number of elements in the tree is four into two eight minus one seven and you can see that here so it follows essentially that we have an exponential number of sub problems we have we are calling the recursive function an exponential number of times and inside the recursive function we are doing inside the recursive function we are doing a constant time work so you can see here that there's no there's no special work that we're doing all we're doing is some comparison and we're doing an addition both of them are constant time so we make 2 to the power of m plus n recursive calls inside each we do constant work so the time complexity is order of 2 to the power of m plus n okay that's a rough explanation we've not gone into a lot of depth because we've covered this over and over in three lessons but the exercise for you to is to verify how exactly it is 2 to the power of m plus n okay so that's our recursive solution and we know we now know that the time complexity is 2 to the power of m plus n let's just copy that over here and the inefficiency as we said in this algorithm is that we are calling the same problem we're calling the exact same problem the lcs recursive function is called with idx equal idx 1 equal to 5 and idx 2 equal to 7 and at x 1 equal to 5 and at x 2 equal to 7 the same time twice so each of these sub problems will be called twice and then each of the sub problems within them will be called twice and of course some of these sub problems will once again get shared so there's a lot of repetition now there's a simple solution here which is simply to remember some of these results okay and this technique is called memoization and you may also just call it memorization because you're just remembering some of these things but memoization is a technical term for it and we remember these solutions in our dictionary called memo so what we are going to do is we are going to follow the same recursive strategy but this time we are going to maintain a dictionary called memo and we're going to track intermediate results within the dictionary and if we find an intermediate result already exists in the dictionary then we will not compute it again okay so let's see so now we write lcs memoized or let's just say lcs memo for short it takes a sequence one and it takes a sequence two and this time we create this dictionary called memo and then we write a function inside it so we will write a helper function a recursive helper function inside the lcs memo function so that it has access to sequence 1 and sequence 2 and we will simply started out with idx 1 as 0 and idx 2 as 0 as well right x1 will track the position in sequence one idx2 will track the position in sequence two now the first thing we do is create using the two indices create a key so we are going to create the key idx one comma idx2 and if the key is present in the memo so this is the way to check if a key exists in a dictionary then we simply return memo of key simple the problem is solved we don't have to solve this problem because it's already it's already something that we've solved if it isn't then we need to solve the problem and save it in the memo now here we know that we can now write our same three recursive cases now if the base case if idx 1 is equal to the length of sequence 1 or idx 2 is equal to the length of sequence 2 then we simply set memo of key as 0 because by this point we have reached the end of the strings there's nothing left for us to compare lf idx sequence 1 of idx 1 equals sequence 2 of idx 2. so in this case this is the case where the current characters are equal so this is if we go go up here and look at the tree once again this is a case like this where the current characters that we are pointing at are equal so in that case we simply return we simply get the result as one plus the result for the remaining with the first character removed so in this case we simply set memo of key to 1 plus we call the recursive function again recurs idx one plus one and idx two plus one great else so this is the case where the two elements are not equal so and this is where we have two options i'm not going to write the two options separately let's just do a max directly here max and we say recurs with idx 1 plus 1 comma idx2 and recurse with idx 1 comma idx 2 plus 1 okay and finally from the recurse function we return memo of key so we have whichever case it is we have computed the result and saved it in the memo so this time these computations will not get repeated again and again and let us now return recurs of zero comma zero because zero comma zero is the entire string and that's it and this is the common strategy that you should apply whenever you come up with a recursive solution and you see the inefficiency coming because of the same problem being called again and again and again this is where you need to apply this technique called memoization right and in this technique you will then be able to simply store intermediate results so it's really simple you just create a dictionary and then you add one or two lines of code here and you make sure to save the result in that dictionary whenever you compute a result the next time you don't have to compute it okay and we can test it out we can test it out with all the test cases evaluate the test cases so lcs memo and lcs tests and you can see that all the test cases pass now not only do all the test cases pass you can see that the time taken is now lower okay so that's nice the time taken is now lower now we went from 450 milliseconds if we just go up here you can see that it took 480 milliseconds for the for finding the longest common subsequence between precipitation and serendipitous but in this case it only took about 0.234 which is 0.2 milliseconds so it is 2 000 times faster even for strings of length 7 or 8 and that's a huge boost let's analyze the complexity here let's uh look at the complexity now a quick and easy way to find the complexity of the solution is to see where the computation how many times the computation can occur now this is where the bulk of the computation is occurring in a recursive call and this computation is avoided if we already have something in the memo okay so that means that the only number of computations that we need to do is equal to the maximum number of elements that can end up in the memo now what are the keys in the memo look like the keys in the memo look like idx 1 and idx 2 great and what values can these take idx 1 can take 0 to m values if m is the length of sequence 1 let's say and idx2 can take 0 to n values if n is a sequence a length of sequence 2. so in total the possible number of keys is m times n the possible number of keys is m times n the possible number of things that you need to store in the memo is m times n and for each of them you do constant work and then the next time you try to access this you do not need to do the work you do not need to call any recursion you can simply access the memoization right so what that tells us is the complexity of this case and in any memoization case in general is equal to the number of keys which in this case is m times n so the time complexity here is order of m times n so we've gone from 2 to the power of m plus n which if you if m plus n was equal to 30 would be 1 billion to m time order of m times n so let's say both strings were 15 and 15 so that would just be 225 operations so we've gone from 1 billion operations to 225 operations simply by storing intermediate results and it's a very powerful technique that we apply all the time so now you can see here that the first time five seven is computed the next time phi 7 does not need to be computed again and that's why this tree here is actually marked out so this is the tree for memoization this the first time 4 7 is computed it never needs to be computed again so this entire tree of computation gets eliminated and similarly this entire tree of computation gets eliminated we are eliminating from 1 billion computations almost all except 225 computations so we are left with practically nothing and that speeds up your algorithm by a huge huge factor so that's memoization and as i said it's really easy to compute the time complexity of memorization just simply count the number of keys and then just track how much work do you need to compute each key assuming that you already have the recursive solutions for the remaining okay so how much work do you need to compute each key using some other existing solutions now in this case that was constant because all we needed to do was compare and add okay and i'll let you write here a simple optimized so a plain english explanation of memorization it's worth a it's a good exercise to try out but what we will also look at is another technique called dynamic programming now the downside with memoization is that it requires recursive calls and while it's not a problem for small cases when you have really large problems recursion has an overhead and the overhead for recursion if you see this way is that for this function execution to complete you need this function execution to complete and this to complete and for this to complete you need this to complete and this to complete right so the idea here is that for each new recursive call takes more space in the memory and it also takes longer because now we have to allocate some memory and then set up that function stack the function stack for the execution of that function so if you have a large tree then you're creating hundreds thousands or possibly millions of open functions all of which have their own memory and that can eat up a lot of memory and sometimes that can also take up a take longer time so the solution to replace recursion is iteration and how do we do that we do that using a technique called dynamic programming so we'll do almost the same thing there are a few changes here instead of using a dictionary to track intermediate results we will create a matrix because we know that sequence one uh the idx one can go from zero to n uh where or zero to n one let's say where n one is the length of sequence one and sequence idx2 can go from 0 to n2 where see n2 is the length of sequence 2 and what we can do is we can use a for loop or a couple of for loops to fill out all these sub problems without having to require a recursion okay and this is how we'll do it so let's say these are the two strings that we're working with this is string one t a c g t and this is string two and these this is what dna sequences look like so what we'll do is we will create a table of size n plus one plus one and n one plus one and n two plus one so you can see that there are n one plus one rows so if if this is of length n one these are n one rows and then there's an additional row and similarly there is there are n2 plus one rows here so if this is of length n2 there are there are n2 plus one columns so you can see these are n2 columns and there is an additional column here and table of inj so let's say table of uh if i and j are 0 so i is a pointer for the first sequence and j is a pointer for the second sequence so i selects a row and j selects the column so table of i and j represents the longest common subsequence of sequence 1 up to i which means sequence 1 so here if let's say i was 1 and j was 1. so this represents the longest subsequence of sequence 1 up to i so all the positions before 1 which means only the 0th position just t and sequence two up to j which means all the positions up to the first position of up to up to one so which means only the zeroth position so which means a okay so table one and uh table i j represents the longest common subsequence of these two of just a and t which is zero on the other hand if we skip ahead a little bit if we skip ahead to let's say this position you can count here i goes zero one two three four five six so this is 6 here and here we have 0 1 0 1 2 3 so this is so this is table of 6 comma 3 the table of 6 comma 3 takes the first six elements which is ta gtca and the first three elements aga of sequence 2 and it stores the result of the longest common subsequence between these two okay so i'll just let you look at the table and maybe even draw the table on a piece of paper and verify that the length three is right you can see here a g a a g a occurs here so a g a is a subsequence of t a g t c a so the longest common subsequence between them is three now what we'll do is we will now compare the next elements of we will now compare sequence 1 of i in sequence 2 of j so let's say we are looking at let's pick an example let's say sequence let's say i has the value i has the value 0 1 2 3 4 i has another value 0 1 2 i has the value 2 and let's say j has the value 1 so 0 1 so if we compare sequence 1 of i so which is g and sequence 2 of a sequence 2 of j which is also g and if they're equal so if they're equal then table of i plus 1 on j plus 1 which is this value right so remember i is 2 and j is 1 so table 1 of i plus 1 so table 1 of 3 is 0 1 2 3 and table and table 1 of i plus 1 j plus 1 is table 1 of three and a table a table of i plus one and j plus one i being two and j being one is table one of three and 2 and table 1 of 3 and 2 is the value 2. so this value is obtained by adding 1 to table 1 of i comma j so because these two elements are equal we we can then say that if we take the longest common subsequence of t a and a and add one to it that will give us the longest common subsequence of tag and a g so the exact same logic as recursion we have simply now reversed it so we now now looking at the last element so that we can keep filling out the last value using some previous values okay so this is one case similarly here's one other case where a and a are equal so the longest common subsequence between t a g t c a and the longest common subsequence between a g a is one plus the longest common subsequence between t a g t c and a g okay one plus this value so that's one case the other case is if they're not equal so let's look at this value for example over here so we have ta gt on this side and then we have okay let's let's look at this one uh we have ta gt on this side and we have a g ac on this side now t is the element here and c is the element here they are not equal so that means the longest common subsequence between these two either does not contain t or it does not contain c it cannot contain both obviously because one of the strings has to end so if it does not contain t then it is this result and if it does not contain t or if it does not contain c then it is this result so we simply take the maximum of these two maximum of these two to get the result for this if these two elements are not equal and that is how you fill out the table you start from the top the first row is zeros because we are we have empty strings and the first column is also zeros because we have empty strings to fill out an element you compare if the two elements are equal and if they are equal we simply add one to the diagonally left top left element if they're unequal then we take the maximum of the element above it and the element to the left of it and that way we fill out the entire table okay so that's the dynamic programming solution and i know this can seem a little bit complicated honestly i still get confused with dynamic programming a lot of times and that's why i like to just draw tables and write things out carefully okay and and especially you have to be specially careful with indices because here we are saying that if sequence i one i and sequence to j are equal then table one of i plus one and j plus one is one plus table i j so b just watch the indices carefully here but let's implement the solution let's implement the dynamic programming solution so let's say lcs dynamic programming so we'll just say dp here and we have sequence one and we have sequence two and the first thing we need is we need a table of results now this table for it let's just grab n1 and n2 so length of sequence one and length of sequence two and now we need to create a table with all zeros how do you create a table with all zeros the way to do it a way to create a list of zeros is this zero for underscore in let's say n1 let's give n1 and n to some values now if you want to create a list of zeros of length and one use you simply say zero for underscore or zero for x you're simply ignoring whatever value you're getting from a range arrange n one and that's going to give you a list of zeros but we don't want a list of n one zeros we want these one these want to be rows so we want each of these to itself be a list of zeros of length n2 so zero for x in range n2 and now we have you can see that we have five rows one two three four five then we have seven columns one two three four five six seven so this is the table that we want to create initially okay now this is a table that we've created this is going to be this exact same table and we're just simply going to start each string from position 1 this time not from position 0 because we want to have this additional row where we don't consider either of these that just makes computations a little easier now we say for idx one in len at x1 and range n1 so that's that's going to iterate over the rows and then for idx true in range n2 and that's going to iterate over the columns and first we compare if sequence one of idx 1 is equal to sequence 2 of idx2 if they're equal then we can fill out table of i plus 1 and j plus 1 as 1 plus table of i j okay and we can see this we can see this here suppose the first elements were equal so suppose this was suppose idx 1 was 0 and idx 2 was also 0 suppose they were equal then this value should be 1 so this value should be 1 plus the diagonally top element and that holds true anywhere within the list so wherever you have two elements equal like g and g are equal here so this value is one plus this value else we have table i plus 1 and j plus 1 is max of table i comma j plus one so you stay in the same row or you you go to the previous row or you go to the previous column which is table of i plus 1 comma j and this is the previous column okay so this is this case where g and a are not equal so if g and a are not equal then we take the maximum of these two values and that's it that is going to fill up the table for us and then we simply say return table we simply want the bottom right element we can simply say return table minus one minus one so this is going to get that last row last column and that's our dynamic programming solution let's do evaluate test cases here okay turns on there's no i okay let's just call this i and j turns out idx 1 is not defined let's just make these i and j now that we're doing this coding live you can see that even after a decade of coding i still make all of these issues says the list a list index is out of range it seems like i plus 1 and j plus 1. ah that's because remember we need an additional row and an additional column to track the case where either of the strings is empty so we need to get range n two plus one here and we need to get range n one plus one here okay that's why it helps to have test cases so that you can fix all of these issues now you have test k0 it passes and test case 1 2 3 all of them pass you can see that all test cases have passed and you can also verify that the amount of time it took is now lower than the amount of time it took for memoization okay and so that's the dynamic programming approach you simply create a table and you fill out the table sometimes just working with indices within the table can get confusing so it helps to work with it on paper and make it clear to yourself and then write it in english that's why we've written it in plain english here and now an exercise for you is to verify that the complexity of this dynamic programming approach is order of n1 times n2 so which is the same as memoization and it's actually more straightforward to see here because you have two for loops and then each of these for loops you're simply doing a comparison and an addition and there's not even any recursion to very very there's not even any recursion for you to worry about so you just do a comparison and you do an addition or you take a maximum pretty straightforward so order of n1 times n2 and it does not even invoke an another function so it does not take up too much memory it does not take up too much time it's very very efficient and this is how you solve pretty much every dynamic dynamic programming problem you write a recursive solution you come up with the brute force solution and keep in mind that recursion is almost always the way to go about creating a brute force solution so you come up with a recursive solution and then you identify you draw the recursion tree and if you see that the same sub problem is being called again and again that is a point where you can introduce memoization so you introduce memoization and sometimes you can just write the memoized solution and that's enough because it's easy to reason about you just put in a memo and you're done with it um even the interviewer or the coding assessment will accept that solution but in some cases you will be asked to then remove the recursion and write it as a in an iterative fashion and that is when then you have to start drawing a table and think about what are the rows and columns in that table need to represent so here the i j element of the table represented the first the first i elements of sequence one and the first rj elements of sequence two what is the longest subsequence between them and we used that to build the next row and the next next column and we then filled out the entire table and we simply used the last value now again this is not very straightforward uh how to come up with this and the way you do that is by solving problems so if you solve five to ten dynamic programming problems you will get some intuition about how to build the tables and it's always very helpful to solve it on pen and paper first especially with dynamic programming so that it's clear to you what each element of the table represents otherwise you may make a lot of off by one errors like missing the plus one here or missing the plus one here and get confused just like i did pretty much and that's the time the time complexity is pretty straightforward in most cases it is simply the size of the table but sometimes you may have to do more than constant work here so keep that in mind see what it is that you're doing inside your loop now inside of inside your loop if you have to go back and check the entire length of the string so that will introduce another factor into the equation so keep that in mind but in most cases counting the iteration should be good enough to give you an idea of the time complexity okay so that's the first problem and let us just commit this and now it's saved to my profile if i just open this up here you can see that now i have this notebook called longest common subsequences and i can share it online whenever you work on a notebook it's always a good idea to make it public put it up on jovian all you need to do is run jobin.com and share it online just press the share button and then you can share it on twitter linkedin facebook or wherever you like so that's the first problem that we looked at now let's come back to lesson four and uh by the way the problems that we're talking about all the problem statements the graphs the images you can see them in the second link here but we will once again open up the problem solving template and now we'll use it for the second problem let me run this once again and we're going to look at the second problem which is the knapsack problem so let's read the knapsack problem it's also called the zero one knapsack problem here's there are many variations of this problem but here's one way to state it that you might come across or something similar you are in charge of selecting a football or a soccer team from a large pool of players and each player has a cost and a rating so there's a selection going on you have to come up with a team for this year and you have a large pool of players each player has a cost and each player has a rating now you have a limited budget so you need to build a team within the budget so what is the highest total rating of a team that you can create which fits within your budget okay so this is the question here you have to maximize the total rating but fit it fit the total cost within your budget so we have two variables here rating and variables rating needs to be maximized cost needs to be simply optimized to the extent that it fits in the budget and just a simplifying assumption here is that you can assume that there is no minimum or maximum team size this is simplification and later you can introduce a criteria there as well that you want to build a team of exactly 10 people and see if you can also solve that problem in a way so that's the knapsack problem let's copy it over and here's a jupiter notebook a fresh problem solving template let's simply change the title here it is also called the zero one knapsack problem because each item can either be chosen or not chosen and let's give it a project name here too let's commit it and let's paste the problem statement here okay so that's the problem statement and this is a specific or a special form of a more general problem statement and we look at the general problem statement in a second we'll when we try to state the problem clearly but here's once again the systematic strategy will apply we will state the problem clearly identify the input and output formats come up with some example inputs and outputs and try to cover all the edge cases then we will come up with a correct solution for the problem and state the solution in plain english it just has to be simple correct solution not too complex then we apply the right technique to overcome the inefficiency and then we so we analyze the algorithm and identify any inefficiencies after implementing the solution and finally we apply the right technique to overcome the inefficiency and then repeat the process of stating the solution implementing it and analyzing it so to state the problem clearly what we can do is we can abstract out the problem in more general terms and that is what is stated here and let's just grab that and we'll take a look so here we have we are given n elements and each of which has a weight and a profit so you have n elements and here's the profit of each element and here's the weight you can of each element so you need to determine the maximum profit that can be obtained by selecting a subset of the elements weighing no more than a given weight w so you have a capacity a maximum capacity let's say the maximum capacity is 15 and you have to select certain elements so that you fill out the total weight is no more than the capacity and the total profit is maximized that's and this is why it's called the knapsack problem it's assuming here you have a bag or a knapsack with a capacity of 15 kilograms and these are the weights of the items and these are the profits now in this case you can see in this example the optimal selection is these four elements which have the weights five three two and five so that you fill up the total capacity of 16 or 15 and the of the solution on the maximum profit that you can obtain is 7 plus 4 11 plus 5 16 plus 3 19. now you can try other combinations and verify that this is actually the best solution do give it a shot so what are the inputs here so we have it's pretty clear we have an input weights so these are the weights of the this is a list of numbers containing weights and then you have profits a list of numbers containing profits and this should have the same length as weights and then finally you have a capacity the maximum weight allowed and there you go and now we have outputs so now the output would simply be the max profit so this is the maximum profit that can be obtained by selecting elements of total weight no more than w or no more than capacity okay great so that gives us a pretty good starting point now we can write a function signature here so we write max let's say def max profit and we can give it weights and we can give it profits and we can give it a capacity and we pass so now we have defined the problem we have stated we have identified input and output formats now we need to come up with some example inputs and test cases once again we have listed out a few test cases here so we will have a few generic test cases where you have just a random set of weights and profits and you identify the knapsack the optimal solution then here's one option where all of the elements can be included you can take everything here's another option where none of the elements can be included you have to think about all these scenarios here's one where only one of the elements can be included then you may also think about areas where you do not use the complete capacity okay you do not use the complete capacity because the optimal solution is actually taking a lower capacity so there may be a way to fill out to capacity but that may have a lower profit than another option which takes less than the complete capacity but has a higher profit so think about some cases here think about some good test cases here and i will just copy over these for now and then what we'll do is we will express these test cases once again as dictionaries so you have test 0 test 1 s2 all of these expressed are dictionaries and these are covering all the test cases that are mentioned here you can see here are some weights and some profits and the capacity is 165 and then the optimal solution is 309 now we are simply asking here for the optimal solution the maximum profit that can be obtained but an extension of this problem is to identify which are the elements that should be chosen and it's a simple extension it's a good exercise for you to try out and you can discuss it in the forums we have test zero test one test two test three and four and five so we have a total of six six test cases let's copy over these test cases here and let's put them here into a single string and that gives us the test cases okay now coming up with the solution so once again the first step is to try and come up with a recursive solution and a recursive solution is again quite straightforward we'll write a recursive function max profit that given an index so this time we have just one sequence so given an index within the sequence so let's say our index idx it computes the maximum profit that can be obtained using the elements from idx onwards so 31547 using all of these elements idx onwards the maximum profit that can be obtained right and using a given capacity so it will take an index so it will take an index and a capacity so if let's say the idx is one so it will then look at just these elements and if the capacity is 10 so it will try to fill the capacity of 10 and that's a recursive function and why are we creating a recursive function like this there's a simple reason now suppose idx has the value 1 and the capacity is 10 or let's say the capacity is 3 then the weight of this element is greater than the capacity so that means it cannot show up it cannot be selected because it cannot fit inside the bag the knapsack that we have so then the solution for this sub problem with idx equal to 1 and capacity equal to 3 is same as the solution for this sub problem with this element removed because you cannot include this element within the knapsack right so if you remove this element and simply consider these elements the remaining elements which essentially means idx plus one so max profit of idx plus one profit uh of weights idx 1 profits idx 1 and capacity is the answer for max profit of weights idx profits idx and capacity because the current weight 5 is greater than the capacity 3 which is which the recursive function has been invoked with so that's one option but the more general case is that you have enough capacity so let's say you have a capacity of 10 recursion was called with a capacity of 10 and you are at idx 1 so then you have two choices either you include this element in your knapsack or you do not include this element in your option because you don't know whether the optimal solution will have this element or not so you try both so there are two possibilities we either pick weights idx this element or we don't and what we can do is we can simply compute the result in both cases and pick the maximum so if we don't pick weights idx then once again if we don't pick this element so the capacity remains the same let's say the capacity was 10 so we simply try out to fill out the capacity of 10 using the remaining elements so we simply call max profit with weights idx plus 1 profits idx plus 1 onwards and the remaining capacity which is 10 but if we do pick the element if we pick the element and we had a capacity of 10 then the optimal then the solution the best solution in this case will have a profit 3 more than the solution for this case and since we also used of some capacity so we need to add 3 in the profit and we need to subtract 5 from the capacity right so if we pick weights idx then the maximum profit for this case is profits of idx plus max profit of weights idx plus 1 onwards profits idx plus on onwards but because we've used up some capacity we reduce the capacity in the recursive call okay and that is why a recursive call takes both an index and a capacity okay i hope that makes sense so here's a recursive tree that tells you the same thing we started the first index and we we have the capacity and if we don't pick the first element then we sim the answer is simply the to the best solution for second index onwards with the same capacity if we do pick the first element then the answer is the second solution onwards with the reduced capacity with the profit added okay and then we simply take the maximum of these two cases so we call these two recursive calls and then we simply take the maximum of these two cases to get back the final result or the final best answer and the final end case is that if we've reached the end if weights idx onwards is empty if the index that we're tracking has reached the very end then irrespective of what the capacity is the maximum profit is in that case is zero so let's try and implement this now let's copy this over as the explanation and let's try and implement the solution let's say let's call it max profit recursive and this is going to take a set of weights it is going to take a set of profits and it is going to take a capacity and is also going to take an index which the index will start out at zero so now if the index is we start with the base case so if i dx equals the length of weights in this case there's nothing left to do we simply return 0 because we don't have any more elements then we check if the weights idx is so the current element is greater in weight than the capacity then it's a pretty straightforward solution we simply return max profit recursive of weights profits capacity plus one sorry capacity and idx 1. so we simply ignore this element because we cannot fit it in the capacity that we have else we have two options we have option one option one is even though it can fit within the bag we don't take it we every because the optimal solution may still not have it just because it fits does not mean we should take it so we look at the option one which is once again the same as this where we ignore this element and then we have we look at option two in option 2 we actually put this element into the bag so since we are putting this element into the bag then we get we get profit from it so we get profits of idx and then we call max profits recursive and this time we call it with weights and profits and now the capacity has reduced a little bit because we have taken this element so now we can now we need to fill the remaining we fill the need to fill the bag with the remaining elements from idx plus one onwards with a limited capacity of capacity minus weights of idx and then finally we just put in idx 1 so that we can start calculating the solution from the next element onwards so that's max profit recursive again not very difficult it is just about six seven lines of code and let's try it out here's test zero let's try max profit recursive with test zero input and we need to get weights capacity and profit all of these out of it the simple way to do that is simply to put in star star and we'll get back all of these will get passed in capacity will get passed as a capacity parameter in weights will get passed against the weights parameter and profits as the profits parameter okay so we've encountered an error and that's completely fine completely fine to encounter an error i see so what we've done here is uh we have not really taken the maximum of these two we've just defined the two options so we do need to take max of option one and option two okay once again this is why helping test having test cases helps and you can see that now we call max profit and we can also add a timer here so max profit it takes 210 micro seconds but it result it returns a result 309 great we get back the result 309 here which is what we expected so our function is working correctly we can even evaluate it on all the test cases so from joven dot python dsa we import evaluate test cases and then we simply call evaluate test cases on all the inputs so we pass in max profit recursive and then we pass in all the test cases as tests now you can see that we have these test cases and each test case seems to be passing just fine all six x test cases have passed and these are the times they took so that's your recursive solution pretty straightforward once you reason it out once you maybe just look at an example draw a tree of recursion yourself work it out on paper the code is in fact in most cases fairly simple and this is what the recursion tree looks like each time we make a choice to either include the element or not include the element and now you can reason the complexity very easily because now we have n elements for each one we keep making this choice so that means we end up with 2 to the power n leaves and from there it follows that the complexity of the recursive algorithm is order of 2 to the power n right so it could be 2 times or c times 2 to the power n but and in the bigger notation it's order of 2 to the power of n so it is exponential and complexity and why is it exponential complexity once again there are it's a possibility here that we may be computing a lot of things repeatedly because we are creating so many of these sub problems so it's possible that we may be creating we may be recomputing a lot of data here so now the task for you or the an exercise for you is to write the memoized version of this so what is it that you need to memoize now the trick here is to look at what is changing within the recursive calls so now in max profit recursive you can see that weights and profits remains the same but it's the capacity and the idx that change so you can take the capacity comma the index the idx as the key in your memoization dictionary and each time you compute so each time let's say you compute this or you compute this or you compute this store the result in the dictionary before returning it and then at the beginning of the recursive function check within the dictionary if this value is already present okay so remember what we did for longest common subsequence we defined a recursive function internally we defined a memo a dictionary internally and the recursive function kept either checking the dictionary or filling the dictionary if it could not find a value and that could eliminate a lot of the repeated work in your problem okay so that's the challenge for you to try out implement the memoize solution and what we'll do is we will go ahead and we will implement the dynamic programming solution so let's just commit our work once again and we've analyzed the algorithms complexity in recursion it's order 2 to the power of n in memoization now that's an exercise for you what do you think the complexity will be well let's apply dynamic programming so let's look at a dynamic programming solution now once again for dynamic programming you have to create a table you always almost always have to create a table for dynamic programming and in this case we can see that there are n elements so there are n rows within the table because we have n elements to choose from and we we have a number of columns going from zero to capacity plus one going from zero to capacity and that's why there are total of capacity plus one columns and in fact what we can do is we can also include another column at the top here another row at the top here which we have not which is not shown here but what n represents n is the number of elements so what n represents or or what the a particular element in the table represents so table of i comma c what it represents is the maximum profit that can be obtained using the first i elements if the maximum capacity is c so if your maximum capacity is c let's say your maximum capacity is three what is the maximum profit that you can obtain using the first two elements so um here let's say we are at this position so using the first two elements of the list within this capacity okay so the first two elements have weights one and two and the capacity is three so you can you can actually pick uh sorry the first two elements are weights two and three and the capacity is three so you either pick this element or pick this element now if you pick this element the profit is one and if if you pick this element the profit is two so the solution is to pick this element and you get uh you fill the capacity three and you get a profit of two you cannot pick both because your capacity is three okay so that's the logic here a very simple visual representation now remember that there will also be a zero throw here which we have not shown but this is something that should be here another zeroth row so the zeroth row represents that you've not picked any of the elements and if you don't pick any of the elements it is simply going to contain all zeros and that's why it's not shown here the first row assumes that you have picked you can pick only the first element so you can you you can't pick the first element till a capacity of two and then from a capacity of two onwards you pick the first element and that has a maximum capacity of one the maximum profit of one the second row or the row number two with row with index two represents the fact that you can pick both of these elements and if you can pick both of these elements once again at capacity 0 none of them can be picked at capacity 1 none of them can be picked at capacity 2 this element can be picked which has a weight 2 and it gives you a maximum profit of one at capacity three this element can also be picked so now you have a choice to pick between the two of these so you might as well better pick this one because this is going to give you higher profit and then finally when the capacity becomes 5 you can pick both of these elements and you can pick both of these elements and that is going to give you a profit of 2 plus 1 3 and so on so you keep filling out this table for each step here or for each set of first i elements you fill out the capacity table and then you use the information to fill out the next row and the next column and so on okay and finally your what we need is using all the elements and using the maximum capacity that we have what is the maximum profit that we can obtain so the last element of the table will give you the result okay so what does the logic look like we will fill the table row by row and column by column now if table of i comma c table of i comma c let's say this is a certain position here table of i comma c can be filled using some values in the row above it okay now if you look at the table of i comma c you you look at look at this element for example yeah let's look at this element here so in here c has the value 3 and then i has the value 0 which is a row that is not shown 1 2 3 4. so i has the value 4 and c has the value 3. so if yeah so if weights of i is greater than c so zero one two three four uh if though if this if this weight so this weight the weight of this element is greater than the capacity so the weight of this element is 4 it is greater than the capacity then this element cannot show up in this maximum profit why because its weight is greater than the capacity so obviously it cannot show up in the maximum profit now if it cannot show up in the maximum profit then the then this cell can be filled using the value above it because in any case you cannot put in this element so you might as well get the result by using the first three elements and in that case the value of this cell is obtained from the value of the cell above it that's one case now on the other hand let's come here you come to this case into this cell in to fill this cell because you have a capacity of 4 you have the option of either choosing this element or of not choosing this element now if you do choose this element let's say you choose this element with a capacity of 4 with a capacity of 4 you get back a profit of 9 and now you have no more capacity left to create more to fill more elements on the other hand if you do not choose this element then that's the same as this value because if you do not choose this element then you have to fill the capacity of 4 using the value of using the first three elements and that simply gives you the same highest profit as the previous cell right so you just consider these two cases whether we choose the element or we do not choose the element now if if you do not choose the element the value comes from above if you choose the element then the value comes from where let's see if you choose the element the profit of 9 comes and you fill out the capacity 4 so you have no remaining capacity but on the other hand if the capacity was 6 and you choose the element then you have chosen the element and you've used up the capacity 4 so you can still use the previous three elements to fill the remaining capacity which is six minus four so which is your capacity of two so you can go back to the previous row and check the capacity two and see how much was the maximum profit that you can obtain with capacity 2 and it turns out that with capacity 2 using the first 3 elements you can obtain a maximum profit of 1. so the maximum profit here when you choose the element is 9 plus 1 n similarly here a maximum profit that can be obtained if you choose the element is 9 plus 7 minus plus from the previous row you pick the element with a capacity 7 minus 4 which is 3 so 9 plus 5 14 okay so that's the logic here sometimes you choose the element sometimes you don't choose the element and in fact the solute the result of this cell is simply the maximum of either not choosing the elements the maximum of this cell or choosing the element and subtracting the weight which is six minus four two so maximum of this and that okay so let's implement this same dynamic programming solution once again do work this out on paper it really helps to work it out in paper but let's say we have max profit dp the dynamic programming solution we have weights we have profits and we have a capacity and then let's say n is len weights so we need to create a table so this is our table our table contains n rows so we have len n and then in for each of the rows we contain we have capacity plus one oh we contain n plus one rows remember we also want to consider the case where we don't consider where we don't take any of the elements and it is filled with zeros and the number of columns is capacity plus one to check the values from zero to capacity so that's our table right now you can check what this capacity looks like let's say n has n capacity i have the values here n in this case is 5 and capacity is 10. we don't need a len here you don't need a len here as well it's all perfectly natural to make these mistakes this should be range not len this should be a range and this should be arranged too yeah now you can see that we have created n rows or n plus one rows so one for each of these and then one more row above containing all which will contain all zeros this is in the case where we don't pick any of the elements and then we've created 11 columns so this is for capacity zero so again the first column will also contain all zeros and this is something that you will often see in dynamic programming you will have an additional row at the beginning or at the end containing all zeros and that is simply to make your calculations or computations easier but what that will lead to is off by one error so you need to be very careful while doing this and now we'll fill out this value using either this value or by subtracting the weight of the element that's here and getting a value from the previous row so now we start iterating so when now we say for i in range n and for j in range c let's just say for c in range capacity it should be capacity table of i comma c and it's actually going to be i plus 1 and c plus 1 because we have these additional rows and columns table of i plus 1 comma c plus 1 is there are two cases here if weights of i is greater than c the current capacity then we can simply look at the previous row so which is this case let's say the weight 3 is greater than the current capacity 2. so then we simply copy over the value from the previous row the same column so we just say table of i comma c plus one we see so our capacity should go from the value of 1 because we don't want to affect the first column so the capacity goes from the value of 1 to a value of 10. so capacity c goes from the range of 1 to capacity and if the weights i is greater than the capacity then we cannot fill the table on the other hand if it is if it fits within the capacity then we have two options the table of i plus one comma c has two options so one is we don't use the current element we don't use the current element and that gives us stable ic once again the other option is we use the current element so we get profit from the current element so profits i but we do not get profits but that reduces the capacity so we then have to pick table of i but now we have to pick c minus weight weights of i okay and that should fill out the entire table pretty much that's a nice thing about dynamic programming you simply just have to write this one solution or this one recurrence and be careful about it and everything else is taken care of by this loop here now we simply return table of minus one and minus one and let's see if that works it's likely that there are some issues here but let's see we have test cases max profit dp with the tests that we have great so we are seeing an issue already i see here that this should be range and this should be range okay one thing that we haven't done here is well it seems like our solution is always zero ah this should be capacity plus one so that we this takes all the values from zero to capacity right so c the iterator should take the values from one two three four all the way up to the maximum capacity and the range does not end so the range does not include the end value so you need to put capacity plus one here okay now with that out of the way you see once again these off by one errors are always going to bug you with dynamic programming i've probably solved 50 or 100 problems in dynamic programming but i still make these errors but with that out of the way you can see now that each of the test cases seems to pass now there may be other cases which you have not accounted for but overall we've covered all the test cases here and we've ended up with now a dynamic programming solution and i'll let you figure out the complexity here but once again it's pretty straightforward because we are filling up this table and filling up this table simply requires this constant amount of work which is a comparison and then potentially another comparison and an addition and a subtraction so like four or five operations so you have this n times and uh you have this n times capacit n times capacity where n is the length of weights and capacitor or w is a total capacity so n times w is the number of iterations and that really also is the complexity the time complexity of the algorithm so that's the knapsack problem and now what you can do is try and figure out not just what is the maximum value but also figure out what are the actual elements that were chosen now you can do this for the knapsack problem and you can do this for the longest subsequence problem figure out the actual longest subsequence and it should be possible to do that with just a small modification now use the forum if you have any questions about the contents of this lecture go back to the lesson page and open up the course community forum here you can see here that this is the lesson for recursion and dynamic programming lesson you can post your question here and you can also discuss ideas on how to figure out what the longest common subsequence is and what the best selection for the knapsack problem is so what do you do next well you can review the lecture video and execute the jupyter notebook the next step is also to complete the assignment now we have released assignments one and two so far if you go back on the lesson page you will find lessons you will find assignments one and two and you can work on them there is sufficient time and also work on optional questions and do participate in forum discussions and or if possible join or start a study group too that's a great way to stay motivated this was lesson 4 of data structures and algorithms in python thanks and talk to you soon hello and welcome to data structures and algorithms in python this is an online certification course being conducted by jovian today we are on lesson five graph algorithms like bfs dfs and shortest paths my name is akash and i am your instructor you can find me on twitter on akashens if you follow along with this course and complete all the assignments and build a course project you can earn a verified certificate of accomplishment for this course so with that let's get started the first thing we'll do is go to the course website pythondsa.com now you can point your browser to pythondsa.com to open up the course page and on the course page you can enroll for the course and you can view all the previous lessons and assignments so do check it out and do check out the course project as well but for now we'll open up lesson 5 graph algorithms now on this page you can watch a video for the lesson later the same video that you're watching right now and you can also catch a hindi version if you wish and here is the code that we are going to use today the first notebook under the heading notebooks so let's open it up and this is a jupiter notebook hosted on jovian you should be familiar with it by now but here you can see that there are some explanations and then there are some code cells where we can write some code you can see that there's some code here now to actually execute and edit this code we will need to run this notebook you can find the instructions to run the notebook right here but the simplest way to do it is to click run and select run on binder now this will take a second or two but this will take your jupiter notebook and create a new machine in the cloud and send your jupyter notebook to that machine for execution this is a free service that you can access via jovian you can also run this notebook on your own computer directly if you wish so for that you can check the run locally option here okay so our jupyter notebook server is now ready so we can now start editing and writing some code let's just go full screen here okay so the topic today is graph algorithms bfs dfs and shortest paths using python now before we talk about graph algorithms let's just try to understand intuitively what graphs are now here's an example of a graph in the real world so this is the railway map of india you can see here all the train stations that you have in india they're represented using these black dots points they're also labeled so each train station points to a city or a village so all these are also labeled and then you can see connections between these stations so these are as you might guess railway lines and you see that there are three or four colors involved so these colors could represent different types of railway lines like different gauge meter gauge broad gauge etc or these could represent different zones so there's some information contained in the connections as well now another important thing is that each railway line between two cities will also have a certain length so that's what a graph is roughly and the kind of questions that you may want to ask here is for example is there a path from new delhi to hyderabad so given this information first of all the question is how do you even represent all this information how because you have so many railway lines connections between different cities so many hundreds of cities how do you even represent this so that you can start writing algorithms to answer these questions right so if you're building a search a trained search website then you would have to answer given new delhi and hyderabad is there is there a way to get from new delhi to hyderabad okay that's the first question that you might ask now if there is a way then the next question might be that what is the path with the shortest number of stops so do you go this way for the shortest number of stops or do you go this way or do you go this way another question could be what is the path with the shortest distance right so sometimes if you measure the distance and if you measure the number of stations the number of stops they may be different along different paths and one may be greater than the other in in certain cases so those are the kind of questions that we want to ask and answer today or another question could be what are all the stations reachable from new delhi within one stop or two stops or three stops or ten stops so those are the kind of questions we'll try and answer and for that we need a way to represent graphs in a more abstract fashion because this same question can be asked in a different context for instance here we are looking at flight routes international flight routes now once again you can ask the exact same thing here is there a way to get from new delhi to vancouver now if there is then how many stops will that require what is the minimum number of stops we can take to get from new delhi to vancouver or what is the minimum time it might take maybe if you you're okay with taking multiple stops but you want to minimize the the the time taken or the distance traveled because you're concerned with the miles or for some reason another thing you could ask is what is the minimum cost if there is a cost along each route okay now here's one more example from a very different domain this is hyperlinks or the internet essentially so you can see here here you have a whole bunch of websites and you have links on websites now links on websites point to other websites and in this case it is a one-way connection you can see that from this particular course website we have a link to ibm but from ibm you may not have a link to this course website now that's an interesting thing that's a slight variation here and this is called a directed graph because each connection here is has a particular direction now this is again interesting to ask is there a way to navigate from cs.umass.edu to ithaca weather if there is what is the shortest way what do what does that path look like so those are the kind of questions that we want to answer today and to do that we will need a more abstract representation of graphs and we start with the simplest possible representation where you have certain points or what we will call nodes or vertices so these are two terms that are used for these points so nodes or vertices a graph has certain nodes or vertices and just to make things easy these could be cities or these could be web pages or these could be something else but just to make things easy what we'll do is we will number the nodes so in our graph if we have 10 nodes then we will number the nodes from 0 to 9. okay this is and they can be numbered completely arbitrarily there's no reason to name number the 0 number this one what's more important is that we should use up all the all the numbers from 0 to n minus 1 if we're dealing with n nodes now why do we do that we'll see in a moment when we try to represent graphs using certain data structures like adjacence adjacency list etc but we want to number our nodes from 0 to n minus 1 and this number is arbitrary this 1 doesn't represent anything in the sense that 1 being greater than 0 or so on okay so these nodes have labels and then you have edges between nodes so an edge is simply a pair an edge is simply something like 1 comma 2. so a pair 1 comma 2 tells you that there is an edge between the node one and node two okay now as we move forward we will also store some information within an edge and we will call that weight of an edge and we will also later look at directed edges and those will get us directed graphs but let's start with this and let's see how we can now represent with this basic structure how we can represent a graph so we can represent a graph using two variables so one is called a number of nodes and the number of nodes is in this case 5 and then we can represent the edges using a list of pairs so in this case the pairs are zero comma one in this case the pairs are zero comma one that's an edge then zero comma four that's an edge too then we have 1 comma 2 so 1 is connected to 2 and the edge in this case is bi-directional so when we are saying 0 comma 1 we're saying it automatically says that 1 and 0 are also connected right so 1 comma 2 and then we have 2 comma 3 and which order we write these in doesn't matter we could have just written 3 comma 2 here as well or we also have 1 comma 3 and then we have one comma four great and then finally we have three comma four okay so this is how we represent this data structure which what we've drawn here is now represented in code using these two variables and we can check here if we simply print the number of nodes and the length of edges we can verify if this is roughly correct so you see we have five nodes and we have one two three four five six seven edges okay seems right to me we could there may be a mistake here but roughly uh we have set things up correctly okay now the question becomes is this question is this representation good enough now this representation is good enough if you want to convey the structure of a graph to someone i could give you these two variables and then without showing you this image and you could use this information to draw the graph on a piece of paper so this representation is complete it provides all the information about the graph but it may not be efficient for example if you want to find out which nodes the node 1 is connected to we would have to iterate over the entire list of edges we would have to go through this one and then check if either of these is one and check if either of these is one and so on so that makes it very tricky to access any information efficiently rather it'll be much nicer to just look at a list of nodes that one is connected to in some way and go from there now if you want to find the shortest path we would first have to find all the nodes that one is connected to and then for each of those we would have to find their neighbors and then for each of those we would find have to find their neighbors and so on so it would get pretty tedious to go through the list so many times that's why and by the way by a neighbor we represent we mean two nodes that are connected by an edge so zero and one are neighbors but zero and two are not neighbors okay so that's a very simple nomenclature that we can use and what we can say is if we track the path we say 0 1 2 and then if there is an edge between both of them we say that 0 1 2 is a path so 0 1 2 in this case is a path but 3 zero one is not a path because there is no path but there's no edge between three and zero okay and we'll see what what we mean by paths and neighbors and so on in some time but to work with graphs more efficiently we will represent them using what's called an adjacency list now the name it explains what it contains so the adjacency list contains a list for each node and it contains a list of all the nodes that are adjacent to that node now again adjacency is the same as an adjacent is same as neighbor so if for each node so for example for the node 0 we we maintain a list and that list contains the numbers one and four indicating that zero is adjacent to or zero is a neighbor of or zero is connected via a direct edge to one and four so that's why you have one and four here and then one is connected to zero two three and four you can see that one is connected to zero two three and four similarly two is connected to one and three three is connected to one two four and four is connected to 0 1 3. now this is more convenient for sure one because since this is an uh this is a list if you want to find let's say which nodes 2 is connected to we can directly access the index 2 within the list and this is why we number the vertices or the number of the nodes from 0 to n minus 1 so that we can access them directly in an adjacency list right so we directly access the number stored next to 2 and so we have one and three here so that's what makes it convenient and one important thing to notice here is that edges each edge goes twice so the edge zero one shows up in the list for zero so you can see here in the list for zero we have one and similarly in the list for one we have zero so each edge shows up in two adjacency list of each of the nodes that it connects okay so now the obvious next question might be to create a class to represent a graph as an adjacency list in python okay this is again a question that you might get asked a step or this might be part of another question that you may get asked where you're asked to perform a breadth first search or depth first search or find the shortest path but the first step you will have to do is define a class for a graph to maintain the information about the graph as the adjacency list okay so here we're creating a class graph and the first thing we'll need inside the graph is a constructor function so we need to put something inside the constructor function and we know that the first argument to any graph any class method in python is self which represents the object that will get created ultimately when we create an object of the class but apart from this what information do you need to create a graph now it's pretty straightforward we can simply work with this information because these two variables together specify the graph completely so let's simply accept num nodes and a list of edges as the information the first thing we can do is simply store num nodes in self.num nodes so that once we create a graph we can access the number of nodes very easily then we need to create the adjacency list so we need to create the adjacency list we'll call it self.data and initially we will create a list containing empty lists because and then we will fill out the empty list step by step so what we need is something like this in this case because there are five because there are five nodes so this is what we need to create the five empty lists now in general the way to create repeated elements is this you can say if you want to create a repeated element like this 0 times you type 0 times 10 and that gives you this list 0 0 or containing all zeros on the other hand if you create empty list times 10 and let's call this l1 and let's see what l1 is it looks like you've gotten an empty list you've got in a list containing 10 empty lists but let's just go into the first element so the first element is this first empty list and inside the first element let us add the value one okay and then let's look at the let's look at the list l1 once again and you see what happens this one gets inserted into all of these lists now what's the problem here now the problem here is that when we do this when we create a list containing an empty list or containing any object then the same object gets replicated 10 times but python does not create copies now when you're working with numbers it's fine because when you're working with let's say the number zero that's fine because there's no internal structure inside zero right so there's nothing you can change inside the zero it's a fixed value fixed immutable value so what so you can you can't really say l one of zero and change its value internally what you can do is you can set l one of zero to another value let's say you can set l1 of zero to one so instead of getting all zeros you get all ones but you cannot take this take the zero and change something inside it on the other hand when you have an empty list here so this is the same list that is showing up in 10 different showing up 10 different times each of the each of the elements in the list outer list is simply a pointer to this same empty list so what we can do is we can go inside this empty list and append something to it so since this is the same object that we are seeing over and over the one gets appended to the first list and because the rest of them are the same object we get back all once inside here okay so this is a com the reason we're spending time here is because there's a common common bug that you may unintentionally execute whenever you want to create an list of empty lists do not use this method so what's the method you should use then so here's one method you can use let's say you want to create a list of empty list of size 10 so you may be familiar with this object this this object called range this function called range what this does is if you view it as a list you can see that it contains all the elements from 0 to 9. okay now if you view the range itself it simply shows you 0 to 10 but when you convert it into a list you can see that internally it contains the value zero to nine okay so you can take this range and you can do something like this put this range or put anything which is iterable inside these brackets the list brackets and then say for x in range and simply put x so what did that do that did practically nothing we simply took x from the range of 0 to 10 and returned x itself so we created a new list like this but suppose we multiplied it by two here x by two so for each element in the range we are multiplying it by two so we get back a new list which is zero two four six eight so this is each element is the double of the elements that we have in the range now what we need is we need just empty lists right so we can simply put an empty list here and we can ignore this value x that we get here so now we get back a whole bunch of empty lists so let's call this l2 and what we are now doing is for each element in the range we are creating a new empty list so this is important so now when you do l2 0 dot append 1 and then check l2 you can see that one was only inserted inside the first list so keep keep out watch out for this this is something that you will probably go wrong with at some point i've gone wrong many times and one last change we can make here is whenever you're not using a variable in python it's always a good idea to just call it underscore you can still call it x but your sometimes somebody reading your code may not understand why you have declared a variable and not used it and assume that maybe you've made a mistake so just to make things very clear it's always a good idea to make something underscore it's also a variable name a valid name and mark something is underscore if it is not being used okay so with that whole discussion about lists we now know how to create a list of empty lists so here you have a list of empty lists or underscore in range num nodes so now we have created a list of empty lists then for each edge in edges we need to do something so we need to insert it into the right lists okay now what does for edge and edges look like so let's see for edge in edges print edge okay each edge is a pair we already know that and when you have pairs or tuples here you can get them get the values out so let's say let's call them n1 and n2 node 1 and node 2. you can get the values n 1 and n 2 out like this so now we can say print n 1 and print n 2. you can see that we are able to get values n1 and 2 out directly within the for loop so let's call this n1 and n2 and now this is a much more pythonic way of writing code so one of the things that we are also learning is how to write code which is more pythonic or which is idiomatic in python and this is again something that will impress people when you use it in an interview or a coding challenge so for n1 and n2 in edges what we need to do is first we get self data of n1 so this gives us the adjacency list for n1 the first node and here we append the value n2 and similarly we do the same for n2 and we append n1 to it and that's it now we've set up the graph let's create a graph g1 let's call this graph 1 maybe and we simply invoke the graph function and then we give it a number of nodes and the edges right so remember self will be passed in by python automatically as the object that is getting created so the graph one object essentially so now the number of nodes is five and we have a list of edges and let's see what graph1.data looks like so there you go you can see that zero is connected to one and four and one is connected to zero two three four and so on now while this is okay it would be nicer to print it like this so maybe let's see if we can print it like this and the way to do that is to define a wrapper function so we define a function called underscore underscore repr and it contains it simply takes self as the input and what we are going to do is we are going to go over we're going to call enumerate on self.data now what does that give us let's just check what enumerate on self.data give us gives us well maybe before we do that let's see what enumerate on a list gives us enumerate on a list gives us this object but let's just get the value out of it in a for loop because you can use an enumerate in a for loop and just print x so what enumerate gives us is it gives us the values from the list but apart from those values it also gives us indices okay so you can get an index i and a value v out of enumerate so then you can see that you can print both i and v here and you will get back the same output so what we can say is we can do enumerate self.data now because self.data contains these elements so what we'll get back is we'll get back pairs let's see here we'll get back pairs 0 comma 1 4 1 comma 0 2 3 4 2 comma 2 1 3 now this is starting to look a lot like what we want okay so we'll just take enumerate self data and these we'll take these pairs so the pairs will be a node so node n and its neighbors the so we have the node n so the node n will first be 0 and its neighbors will be 1 and 4 node n will be 1 and its neighbor will be 1 and 2 and so on and then so for n comma neighbors in enumerate what we'll do is we simply create a simple string and here we are using string formatting we are simply creating this string where we put this here we place a placeholder where we put n and then here we put a placeholder where we put neighbors again let's just see what that looks like and this is the best thing about jupiter while you're writing code you can test your code right then and there simply by creating putting data into a new into a new cell so let's see graph1.data so you can see here that now we have now we have converted that enumerated list into a list of strings so we have a string here this is the string 0 pointing to one comma four this is a string one point this one pointing to zero two three four and so on but this is still a list of strings what we need to return from the wrapper function is a single string so the way to join them together whenever you have a list of strings and you want to join them together all you need to do is you say what you want to join them with so we want to join them with a new line and then call the join function on that string and return that right so that is our wrapper function and we'll see its users in just a moment and similarly we have another function called str now wrapper is used when we simply type graph one so when we type graph one this is the output of the default wrapper function now this will get replaced by the wrapper function that we are defining but when we do str of graph one or when we do print of graph one or when we insert graph1 into string that is when the str function is used now we will simply use the wrapper representation so let's just put self dot underscore underscore rdpr and that's it okay so let's see now let's put let's type graph one here and you can see that now we have this representation printed using this wrapper function that we've defined so we have zero one four one zero two three four two connected to one three three connected to one two four and four connected to zero one and three okay so now we have a graph data structure that we've implemented using a class so the adjacency list and we have a nice way to print it out and this is just good programming practice now you don't have to do this in a coding competition or you don't have to do this it's good if you do it in an interview if let's say you're able to type this out quickly but when you are working when you are working on your own problems or on your own code or on a project always make sure that any classes you define have a good string representation so that when you type the name of a variable you understand what it represents and you don't have to spend time thinking about it make it clear to yourself okay so that's the adjacency list and we'll see how that is useful in just a few moments but here are a couple of questions for you try writing a function to add an edge to a graph that is represented as an adjacency list okay so here we've specified all the edges right in the beginning but can you write a function add edge which takes a couple of nodes and it inserts an edge between those two nodes and here's a hint this code might be useful so do try that out now here's another one can you write a function to remove an edge from a graph which is represented as an adjacency list here you may have to use the list remove functions to remove a particular element from a list but these are two good exercises to complete here okay now before we continue let's just save our work and we know that this notebook is running on binder which is a free service so we'll just save our work by running jovian.commit and what that will do is that will capture a snapshot of this notebook all the changes that you've made and put this on your jovian profile now this will go on your jovian profile from where you can continue running it continue executing it from where you have left off okay now another common representation for graphs is called the adjacency matrix which is slightly different from adjacency lists in this case for example the same graph here is represented using this matrix so what we do is we create a matrix of size n by n if n is the number of if n is the number of nodes in the graph and then for each node for instance since we have zero and since we have a edge between one and two so if you take the first row row number one and column number two you put a one there otherwise if there's no edge for example there's no edge between zero and two you take the zero throw and column number two the you put a zero there okay so you put a one wherever there is an edge between the two nodes and you put a zero wherever there isn't you can see that there is this reflection reflexive property here because one two is one and two one is also one because these are undirected edges now of course if this is a directed graph this would be different okay so an exercise for you once again is to represent a graph as an adjacency matrix in python shouldn't be too hard all you have to do is instead of so in adjacency list we initialized a list of empty lists here you may want to initialize a list of zeros a list containing lists of zeros okay and then you may simply just want to fill in the zero once in the right places now adjacency matrices have their own benefits sometimes they are more useful for example when you want to immediately check if there is an edge between two vertices or two nodes you can quickly look up look it up in the adjacency matrix but in the adjacency list you will have to get the list for one of them and then search through that list which is fine for most cases but in some cases you may just want an adjacency matrix as well so that's one other way you can represent a graph and that's an exercise for you okay so now we know we've represented graphs and now we can start looking at some graph algorithms and probably the most common graph algorithm something that you will ultimately get asked in one interview or the other if you're interviewing with a bunch of companies is breadth first search and breadth first search well this is what it looks like so suppose you have this this is a real world graph that we're looking at so these are cities in germany and you can see that there are roads between these cities and we have lengths of each road now we can ignore the lens for now what's important is that these cities are connected to each other but not all cities are connected to all of them all of the others so starting from frankfurt you may want to find out which are the cities that are that you can reach from frankfurt without stopping so which are the cities that are one edge away from frankfurt and if you look at it this way it turns out that mannheim castle and wurzburg are the three cities that are one edge away from frankfurt right so if you start drawing this tree of sorts so you will find that mannheim wurzburg and castle are one edge away okay then you might ask which are the cities which are two edges away from frankfurt so now the cities that are manheim is connected to karlsruhe and wurzburg is connected to these two cities and then castle is connected to this city okay so here you have these other cities and then you might ask which are the cities that are three steps away from the from frankfurt and that would be the remaining two cities augsburg and stuttgart okay now i'll let you think about this but what you will find in this way as you go step by step by step like first you're finding all the cities that are one step away so all the nodes that are one step away from a source node then you're finding all the nodes that are two steps away from a source node what this will give you is ultimately you will end up for each node you will find out how far away it is from the source and that will be the length of the shortest path between the two okay and you can verify that i'll let you think about it for instance if you see you can go to castle by going this way from wurzburg to nurenberg to mission to castle but that would not be the shortest path but binary search the this is called breadth first search brett first search will always discover the shortest path because we're first finding all the nodes at distance one and then we're finding all the nodes at distance two and then we're finding all the nodes at distance three and if a node at distance three has a shorter path then it would have been already found when we are finding nodes of length one or two or distance one or two okay so that's brett first search so here's one problem that you might face in an interview implement breadth first search given a source node in a graph using python and here is some pseudo code this is so it's always a good idea to write or explain your approach in plain english before you implement it so that you do not make mistakes while coding so here for here is the pseudo code so if you have to write a function bfs which takes a graph and a root or a source node so first we say create a queue so we're creating and this is taken by from wikipedia so first we create a queue and what's a queue well a queue is a very simple data structure a queue is simply a list and it follows a first in first out of policy so when you have a list and you want to add something into a queue it's also called enqueue the enqueue operation so when you want to add something into a queue you add it at the end okay so you have a list and then you simply keep adding things at the end you just append things at the end of a list but when you want to access something from a queue you do not access any value directly no you always access the first available value okay you access the first available value in this case what what is called the value in front and when you access a value it gets removed okay so it's so in this way you can see that it implements the the first gen first out policy like if first we enqueue one and then we enqueue three and then we on q4 and then we want to dq and when we want to dq we simply get the first value that was inserted which is 1. then maybe we enqueue a few more numbers 5 2 7 then we dq and then we get back the first value that we had inserted which is not yet dequeued so then we get back four or whatever was the was the second value inserted initially right so that's a queue and we let's see how a queue is useful so we create a queue and then we mark the label we mark root we label the root node as discovered okay so we need to somehow track which nodes have been discovered or visited and first what we'll do is we will mark the root node so let's say we're starting from the node three we will mark three as discovered so three is now discovered and as soon as we mark something as discovered we will enqueue it okay then while the queue is not empty which is why we have not accessed all the elements in the queue or well we have not dequeued all the elements from the queue we dequeue an element so we dequeue the first element which has not yet been removed from the queue and if we are looking for a particular goal node then we can simply end there like we found that node but we are not looking for a goal node so let's remove this code yeah so we get we get the first element or the first node from the queue which is not yet dequeued and then so for example initially we just have three in the queue so then we get back three we get three back from the queue then we check all the edges for three so we check that three is connected to one and three is connected to two and three is connected to four so we see all the edges for three and if the other end of the edge we check for each node let's say the other end of this edge is 2 we check if 2 is not yet discovered or not yet visited then we enqueued 2 into the queue similarly we check for 1 and if one is not yet already discovered we enqueue one into the list similarly for four we enqueue four into the list okay so we have dequeued three so three is no longer in the queue or we've moved forward are we no longer going to get q a three out of the cube but now we've enqueued two one and four and two one and four we now understand they are at distance one so when we pick the next element of the queue we dequeue the next element the first in first and first out we get back 2 and then we mark 2 as visited great now we visited 2. oh no we we mark as soon as we are adding something to a queue we also mark them as visited because we've identified that 2 1 and 4 are all at distance 1 from 3 and we've added them to a queue so we mark them as visited now when we get 2 out of the queue in the next iteration we check if there are any nodes which 2 is connected to those are not yet visited so 2 is connected to 1 but 1 is already visited so there's no need to enqueue it again and then 2 is connected to 3 but 3 is already visited so there's no need to encue it again and so we just move forward then we go to 1 and when we go to 1 we realize that 0 is not yet visited so we enqueue 0 4 is visited so we don't enqueue four okay and that's how we proceed so now what you should do is you should draw this on a piece of paper and work it out just write on a piece of paper what would be the first element that gets inserted and what will be the elements that we will insert into the queue etc etc but this is the algorithm here exactly what we what we just discovered so we dequeue in a vertex for all the edges that start from the vertex v or the node v if the other end of the edge is not labeled as discovered then mark it as discovered and enqueue it into the queue let's implement this let's see if we can implement this live so we are implementing bfs where we will get a graph and a source node the first thing we need to set up is a queue so the queue is empty then we set up discovered and discovered will be false initially and it will have the length so we want to mark it false for all the elements okay and remember now we can use this notation here because false is an immutable value so it doesn't matter so so we don't really need to use the range or the list comprehension notation here then here let's come here so we mark the label root as discovered so discovered of source let's just call it root so that we don't get confused with the terminology so we mark discovered of root as true great then we insert or we enqueue the root so we we type q dot append now enqueue simply means adding something to the end and you know how to do that in a list you simply call q dot append so q dot append root great now python list by default do not support a dq operation so what we will do is we will set up an index which will track the first available element in the queue okay so whenever we dequeue an element we will increase the index so that we move forward so here we have the index idx equal to zero so now while there are elements in the queue which means while the next available index is less than the length of the queue first we will get the current we will dq so dequeuing simply means getting the getting the first in element the element that was most recently inserted and has not been dequeued so we get current is q of idx and then we can also increase idx so as soon as we dequeue something we update the index so you can imagine that the index starts out here and when we dequeue this or delete this then we get that value out and then we update the index to the next position okay so now we have the current this is the dq operation then what do we have next now we want to check all the edges of current right so we are going to say for so remember we have the adjacency list representation so we will get for node in self.data current so self.datacurrent contains a list of all the nodes that are connected with the current node so for node in self.datacurrent if not discovered node so if you have not yet discovered the node then we first mark it at this as discovered and then we add it to the queue so we do q dot append node okay so what you end up with this way is first you have the source that got added to queue and then we inserted all the inserted all the nodes which were at a distance one from source and then we insert then if you follow the trajectory you'll see that we we will insert all the nodes that are at a distance to from q and so on right so ultimately when we end up with this entire process we will have the q and the queue will contain the list of nodes as they would be visited in a binary in a breadth first search okay so we can simply return the queue here so let's try it out so we have graph one and let's call bfs and graph1 is this graph so let's grab this image as well so let's simply copy the code for the image and come down here let's come down here and put the image here okay let's call bfs on graph one starting at the node three okay of course this should be called graph.data so because graph is the graph that we're working with so we need to check graph or data here okay so we start out with the note 3 and you can see that three first causes one two and four to get inserted and then one causes two to get inserted okay now that's bfs for you it's pretty much done at this point but what would also be helpful is maybe to keep track of what is the distance of each node right so we can also track we can also keep track of a distance so let's say we have a distance which we initially set to none or yeah which we initially set to none and we will track a distance for from each for each node so we have the distance here and initially we are going to set the distance for the root to zero of course because the root is at zero distance from itself and the distance here means the number of edges right then when something is discovered so when we are discovering a node and that node was not already previously discovered that means that the distance for that node is one more than the distance for the current node which caused it to be discovered right so the distance for so for example if you're starting with three the distance for one is one more than three which caused one to be discovered and the distance for zero is going to be 1 more than 1 which caused 0 to be discovered so that's the distance great we've now also tracked the distance one other thing that would be nice to have is what is called the parent if you see if you go back here you can see that it would be nice to know what led to carl's room being discovered was it mannheim wurzburg or castle so that we can work our way backwards and find a path from frankfurt to carl's room okay so for that what we can do is we can keep track of a dictionary of a list called parent once again we will have no parents uh by default so parent none and whenever we find a node and that node was not already discovered then we can set the parent of that node to the current node which caused it to be discovered okay and now we can return from the queue the distance and the parent so let's see if that works okay so it seems like now we have these are this is the this is the order in which the nodes are being visited you can see that 3 is the first node to be visited and 3 has and if you want to check the distance of 3 you can see that the distance of three is zero so this is distance is given in the order of the nodes in the order of the original numbering of the node so you can see that three is at a distance zero from itself obviously then you have one two and four now if you want to check the distance of one just check the index number one here so one is at a distance one if you want to check the distance of two now that is at a distance of one as well you can check here and then you want to check the distance of four 4 is also at a distance of 1 right so all of these 1 2 and 4 are at a distance of 1 from the root node 3 and also you can see here that the parent of 1 remember these are 0 1 2 3 4. these are the indices of the nodes so the parent of one is three and the parent of two is three as well and the parent of four is three three itself does not have a parent that's why this is none and finally the last node we visit is zero and it is at a distance two you can see it is the this distance here is indeed the highest and the parent for zero is one right so because one was the first node that caused zero to be visited it could have been four two but in this case just how we implemented it one was the first node which caused it to be visited so one is the parent of zero so if you now want to find the path from three to zero you can look at the parent of zero that would be one and then you can look at the parent of one that would be three and we are done so we can work backwards from the target we can keep checking the parent after parent of the target and that will give us the entire path so now we have the path we have the distance and we have the order in which these nodes will be visited so you may get asked bret first search in all these different variations but roughly this is what the code looks like and you can see here that the code is not too long now we have created all these additional additional lists but you don't really need them so the code is about 15 lines of code 10 to 15 12 to 15 lines of code not more than that so that's bfs again if you're working on a bfs problem it always helps to first state it in simple words and work it out with an example and then start coding so that you do not make mistakes while coding now one question that you can work on is to check if all the nodes in a graph are connected this may not always be the case so for example here you can see that all the nodes in the graph are connected but sometimes you may have a situation where some nodes are not connected for instance if these edges 1 1 2 and 3 2 weren't present then 2 would not be connected to 0 and maybe 2 is connected to 5 and 6 etc so here is one graph where not all the nodes are connected to each other you can see that there are nine nodes but there are only eight edges and if you look carefully you will see that 0 1 2 3 0 1 2 3 are connected but there is no connection from these nodes to 4 so 4 5 6 are then connected separately and then seven eight are connected to each other but not to one another right so can you use breadth first search to determine if all the nodes in a graph are connected i would reckon yes look at this q now this queue gives you all the nodes that starting from the source node are connected to the source node by zero one two three or so many steps if something is not connected it will not show up in the queue so you can simply check the length of the queue and see if that is less than the total number of nodes and then use that to determine if all the nodes are connected or not now another related question that you may get asked is to find the number of connected components in the graph now what's a connected component if you take a set of nodes that's connected that's one component and if you remove that then you look at the next set of nodes that's connected that's two components if you remove that then you take the next set of nodes that connected that are connected that and that gives you the third connected component and so on so in this case for example you have this is one connected component you can check by drawing the graph and then this would be one connected component and then these would form one connected component so zero one two three would be one connected component four five six would be another and seven eight would be another can you find the number of connected components or even can you list all the connected components of a graph using bfs yes you can again a very simple way to do it is just pick the first node perform bfs from the first node that gives you the connected component that contains the first node then find the first index which is the first node which is not yet visited start bfs from that node now that will give you the connected component for the second node and then find and then keep doing keep repeating this till all the nodes have been visited okay that's another question that you might get find the number of connected components or find a list all the connected components in a graph the bfs is a very versatile algorithm that can be applied to solve pretty much most graph problems that you may get asked in an interview so do do work on a few bfs problems and get some practice with it now another way to work through a graph to look through a graph is what is called dfs and this is the way in which you would normally explore a maze where you start out in one direction and then keep going so for example we started out here and then we kept going till we hit an end right so you can see here that we kept going until we hit an end and then we turned back and then we tried the next path and then we turned back and tried the next path and so on so we go like this then we turn back we try 5 go like this turn back we try 8 then we turn back try 9 10. that's another way to go about it and it's some cases in some cases bfs makes more sense in some cases dfs makes more sense and you can in most cases both of them work just fine for most problems so you can implement either one when you are faced with a graph problem so let's implement dfs or depth first search okay now here is a depth first search it's pretty straightforward you have you pick a node and then you pursue the the node and the next node then the next node and so on among the edges you pick one node and then once once you've exhausted the path along one edge you come back and try the next edge and then you come back and try the next edge so there are two ways to write it there is a way to write it recursively and then there is a way to write it without recursion and i'll leave it as an exercise for you to write it recursively but what we'll do is we will write it without recursion and you write without recursion we will use something called a stack we will use a stack and a stack is another data structure very simple list like data structure but it's just like a queue but it's different instead of being first in first out which is what we do in a queue in a stack we perform last in first out so here's how it works you start with an empty stack so you can think of it like this container or a cookie jar and you start putting in things into that jar you put in one and you put in two and you put in three so now when you have to remove an element from the stack or you want to access an element from the stack the only element that you can access is the element that was inserted most recently so last in first out that's a stack how is that going to be useful it's pretty straightforward if you think about it because this node when you start from this as the source you will add all these three into the let's say you add these three into the stack now if you add these three uh let's add them in this order so you start with this node then you add this this and this so you add these three into the stack then the last in value was two okay so then what you do is you extract two out and then you insert everything that two is connected to into the stack so you insert three into the stack and then you the last in value was three so you insert you take out three and then you insert four into the stack then the last in value was four then you take out four and you have nothing left to insert so now this entire path has been exhausted so then you end up with this five now when you end up with five you can insert its name neighbors eight and six into the stack and once six gets inserted into the stack uh then you take out six and you put seven into the stack and so on right so you can see how depth first search is working using a stack and roughly this is what the procedure the process looks like you start a stack it's empty push push the current source let's say the root node which you are starting with put the root into the stack now while the stack is not empty pop the stack so get the last 10 value from the stack and that gets removed as soon as we call pop then if that node is not already discovered then we mark it as discovered and then for all the edges from v to w so for all of its neighbors we simply push them into the stack right so that's it that's all we are doing all of its neighbors which are not already visited we can simply push push them into the stack okay so let's do that let's implement dfs and once again we will keep this picture in mind so let me just grab this picture here as well this is one of the nice things about jupiter that you can take these images and simply include them within your jupyter notebook while coding so that you don't make any mistakes so let's say we're writing define dfs and once again let's assume that we are going to start from 3 and this picture is graph one so let's say we are starting from three so define dfs graph and we have a root node that we want to start with and the first thing we want to do is we want to create a stack and you can use a list as a stack adding you can simply add things to the end and then pop them from the end so we create the stack and then we find discovered we mark discovered as false for every node len graft or data then we say stack dot insert so stack dot append so we simply add the number three to the end or the root number to the end so stack dot append root and then we don't mark it as discovered yet now this is the interesting thing in dfs because remember when you start out with three you want you don't want to mark four one and two all of them as discovered you want to put them into the stack but only when they come out you want to mark it as discovered because you want to discover four and then you want to discover zero before you discover one so that's why we put these into the stack but we don't really mark them as discovered just yet so that's why we are not marking the root as discovered then while lens stack is greater than zero we get the current value so the current value would be stack dot pop so interestingly python lists two support a pop operation so if you have a list and then you do l1 dot pop you can see that the value v that you get from l one dot pop is the value two and l one now has the value five comma six okay so you can use a dictionary or you can use a python list like a stack in fact we can even try append here to see the entire process let's say we are appending three and then we are popping three so we get back three and five six two remains so we pop the current node and then we mark it as discovered so we mark it as discovered here discovered of current is true and we may also just want to store that this is the result that we have so we may also just want to create a result list where every time we pop something we are also going to add it to the result list so let's say result dot append current and then we are finally going to return the result okay but here's the main logic so for all the nodes in graph dot data current we are simply going to push those nodes into the stack so we are simply going to say stack dot append node okay so what we do is we start with three and we then pop three and add it to the result and then we put one two and four into the stack we don't mark them as discovered yet then we pop one and then we put all of these zero two three four into the stack we don't mark them as discovered yet we mark one is discovered now then we pop zero because sorry then we pop four not one because we insert one to four so four is the last inserted value so then we pop for four we mark it as discovered and then we insert zero one and three now you can see that there is some repetition here we're also inserting three once again so just to avoid that what we can do is we can say if not discovered node only then add it to the stack right there's no point in adding something to a stack if it is not all if it is already discovered so now with that in mind let's see we start with three and then we insert one two and four great four is the last value inserted so three is discovered now four is the last value inserted so we pop four and then we insert 0 1 but we don't insert 3 because it's visited so now 1 is the last value inserted then we pop 1 and we try to insert some of these other values it seems like everything is already inserted so nothing will get inserted then the only thing that remains is zero so we pop zero then we pop once we have popped zero we are going to pop four so the order in which we expect to see things is three four one zero two i believe let's see dfs graph one starting at the node three okay so it looks like we have zero one so it looks like we made a mistake because we got some repeated values here and that's because we may want to just check if not discovered current we may want to just add this check and put everything inside this check so that any older values that have been inserted into the stack which are already visited later sometime through another value in the stack that gets ignored so we end up with three one three four one two zero right so it goes like this first we go from three to four to one to 2 and then we go from 3 to 4 to 0. so that's how it goes now a challenge for you is to also implement distance now in this case the distance will not really make sense because this is not the shortest distance anymore so when you want to get shortest distance from one node to another then you want to use bfs not dfs because if you track distance here you may end up going by dfs three to four to one to two and that is going to give you a distance of or a distance of three to getting to two although the shorter distance is one so maybe distance doesn't make sense here but what you may want to put in is the parent you may want to track the parent for each node should be simple enough to do whenever you are popping something you may just want to track its parent okay that's an exercise for you another exercise that you can try is to write a function to detect a cycle in a graph now when you're performing dfs let's say you are going about performing dfs starting at one and you do this and then you end up here back at one right because you go to from one to two two to zero and when you notice that zero points to one which is already visited that gives you an indication that there is a cycle in the graph a cycle is simply a path which leads from a node to itself so 1 2 0 1 is a path and a path is something a path is a sequence of edges so 1 2 is an edge 2 0 is an edge and 0 1 is the edge so this is a path but 1 2 and 2 4 is not an edge so 1 to 4 is not a valid path right so cycle is simply a path that leads a node leads from a node to itself so the challenge for you is to write a function to detect a cycle in a graph another challenge for you is to detect maybe the number of cycles in a graph okay so that's another thing that you can try out but we will move on to another problem now we will talk about weighted graphs and get closer to that example of the railway map that we looked at initially so here you have nodes so you have nodes numbered from 0 to 8 so you have a total of 9 nodes and you have edges too now these edges also have weights and this could be distances for example the railway line or this could represent any other information which is of value to you right so you decide what edge weights are what they mean in the abstract representation we simply call them weights so this is a weighted graph and here is an example of how we can convey the information about a weighted graph i can give you the number of nodes and then i can give you a list of edges so the first two elements of each edge tell you which nodes are connected like the nodes 0 and 1 are connected here and then the last element of the list or the third element of the list tells you if it is weighted if there is a weight associated with the edge okay so you have 0 1 3 and then you have 0 3 2 so 0 is connected to 3 and it has a weight 2 and so on and you can verify that there are 10 edges here and these are the 10 edges with the 10 weights so that's one variation that we see in graphs here is another variation this is called a directed graph in this case edges have a certain direction so this corresponds to the example of hyperlinks where we have pages web pages on the internet and one page can link to the other but the other page may not necessarily link back they may in which case you may have a bidirectional edge but in most cases there you would have a single unidirectional edge so you have 0 1 1 2 and 2 3. now directed graphs can be represented just the same way as undirected graphs all we need to do is we need to provide some information that this is a directed graph right so you can simply say uh directed equals true and that will simply and once you provide all these all this information that can then specify to the person who is going through this data that this is a directed graph right uh so here's how it's exactly the same as a normal undirected graph but when we create the adjacency list we can have a graph we can have a node from 0 to 1 but we should not put 0 into the adjacency list for one because there's no way from uh to there's no direct h from one to zero there's only a direct edge from zero to one so keep that in mind and similarly in the adjacency list now you will not set both the values zero one and one zero two one you will only set one of them corresponding to the one direction unless of course there is a bi-directional edge and what we can do is we can even combine directed graphs and weighted graphs so here's the here's what we'll do we will define a class which can represent weighted and directed graphs in python so we'll use it to represent undirected graphs directed graphs and weighted graphs all of these and we will take some information in the constructor to capture this detail so let's say let's create a class graph once again we will create a constructor now this has the self which is the object that gets created always the first argument to any method in a class in python then we take the num nodes then we take the edges and then we take a couple more arguments we take a argument directed which has a default value false and we take the argument weighted which has a default value false okay and we're going to store the information self dot directed let's store self dot num nodes as num nodes self dot directed as directed self dot weighted is weighted okay so now we come to the edges so for edge in edges what do we do now an edge can either have two values or three values if it is weighted if it is unweighted then it will have two values if it is weighted then it will have three values so we need an if condition here if self dot weighted then include weights else work without weights okay now we may want to also because we need to create an adjacency list so we will create self.data just as we have been doing so far and in self.data we will create a list of empty lists as we have done for underscore in range num edges now what we'll do along with self.data we will also create something called self.weight and self.weight will store for each corresponding value in the adjacency list it will store the weight of the edge between the two elements so for under and you will see how it works in just a moment num edges okay so we have self.data and self.wait and this will make it easier another way you can do it is instead of storing single values you can store tuples inside self.data which will correspond to the node and which will also contain the weight right so that both these are both ways to do it i'm just doing it this way but you can do it the other way as well where you can store tuples directly inside self.data suppose it is weighted then first we get the values out of the edge so node 1 node 2 and weight from the edge remember the edge is a tuple if and then first we set self.data node1 and append to it node2 and then we also set self.wait node1 so at the exact same location where we have node 2 at the exact same index we store the weight between the of the edge between node 1 and node 2 which is weight okay so now we've stored one direction which is node one to node two we may also need to store the second direction so if not directed so if if the graph is not directed only then we need to store the second direction so we just say self.data node two dot append node one and then self dot data node two dot append wait okay and that's the case when it is weighted if it is not weighted well the code is actually simpler so we simply get node 1 and node 2 from the edge and we say self.data node 1. dot append node 2 and then if not directed so there's no wait here so we simply check if the graph is not directed self.data node2 dot append node one okay so there's a bit of code here but the code is again fairly straightforward it's just a couple of things that we have to take care of whether it's weighted or not whether it's directed or not but now that we've done this we have a fairly generic representation for a graph right so now we can take this graph and remember graph one the graph one had this information so similarly we can we can create this graph we can use this graph class to represent graph one but we can also use it to represent one of these which is a directed graph a graph with weights or a graph with directed edges or a graph with both a graph with both weights and directed edges which we'll see in just a moment now one thing that we'll also do here is create a nice representation so let's just create a representation here now i'm not going to get into the code of this but roughly what we want is we want while showing the graph if there is a weight we also want to show the weight we'll show the weight alongside the other node so let's see we create a result the result will be this the empty string and then we'll return that result then we are going to say for i comma nodes comma weights in enumerate self.data and self.weight so now this is an exercise for you to figure out what exactly this is doing and you can apply the exact same technique take create a new create a new cell and put this data into a cell put the zip into a cell and then see what that represents if if you're not able to if it doesn't show something then try converting it into a list or using it in a for loop and then put enumerate around it and see what that represents so that you understand what i nodes and weights represent but i am simply going to write it here so that you see the final result okay so let's take nom nodes one once again and edges one it was called num nodes and edges so this was the initial data that we were working with let's create graph one and of course we want to do this only if it is weighted so if self dot weighted if it is not weighted then we have a different case where for i common nodes in enumerate self.data result plus equals okay let's see so graph one we are going to use the graph and we're going to pass num nodes edges and by default weighted and directed are both false so we don't need to specify them and let's see graph one this should be num nodes so you can see with live coding we always make mistakes and it's almost always bound to happen that's where jupyter notebooks are very helpful and it's always helpful to just test your function while you're writing it okay so now we've created graph one and graph one you can see is an undirected graph you can see that zero points to one and one points to zero then let's look at graph two so we're going to grab this data this contains let's call this num nodes two and edges two this is a graph with weights so now let's create a graph too graph and here we pass in num nodes to edges too and weighted equals true and let's say graph two okay there's a small change here yeah so now you can see for graph 2 this was the graph we were looking at here this graph let's grab this image as well yeah this is the graph that we were looking at and you can see that zero is connected to one and three and so zero is connected to one three and eight one three and eight and there are also weights associated so zero one has the weight three 0 3 has the weight 2 and 0 8 has the weight 4 and so on there seems to be something off here because 0 1 only seems to be connected to 0. i think we may have made a mistake somewhere in the code okay so we may just have to debug this code it seems like we may have made a made a small mistake somewhere because zero one uh one seems to be connected only to zero but one should also be connected to seven i don't see why that did not show up here this is the curse of life coding and that's why i have created a working i have some working code here i'm simply going to grab the working code right now and we'll just replace that but see if you can detect the bug in the code okay we don't the version i have does not require you to specify weighted so we can simply skip weighted here it detects automatically if the graph is weighted still something wrong here let's just quickly verify what's going wrong so we are going through the list of edges here and we are appending maybe let's just print graph two dot graph2.data maybe the issue is in the representation and not in the code graph two dot edges ah there seems to be some issue in the weight here so we may not have inserted the weights correctly ic so this should be called weight this should be called weight and so should this be called wait oh there was a syntax error here okay i think we fixed it finally let's see this should be called weight so we have an edge here we have too many values to unpack ah we simply pass weighted equals true finally and we need to make this a list it's finally done some good hardcore live debugging but we have this finally and again you get to see that when you're coding you will you will make issues you just need to but if you have a clear idea of how you've written the code it's easier to narrow down the issues by looking at the errors but let's see this graph here so we have zero connected to one three and eight and that's you can see that here one three and eight are zero connected to one three eight with the weights three two and four then we have three connected to zero two and four so we have three connected to zero two and four and we have six connected to five and eight you can see six connected to five with the value eight so great we have now represented our graph properly and this is why a representation is really useful because now we can check if our implementation is correct before we go on and implement any graph algorithms we can check if our representation is correct let's try one more let us also try this directed graph so we're going to grab this code and put it here let's call this num nodes three edges three and directed three let me grab this graph code here as well we are working with this graph and let's create graph three so for graph three we have graph and we pass in num nodes three we pass in edges three and you can verify that the edges are set up correctly and we just specify directed equals true so we don't really need this at this point we can just say directed true and weighted by default is automatically false so we have graph 3 here you can see that 0 is connected to 1 and 1 is connected to 2 but not to 0 so now we haven't inserted the opposite edge and then 2 is connected to 3 and 4 then 3 is connected to 0 and 4 is connected to 2. great so we've implemented we've now set up another graph and now here similarly you can check that if you have a weighted directed graph the code is still going to work fine that's an exercise for you and at this point let us just save our notebook using joven.com so the next question that we are going to look at is called the shortest path question and this is really what we started out with let's say you have a bunch of nodes and this is we have taken a directed graph here but you need not have a directed graph you can do this with an undirected graph too and that will be an exercise for you but you do need weights here now whenever you're talking about shortest paths in terms of weights that is when this algorithm makes sense now if you do not have weights in the graph then the shortest path can be found simply by performing breadth first search okay so whenever you're asked to find the shortest path the first question you should be answer asking is is there a weight involved or are there no weights now if there are way no weights involved then we're simply concerned with the length of the path the number of nodes in each path and in that case you can simply perform a breadth first a breadth first search but if you have weights whether it's directed or undirected then breadth first search alone may not be enough right because it may turn out that certain parts for instance you go from zero to three so you you go if you go via zero two four and three the length of the path is two plus three five plus four nine but if you sorry the yeah the total size the total size of the length of the path is two plus three five plus four nine but the number of nodes is four zero two three four on the other hand if you go via 0 1 3 in this case the number of nodes is smaller so there's just one in between so 0 1 3 there's just three nodes total but the length of the path is 14 which is far higher right so this could represent that you go to a far off place of via a train and then take a train to something that was actually closer even though there were more stops in a different route okay this is what we're going to implement now we're going to implement an algorithm to identify the shortest path from a given node to a given target okay so now this time we're going to focus our search between a node and a target so what is the shortest path in terms of the total weight of the path not in terms of the number of nodes in the path keep in mind what is the shortest path in terms of the total weight that we can find from a starting node to an end node and roughly the strategy goes like this and the strategy is called the distraus algorithm roughly the strategy goes like this you have the source node and the source node is at a distance 0 from itself so there's nothing there really but the first thing that we know the first and the only thing that we know is that for one of the siblings for one of the neighbors of the source node the direct edge will be the shortest path so for example we have one and we have two now you have directed you have direct edges from you have direct edges from zero to two and you have a direct edge from zero to 1. 0 to 2 has the weight 2 and 0 to 1 has the weight 4. now in this case suppose we had an edge from 2 to 1 and that edge had the weight one then you could go from zero to two with a weight two and then go from zero two to one by a weight one and the total weight you would incur to get to one would simply be three and that would be smaller than the shorter smaller than the direct edge right so even if we are looking at direct connections of the root we can't say that the direct edge is the shortest path except for one of the nodes right so if we just look at the node where the edge weight is the smallest so you start at the root and you look at the edge with the smallest weight then we can say for sure that the shortest path from the root to the next node to the node to is the direct edge why because this direct edge is smaller than or smaller than equal to any other direct edge so any other path that comes to two indirectly will contain an another direct edge and then some other edges right so it will have a length greater than or equal to this direct edge right so that's the key inside here that at every point you maintain a group of visited nodes so in this case initially just two zero is visited and then you find the first node which is at the closest distance from any node within the visited group okay so for example if we start out at zero and then we look at 1 and we look at 2 we see the smallest edges 2 so we add 2 into our visited group because we we know that this is the shortest path from 0 to 2 and at this point now we take all of the siblings or all of the neighbors of two and update their weights now because we know that zero to two is a direct shortest path so we can update the distance for four that four could be at a potential distance of two plus three five or there could be a shorter path so we've not yet added it we'll just update four and similarly if there was a edge to one we can update the distance of one and we can say that the distance of one is either 4 which was a direct edge or it can be 2 plus 1 if there was a direct edge from 1 so now we will get to know that 1 is at a distance of 3 which is smaller right in this case it's not but suppose there was a direct edge from 2 to 1 of weight 1 we would get to know that 1 is at a distance 3 so each time you add a new node as you mark a node as visited you you update the weights of or update the distances of all its neighbors and then you simply find the next node with the smallest distance right so you will find that the next node with the smallest distance in this case is four and then you update the numbers of four there's only one neighbor the next node with the smallest distance is three you update the weights of three and so on so that was shortest path in a directed graph but here let's see a shortest path in an undirected graph where we have more such cases let's just watch this from the beginning let's wait for the animation to start again so we start at zero then we check two okay we mark two as updated then we check nine then we mark three as updated then we update the distance of 14 but now we can see here that we have another path to go to 2 or we go to 3 that's why we track that and finally we get 2 we mark 2 as visited now we are considering 3 and using 3 we are updating the weights of all the other graph all the other nodes and then we are marking 3 as visited then we are using 3 to mark 6 as visited and so on right so at each point you have a group of visited nodes and you have distances for all the nodes that are connected with the visited nodes and then you pick the first unvisited node with the smallest distance okay now let's read the algorithm you first mark all nodes as unvisited and then you create a set of all the unvisited nodes and you call it the universal set so a set of all the unvisited nodes is called the universe call it the unvisited set assign to every node a tentative distance value now set it to zero for the initial node because the initial node is at a distance 0 and set it to infinity for all the other nodes so we now set the distance to infinity because we've not yet visited the nodes we don't know their distance then you set the initial node as the current node so there is a always a current node that we're looking at in this case we'll start with the initial node now for the current node consider all of its unvisited neighbors and then calculate their tentative distances through the current node right so you have the current node and the current node is connected to a lot of unvisited nodes and if we look at each unvisited node we know the distance up to the current node because the current node is visited and using that we can calculate distances for the unvisited nodes now if the unvisited nodes have distances set to infinity then we know that the distance from the current node distance y for going wire the current node is going to small is going to be smaller than the distance infinity that has been set but on the other hand if the if a distance has already been set for an unvisited node through some other node then we can simply compare whether it is better to go through the current node or whether it is better to retain the retain the distance that was obtained by some other node and just maintain that right so in this way we simply update the distances of all the unvisited nodes that are neighbors of the current node okay so for example if the current node is a and it is marked with a distance of six and then there is an edge connecting it with a neighbor b and then that edge has the weight or the length two then the distance to go to b through a from the source will be six plus two eight right so from the source to a is six a to b is two so the distance if you want to go to b through a will be six plus two eight on the other hand if b was already previously marked with a distance right so it was not visited but it was just marked with a distance greater than eight then we know that we have found a shorter path via a so we update its distance to 8. on the other hand if we have a value let's say the value of for visiting b by an another node d was seven so we keep the distance as seven right so we're simply updating the distance we are not yet marking these new we are not yet marking b as visited now when we are done updating all the distances for the current node then we mark the current node as visited and of course we remove it from the unvisited set right so we mark the current node as visited then a visited node will never be checked again because once you have visited a node you have found the shortest paths to it and you have used it to update the distances of all its neighbors you never need to visit it again so then find the first unvisited node find the first unvisited node that is marked with the smallest distance right so now we have a bunch of visited nodes and then we have a bunch of unvisited nodes many of those unvisited nodes have been marked with a distance so you simply get the first unvisited node with the smallest distance and make it the current node and then repeat the process okay so you start out with zero you see that you can mark two as you can mark the distances of four one and two so one gets the distance four and two gets the distance two now then you mark zero as visited now you see that the node with the least the unvisited node with the least distance is two so you get two and then you mark the mark the edges from two so you mark the distance for 4 as 2 plus 3 5 and suppose 2 had an edge to 1 then you would mark the distance for 1 as 2 plus 1 if 1 was the weight of the edge let's say you would mark the distance for 1 as the minimum of 4 and 2 plus 1 so which will be 3 so you can mark the distance for a 1s3 and that's it and then you remove 2 from the unvisited set next you find the next unvisited node the which has the lowest distance so if this edge existed that would be one but if since this says if this h does not exist that would be four so you get four and then you mark the distances for the neighbors of four and so on okay so what we'll do is we will create this we create this graph here which contains okay there should be a graph here that we can look at yeah so we'll create this graph here which contains uh 0 to 6 which contains 6 nodes 0 to 5. this is the graph we are creating let's just put it here this graph yeah so this is a graph that we'll work with and let's start writing our shortest path algorithm so def shortest path and we have a graph and that's it we have a we have a start node so let's call it source and then we have a target node that node that we want to get to so we want to go from zero to five and as soon as we have the as soon as we mark the target node as visited our algorithm is done right so first we mark everything as unvisited by setting visited false times lan graph.data so here we have marked visited then we have distance so we take we take the distance as infinity so here's a way to create infinity in python you just say float in and once again we set all distances to infinity then we are going to maintain a queue so because we have this first in first out kind of structure so we're going to maintain a queue the first thing we'll do is we will mark the distance for the source node as zero then we can insert the source node into q so q dot insert or q dot append source and then we'll set an index to keep track of what is the next element that we need to dequeue so the first element is what we need to dequeue so while index is less than zero and not visited target so while index is less than the length of the queue and the target is not visited so what do we need to do we need to get the current element from the queue so we simply get q of idx and then we increment increment idx by 1 so we increment idx by 1 here then we need to take all the neighbors of queue all the neighbor we also need to finally mark it as visited so let's just put in visited current equals true here but in between what we need to do is we need to update the distances of all the neighbors and then we also need to find the next node with the find the first unvisited node with the smallest distance okay so to update the distance of all the neighbors we have written a function called update distance so we'll call this function update distance or update distances where we will pass in the graph and we will pass in the current node and we will pass in the distance matrix or the distance array and we pass it in this way and what update distances does let's look let's look at it here and again it's always a good idea to extract out specific pieces of logic into separate functions so here you we're calling update distances where we have a current node and then we have the graph and then we have the distance so we get the neighbors of the current node using graph dot data draft or data current will give us the neighbors of the current nodes then we get the weights of of the neighbors of the edges connecting the current node to its neighbors so we get the weights as well now we go through each list of neighbors so for i common node in enumerate neighbors and then we check we get the weight so we now we have the node and we have the weight so we have for each edge the node that it is connected to and the weight of the edge and then we check the distance for the node if the distance for the node let's say hasn't already been said then it is infinity so in that case distance to the current node from the source plus the weight of the edge from the current node to the next node will be less than the distance so if the distance flow of current plus weight is less than the distance we simply update the distance of the node on the other hand if the distance of the node has already been set via some other node and that is less than the distance via the current node then we do not update the distance okay so that's all we are doing here and we can ignore this for now we'll come back to it but this is performing exactly that update distances function that we talked about then next we want to find the next unvisited node so here we have a function called pick next node which has a list of distances and it has visited so we want to track the minimum distance so we first set a variable called minimum distance to the value infinity and then we set a variable min node so this is the node with the minimum distance to the value none then we iterate over the all the lists all the nodes in the that we have in the graph so from 0 to n minus 1 and we check that if the node is not visited and the distance of the node is less than the minimum distance we've obtained so far then we set that node to the minimum node and we set the minimum distance to that value okay so we track the minimum distance the running minimum distance by going over all the nodes in the graph and we keep track of which node has the minimum which unvisited node has the minimum distance so finally what pick next node gives us is the first next unvisited node okay so here we can get next node is pick next node and we give it the distance and we give it visited okay so now if there was a next node it's possible that there is no next node because we've probably already visited everything that we can visit so if there is a next node then we enqueue it so we say q dot append next node and that's it that's pretty much it so that is our shortest path algorithm we create a visited list we create a distance list we create a queue where we will add things so this this will be all the all the nodes that we have visited will go through it or go through this one by one and the q in order will give us a list of all the nodes in their order of distance from the source node now what we need to return here is we simply need to return distance of the target and since that was what was asked here let's also mark current as visited through here soon enough so that we don't end up visiting current again and again all right so let's run the shortest path algorithm then here we have a graph this is the same graph that we see here now we can create a graph graph 7 and this is weighted and directed so we will pass in graph we will pass num nodes seven we will pass edges seven and then we will pass weighted equals true and directed equals true and this is graph seven okay this seems like it was it worked out right zero is connected to one and two with the weights four and two respectively and 5 is connected to nothing 4 is connected 3 is connected to 5 4 is connected to 3 okay this looks fine so now we can say shortest path in the graph from let's say from 0 to 5 in graph 7 and it says that the length of the shortest path is 20. so you have 2 3 four eleven so two plus three five five plus four nine nine plus eleven twenty so that seems to be right what would also be nice to get is just to see what that path is and for this we can introduce something called a parent so here we can simply have another thing called a parent which is set to none for each element so visited let's call this parent and let's set it to none by default and all we need to do is whenever we are enqueuing a node we need to track why it got enqueued right so if an if a node is getting enqueued then it is probably getting enqueued so sorry not whenever we are enqueuing whenever we are updating the distance of a node we need to track why why its distance got updated so inside update distances whenever we update the distance of a node we also set the node the parent of the node to that current node from which the distance got updated right and that's all we need to do when we update the distance of a node we need to track why did we update this distance by which node we did we come to update this distance so this way we have now tracked the parent and let's return not just the distance of the target but let's also return the queue and let's return let's just return the parent for now i think this should be fine okay so now you have the parent for each one so if you look at the fifth element 0 1 2 3 4 5 you can see that the parent of 5 is 3 so it seemed like we arrived at 5 from 3 and then if you look at the parent of 3 so zero one two three the parent of three was four it seemed like we arrived at three from four then you look at the parent of two it seems like we arrived at from at four from two then you look at the parent of two and it looks like we arrived there from 0 and 0 was our source so the path is if simply going reverse 0 2 4 3 5 okay and that's how you get the shortest path and not just the shortest path distance now notice that 0 itself does not have a parent because that was the source now you can repeat this with another graph let's say we take this other graph that we had this was graph two so let's grab this image here so let's get graph two and let's say shortest path graph two and let's get the shortest path maybe from zero to seven so it seems like there are two parts one goes by one and one goes by a six two three three two and seven so let's get the shortest path from zero to seven okay so we started out with zero and we end up at 7 so 0 1 2 3 4 5 6 7. it seems like the parent for 7 was 1 and then the parent for 1 was 0. so it's clear that it picked the path 0 1 7 and the total length of the path was seven sounds good we can try another one we can try two and eight so there are a couple of ways to go from two to eight one is to go wire three so you can go to six to other three ways actually but six two uh six z uh you can go at three zero and eight or you can go at three four and eight let's see which one it picks okay so now zero one two three four five six seven eight so the parent for eight is five oh sorry zero one two three four five six seven eight so the parent for eight is four so we came to eight wire four and then the parent for four zero one two three four apparent four four is three so we came to four wire three and then the parent for three zero one two three the parent for three is two so we came to three y six wire two so two three four is the path and the length should be eight plus one nine plus six fifteen great it seems like we've figured out the shortest path once again and this time this was an undirected graph okay so as long as you have weights you can apply this algorithm and this algorithm is called the dystra's algorithm and that's it so that's all we're going to cover today now one thing that we have not looked at very closely is the running time complexities so let's do a quick look at that let's do a quick look at let's say bfs and see if we can identify and get or guess the running time complexity and the full proof is left to you as an exercise but roughly it looks like this this is the main this is the main loop here so where we are going through the queue so the number of times this may happen is n which is the number of n which is the number of nodes and the number of times this might happen now inside each for each node inside bfs remember that we check a full list of nodes inside each node for bfs so the number of times this may happen is equal to the number of for each node we may perform an additional number of steps equal to the number of nodes it is connected to right so if we have n nodes so we have n while loops and then if we have a total of m edges and let's say those m edges are split across if i count the number of edges for each node the number of edges is e1 e2 e3 e4 and so on and then we so the num the num the size of this loop for the node n1 is e1 the size of this loop for the node n2 is e2 the size of this loop for node n3 is e3 so if you add up the list of all the edges e1 plus e2 plus e3 plus e4 so the total number of iterations inside this for loop turns out to be you can see here the total number of iterations inside the for loop will turn out to be the total the sum of all the adjacency lists okay and this total sum of all the adjacency list is equal to twice the number of edges you can see here the number of edges is one two three four five six seven and you can verify that the number of elements of all the adjacency lists put together is 14 because each edge is represented twice right so we end up if we have n so if you have n f n vertices and m edges we end up with n plus two m operations right so each of the n operations to start the while loop and then each of the two m operations those are to iterate over each adjacency list right and now when we are talking about complexities we can ignore the m if m is the number of edges we can ignore the factor 2 associated with it so what we end up with with is order of n plus m so order of n plus m is the complexity of breadth first search and now by this by this point you should be able to just work it out by looking at the code so do try it out and if you if it's not clear do ask on the forum but order of n plus m is the complexity of brett first search and you will find a similar complexity for depth first search as well order of n plus m for the shortest path algorithm however the complexity will be different because in the shortest path algorithm let's see it here in the shortest path algorithm what we do is we go over all the vertices so that's we insert each vertex or each node into the queue once and then we take it out once so this contributes a factor n then when we are saying update distances then it also contributes the factor m but when we are picking the next node we may we visit all the vertices once again right so here we are performing n operations inside when we are picking the next node so that gives us order of n square plus nm n square plus m yeah something like that so order of n square plus m or n plus m into n so that's you those are some complexities that you will see reported for shortest path and a way to improve this a way to improve the picking of the next node is to use what is called a min heap so that you don't have to look through the entire list of nodes each time to pick the next node but you can simply pick the next node in a very short time so there's a data structure called a min heap that you can look at the min heap allows is used to keep track of a bunch of numbers and easily track the minimum so you can keep a bunch of numbers around in a binary tree like this and the root will always be the minimum and the numbers on the left and right will always be larger than the root and then the same will be true for each subtree as well an insertion into this heap is of order login and deletion into this heap is of order login as well and then the min max in this case fetching the min or the maximum value is of order one so instead of mainta instead of looping through the entire list of nodes each time what you can do is you can simply insert nodes into this min heap and delete nodes from the min heap when they become visited and getting the next node is as simple as fetching the minimum value okay so check this out this is not something that will generally get asked this is a more advanced concept in fact even the distraught shortest path algorithm it's very unlikely that you will get asked but do review it and do try as an exercise if you want to go further try implementing and improving the distress algorithm using a binary heap so that will take the complexity for from m plus n times n to m plus n times log n okay and that may be better so do check that out that's obviously going to be better for larger graphs so do try to implement it in fact inside python there is a built-in heap called the heap queue data structure and that will that will optimize the pick next node operation in the distress algorithm so that concludes our discussion of graphs here there's a lot more in graphs graph theory is an entire course in itself but since this course is particularly concentrated on data structures and algorithms from the perspective of coding interviews and coding assessments this is as far as we need to go so what you should do is you should practice more graph problems related to breadth first search and depth first search that is really something that you need to become very familiar with breadth first and depth first search and shortest path may be in sometimes some some really hard interviews you may get asked the shortest path as well so do familiarize yourself with that but apart from that you don't really need a lot more but there are other algorithms you can look at minimum spanning trees you can look at topological sorting you can look at connected components that's another path you can look at detection of cycles and there's something called disjoint sets so there's a huge huge number of topics that we can cover in graphs but we'll stop our discussion here so what do you do next review the lecture video and execute the jupiter notebook complete the assignment and attempt the optional questions and finally participate in forum discussions very important if you're stuck at any point just go on the forum ask a question you can also share your code as long as it's not working to get help and you can also join or start a study group to learn together with friends and you can also find us on twitter at chopin ml and akash in s and the next lesson is data structures and algorithms in in data structures and algorithms is python interview tips tricks and practical advice thank you hello and welcome to data structures and algorithms in python this is an online certification course being conducted by jovian today we're on lesson six python interview tips tricks and practical advice this is the final lesson of this course so i hope you're excited my name is akash and i'm your instructor you can find me on at akashenist if you've been following along with this course and you have been working on the assignments and if you complete a course project as well then you can earn a certificate of accomplishment for the course which you can find on your jovian profile and also add to linkedin or download as pdf so let's get started first thing we'll do is go to the course website pythondsa.com so this is the course website pythondsa.com this is where you'll find all the information about the course you can watch all the previous lessons lessons one through five and you can also check out the previous assignments assignment one two three and you have the course project as well let's open up lesson six now on lesson six you will be able to find a video recording of the video you're watching right now and here is the code that we will look at today so today we will do something different we will simulate the experience of being in an interview so while we have given you a problem solving template and we recommend that you follow this template for any project or any notebook that you work on any coding problem that you work on and here on the problem solving template we also have a method something that we have been applying throughout this course to different kinds of problems different kinds of data structures and algorithms but in an interview obviously you will not have this template so we'll see how to apply this method during an interview and before we do that let's revise the method so that we can recall it from memory when we are working on the interview problem so here is the systematic strategy that we have been applying so far for solving problems and do check out the previous lessons if you haven't seen them for examples of how to apply it in detail so the step one is to state the problem clearly in your own words and identify the input in output format and then the second step is to come up with some example inputs and outputs and try to cover all the edge cases that you can think of you want to think of all the possible scenarios and that will help you write your code properly then step three is to come up with a correct solution for the problem and state that solution in plain english and then step 4 is to implement the solution and test it using some example input this is important while you're practicing but initially when you come up with a correct solution it will be a simple solution what is often called a brute force solution and in an interview setting you may not have the time to implement it from scratch so you may skip if the brute force solution is too straightforward then step five is to analyze the algorithm's complexity and identify any inefficiencies in the algorithm so what you can do in an interview is come up with the correct solution and describe it to the interviewer and then analyze its complexity directly and start identifying inefficiencies and then move on to apply the right technique to overcome the inefficiency so this is where you need to identify what which one of the techniques that you've learnt in this course do you need to apply is this a binary search problem is this a divide and conquer problem is this related to binary search trees is this something that you can solve in a similar way you'd solve sorting is it important to look at the worst case or average case complexity is this a graph problem or is this a recursion or is this a dynamic programming or a memoization problem so all of these things are something that you have to think about and as you practice more and for more problems so for each of the lessons if you try and practice about five to ten problems then you will start to recognize these patterns and when you're on step six when you're trying to come up with the right technique to overcome the inefficiency the ideas will automatically come to you so practice is very important to succeed in step six and once we have determined how to overcome the inefficiency through the right data structural algorithm then we state that solution implement it analyze the complexity right so this is how your a coding assessment or an interview should proceed for you and let's see let's pick up a coding problem and let's go from there so here we have a coding problem python sub array with the given sum and we read the problem but before that you can see that here this notebook is fairly empty and we what we're trying to do is we're trying to simulate the situation where you are on a call with somebody and they are interviewing you and typically they would be using some platform like a collab edit or maybe a platform where you can also run the code or a platform where the question is somewhere let's say on the right it's already printed it's from a pre-selected database on on the right on the left and on the right you can type your code and you can experiment with it now we're not using any third platform here what we'll do is we'll simply simulate that in our jupyter notebook okay so now we have this notebook running we've clicked the run button on the jobin notebook and here we are now the question is and this is a question that was asked during a coding interview for amazon of course a lot of other companies may ask similar questions too you are given an array of numbers and these numbers are all non-negative you need to find a continuous sub array of the list which adds up to a given sum this is how interviewer might state the problem to you and then they may also tell you an example sometimes they don't and if they don't it's always a good idea to ask for example now you might sometimes feel that maybe if you ask too many questions the interviewer might think that that you don't know this or you're dumb in some way but that's not true it's actually the opposite the more questions you ask the better the interview the better the interviewer is able to convey what they want right now they're busy they're doing five interviews a day and they have their entire day's work sometimes they may just fail to state the question in its entirety and if you don't ask for clarifications you may assume the wrong thing and go ahead and implement something that's completely wrong and that completely deals your interview and trust me it happens more often than you might think okay so we here is one example so let's say if the interviewer did not provide an example you can ask them can you please give me an example for this problem and then they come back to you and they say suppose we have this array one seven four two one three eleven and five these are these are all numbers and they're all non-negative some of these could be 0 as well but suppose we have this array and i give you the number 10 that i want you to find the large i want you to find a continuous sub array of the list which adds up to the given sum which is 10. so then they might also tell you that in this case the solution is this sub array starting from position 4 starting from the number 4 and going all the way up to 3 and you can check that there are no other ways to create 10 like if we took 1 7 that would be 8 and 1 7 4 would be 12. on the other hand 7 4 2 would be 12 again but 4 2 1 3 turns out to be 10 and once again on the right you will not be able to create the total of 10. so this sub-array is what you have to return now what does it mean to return a sub-array to return a sub-array means to return the indices which is the index of the starting term or and the index of the ending term and sometimes we know in python when we're working with ranges typically the end index is outside of the actual data so you could return the index of 4 in the index of 11 so that we so the index of 4 is 0 one two so two is the index of four three four five six index of eleven six so if you return two and six and then i try to access the two to six two colon six range of the list then you will get get this list four two one three in fact that's something that we can very quickly verify here let's say l1 so you have one seven four two one three now if i say that the start index is the start index i and the end index j are 2 and 6 respectively and you can see l 1 of 2 to 6 is 4 2 1 3 right so although j is outside so that doesn't get included when we put it as a range and then we put in 4 2 1 3 and you can also verify that the sum is 10. all right so that's the problem now i've explained it to you in a lot more detail than an interviewer would but this is the process that you have to apply in your own mind what and sometimes what you can also do is you can repeat the problem back to the interviewer that's a great idea you you they've stated the problem to you they've maybe given you an example now you state the problem yourself in simple words remember that was step one so in the same way that i just have you can state the problem and then you have to figure out what are the inputs and the outputs so the input you have an array or array is also a list in python so let's say arr zero let's create let's make this the first an example first input and that would be one seven four two one three and then the target so your target sum is ten that's the input here and then the output that we want to want is so this is the output 0 that would be 2 comma 6 as we've just verified so this is the input and output format always makes sense to just create some variables for that before you start coding the next step is to think of what are all the cases that a function should be able to handle but actually before we do that we should also write a function signature because we know what the input looks like we know what the output is going to look like and we know what so we know what the function should look like so we can just say def and let's call this sub array sum and it's going to take an array it's going to take a target and there's going to be some logic inside it okay all right so that was step one sorry i forgot about the function signature but it always helps to just write the function signature because if you've misunderstood the problem still the interviewer can immediately correct you and tell you hey but you haven't taken a certain input or you've assumed an input which i have not provided okay all right so now we have the function signature now step two remember step two was come up with an exhaustive list of test cases to test the problem so you can do this in comments you can just create some comments and you can say i'm thinking about the problem and i'm just trying to think what are all the cases we need to handle and this is a great quality this is not something people do often but they should because this indicates that you are doing what is called test driven development which means you are thinking about all the ways in which your code code might be used and accounting for those before writing the code so kind of working backwards and it's a very useful way to avoid errors so now the first one could be a generic array where the sub array is in the center somewhere in the center right so which is what we have already seen here now the sub eric would be in the center or the sub array would be at the start or the sub array could be at the end or it's possible that the sub array there is no such sub array so there's no sub array which adds up to 10. you may also have the situation where you have a few zeros so you have a few zeros in the list that's one option here's one thing that can happen this could be that there are multiple sub arrays with the same sum now this is where you might want to just clarify with the interviewer hey what happens if we get two sub arrays which add up to the same number the target and the interviewer might say find the shortest one or find the first one or find any one but it's always good to clarify that one option could be that or you could also ask them what is what happens if there is no sub array that adds up to 10 and then they may tell you you can return none none or you can return -1 or whatever it is or assume that there is always a sub-array so that will help you write your code and then you can obviously you may have to work with the empty array you may also have to work with the sub-array is a single element and whenever we say arrayed we also mean list in python they're practically speaking the same thing for our purposes okay we've listed quite a few test cases and in that process we've come across a few more questions which we've clarified so now we're ready to start solving the problem now at this point what you may want to do is maybe just ask for a couple of minutes and keep a pen and paper close to you i'm going to use this tool instead so i'm going to use this tool instead so keep a pen and paper close to you so that you can work on this problem now let's come up with the simplest possible solution right so we have about two three minutes to come up with the solution and often the simplest solution is pretty obvious so in this case one simple solution could be if i could simply try every sub array then i will find at least one if that adds up to 10 if there is one so all i need to do now each subarray is defined by a start index that is where the first element of the array is and then and n index the end index is just next the next index the first index which is not in the array right so that's how we define the sub array remember so all we need to do is try all such values so all such values i i comma j where i goes from 0 to n minus 1 and where j goes from remember you could start out with the empty sub array so which means j also has the value i so here we are saying i and j both have the value 2 so l1 of 2 to 2 becomes the empty array so j grows from i to all the way beyond the last element which means if the last elements index is n minus 1 so j can go all the way up to n all right so i goes from 0 to n minus 1 and j goes from i to n and each time we start at an i and we check e j so we check j equals 0 and j equals 1 equals 2 j equals 3 4 5 and so on then we move i again and then we start over again and then we say we start with j equal to 0 j equal to 1 j equal to 2 3 4 okay and and we keep doing this to we find an array and we have exhaust this way we'll test all the sub-arrays so the problem is solved so that's the brute force solution and what you should do first of all is explain that brute force solution it may seem that this is an obvious solution what's the point of explaining it but to mention it because at this point the interviewer knows nothing about you so they don't know if you can even come up with a solution to the problem right they're trying to assess can you think about problems and they're trying to assess can you write code now if you don't tell them the brute force solution then they don't even know if you figured out the brute force solution so do tell them the brute force solution and generally you do not have to code it you can do the analysis in your memory in your mind and you can sort of write the code in your mind picture the code and based on that come up with the complexity analysis and directly say that the brute force algorithm will have such and such complexity okay now we will just write the code right now just to be very clear about it in case you have not you are not yet clear on how to write the code but in an interview this is the part which you can skip in the interest of time so f sub array i think it was called sub array sum sub array sum and let's call this server sm1 the first approach that we're taking here we have array1 and that's it we have array and then we have our target and we're saying remember that start i from i goes from 0 to n minus 1 that was the first thing so for i in range 0 to n minus 1 and what's n well n is simply the length of the array length of the array then j goes from i to n oops so i made a small error here this should say 0 to n because even in a range the last value is not taken so j goes from 0 to i to n so for j in the range i 2 this should be n plus 1 then because we want j to go all the way up to n and now we simply check if the sum of array i to j and then we've seen this array i to j is going to give us all the indices starting at i but ending just before j so if the sum of array i to j equals target then we found the answer return i comma j that's it so check if sub array sum equals target and if not let's just return none none maybe this is what we agreed but let's return none and that's it so that's your that's your code it's about one two three four five lines of code maybe six but that's a brute force solution if it's really short it doesn't hurt to write it because it then it's going to sit there and at least as a reference you have it but it's something you can discuss with the interviewer should i i mean if you if you are clear about the brute force solution and you can tell its complexity then you don't have to write one other tip is whenever you're coding it's always helpful to simply add a small comment above so that even if the interviewer is not able to follow your code they can just follow your comments and they can tell if your general strategy is correct right once again reading code is hard and especially when you are not familiar with the coding best practices in the industry the code that you write is sometimes difficult to read so while you learn how to write good code in the meantime it always helps to just mention comments makes it makes their job easier makes them easier it makes it easier for them to evaluate you otherwise you may spend five to ten minutes talking about something in your code which either they misunderstood or you made a typo etc okay so we have here the sub array sum one we've implemented the brute force solution maybe let's also check out some cases in and see if this root 4 solution works correctly so in an interview if you have the ability to run the code you can just run a few samples so let's say i simply take array 0 and target 0 and you get the value 2 6 and remember output 0 also has the value 2 6. so great it seems like our our technique work let's test a few more cases just to be sure sub-array at the end subway at the start let's see if we can fix that so here is array 0. now if i take this remember 4 to 1 3 oops i think i didn't complete it let me also put in 11 comma 5 here yeah so remember 4 to 1 3 is the solution now if we simply take 4 to 1 3 11 5 and call sub array sum and put in this number here and put in once again the target zero was 10 oh this should be subway some one okay yeah so now you can see four two one three is zero one two three which is the range zero to four so it seems to have worked correctly let's do the same thing now list this time let's put this at the end so one seven four two one three eleven five this works fine two two six let's try another one let's try maybe 17 and that probably cannot be found oh it can one two let's see one zero one two three four five probably the sum of all of these four let's do six plus four ten okay now maybe there's a problem here because it seems like 17 is not the right sum so you have 1 plus 7 8 and 8 plus 4 12 12 plus two fourteen fourteen plus four eighteen okay so this seems like a mistake then and we can even check this out so we have l1 that's that let's call that l2 and 2 oh it says one to six i think i misread it so we are ignoring the 0th element so this does add up to 17 okay so 17 does show up let's try 18 which takes up the entire array works fine let's try maybe 4 which should just take the single number so that works fine too let's try 19 that should be none none okay we've tested this extensively and overall our solution seems correct this is the process whenever you write any code you should also test it out and it also gives more confidence to the interviewer but if you do not have the option to test it out if you do if you are not able to run the code right now then simply walk them through an example yourself like look at this example and then walk them through the exam okay so now we have the brute force solution the next step is to analyze the brute force solution now let's analyze it so you have here one for loop and we know that counting for loops helps us count the number of operations then we have another for loop so one for loop can go from zero to n so this may run n times then we have another for loop which goes from i to n plus one let's approximate here and say that it can run at most n minus 1 times or n times so n and inside each of these up at most n and then inside the second for loop you have the sum so this is very important now always carefully observe the operation inside your for loop so you have a sum which can be on an array of i to j now remember i can be 0 and j can have the value n that means in the the largest area that you can work with will have approximately the size n as well right so you have n and inside each of those you do n other loops and inside each loop you do work you do n additions right up at most n addition so that roughly gives you that this is going to be n times n times n so this is going to be an order n cube solution so if you are able to arrive at the order n cube solution at the order n cube complexity without implementing the solution great you have learned it but if you're not able to arrive um at the order n cube solu at the order and cube complexity for the brute force solution then you probably need a little more practice because this should become second nature to you just looking at a problem identifying the simplest solution and then finding the complexity of the simplest solution all right so now we have implemented it tested it and we've identified the complexity remember the next step find the inefficiency and overcome that inefficiency by applying the right technique so let's find the inefficiency then here we have let's say we are at this position so let's say you are looking at seven four two let's say i has the value 1 so you start out with i equal to 1 and j equal to 1 in the inner loop then what we do is we increment j by 1 and then we calculate the sum and this sum is 7. then what we do is we increment j by one more and we calculate this loop and this sum and this sum is seven plus four eleven then we increment this window once again and then we calculate this sum and that is seven plus four plus two so seven plus four eleven plus two 13 and then we move this and then we check it again so we are doing this over and over and over many many times right each time we are doing 7 plus 4 plus 2 plus 1 and 7 plus 4 4 plus 2 plus 1 plus 3 that seems like a lot of additional work maybe we can just avoid that what we can do is we can when we start out with a j we can keep a running sum and each time simply before incrementing j add this upcoming element which is the jth element into that running sum right and that way we don't have to do that entire sum inside each of the inner loops so that's one optimization and this is how you should explain it that's one optimization that i have come up with the second optimization that we can come up with is that the moment the sum the running sum that we're calculating the moment the sum becomes greater than the target value we can skip all of these right so we know that 7 plus 4 is greater than 10 and we know that the array only contains non negative number so what that means is 7 plus 4 plus any of these numbers is always going to be greater than 10 right you can obviously you can see this the number is not going to decrease if we keep adding positive numbers and so as soon as the running sum crosses this value we can break out of the inner loop we do not need to continue and look for higher values of g two optimizations helps to just write them down maintain a running sum so that you don't forget it and find it and second optimization is when some exceeds target break inner loop okay so now we have applied an optimization simply by just looking at the data in a lot of cases it's very straightforward you don't even have to apply any special technique and in this case we found these couple of optimizations so let's apply them so what we'll do is we'll define def subarray sum2 and here once again we have the array and we have target and this time we get the length of the array now once again i goes from the same value so i goes from 0 to n minus 1. nothing changes here so for i in range 0 to n minus 1. now here is where we want to start a running sum so s equals 0 this is our running sum then for j in range remember we start out with i and we'll go all the way oh this i keep making these mistakes all the time and by the way these are called off by one errors here what we did was i wanted to go to at the address n minus 1 but because ranges do not include the final value i put in what i put in n minus 1 was wrong i should be putting in n and i make these mistakes all the time even after many years of coding so always watch out for off by one errors anyway so j can j can take the range of zero of i to n so here we should put in n plus one and now first we want to check if the running sum is equal to the target right so so assume that we've been calculating the running sum step by step and we'll write and and at this current point the sum has become equal to target now if the sum has become equal to target then we simply return i comma j because this sum includes the sum from index i all the way up to just before j so initially the j also has the value i so the sum is 0 which makes sense but if that is not the case we check if it is greater than the target so is it possible that our sum has already exceeded the target in that case we don't need to continue this inner loop we can break out of this inner loop and the way to do that is by simply typing break and then if neither of these held true if neither of these was true so which is that the sum was not equal to the target it was not greater that means it is still less than the target so that means we need to then add array of j into the sum so we can say sum plus equal to oops sum plus equals area of j which is the same as sum equal to sum plus array of j in any case array of sum plus equal to array of j so we have added the jth element now remember if this is the pointer j we added the jth element and then we we will set j to j plus one that will happen automatically when we come into the next iteration and the next iteration will once again check if the sum is equal to the target if it is equal we return i comma j otherwise we check if it is greater than the target if it is so we break if it is still less we increment we move j once again so we add one and then we move j once again and then we check again right so that's our running sum looks good now once again if we were if it was found it would have been returned somewhere here since it seems like it was probably not found so if we come to the very end so here we return none none okay and once again let's test it out so let's try subway sum two it gives you two six subway sum two of none okay seems like there is an issue here yes so this is why you need test cases so it seems that array j took up an invalid value so why is that well that was because j can go to the point of n so the maximum value j can take is n so which means that you have already arrived at this position so now you can no longer increase the sum further right so if you've arrived at this position but you've still not reached the total of 10 then that means you may need to increase it further but you can't increase further so there's no number here to add so what we should do is we should here add a check if j less than n since j can go all the way up to n and that's it so we had a small bug and we fixed it now again this is something that you should work out for yourself on pen and paper so even while doing the optimization you can ask for a couple of minutes play around with it on pen and paper write a few examples relax you can even take up to four five minutes and if you if you're not getting any ideas you can simply talk to the interviewer you can speak out loud explain your thought process and in a lot of cases they will give you a hint because they want to see you succeeding okay so now this is the second implementation let's see okay this time it worked none none four two one three let's put in 10 here this should give you the value two comma six let's put in this so that's zero comma three let's test this out two so that's zero comma four yeah 0 comma 4. so it seems like it's working just fine yeah so seems like this is working pretty well so now we have the second optimized solution so let's look at the optimized solution and analyze it so we have one loop and then we have a second loop these two are the same but inside the second loop we are simply doing a constant operation we are just doing some comparisons and one addition not up to n additions so the complexity goes from order of n cube to order of n square by maintaining a running sum great now this at this point when you've described the solution to the interviewer and maybe also coded it you might ask them is this good enough and they can see that you've ex you've thought about it you've found the solution and you have tested it and it tests well and at this point they may just say i'm happy with the solution this is good enough or they may say can you do better now when they say can you do better most of the time it suggests that there is a better solution so let's see let's think about it a little more and let's see if there is a better solution now you can to can you do better we apply the exact same technique we have analyzed the complexity and now we need to look for inefficiency okay now we have removed the inefficiency on this side which is as we move j that is when we reuse the previous sum to compute the next sum so we've removed the inefficiency on this side and we've also added this also added this condition so that j only goes up to a certain point now of course in the worst case j may always go up all the way to the end but at least in a lot of cases j will not j will not not go beyond a point where the sum becomes larger than the target so these are good optimizations but what about i what about the left window now look at this here when you have seven four or let's start out all the way at one so we have one that's so first we started with the empty empty sub array that has the sum zero then we increment j so now the sum becomes one then we increment j now the sum becomes eight then we increment j once again and now the sum becomes 12. okay the sum has become 12 now that's a problem so what do we do what we are saying is we will take i and set it to the next value and then we will bring j back to zero or back to the value i so that we start with the empty sub array once again so now when we do seven and when we so that's that just has a value seven and when we do this we have to add up seven plus four now here's something that we could have done instead now as soon as the value became larger than the target value we could have simply moved this here does that make sense let's think about it so till this point this total was less than 10. as soon as we added this number on the right this total became more than 10. now we know that this product became more than 10 that means that if we slide this window if we slide the left window forward one step then the total may become less than 10 right it may still become stay larger in this case it stays larger or it may become less than 10. so if the total now becomes less than 10 then we can once again move this but the total has not become less than 10 so we will move this instead so now the total again is less than 10 so we can once again move this and now the total is still less than 10 so we move this now the total is still less than 10 we move this and we encounter 10 here but suppose we had not encountered 10 suppose this number was 4 instead then what we would have to do is move this right and now now the number becomes less than 10 so we always go we always try to maintain a window of size less than 10. the moment the window becomes greater than 10 we keep trying to reduce its size further to less than 10 right or exactly 10 as well it's possible that the size may become exactly 10 and then the problem is solved but we keep trying to reduce its size to a value till it becomes less than 10. so to revise the algorithm we start out with both i and j at zero then we increment j while the running now we have a single running loop and a single loop essentially so we increment j while the sum is less than 10. the moment it becomes greater than 10 we start incrementing i the moment the sum becomes less than 10 or less than target we start incrementing j and if we encounter the point where the sum equals 10 we have found the answer so that's the algorithm so let's write it sub array sum three now this is the array target now we have we have i we have j and we have sum all of them let's call it s because sum is a reserved word in python an existing function so all of these have the value 0 then we say while i is less than len array let's call that n so let's create n equal to len a r r i is less than n and j is less than n plus 1 remember because j can take the value n as well it is the exclusive end index now at this point you want to check first so if the sum s the current sum running sum is equal to the target then we simply return i comma j l if some is le less than the target then we simply increment j okay so now we can move the window forward so we we are incrementing j if the sum is less than the target so we increment j but before we increment j we should add the jth element to maintain the running sum so here we say s plus equals j or array of j and remember j can take the value n as well so that's where we do this only if j is less than n if there is indeed an element for us to add this is an error we faced last time and you will discover this when you write the test anyway and then we say l if s is greater than target and we can also just say else here but just for clarity let's say alif in this case what we want to do is we want to move i forward so suppose we end up in a situation like this and we want to move this forward for that we need to subtract s array of i first so we s we say s minus equals which is equal to s minus which is the same as s equals s minus array of i and then we increment i so we move the left window forward as well so we then repeat this so we first move j to a point then we as soon as we cross the target we start increasing i and then we keep doing that till we match the target and then finally we return none comma none if we have not found it so that's r sub array sum three this is seems like the most optimized solution and let's test it out so here we have sub array sum three and let's test sub-array sum three here as well seems like it worked let's see so if you put in ten here you get two comma six let's say this is four two one three zero comma four let's put in 12 here that doesn't show up let's put in 17 here zero comma five thirteen one comma five let's try 19 that's three comma six let's see one plus three plus 7 plus 9 yeah that has the value 19. let's throw in a zero there and see if it works with zeros that's three comma seven works fine and let's see if it doesn't work out yeah okay great so this solution is correct too again if you don't have the option to run the code you can simply pick one example and walk through the working of the example now we have sub array sum three and once again we are ready to analyze the complexity and in this case the complexity would be this somewhat tricky is a little bit unusual because there is a while loop with two variables but remember that in each while loop either we exit which is the best case so we can ignore that or we either increment j or we increment i right so we increment j or we increment i and if we increment uh so j can go from the value 0 to n and i can go from the values 0 to n minus 1. so the total number of increments can we and we can do is the sum of the number of possible values of i and the number of possible values of j right remember this is not a product this time because this you do not have a nested loop so for each value of i you're not doing this rather you are incrementing each one and i only one of them each time so the sum of total number of values i can take is n the total number of values j can take is n plus one so the total becomes this number of iterations becomes two n plus one now of course there's the you can verify that a constant amount of work is being done here so we finally end up with the conclusion that this is an order n algorithm so this is finally an order n algorithm this is a good example of a problem where the step-by-step solution coming up with a simple solution and then thinking about the inefficiency in the problem and then applying in this case just common sense to solve the inefficiency step by step leads to the perfect solution and a very good solution in fact so you start out with the order n cube solution the order n cube is going to be pretty slow when you start hitting let's say even a thousand even a thousand elements if you have ten thousand elements it will take forever it will take maybe an hour or so if you have a million elements it will take hundreds of years on the other hand order n can work fine all the way up to a billion elements right so there's a huge difference between the sub array sum one two and three so where some three can work instantly for a billion elements so where is some one will take forever even for a hundred thousand elements and sub-array two is in between and you can do the math and this technique where you can almost certainly tell what the next strat what the next step is so this was not really related to any of the algorithms or data structures that we have talked about this is what is called a greedy approach where you know some optimal strategy about the problem in this case you know that we can calculate the sums by maintaining a running sum so we just do that and then you also know that as soon as it becomes greater than a target we need to break out and then you know the next thing that when it becomes greater than target rather you can simply up update i so this is what what is called a greedy approach where you somehow know that just doing this will fix it right there's no real technique to be applied and these problems are somewhat tricky but you get the hang of these problems as well if you search for greedy problems online you get the hang of these by solving a few practice exercises so that's our first interview problem and we've solved it in about 45 minutes and this is approximately how long you will have for an interview a typical 45 minute to one hour interview we'll have about a couple of minutes of introduction maybe a few minutes just you talking about a project and the interviewer asking you questions but then the next 30 to 40 minutes will be dedicated towards solving a problem and this is what roughly the process will look like let's do one more example let's pick another interview question and let's see if we can solve this one so this is slightly different so this gives us one more variation to study by the way to run these you simply click the run button and select run on binder okay so this is an interview question that was asked during a coding interview at google and the question is given two strings a and b find the minimum number of steps required to convert a into b so what you can do is you can perform operations and each operation is counted as one step and the operations you can perform on a word are these you can either insert a character into the word or you can delete a character from the word so for instance here you can see that we are trying to convert intention into execution so either you can insert a character for example you could insert c here or you can delete a character for example you can delete i here or you can replace a character that is you can take n and replace it with e you can take t and replace it with x and e does not need to be replaced and here we've inserted c and then here we substituted n for u right so we have taken the word intention and by performing a few changes character by character by either inserting deleting or replacing a character we have converted it into the string execution so the number of steps required here is one two three four five now here's a challenge for you try and work this out on paper and prove that this is the best solution so because we need to find the minimum number of steps required to convert a to b okay so that's the problem and this is a moderately hard problem and variations of this show up as well so let's start applying the method now when you hear the problem a solution may not strike you up front that's perfectly all right don't panic sometimes when you're not able to immediately come up with a solution or identify how to solve this problem you enter a sort of panic and then you are unable to think don't do that remember have faith in the method and we will apply the method and come up with a solution step by step so the first thing is to state the problem in your own words so given two strings we need to perform operations a series of operations on the first string the operations could be deletion of a character substitution of a character with another character or insertion of a character and through these operations we need to convert it into a second string okay we have understood the problem if the interviewer had not given an example either you can state the example or you can just ask for an example whatever makes works for you so we've stated the problem now what are the inputs to the problem so the inputs are two strings so the inputs are strings like intention and execution so let's see maybe let's call them str1 this is intention str2 this is execution now one thing you have to be careful about here is you do not want to capitalize because sometimes what might happen is this i may match up with an i here in the in the proper solution but python obviously treats small and capital letters differently python doesn't know what's that the i which is lowercase and the i which is uppercase is the same so you will not be able to compare them so just to keep things simple either make everything uppercase or make everything lowercase but yeah this is what the input looks like and the output is going to be a single number so the output is simply going to be the edit distance so let's just call it output one and it is going to be the number five and here is something that you can verify so that's the input that's the output and function signature so of course this term edit distance is how this problem is described but here there is no edit there's no concept of edit distance that's mentioned so you can give a function name that makes sense for this problem so find the minimum number of steps required to convert a to b okay so let's just call it min steps for now so the function definition would be min steps and this would take an str1 and this would take an str2 and it would return an output for now we'll just put in pass here all right so now we have already clarified the problem if you had any questions this is this would have been a good time to ask the interviewer and make sure that you have a clear understanding now you have stated the input output and function signature the problem has been communicated back and forth properly the first step is done the next step is to list out some test cases right once again a very good quality listing out some test cases so you can say that now i'm just going to list out a few cases that i want my function to cover so that they will help me it will help me while writing the code now one is the general case which is listed above so this would be intention execution and we can take a few more examples like this now one example could be where no change is required so you are given the same strings one case could be that all the characters need to be changed so these are the two extreme cases one is no change is required and second is all characters need to be changed maybe added remove deleted lots of such things then you can check both strings of equal length so in this case they are in fact of equal length unequal length you can check both strings of unequal length one of the strings is empty your function should be able to handle that too then you may check things like it will if something only requires deletion if something only requires addition or if something only requires swapping right such things i guess this is pretty good at this point so now we can probably move forward so we have stated some test cases now you don't need to create all the test cases right now in an interview it can take a little take a bit of time so let's just move ahead and the next step is to come up with the simplest solution to the problem which is also called the brute force solution so now we have a lot more information about the problem in this mean time probably it has sunk into you and you may have been able to think of a brute force solution but if not don't worry there is a simple trick i'll tell you which you can apply whenever you are stuck and you can't think of a brute force solution so we're looking at you're looking at it in tension execution what am i going to do am i going to start from the left and right how do i check which one is how do i know if this is going supposed to be inserted or executed or replaced or substituted or deleted so the simple trick is whenever you're in doubt think about recursion see if there is a way to solve this problem recursively and what do you mean by solving a problem recursively can you reduce the overall problem to a combination of one or more sub problems so if you take a portion of the input and can you solve the same problem on the portion of the input and then use that to solve the overall problem okay so let's see let's see if there is a recursive solution possible here so here i have the same thing intention and execution now with recursive solutions normally you either start by looking at the first character or the last character so let's look at the first character character of each string right so we given these two strings and we need to find the number of operations to change this string into this string let's look at the first character now suppose the first characters were in fact equal suppose this was not intention but it this was n tension and this was execution so now we compare the first characters and we know that the first characters are equal okay so if the first characters are equal then obviously neither of them needs to be deleted or removed or obviously this character does not need to be deleted or removed or switched it's already matching so what we can do is we can just ignore the first characters and we can simply look at the remaining string so intention and execution because the first characters are already equal let's write that down so that we don't forget it and this is the recursive solution now this is where you can take a moment to work this out on pen and paper and that's perfectly all right what helps us to just talk keep talking about what you're doing but for recursion now first thing we know is if the first character is equal then ignore from both so just ignore the character first character of both strings and simply recursively solve the problem for the sub list or the sub string without the first characters in each of the strings right so you exclude e and exclude e from this and solve the problem for these two perfect now suppose the first character isn't equal so that's another case now right so that is the case where you have intention and execution so if the sub if the first character is not equal then either the first character has to be deleted or the first character has to be swapped so you may you may have to swap i with e or the first character or maybe something needs to be added before the first character okay now let's see one by one so if the first character is not equal either it has to be deleted or swapped or a character inserted before it there are only three possibilities right of course it's possible that we may do some other things insert characters after after it and so on but at that position after applying an operation either the first character will get deleted or the first character will get swapped and will be changed to e or the first character will now change to something else and the first original first character will become the second character now let's look at each case the first cases it has if it is deleted now the power of the beauty of recursion is that we don't need to guess which solution it is we can try all three recursively and then simply pick the best one so suppose we choose to delete the first character so suppose we say that we are deleting the first character now what that means is we've performed one operation and we've deleted the first character so now what we are left with is this so now what we end up is the second the string has remained the same only the first string has changed where we have lost the first character now what we end up with is with a sub problem where we need to find the minimum number of steps to change n tension nte and tion into execution okay so in this case if it has to be deleted then recursively find then recursively solve after ignoring first character of str1 okay that's one possibility and the you get the recursive solution and you simply add one to it that tells you the solution if you delete the first character the next option is that we change the first character i to e now if we change the first character i to e so one operation has been performed and then now these two have become equal now that these two have become equal we can move this forward and we can move this forward so now we can simply recursively solve the problem for n tension and execution find the minimum edit distance between the two and simply add one to it to get the number of steps required to change intention to execution by swapping the first character right from i to e so in this case you recursively solve after ignoring the first character of each right so it is one plus in both cases it is one plus the recursive solution after ignoring the first character of each because the one operation is something that has been performed okay now the final case the final case is you have intention and execution now we decide that we are going to shift the string forward and we are going to include we are going to introduce an e here so we are going to introduce e here so now what happens is the e is matching the e now i has gone on to the first position i has gone on to the next position here so effectively what has happened is that we need to recursively solve the problem for the original string in tension and the second string with the first character removed because we have inserted something before the first character in the first string so that is going to match with the first character of the second string and hence we simply need to recursively solve the problem for these two in this case what we are doing is the solution is one plus recursively solve after ignoring the first character of str2 okay sounds good looks like we've done that now what's the end solution going to look like the end case remember in recursion this is all well and good but at some point we are going to hit some kind of an end so let's see let's see if we can define such an end scenario so maybe let's say we have been performing recursion and then we ended up at a situation like this where there is nothing left in the second string but you still have some characters left in the first string right so you are at this position now and here this is gone there's nothing left in the second string so in this case to change recursive to change tion into the empty string all we need to do is delete all four so if you have a few character if you if the second string becomes empty then you simply find the number of remaining characters in the first string and delete them so that is the number of operations or the other possibility is that the second string still has some characters but you've run out of characters on the first string if you run out of characters on the first string but the second string still has some characters then in that case what you need to do obviously is you have the empty string and you need to take this convert this empty string into t i o n that is a recursive problem we are solving so you that you can do by adding t i o n great so you add t i o n and that is again going to be four steps which is the number of characters remaining in the second string okay so these are the two end cases now of course if both of them are empty then the answer is zero but if either of them is empty the answer is the number of remaining elements in the other one let's write the solution now we figured out the solution it took some time but again this is not a very straightforward problem so there are a few cases to figure out and while you are doing this while you are identifying each case either you can say it out loud to the instructor or you can write it as a comment whatever you feel more convenient with because the interviewer cannot see the work that you're doing on paper so it's very important for you to be able to convey it and that is why all this while in this course we have been saying that you need to express the solution in simple words because you need to tell the other person that you know the solution and they should be able to understand what you're saying without looking at your work without looking at the images that you've drawn and a great way to do it is either by writing or by speaking let's define it then def what's it called min steps and min steps is it takes str1 and str2 great now we are doing recursion and in recursion what we're tracking is the which character we are currently at so we could be at the zeroth character or the first character or the second character in string one and we could be at the zeroth character second first character second character in string two right so the the starting point of this window determines the sub string that we are solving the problem for so ideally we when we want to solve this problem for these two sub strings we can simply pass those sub strings but creating sub sub list or sub strings as a cost because you have to copy those characters out and then allocate some memory and put them into a new place so an easier way is to simply keep a pointer so we will keep two pointers i1 and i2 and these will signify that we should be skipping while computing min steps we should be skipping the first i1 characters or we should be starting from the i want index and we should be starting from the i tooth index for str2 okay so in your window if the i1 index if the starting if the starting index is equal to the length of string one so this is the end case and remember the end case while coding is always written first so if this is equal to length of str1 then we have known we have seen here that we need to perform these many additions so we simply return in this case str len of str2 minus i2 right and you can verify that this is the amount number of additions required alif on the other hand i2 is equal to len of str 2 so which means that you have exhausted the second string but the first string still has some values left so in this case you need to remove the delete the return remaining values in the first string so you just type len of str1 minus i1 right so these we have now solved the trivial cases now let's see lf str1 of i1 and str2 of i2 which means the first characters of each sub string that we are working with right remember we are just using arrays as a we're just using indices as a optimization what we really want to work with the substring so the first character of each substring str1 of i1 and str2 of i2 is equal now the first character is equal e and i are equal then we simply ignore both and solve the problem for the remaining string so we simply say return main steps and we pass an str1 we pass in str2 and then we simply pass in i1 plus 1 here and we pass in i 2 plus 1 here so what this is saying is that now we want to recursively solve the problem or a substring starting at i plus i 1 plus 1 so we have ignored the first string of the current substring and similarly we have ignored the first character of the current substring or of the second string okay so we ignore the first characters and that's it and there are no steps required here no operations required here right now because the first characters are equal now finally this is the final case else here we want to return one so we have to perform one operation either it is an insertion deletion or swap and what we can do is we can recursively check the cost or the number of minimum steps required for each case of insertion deletion and swapping and simply pick the minimum one and if to do it we add one then we get the total minimum number of steps we need to perform for the entire list right so again recursion is very useful because you can simply assume that you have the function which solves the problem and you simply need to take the result of the sub problem and combine them so we take the minimum off the first option is if the first character of str1 has to be deleted so which is let's say we choose to delete i if we choose to delete i then that means we have to solve the problem for these two so we say 1 plus recursively solve the problem after ignoring the first character of str1 so we solve min steps for str1 str2 now since we have deleted the first character of str1 we can skip ahead into the next because we are solving the problem now for the from starting from the next index and i2 remains the same right so remember here we have not affected i2 so we need to solve this problem recursively so this was the case of deletion next we have the option where you have swapped the first character so we have taken e and we have converted the it in we have taken i converted it into an e if we did that so then we can say that okay now these two characters are matching so now we can simply recursively solve the problem or the next character onwards after ignoring the current character so this becomes str1 plus str2 plus i1 plus plus 1 plus i2 plus 1 great so this is swap or replace and you might notice that this is this turns out to be the same recursive call as this except that we will add one to it because we have done the swap and finally if you are adding so if you're ad inserting so finally if you're inserting here something so if you are inserting e here let's say so in this case what we'll do is now we'll recursively solve the problem for intention and execution without the e in front so we skip the first character of the second string so we have main steps str1 str2 i1 and i2 plus 1. so this is rather nice and symmetric and that's it so this should be it let's run this okay there is a syntax error here that's perfectly fine there needs to be a comma here that's fine too i make a lot of syntax errors all the time and of course off by one errors i'm sure there are a few but yeah this is the minimum number of steps and this is a recursive function not too bad two four six around eight lines of code and let's test out some of the test cases here i'm just going to copy the test cases out here below and let's try as a general case which is intention and exception so let's see main steps intention and exception it says 5 4 okay why does it say 4 maybe let's test let's test a more simpler case first which is one of the strings being empty let's say we have intention and one of the strings is empty so we will need to delete uh let's just say int and one of the strings is empty this looks fine we will need to delete all three of these and that in some way tests out this case where or sorry tests out the second case where the second string is empty now we can test this case in this case also the in this case also the solution is three great looks fine let's test this case where str1 i1 and str2 i2 are equal so if you have integer and let's say you have india so i n i n would be the same so these would get skipped and here is where the recursion would kick in so t would have to be changed to d and then you would have to add i and e okay that looks fine too and let's check in tension and exception once again i don't know what's wrong here let's see so is it possible to do it with four i don't know it's maybe possible to do it with just four changes if you change i you delete i and then you delete n and then you delete t delete i substitute these two i don't think it is possible with just four changes so there's probably an issue [Music] i don't know what's wrong here it's possible i may have made a mistake here let me try another saturday and sunday okay so saturday satur needs to be changed to sunday s u n now s is the same so a t u r needs to be changed to u n so u remains the same now if we can what we can do is we can probably delete a delete t and take replace r with n so this seems to be fine all right so we'll probably unless i'm not seeing this so you have in tension and you have exception unless i'm not seeing something it seems like we may have made a mistake one one thing we could do is we can simply print out the strings that we are checking so let's see str1 is i1 onwards and str 2 is i2 onwards we are first checking intention and exception then we check let's also print the result here okay so at this point i would probably look through the loop here and see if the see if it is correct coming properly so you have intention and exception first we delete i then we delete n then we delete n then we delete t then we delete okay then we compare e and e so then we come back to n and exception and so on i think we'll have this might take some time to fix we'll come back to intention and exception but supposing we've solved the yeah supposing we've written the recursive solution correctly and i do have the recursive solution here so let me just grab that and put that in here let's see what's different okay probably the answer is 4 because i'm still getting 4 but supposing we have the recursive solution here so we have min edit distance this is the recursive solution and now what you need to do is you need to find out the complexity of the recursive solution now to find the complexity of the recursive solution what we can do is simply look at the recursive calls in the worst case so how you start out is you start out with a string of length n1 let's say and a string of length n2 we have one string of length n1 and one string of length n2 then you call either you call this min edit distance with i1 plus 1 and i 2 plus 1 so str1 and str2 you call them with i 1 plus 1 and i 2 plus 1 so that's one possibility or you call three recursive calls now one recursive call is the good case where these two match up so we want to look at the worst case where these two things don't match up so in that case you make three recursive calls right so you make three recursive calls and in each recursive call you are then going to reduce the problem size by one so you're either going to decrease i2 or you're either going to decrease the size of the first string or you're going to decrease the size of the second string or you're going to decrease the sizes of both strings right so just to keep things simple let's assume that in all three we are decreasing the size of either one of the strings by one so we are decreasing the total problem size which is n1 plus n2 by one right so the number of levels of recursion is going to be the total number of total length of each of the two strings so let's maybe just draw that graph here as well so let's take this so here you have n1 comma n two so let's assume these are the lengths of the two strings now n1 plus n2 what happens to it is that this n1 plus n2 calls three recursive functions so there are three recursive functions so let's just draw those three recursive functions so we have those three recursive functions here let's take this two and then those in those three recursive functions what we have is either you reduce either you reduce the size of the first string or you reduce the size of the second string or you reduce the size of both strings so either you end up with n1 minus 1 and n2 and let's reduce the size of that or we end up with n1 and n2 minus 1 or we end up with n1 minus 1 and n2 minus 1 okay so these are the three recursive calls that we're doing and then each of these will once again make three more recursive calls and so on now what is the depth overall depth of this recursive call now because we can see that each time the size of the problem reduces by 1 so if the size of problem is n1 by plus n2 in this case it reduces by 1 in this case it reduces by 1 and in this case it reduces by 2 but for simplification let's say it reduces by 1 here so the total size of the problem the total number of levels in this tree is going to be n1 plus n2 so you have three problems in the first layer the second layer will have three square problems the third layer will have three cube problems three times three times three and similarly you can go ahead and you will find that at the last layer you will have three to the power n plus n two minus one layers right and if you then add together all the layers what you end up with is that total total number of sub problems is 3 to the power n 1 plus n 2 right so you have a total of 3 to the power n 1 plus n 2 sub problems that you end up creating and because of that you have the complexity 3 to the power of n1 plus n2 in this case so that's that's the complexity so here we have a recursive solution and then we have the complexity of the recursive solution which is exponential 3 3 to the power of n1 plus n2 now at this point it will make sense to add memoization so whenever you see recursive solutions and you see repeated problems for example here itself you can see a repeated problem and then you can see that this problem will get repeated inside this problem and inside this problem too so there are a lot of repetitions and all we need to do is remove some of those repetitions and to remove those repetitions we can use memoization so what happens in the memoize solution it is exactly the same as the recursive solution but before doing any computation we check a memo we check a dictionary if we already have the solution for the changing variables which is i1 and i2 and if we have those if we have those solutions what we need to do is just return them directly if we do not have those solutions we need to compute the solutions put them in the memo and then return the value from the memo so let's write the memoize version so we have min edit distance with str1 and str2 and this we are calling memo okay this we are calling memo so now we have a memo the memo is going to be a dictionary and the dictionary is empty and then we define a function recurs so instead in memoization normally you have to write a recursive helper function now you can either write this outside or inside i like writing this inside because well it will have access to str2 and they do not need to be passed in so here we have i1 and i2 and first thing we do is we create a key so the key is i1 comma i2 now if key in memo which means if we have already computed the solution then we simply return memo of key if not then we have all the other cases so now we have alif now we can check if i1 equals len of str1 in that case don't return set the memo of key to len of str2 minus i2 lf i2 equals len of str2 then we return memo of key is len of str1 minus i1 lf okay in this case then we check if the first elements are equal so we have the exact same logic you can see the same cases coming up here so if you have str1 of i1 equals str2 of i2 in this case we have memo of e equals we simply ignore the first character so we increment i1 and i2 so exactly what we have done here so we simply call recurse this time with i 1 plus 1 and i 2 plus 1 right so we always call the recursive function but inside the recursive function if it has already been computed it will return from the memo and finally if we have and this is the final case which is where they are not equal so here memo of key becomes 1 plus min of let's see here so we have recurse so the insertion cases we will ignore the first element sorry the deletion cases we will ignore the first element of the current range from the first string so we recall recurs with i 1 plus 1 and i 2 otherwise we call recurse with i 1 plus 1 and i 1 plus i 2 plus 1 this is the case where we swap the first element of the first string so we can just recursively check after ignoring the first element of each and then we have recurse with i1 comma i2 plus 1 i2 plus 1 and there we go and that's it so now we have stored it in the memo and then we simply return memo of e at the very end and finally we call recurse 0 0 and that is our solution and there is a syntax error you can fix these syntaxes they're easy to fix and i've just realized that the solution in this case might actually be 4 because what we can do is we can change n to p so that's one step we can replace i n t with e x c so we replace i and t with exe that's three changes we don't change e and we replace n with p the solution is four so our solution was correct there was no issue there in fact this is not the best solution this is a sub optimal solution so this output should be four and that's okay this is something that happens all the time where you miss something and you just assume that you just say that you're going to come back to it at the end and then you move forward assuming that that code was right and then you realize that either you were correct or what your mistake was it's probably going to happen in one of five interviews anyway okay so now we've written a memoi solution great and we can start checking the memoi solution now so minimum edit distance memo let's call main edit distance memo and we get back the value 4 looks fine let's try saturday and sunday as we have so that's three so what you will do is you will leave a as it is change a t u r to u n by removing a t and changing r to n that seems fine let's test out some cases like this okay this is three six eight characters so that seems right we simply delete all the characters let's check out this here also eight characters we have to add eight characters let's say we have abc and xyz so this should be three if it is x y z k then maybe that will be four what if it's x y z a in this case also it's four so this seems to be working fine we have now taken the recursive solution identified the inefficiency calculated the complexity which was exponential identified the inefficiency and the which was repeated sub problems and then fixed the inefficiency by calling min edit dist by using memoization and now how do you compute the time complexity of memoization well the argument is if you only need to compute the solution for a key once and the computation apart from the recursive calls simply involves some comparison and a fixed number of comparison and an addition so the time required to compute assuming you have the recursive solutions is constant so that means if you simply count the number of memoizations that can possibly occur that gives you an upper bound on the total num number of operations it will be some multiple of that some constant multiple so i1 can take the value 0 to n 1 where n 1 is the straight length of string 1 and i 2 can take the value 0 to n 2 where n 2 is the length of string 2. so memo the keys in memo can be i 1 comma i 2 so you have n while n 1 values for i1 and 2 values for i2 so that makes it n1 times n2 that's the number of keys and that because there's a constant amount of time additional time required to compute the solution for a key that is also the complexity so the complexity is order n1 plus n2 so we've gone from 3 to the power of n1 plus n2 which grows very quickly even for 3 to the power of 3 to the power of 10 is pretty high we can check it out here 3 to the power of 10 is something like 59 000 3 to the power of 100 so if you have n one plus and 2 then that's e to the 47 that's going to be a lot of operations on the other hand if or in with memoization it is only going to take let's say 100 is split as two strings of length 50 and 50. only going to take 2500 operation so where it was taking 10 to the 47 operations now it takes only 2 500 operations which is pretty small so you can still work with lists of size up to 10 000 or 100 000 very easily using the memoi solution so that covers this problem and keep talking through your solution even as you're stuck even as you're confused just as i was it's helpful to just keep spend maybe two or three minutes trying to solve the issue and if you're not able to solve the issue just say that this is something i'll fix later and then move on assuming that you've fixed it and then keep talking and keep continue keep working on the solution and at some point later it's possible that the solution might like you now at this point you may be asked sometimes to implement a dynamic programming or an iterative solution like the when you talk to the interviewer and you're telling them that this is how i'm thinking i'm doing i'll do a recursive solution first and i can see that maybe there are going to be some sub problems there then i'm going to then apply dynamic programming so you can just check with them and in most cases they will accept a memoization solution because the dynamic programming solutions can take a little bit of time to solve to implement and they're always off by off by one errors and it's also difficult to explain the solution so you can most most cases get away with memoization but if they do ask you to do it with iteratively with dynamic programming then you'll have to go ahead and implement the dynamic programming solution so once again take a couple of minutes now and work it out on a piece of paper and then go back to them now for dynamic programming remember you have to create a table essentially so what the table will look like in this case is let's see if we can simulate a table what the table will look like is let's create a new sheet and in this sheet let us put the two words which is intention okay and let's put the word exception as well move this down too and let's also put in the indices ultimately this is what a dynamic programming looks like programming problem looks like you are ultimately going to create a table here and how we'll start filling the table is the ijth element so let's say this element int int e e x c e so this element represents the edit distance or the number of operations required to convert i n t e into e x c e and how do you check what the solution is now you know that e and e are equal so the final elements are equal so what that means is we look at this value then this value should tell us what is the minimum edit distance between exc and exc now since we can simply add e to each string and get this solution that means this solution is equal this value should is equal to this value all right so in the case where the corresponding elements are equal we simply copy over the value diagonally left top left value onto the current cell the other option is if they are not equal so let's say if we are here where here you have n and here you have p now there are three possibilities you you want to find the minimum edit distance between i and t e n and e x c e p now n is not equal to p and this is the original string so either we delete n now if we delete n then we need to find the solution for i n t e and e x c e p so if we delete n then this value will become one plus this value that's one possibility or another possibility is that we swap n so we swap n for p so now you get this becomes p and this becomes p so this value will becomes will become 1 plus this value because now we can ignore the p and simply get this previous solution for e x c e and i and t e so this value becomes one plus this value or the final option is that you can insert something just before n so if you insert something just before n which is going to be p so if you insert p just before n so you insert p just after n naught before if you insert p just after n then you have p after it already so you can just look at this value and this value is going to be one more than this value in the case that you insert something insert p after n right so there are three ways to come to this value either by deleting n or by inserting p or by changing n to p and what you can do is you can take the minimum of three values three or these three values and add one to obtain this value so that's the logic roughly speaking and you start from the left so you see okay e and i they are unequal so you need one operation to change them and there's nothing else to consider so that's done then e and n they are unequal now you need what you can do is you can either delete n if you delete n then you simply need to check e and i and you know that the solution for e and i is one so this would be two another another option is that you could possibly insert something but if you insert something the length of i n is going to increase so that's going to cause a problem so you can't insert anything another option is you change n with e but if you change n with e then you will no longer be able to if you change n with e then you will no longer be able to use this solution right because now you will have to match i with the empty list so that's going to be one as well so overall you end up with two and this is how you start filling the list so you start filling up from left to right and left to right and keep going top to bottom and as you fill out this list finally you will fill out this final value exception and intention and that will be your solution so that's the dynamic programming solution and you can see that it's getting tricky to convey the entire solution because there are so many cases involved here so typically you will not find dynamic programming solutions to requested in interviews and it will help you to just stick to the memoization solutions all right so with that we have covered two common interview questions and you can keep going so the idea here is to just apply the method remember the remember the method the problem solving template that we've covered state the problem identify input and output formats write a function signature come up with some example inputs and outputs or at least the scenarios come up with the correct solution stated in plain english implement the solution test it using example inputs and fix bugs if you face any then analyze the algorithms complexity and identify inefficiencies and finally apply the right technique to overcome the inefficiency and you repeat the process going back and stating the solution implementing analyzing and repeating now you in some cases you do not need to implement the root force solution if you don't have the time but when you're working with recursive solutions it always helps to implement brute force first before you do memoization or dynamic programming and some tips ask questions as many questions as you can as many as you need to clarify the problem show an example follow the method don't panic if you get stuck at a certain point give it a couple of minutes sometimes you can even ask the interviewer and they may be able to tell you that maybe what your error is or maybe you're not stuck at all what you're simply assuming something incorrectly but beyond a few minutes what you want to say is that let i'll fix this later assuming this is correct let's move on and then talk about complexity and optimization and such and such very important is to state the brute force solution to the interviewer and if you are unable to figure out a more optimal solution then the best thing you can do is to offer to implement the brute force solution so that you can at least demonstrate that you are able to write code and it's all right in a lot of cases you will not be able to figure out the optimal solution and in some cases there may not be an optimal way so there are some there are certain problems where there is just one way and that is the hard way or the brute force way and this is typically very true with a family of problems called backtracking something we've not really covered in a lot of detail but it is also another form of recursion so what do you do next so the next step for you ish is to review this lecture video and solve these problems yourself or take more problems ideally what you want to do is you want to take all the five different techniques that we've covered and let's quickly review what those five techniques were the first one was binary search so we looked at linear search and binary search which is a form of divide and conquer and along with that we also understood the complexity and big o notation and then you had some homework on linked lists and python classes but binary search is something that comes up often and the hint to detect binary search is simply to look for order whenever you see something being something being mentioned mentioned as sorted now that is an indication for you that this may be binary search sometimes what you may have to do is you may have to get things into a sorted form maybe by taking replacing elements by sum of values till that element or so on and once you get things into a sorted form maybe then you can do binary search that's one way to go about it and once again just do five to ten problems on binary search and you will be able to identify pretty much any binary search question in an interview then the next topic that we looked at was binary search trees traversals and here is something that is generally asked very directly so you will be given a question like binary search tree do something with the binary search tree and you can answer that question directly we've covered a lot of different things here so do check out lesson two for all the different things you can do with binary search trees traversals balancing and most of these are recursive solutions so it's also a good exercise on recursion and we've also looked at balanced binary trees and how can we optimize them further then you had an assignment on hash tables so hashing is a again a common question that is often asked so we built hash tables from scratch in python and we also handled collisions using a technique called linear probing and so this is something you can check out in assignment two so you may get asked just to implement a hash table in python or implement collision resolution in a hash table in which case you can use linear probing then you have the sorting algorithms where we looked at bubble sort and insertion sort merge sort using divide and conquer and quick sort where we had a quadratic worst case complexity but a logarithmic average complexity and that's a good thing because merge sort although it is logarithmic in the worst case it still takes up a lot of space and space allocation is slow and you may also not have the memory that's why we sometimes use prefer quick sort over merge sort when we are constrained for space then assignment 3 is pretty interesting where you will implement an optimal algorithm for polynomial multiplication using divide and conquer so do check out assignment 3 as well then we looked at dynamic programming we looked at recursion memoization subsequence and abstract problems and then we finally also didn't cover backtracking and pruning but we there are some questions there in the lesson notebook which you can try out which use backtracking and pruning as well then we looked at graph algorithms the last time which was graphs and adjacency list and adjacency matrices we looked at the depth first and breadth first search and how to implement them and we also looked at shortest paths and directed and weighted graphs this is a very important topic breadth first in depth first search you will get many questions related to these so do solve maybe five questions on each of these topics and you should be good with most graph problems asked in interviews now this project for you the course project if you haven't seen it already is to pick a coding problem so you can pick a coding problem from an online source like lead code hacker rank geeks for geeks etc and then use the problem solving template that we've shared with you this problem solving template as a starting point so just give it a name and then write the problem statement and implement the solution step by step to use the problem solving template to solve the problem using the method you've learned in the course then document your solution add explanations wherever required perform the complexity analysis all of this you should add in the jupyter notebook and then publish your notebook to your jovian profile and finally you can submit the link to your jovian notebook here do submit the link to your jovi notebook here and you can check out the discussion where you can change where you can post what you what you're working on so do post your notebook as well and finally today we have looked at a couple of real interview questions from amazon and google and how to go about solving them and we also addressed a few issues that we faced along the way so that was a helpful exercise and that's it so now you can review the lecture video execute the jupyter notebooks complete the assignments and attempt the optional questions so that the topics that we've covered they get consolidated and you do not ever have to look at this lecture again right practice is what really reinforces and consolidates your learning complete the assignments and attend the optional questions to practice and participate in forum discussions also very useful when you participate in forum discussions why by answering questions a lot of your own doubts get cleared so do participate in forum discussions and then join or start a study group if possible getting together with a group of four or five people is great it really helps you focus and uh improve your understanding by discussion so that's data structures and algorithms in python with that thank you very much for joining us on this journey as we learn data structures and algorithms in python a very useful topic to improve your coding skills and also something that you will almost certainly encounter in one of your interviews no matter which company you're applying to so i hope this is helpful to you do let us know on the forum how this course helped you if it did you can let us know in the youtube comments as well if you have questions if something was not clear do post that too when we make sure to come up with clearer explanations and clearer examples the next time if you have any feedback for us do post it in the comments or send us an email at support jovian.ai with that i will take leave and i will see you in the forums this is not the end of our journey with you so do stay active on joven there's a lot of great activity happening do check out the forums the newsletter and stay tuned for our next course thank you and goodbye