alright so the first thing that I want to do is look on canvas just to show you guys where to find what it is that we're going to be going over make a syllabary so you go under file you'll see that all of our topics or we hides in the borders no topics we'll have more stuff files than others we're going to be working out of these out pages which is our first topic yes it are all yeah I'll have course you enjoy also a long as we get drill yeah and we're doing search infection all right so we're going to be working out of the topic outline for intelligent agent which is what we have your evidence downloaded the record lovely every time through that night really this is a new version of word it colors the top blue releases aversion hi Lucas so here is it don't as we must make it in our collection so I mentioned in the introduction half that AI branches across a lot of different areas and we're going to touch on a lot of this areas over the course of the semester but in order to have intelligent or cleansing dialogue about what it is that AI effects we have to cover some basic terms okay because as we explain how the concepts work these terms are going to become important because they distinguish between logically okay so we're going to talk about these and then there's a model down here that we're going to look at it kind of shows everything put together so let's look at the bottle cars and here's what we've got so two broad objects we have our agent here over the environment here and what this shows is how this you interact okay so if you're thinking about playing the rough pac-man we have a fourth goats with back then those goats could be considered agents in the game board would be the environment all of the things that make up the game from the Royal College you collect Otto okay if we are a robot walking around Mars trying to discover up take pictures whatever the robot is the agency and the planet itself including not only the surface but the atmosphere the air all essence of that all forms the environment and what this model shows is how the two interact because this is the very ground level of AI we could take any piece of artificial intelligence in the world and map it to this home whether it's a Roomba or rather if vacuuming or something playing game or Tesla car driving itself or Roger any of those can fit into this wall okay so let's first define the agent is anything that included perceiving its environment through centers and acting upon that environment through actuators for scenes or receive information through sensors and acts on the environment using actions can you name some edges we just talked about couple few names of agent in a game sort of computing system anything that might fit this category hey buddy NPC design player characters and video games absolutely right now what makes them an agent is they have to not only receive input from the environment but they have to also act on the environment in some way right so would a tree be an agent not if it doesn't they get so this is important because a lot of times there's some confusion about whether or not something is classified as an agent or somebody looks at something like oh that's AI but it's really not okay and so the very basic definition requires that the agent received or perceive some sort of input from the environment and also acts upon the environment in some way okay so NPCs that are stadium some sort of camp and they see something run by oh my god of the fire only to attack them and then they run after it that is an agent with somebody like a tree or a bush or a river that just does what it does does it receive input that's not a nation okay something light and calculator doesn't interact per se with the environment the person interacting with passion there's a lot of great area to you can make the argument well the calculator is displaying something but it's displaying it because it's like and reflect on the phone what are some other examples you got to be this yes absolutely so I don't know if we want to keep it a second quartile but we all just theory receives input typically in the form of voice say hearing what's the date okay in that situation what would the actual interview how would series affect the environment yeah so the pay the data is the for series tell me to do make that we go to ages right first that these are perceptual input now not every agent if you have the base set of inputs that they can receive where you have different sensors some agents might only have sites those agents might be able to see in here so I only have touch right we all know the five senses or six if you're special all of those can be raised that agent received engine okay when we're talking about setting up system a lot of times these are things that we take into consideration if we are building robots that fight to the death wasn't in that sure not a lot oh so we're going to build I think is not about you controllers remote if we wanted to build like intelligent lenses not on their own we would need to come up with sensors that they could figure out what's going on and then they get intelligent choice about what to do so if we were going to set up a robot killing robot what type of sensors we're good of projection we want to consider site or the other so it smells important probably not as much as others I mean you could maybe come for something crazy where it smell something burning how you identify our sound we may be out of this sound probably not up there because vibrations would be more field well I guess it's a fine line between sound vibrations and time there you go so certainly some are more important than others right actuators this is what an agent you uses to affect the environment right so if we're talking about they may have a cigar Roomba things are going backing for you anyway I want on does it work is only once again certain area like it will keep that community merrylegs okay what let's talk about what a Roomba actuators with it how does it affect the environment I don't think that sensor play so the bosses on the like that okay so that would be sensors because the ability to proceed using what about how would it affect the environment arguably clean okay just a little back in mechanics in our computer the we know various so we can remove Anakin please weekly work out an actual robot walking around what would it back you interspace yes hands upbeat rap stop move with its leg now a lot of thing that we have to talk about is this Center question line this is the agent function this is where the decision-making occurs so as you look at this is set up in a circle quiz matter how this is drawn is the circle either right okay the environment is separate from the agent the agent receives something from the environment through assistance this feeds something is a hey I'm acute away from the wall or hey I hear a sound over there it takes that piece of information that it receives from the environment and it passes it to its agent program it uses that to make a decision about what to do now it might decide not to do anything but once it makes a decision how we kind of deviate from the calculator what does it make the decision then it decides to act on the environment using a sexual show think of acting for a second it detects that there is dirt emergent even do that which is just vacuum all right ruff look let's use your NPCs yes but yeah go ahead central hall okay so let's say our vacuum is going this way it's detecting that there's a roll but the wall is 10 feet away - it does nothing or continues through for reference and once it gets to the point where it detects the wall is only play having a number an inch away from it it makes the choice to change its direction and rotate 90 degrees well is it like we build the map of area till I go forward hit a while back up turn ten degrees before we get a walk back up to maybe I look like you I mean are that way you are one of you trap they have a memory okay zero power oh yeah and you would think that that's not terribly difficult where we talk about searching next spot and believe and get into a stronger class after that you're going to see this but here when difficult to program something like that and once you have the room set up it would be very easy to get from one place to another it's just the decomposing of the environment presentation part Bourget room alright any questions on this model is this really important okay and this will be on your test it's going to now this is the foundation for how agents interact yes enough right there you look on paying 30 50 bucks but don't have it document but the one page for you ah okay all right we're good have some simple like sittin here now so very awkward you if Wendy wants to follow chair to make me feel better about stripping be okay with that the extra courses don't complain at the end of semester when you want extra credit offered I look here at all the terms that we're going to buy so environmental states in this is really what we're going to talk a lot about space discussion environmental States is a snapshot of the environment and everything that's in it so we talked a minute ago about the Mars rover the robot on Mars that we dressed if you took a snapshot the environment said what is this state of that environment that would include the current position of the robot along with every position rotation state of it's orange whatever can do it would be the temperature it would be the time of day I mean you can include tons and tons of stuff in a snapshot okay that is what an embargo statement and so if we look here at this example right the very simple environment there are two different tiles for this environment and the two different tiles to either be clean or dirty this I'm guessing sir this is the fashion so what would an example of a state of peace in the vacuum cleaner what type of information would be included in a state right so specifically what we're talking about state are we talking about abilities no let's say is a snapshot of the entire environment including the characteristics of each specific location so if I said what is the state of this environment you would say the robot is in the location a or child a when everyone home valet is dirty I'll be is dirty that makes that we don't need to say anything else because we're not giving any other pieces of information so that's what we need to keep track up there very good with that so a state is a snapshot of the entire environment it's the current configuration of everything that we can see and perceive first set sequence is the entire history of what the agent in procedure and what that means is it is basically a listing of all of the previous States for example look here first up sequence we listen here throw a is clean hypothetical it's not them but if a is clean your dress this a is very now why did you use the word duck instead of clean what in this will say clean with a is dirty then we will clean it B is clean then we would left it B is dirty solution this is clean and a spleen we would run right this doesn't show we obviously can't with all the time it would be a massive amount level state because this is showing all of the thoughtful action a if the agent is a and then it was left is still a there is no so is everything we can't learn with all these goodies for every state for every action I don't know everybody was up how many questions in our the beginning part of this course is a lot of theory stuff like this we've got and I know it's not incredibly exciting we have to cover your show so that we can get into stuff that is for that because it without a foundation and without least terms I can't begin to get into more complex stuff and this is the basis for explaining that all right like learning your alphabet before you longer but there is to play 30 not really infinite but it shows kind of for every for every state for every sequence of actions you can look at it in an infinite it's just trying to give you an idea of the first step sequence to think that this country and its talk about behavior per se we're this idea of rational agent the agents are rational if they seek to do the right thing that's important you need to know that and you know your test or at some point I'm going to say what do you rational agents do or what makes an cases rational and you'll say an agent is rational if it does the right thing what's the problem with that oh yeah but even in our society in the world we would agree that everyone should do the right thing and in fact most of the people even those that break the law don't think that they're doing anything wrong so if we are attempting to create artificial objects agents that mimic intelligent human behavior they can be difficult if we and I realize that will hard time deciding what the right thing to do is that is going to translate and you want having a difficult time selling robots or agents what's the right thing to do so if we are defining rational thought based off with something that is subjective we're going to get different answers about what they should do them except so what does it mean to do the right thing the larger question what does it mean to do the right thing okay to do what your intended purpose is okay now that would be even more of a loaded question if I translated that into human so I took that definition which I think we're all agree that's a good definition I took that and translated it to human life and that's it would you all agree that the right thing to do is what our purpose that naturally makes the question what is our purpose and who is it that defines that perfect that question goes down way different direction okay are we happy to talk about that but not in the front hook up yeah okay we were not a good definition the action that does the most good and I think on the surface again we're all great the other throats between two different actions this one has dismissal in this Google what's the more good what's the one [Music] but for mortgage but again how are you defining what's good while going around just top hale-bopp what is the right step okay maximum efficiency as determined by its creator this is the second time in attempting to define what the right thing to do is we have brought in some sort of creator programmer someone that is offered what it is that they're supposed to do and I think you guys are on the right track and I would agree with you that because this stuff is so subjective we have to have some sort of basis for what is right someone has to say this is the right thing and the only way we can say that the agent is doing what is rational is if what they are doing matches up with what we have said they should be doing now whether or not what we've said that they should be doing is the right thing is topic for another course most likely in a different department but for the sake of this class we will say the ones that define what the right thing to do is is the one that sets what we are going to call the performance notion right the definition performance measures how a series of environmental states are evaluated and it selects an actions or series of actions and each action will cause the environment States to change the performance measure is how we determine is the agent is doing the right thing the performance measure as it says evaluate a series of states right so we're looking at the vacuum world which we're going to go back to that just a second if we are evaluating a series of states we're evaluating a series of actions that the robot vacuum hit shows the survey we can look at how those states evolve so does the environment get cleaner does the robot make unnecessary actions and we can use that as the basis for determining whether the agent is rational right but first we have to define how it is that we are going to measure their performance how are we going to say this is the right thing so we have our vacuum world maybe goes back and forth between those two titles and clean what is the performance measure how clean is it all the actions that they're making caused me to become cleaner all right yeah look at this part right here say this is something I want you to kind of tuck away for later your last assignment in this class will be the bummer melts which we talked about before that's where you are going to program an agent to play a game against somebody else's program agent when you do this and you're going to the city set up a program and your agent or you're always playing the game you're evaluating is he doing the right thing as you might be a situation where few times you're going towards you on this side you want to go push the button where you have three bottles coming on this side these bombs are further away but there's three over here these bubbles are closes there's only two what's the right thing here you can make an argument for us and so what you have to do is establish a performance measure as the basis for whether or not what they're doing is rational and when you do that when you're coming up with your little performance letter it's better to design the performance measure with the desired outcome in mind rather than how the eight rather than how you think the agent should act one focus is on the route the other focus is on fixing symptoms of a problem Yup'ik so here is not a liability side but giving it a blanket up blowing up various items not as neither right so don't focus on individual actions to the agency's mission focus on the underlying goal that influences the reduction it's like I guess some of these parroting because even though I'm getting hold of it if my daughter is faster to do something go pick up that foot put it away and she argued Christmas tutorial okay you can attempt to fix the behavior which would be what this is you shouldn't argue this and that's not the problem and that's not what you should focus on when you're doing the performance issue the problem of the character flaw DISA Beauty she's not arguing because she wants to argue about it she doesn't need an explanation about why we should be picked up she just doesn't want to do it so what should be addressed is the disobedience not the act of arguing and this is the same thing when you are doing this you have to focus on the underlying gold don't pick out a specific action the agent cookin - oh we should have done that well he did that because that's what he will thought was the best thing so if that thing he's doing then the goal that you gave them is wrong except I'm not trying to get too deep but just rather stuff to click rate we've got to get this stuff going for basically we were good so far we will take a break coming up we will take a break about halfway through so no you can only fit for so long the Wolves will take a break down there all right so I want to officially define what a rational agent is important and I'm not ever going to ask you to spit the definition back okay so on your test if I say what's a rational agent I'm not looking for this if I say on your test tell me what a performance measure is I'm not looking for this set definition I want you to explain it to me in your own words now you might use some of the same words that's all I'm interested in your ability to describe what these things are because that shows in if you have an understanding them right so graduation for each possible perfect sequence will deceive everything is you irrationally it should direct an action that is expected to maximizes maximizes performance measure give them the evidence provided by the process sequence and whatever built involves the hmm for example if we go back and look at our vacuum world here if the first step sequence that the history of one can proceed is a is Queen moves right B is clean what does it logically complete yeah so what should it do here's the thing what if we value cleanliness over the use of electricity julie has a rechargeable battery employer what should it do that we're back to a so you have a performance measure influences what it does so in that situation it was in a his claim the logical thing is go through deviancy a businessman we're going to be in this cleaned as well if the agent then moves back to a or I'll go to shows you that it should turn off don't focus on that action focus on why it was there was because the performance measure that we gave it was to value cleanliness over the use electrician reason things yes let's do like every agent that you serve is going through the rationalization to go look like you can be pretty that's what everything is based off of their performance he leakin definition a rational agent performance measures mentioned a couple of times in the definition because what makes it rational is what you've taught it the right thing to do if we look at all kind of different people groups over the world clearly there is a wide disagreement about what the right thing to do after all we've got more sleep got all the stuff going on if you talk to them individually they are all doing the right thing right you're all doing the right thing because the basis for measuring your performance has been defined different and we're doing the same thing so I suppose you like this let me recap this report here's what I want you to take away from rational agent seek to do the right thing the right thing is defined by the performance pressure everything that we just talked about the last 50 points can be summed up in those specific rational agents seeks to do the right thing and the right thing is defined by the performance of any questions on that has anybody not played Mario wait jumping on most anybody not play the game everyone's at least cut your scene so in this place very good if you had it okay okay okay let's talk about old school like the first Nintendo anybody alive when it came up [Music] yeah I wanted to say $85 too let's go out and burn what were you going to help very much Wow run foot walk okay I still let me go this is not what is that bar here like eat it so let's say hypothetically we've been given the job of rewriting the AI scripts or enemy tomorrow you guys remember her very first level work throw it off you jump out these coins little Goomba comes out now in the original there were just a handful of different enemies yes beginning part of the game he has a turtle and you have the goobers did they say hate to say what in terms of their movement right how did the Google say if they hit something what happened all right sir take it they come down and hit a show much to value around the China market are goombas intelligent agents [Music] I agree if I can look at that chart we did now they can perceive stuff it can detect an equalizer they can actually environment because then you lose when that center part is missing now let's make a decision about their actions they do the same station over they don't take the information that they receive and make a decision about might beg the question what they turn around when they hit an object they don't have a choice that is a function of the environment right now if when they got close they decided to turn around or they didn't always turn around and that would be different right so let's focus on windows let's focus on the humor and let's see about rewriting how they work so what is it that they can receive here we talk about the things go back up to the top what is it that they can set well I don't like that now and this is this is real to rewrite this so what do we want them to be able to let me help what they do romário ones right what are the right actions for the agency [Music] now let me preface this whole when you are designing AI for games there are different rules because you have other things to take into consideration for run frustration skill level difficulties from this class for the most part so right now we're going to take those and set them aside when we talk about building intelligent rational agents we want them to be as smart as effective as possible in games that's not always a good thing and we could make checkers tic-tac-toe connect for we can make agents to play those games that would destroy any human would be fun so we're talking about programming for games there are different rules ok let's just cool facade for now and try to make the most obnoxious Goomba possible pursue it tomorrow so what is the right action for the set now we're not going to give the Goomba new action he just move left to right like you're going to throw fireballs at Mario we're not going to change that but given what he likely has access to location of Mario how can we adjust his behavior to make him a little bit more intelligent so we can increase the walking speed okay so we can add some prediction into that yes Mario behind in return okay very good right now all the Goomba does is just walks in a straight direction the only way you can hit Barrios is where is baby just standing there so the first thing that we will do is we'll move towards them and we'll take it one step further and we were trying to predict where he's going to be instead of moving where he is we'll move to where we think efficient we could vary our walking speed to make it more difficult for Mario to predict where we're going so here's the trap and you're going to do the same thing when you're making your bottle about character you can focus on action or if you focus on goal the difficult thing is if you focus on action and there are a lot of different actions you're going to have a hard time balancing and juggling because one action might prevent another action from occurring it is better instead to focus on a goal and let the action naturally flow from your goal so what is the goal Elizabeth Marta not from below so we will say that our performance measure has to be based off of that goal in life so we're going to define a performance measure we know that a performance measure is why it's a definition of a performance over the right thing to do the evaluation of a series of states so basically what we do is we look at what our Goomba just did over the past however Wednesday we will look at its past 10 action we're giving this state what do we do due to this state when you get given this day where did he do so moving closer tomorrow is a good thing right so in which case you want to move away now if you start to take off and you can set up a performance measure as a series of quick values so maybe you get as one for moving closer to Moorea or maybe you get a point value based on how close you are to mario so as long as you're leaving portion of what i visit unless you start a party and then you lose a hundred point if you are within the quick of you goes are short this is how you define what the right thing to do if you haven't set up to wear and most of the time you start with arbitrary numbers looks right we will give up to a hundred points for being close to market we're closer tomorrow we are the closer we get to earning up to 104 okay the things that we see this Mario and I'm here and offer and I will do by November here if I'm this far away they've got to get beautiful on its own here maybe I get 10 I'm here they're going to get 50 the close range over the question I get to the hundred point okay what I lose fifty if she's right about it - right a button when I look at this and be and I look at how its acting I see that goombas still moves directly under Mario even though we're taking away the 50 points and all I don't want them to do that you need to look at the performance measure the amount of points that we're taking away for being neat might not be enough maybe we're four for mourning too many points for being too close so you adjust your performance initial guess see how that works let me go to agents rational diagram not necessary if it is doing if each action it takes either increases or she stands as performances okay that's how you do it so if what I'm doing doesn't match up with what it should then it's not Russia what things if we set up our agents to do with some forms measure aesthetic will be rational but that's our measure for Roberto Osuna effect okay and don't always look at this as the virtual assistant performance ledger is the same person assess up the program a lot of times they're cheating for people this is how you get issues where it's not rational if you're working with your partner and you're like here is the performance measure for what we need they may be working for a company maybe the game is effort and you're Bob gage machines that use the performance measure for the AI in this level okay maybe you don't set it up right we're maybe you're an independent firm that evaluates AI in game and you have a performance measure you're looking at your like this is not you're not being right they don't always assume that the person writing the agent program has access to good that's left up before real sandpaper so if we go back to [Music] ah yes you are you given up inside prevention or anywhere you literally have a sailor so if we go back to Alaska world we give the performance measure that does bring at one point for each clean square at each time step over period of a thousand concepts the balance of the world is known so if you been tile he knows that if he was black she's not going to go anywhere but is this distribution and initial location of the agent or not so he doesn't know where the dirtiest or whether or not the adjacent tile has sorted you can move left ripe or clean need to can correctly perceive its location and whether or not that location contains true if those are the rules of the apartment so using those rules would you say that our agent is rational based off of this so this is what he received in the past yes he do for I think based off of the performances so here we see a separation of two this is an external performance agent for one point for each clean square for the these two waste collected why is the e continuing to stay here what would ages do what sir help me working either let's do nothing is a lot of options a lot of times the way you discourage this type of behavior is like taking away a port every time you move you oftentimes need to increase the reward for doing what you're supposed to do it please so that seeking out unknown location is still more beneficial to stay idle but if you know that all of the places are clean moving at that point becomes unnecessary alright let's take it right here and we will pick back up with some other stuff