Welcome to CSU 29 Machine Learning. Some of you know that this class I taught at Stanford for a long time, and this is often the class that I most look forward to teaching each year, because this is where we've helped, I think, several generations of Stanford students. come expert in machine learning, got built many of their products and services and startups that I'm sure, many of you or probably all of you are using, uh, uh, today.
Um, so what I want to do today was spend some time talking over uh, logistics and then, uh, spend some time, you know, giving you a beginning of an intro, talk a little bit about machine learning. So about 239, um, you know, all of you have been reading about AI in the news, uh, about machine learning in the news, um, and you've probably heard me or others say AI is the new electricity. Uh, much as the rise of electricity about 100 years ago transformed every major industry, I think think AI or really we call it machine learning, but the rest of the world seems to call it AI.
Machine learning and AI and deep learning will change the world. I hope that through 239, we'll give you the tools you need so that you can be many of these future titans of industries that you can be one to go out and build, help the large tech companies do the amazing things they do, or build your own startup, or go into some other industry, go transform healthcare, or go transform transportation, or go build a startup. our driving car and do all of these things that, um, after this class, I think you'll be able to do.
You know, um, the majority of students applying, the- the demand for AI skills, the demand for machine learning skills is so vast. I think you all know that. Um, and I think it's because machine learning has advanced so rapidly in the last few years, that there are so many opportunities, um, to apply learning algorithms, right? Both in industry as well as in academia. I think today we- we have the English department professors trying to apply learning algorithms to understand history better.
We have lawyers trying to apply machine learning to process legal documents. And off-campus, every company, both the tech companies as well as a lot of companies that you wouldn't consider tech companies, everything from manufacturing companies to healthcare companies, to logistics companies are also trying to apply machine learning. So I think that if you look at it on a factual basis, the number of people doing very valuable machine learning projects today is much greater than it was six months ago, and six months ago is much greater than it was 12 months ago. And the amount of value, the amount of exciting meaningful work being done in machine learning is, is, is very strongly going up. Um, and I think that given the rise of, you know, the, the, the amount of data we have as well as the new machine learning tools that we have, um, it will be a long time before we run out of opportunities, you know, before- before society as a whole has enough people with a machine learning skill set.
Um, so just as maybe, I don't know, 20 years ago was a good time to start working on this Internet thing, and a lot of people that started working on the Internet like 20 years ago had fantastic careers. I think today is a wonderful time to jump to machine learning, uh, and- and- and the number of- and the opportunities for you to do unique things that no one has- no- no one else is doing, right? The opportunity for you to go to logistics company and find an exciting way to apply machine learning will be very high, because chances are that logistics company has no one else even working on this, because, you know, they probably can't, they may not be able to hire a fantastic Stanford student as a graduate of CS229, right? Because there just aren't a lot of CS229 graduates around.
Um, so what I want to do today is, um, do a quick intro, talk about logistics. Um, and then, uh, we'll- we'll spend the second half of the day, you know, giving an overview and- and talk a little bit more about machine learning. Okay? And, uh, oh, and I apologize. I- I think that, uh, this room, according to that sign there, seats what, 300 something students?
Uh, I- we have, uh, uh, like, uh, not quite 800 people enrolled in this class. Um, so if there are people outside, and- and all of the classes are, uh, recorded, broadcast on the CPD. Uh, they usually, the video- is usually made available same day.
So for those who can't get into the room, my apologies. Um, there, there were some years, um, where even I had trouble getting into the room, but I'm glad you let me in. Uh, but, but I'm, but, but hopefully you can watch, you, you better watch all of these things online shortly. Oh, I see. Yes.
Yeah. I don't know. Uh, it's a bit complicated. Yeah. Uh, yeah.
Thank you. I think it's okay. Yeah. I, I, we could, yeah.
Yeah. classes people can squeeze in and use at the NTC. So for now, it might be too complicated. So quick intro is, um, oh, I'm sorry. I should have introduced myself.
My name is Andrew. Uh, uh, uh, and I want to introduce some of the rest of the teaching team as well. is a class coordinator.
Um, she has been playing this role for many years now and helps, uh, keep the trains run on time and make sure that everything in class happens when it's supposed to, uh, uh, so, so, so should be, uh, and then, uh, we're thrilled to have you guys wanna stand up? Uh, be the co-head TAs, uh, our respective the PhD students working with me. And so bring a lot of technical experience, technical experience in machine learning as well as practical know-how on how to actually make these things work.
And with the large class that we have, we have a large TA team. I maybe I won't. introduce all of the TAs here today, but you meet many of them throughout this course here.
But the TAs expertise span everything from computer vision, to natural language processing, to computer technology, to robotics. And so, um, through this quarter as you work on your class projects, I hope- hope that you get a lot of, uh, help and advice and mentoring from the TAs, uh, all of which, all of whom have deep expertise not just in machine learning, but often in a specific vertical application area, um, of machine learning. So depending on what your projects, we try to match you to a TA that can give you advice, uh, that most relevant to whatever project you end up working on. Um, so, you know, goal of this class, I hope that after the next 10 weeks, uh, you will be an expert. fit in machine learning.
Um, it turns out that, uh, uh, you know, um, and, and I hope that after this class, you'll be able to go out and, uh, build very meaningful machine learning applications, uh, either in an academic setting where, uh, hopefully you can apply it to your problems in mechanical engineering, electrical engineering, and, uh, English, and law, and, um, uh, and, and, and education, and all of this wonderful work that happens on campus. Uh, as was after you graduate from Stanford to be applied to whatever jobs you find. Um, one of the things I find very exciting about machine learning is that it's no longer a sort of pure tech company only kind of thing, right? I think that many years ago, um, machine learning, it was like a thing that, you know, the computer science department would do and that the elite AI companies like Google and Facebook and Baidu and Microsoft would do.
Uh, but now it is so pervasive that even companies that are not traditional tech companies see a huge need to apply these tools. And I find a lot of the most exciting work these days. And maybe some of you guys know my history, so I'm a little bit biased, right? I let the Google Brain team which helped Google transform from what was already a great company 10 years ago to today, which is a great AI company. And then I also let the AI group at Baidu and let the company's technology and strategy to help Baidu also transform from what was already a great company many years ago to today, which is a great company.
China's greatest AI company. So, having let the, you know, built the teams that led the AI transformations of two large tech companies I feel like that's a great thing to do. But even beyond tech, I think that there's a lot of exciting work to do as well to help other industries, to help other sectors embrace machine learning and use these tools effectively.
But after this class, I hope that each one of you will be well qualified to get a job at a shiny tech company and do machine learning. there or go into one of these other industries and do very valuable machine learning projects there. Um, and in addition, if any of you, um, are taking this class with the primary goal of, uh, being able to do research, uh, in machine learning. You know, so, so actually some of you I know are PhD students. Um, I hope that this class will also leave you well equipped to, um, really read and understand research papers, uh, as well as, uh, you know, be qualified to start pushing.
forward, um, the state of the art. Um, so let's see. Um, so Today, uh, so- so just as machine learning is evolving rapidly, um, the whole teaching team would've been, uh, constantly updating CS229 as well. So, um, it is actually very interesting. I feel like the pace of progress in machine learning has accelerated.
So it- it actually feels like that, uh, the amount we changed the clause. year over year has been increasing over time. So- so if you're friends that took the class last year, you know, things are a little bit different this year because we're- we're constantly updating the class to keep up with what feels like still accelerating progress in the whole field of machine learning.
Um, so, so, so, so there's some logistical changes. For example, uh, uh, we've gone from, uh, we used to hand out paper copies of handouts, uh, that we're, we're trying to make this class digital only. Um, but let me talk a little bit about, uh, prerequisites as well as in case your friends have taken this class before, some of the differences for this year, right?
Um, so prerequisites. Um, we are going to assume that, um, all of you have a knowledge of basic computer skills. and principles uh, so you know, Big O notation, Q's, Stacks, binary trees, hopefully you understand what all of those concepts are. And uh, assume that all of you have a basic familiarity with um, uh, probability, right?
That hopefully you know what's a random variable, what's the expected value of a random variable, what's the variance of a random variable. Um, and if for some of you, maybe especially the SCPD students taking this remotely, if it's been, you know, some number of years since you last had a probability and statistics class, uh, we will have review sessions on Fridays where we'll go over some of this prerequisite material as well. But so hopefully you know what a random variable is, what the expected value is, but if you're a little bit fuzzy on those concepts, we'll go over them again at a discussion section on Friday. Also assume that you're familiar with basic linear algebra. So hopefully that you know what's a matrix, what's a vector, how to multiply two matrices or multiply a matrix and a vector.
Um, if you know what is an eigenvector, then that's even better. better if you're not quite sure what an eigenvector is, we'll go over it that, that you, you, you, that yeah, we'll, we'll go over it I guess. And then a large part of this class is having you practice these ideas through the whole works, as well as I mentioned later, a open-ended project.
And so one there, we've actually until now we used to use uh, MATLAB, uh, and Octave for their programming assignments. Uh, but this year we're trying to shift the programming assignments to, uh, Python. Um, and so, um, I think for a long time, uh, even today, you know, I sometimes use Octave to prototype because the syntax for Octave is so nice and just run, you know, very simple experiments very quickly.
But I think the machine learning world, um, is, you know, really migrating, I think, from, um, MATLAB Python world to increasing- excuse me, MATLAB. octave world to increasingly a Python maybe and- and then eventually for production, Java C++ kind of world. And so, uh, we're rewriting a lot of the assignments of this class this quarter. Um.
have been- have been driving that process, uh, so that- so that this course that you could do more of the assignments, uh, uh, maybe most- maybe all of the assignments in, um, Python, uh, NumPy instead. Um, now, a note on the honor codes, um, we ask that, you know, we- we- we actually encourage you to form study groups. Uh, so- so, you know, I've been, um, fascinated by education for a long time.
Spent a long time studying education and pedagogy and how instructors like us can help support you to learn. more efficiently and one of the lessons I've learned from the educational research literature is that for highly technical classes like this, if you form study groups, uh, you will probably have an easier time, right? So- so CSU 2009 would go for the highly technical material. There's a lot of math, some of the problems are hard, and if you have a group of friends to study with, uh, you probably have an easier time, uh, uh, because you can ask each other questions and work together and help each other.
Um, where we ask you to draw the line or what we ask to do relative to the Stanford's, uh, honor code is, um, we ask that you do the homework problems by yourself, right? Uh, and- and- and more specifically, um, is okay to discuss the homework problems with friends, but if you, um, but after discussing homework problems with friends, we ask you to go back and write up the solutions by yourself, uh, without referring to notes that, you know, you and your friends had developed together. Okay.
Um, the class's honor code is written clearly on the class, um, handouts, posts. sit digitally on the website. So if you ever have any questions about what is allowed collaboration and what isn't allowed, uh, please refer to that written document on the call's website where we described this more clearly.
But, um, all the respect for the- Stanford Law and Code as well as for students doing their own work, we ask you to basically do your own work. It's okay to discuss it, but after discussing homework problems with friends, ultimately we ask you to write up your problems by yourself so that the homework submissions re- reflect your own work, right? Um, and I care about this because it turns out that, uh, having CS 239, you know, CS 239 is one of those classes that employers recognize.
Uh, I don't know if you guys know. know but there have been, um, companies that have put up job ads that say stuff like, so long as you got- so long as you complete the CS239, we guarantee you get an interview, right? I've- I've seen stuff like that. And so I think, you know, in order to- to maintain that sanctity of what it means to be a CS239 completer, I think, um, I ask that all of you sort of really do your own work, um, or stay within the bounds of accepted- of acceptable collaboration relative to the honor codes.
Um, Let's see. And I think that, um, uh, if, uh, you know what, this is, um, yeah. And I think that, uh, one of the best parts of CS229, it turns out is, um, excuse me. So try, sorry, I'm gonna try.
Looking for my mouse cursor. Sorry about that. My- my- my displays are not mirrored.
So this is a little bit awkward. Um, so one of the best parts of the class is- oh, shoot, sorry about that. All right.
Never mind. I won't do this. Um, you could do it on- you could do it yourself online later.
Um, yeah. I started using- I started using Firefox recently in addition to Chrome. Anyway, just to mix that. Um, one of the best parts of, um, the class is, um, the class project, um, and so- So, you know, one of the goals of the clause is to leave you well-qualified to do a meaningful machine learning project and so, uh, one of the best ways to make sure you have that skill set is through this clause and hopefully with the help of some of the TAs.
Uh, we want to support you to work on a small group to complete a meaningful machine learning project. And so one thing I hope you start doing, you know, later today is to start brainstorming, maybe with your friends, some of the class projects you might work on. And the most common class project that, you know, people do in CSUG9 is to pick an area, pick an application that excites you and to apply machine learning to it and see if you can build a good machine. machine learning system for some application in the area.
And so, um, if you go to the course website, you know, cs229.stanford.edu, and look at previous year's projects, you- you- you see machine learning projects applied to pretty much, you know, pretty much every imaginable application under the sun. Everything from, I don't know, diagnosing cancer to creating art to, uh, lots of, um, uh, projects applied to other areas of engineering, uh, applying to application areas in double E or mechanical engineering or silver engineering or earthquake engineering and so on. uh, to applying it to understand literature, to applying it to, um, uh, I don't know. And- and- and- and so, uh, if you look at the previous year's projects, many of which are posted on the course website, you can use that as inspiration to see the types of projects students complete- completing this class are able to do. And I also encourage you to, um, uh, you can look at that for inspiration, you know, to- to get a sense of what you'll be able to do at the conclusion of this class, and also see if, uh, looking at previous years.
this project gives you inspiration for what, um, you might do yourself. Uh, so we ask you to- we- we invite you, I guess, to do class projects in small groups. And so, um, after class today, also encourage you to start making friends in the class, both for the purpose of forming study groups as well as for the purpose of maybe finding a small group to do a class project with.
Um, uh, we ask you to form project groups of, um, up to size three. Uh, uh, most project groups end up being- size two or three. Um, if you insist on doing it by yourself, right, without any partners, that's actually okay too. You're welcome to do that. But, uh, but, but I think often, you know, having one or two others to work with may give you an easier time.
And, uh, for projects of exceptional scope, if you have a very, very large project that just cannot be done by three people, um, uh, sometimes, you know, let us know and we're open to, uh, uh, with, with, to some project groups of size four, but our expectation, but we do hold projects, you know, with. with a group of four to a higher standard than projects with size one to three. So- so what that means is that if your project team size is, uh, one, two, or three persons, the grading is one criteria. If your project group is, uh, bigger than three persons, we use a stricter criteria when it comes to grading class projects, okay? Um, oh, and that- that- that reminds me, um, uh, I know that, uh, let's see.
So for most of you, since this- since this started at 9.30 AM on the first day of the quarter, uh, uh, for many of you, this may be- this is probably your very first class at Stanford. For how many of you, this is your very first class at Stanford? Wow, cool. Okay, awesome. Great.
Welcome to Stanford. Uh, and if someone next to you just raised their hand, uh, actually, raise your hand. So I hope that, you know, maybe after class today, if someone next to you raise their hand, uh, help welcome them to Stanford and then say hi and introduce yourself and make friends after that.
Yeah. Cool. Nice, nice to see so many of you. Yeah.
Um. All right. So, um, just a bit more on logistics. Uh, so, um, let's see. In addition to the main lectures that we'll have here, uh, on Mondays and Wednesdays, um, CS339 also has discussion sections, uh, on held on Fridays that are, and everything we do including the, see all the, all the lectures and discussion sections are recorded.
and broadcast through SCPD, uh, through the online website. Um, and one of- and, uh, discussion sections, uh, are taught, uh, usually by the TAs on Fridays, and attendance at discussion sections is optional. Uh, and what- what I mean is that, um, you- you know, you 100 percent promise there won't be material on the midterm, that will sneak in from this kind of section, so it's 100 percent optional.
Uh, and you will be able to do all the homeworks and the pro- projects without attending the discussion section. But- But what we'll use the discussion section for, for the first three discussion sections, so, you know, this week, next week, the week after that, we'll use the discussion sections to go over prerequisite material in greater depth. So, uh, go over, uh, linear algebra, basic problem statistics, teach a little bit about Python NumPy in case you're less familiar with those programming frameworks.
Uh, so do that for the first few weeks. And then for the discussion sections that are held later this quarter, we'll usually use them to go over more advanced optional material. Uh, For example, um, CS239, a lot of the learning algorithms you- you- you hear about in the class rely on convex optimization algorithms, but we want to focus the class on the learning algorithms and spend less time on convex optimization.
So if you want to come and hear about more advanced concepts and convex optimization we'll defer that to discussion section, uh, and then there- there are a few other advanced topics, uh, hidden Markov models, time series, uh, that we're planning to defer to the, um, Friday discussion sections. Okay? Um, so, uh, let's see.
Um, cool. And, uh, oh, and, um, uh, final bit of logistics, um, uh, for there'll be- there are digital tools that some of you will have seen, but, um, for this class we'll drive a lot of the discussion through the, uh, online website Piazza. How- how many of you have used Piazza before?
Okay, cool. Most of you. Wow, all of you?
That's pretty amazing. Okay. Um, good. So, so, uh, online discussion board for those of you that haven't seen it before, but, um, definitely encourage you to, uh, participate actively on Piazza and also to answer other students' questions.
I think that one of the best ways to learn, uh, as well as contribute, you know, back to the class as a whole is if you see someone else ask a question on Piazza, if you jump in and help answer that, uh, uh, that- that often helps you and helps your classmates. So I strongly encourage- encourage you to do that. Uh, for those of you that have a private question, you know, sometimes we have students, um, uh, reaching out to us to- with- with a personal matter, uh, or something that, you know, is not appropriate to share on the public forum, in which case you're welcome to email us at the class email address as well. Uh, and we also- and- and the class email address, the cl- Teaching Slavs email address is on the course website. You can find it there and contact us.
But for anything technical or anything reasonable to share the class, uh, which includes most technical questions and most logistical questions, right? Questions like, you know, gee, can you confirm what data is midterm or, or, you know, what happens? Uh, uh, can- can you confirm when's the handout for this going on and so on? For questions that are not personal or private in nature, uh, I strongly encourage you to post on P-Answer rather than emailing us because, uh, statistically, you actually get a faster answer, uh, posting this on post- posting on P-Answer than, than, you know, if- if you wait for one of us to respond to you.
Um, and we'll be using great- scope as well, um, uh, to, for, for online grading. And if you, if you don't know what grade scope is, don't worry about it. We'll, we'll, we'll send you links and show you how to use it later. Um, oh, and, uh, uh, again, relative to one, one last logistical thing to plan for, um, unlike, uh, previous, um, uh, years where we taught CS239, uh, so we're constantly updating the syllabus, right, the technical content to try to show you the latest machine learning algorithms, Uh, and, uh, uh, the two big little changes we're making this year, I guess.
One is, uh, Python instead of MATLAB, and the other one is, uh, um, instead of having a, uh, uh, midterm exam, you know, that's a timed midterm, uh, we're planning to, uh, have a take-home midterm, uh, this quarter instead. So I, I don't know. Some, some people just breathed in sharply when I said that. I don't know what that means. Was that shock or happiness?
I don't know. Okay. Don't worry. Midterms are fun.
You love it. All right. So that's it for the logistical. Aspects. Let me check, so let me check if there are any questions.
Oh yeah, go ahead. Oh. Yeah. So, that's interesting.
Uh, let's see. I think it's offered in spring. Uh, and one other person.
Uh, yes. And I was teaching it. So, someone else is teaching it in spring quarter.
Um, uh, I actually did not know it was gonna be offered in winter. Yeah. I don't believe it's offered in winter.
Yeah. Right. Yeah.
Yes. Yeah. So, so I think a great guy and- Teaching it, uh, in, uh, sorry, I'm not sure if I'm saying it right, uh, teaching it in, uh, spring, uh, uh, and I don't think it's offered in winter.
Cool. Anything else? Go ahead. Will this one section be recorded?
Yes, it will be. Oh, and by the way, if, if, if you wonder why I'm recording the, I'm repeating the question, I know it feels weird. I'm recording for the microphone so that, so that people watching this at home can hear the question. But both the lectures and the discussion sections will be, will be recorded and put on the website.
Maybe the one thing we do that's not recorded and broadcast are the office hours, right? Is that right? Yeah.
Oh, oh, but I think this year we have a 60. 60 hours, how many hours? 60 office hours. Uh, 60 office hours per week, right? Yeah.
So, so, so hopefully, I, I just, again, we're constantly trying to improve the class. In previous years, one of the feedback we got was that the office hours are really crowded. So, so we have 60 hours, 60 hours of work, about 60 office hours slots per week this year. That, that seems like a lot. So hopefully, if you need to track down one of us, track down a tier to get help, hopefully that'll, that'll make it easier for you to do so.
Go ahead. Say it again. Well, oh, well, logistical things like when homeworks or DUP covering lectures. We have, yes, so we have four planned homeworks.
Oh, sorry. Yeah, sure. Yeah. And if you go to the- if you go to the course website and click on the syllabus link, that- has a calendar with when each homework assignments go out and when it will be due. Uh, so for homeworks and, uh, project proposal due a few weeks from now and, uh, and then final projects due at the end of the quarter.
But all the, all the exact dates are listed on the course website. Thank you. Go ahead.
Uh, sure. Yes. Difference between this class and 239A. Um, let me think how to answer that. Yes.
Uh, so yeah, I know. I was debating earlier this morning how to answer that. I was asked that a few times.
Um, so- I think that what has happened at Stanford is that the volume of demand for machine learning education is just, right, skyrocketing because I think everyone sees, everyone wants to learn this stuff. And so, um, uh, uh, so. So we've been- so the computer science department has been trying to grow the number of machine learning offerings we have. Um, uh, we actually kept the enrollment of CS239A at a relatively low number, at 100 students. So I actually don't want to encourage too many of you to sign up because, uh, I think we might be hitting the enrollment cap already.
So- so please don't all sign up for CS239A because, um, uh, we- 239A does not have the capacity this quarter. But, uh, CS239A is a, um, much less mathematical and, uh, much more applied, uh, uh, a relatively more applied version of machine learning. And, uh, so I guess, uh, I'm teaching CS239A, CS230, and CS239 this quarter.
Of the three, CS239 is the most mathematical. Um, it is a little bit less applied than CS239A, which is more applied machine learning, and CS230, which is deep learning. My advice to students is that, um, CS239, uh, CS239A, Let me write this down.
I think I'm- right. So CS229A, uh, is taught in the flipped classroom format, which means that, uh, students taking it will mainly watch videos, um, on the Coursera website and do a lot of, uh, programming exercises and then meet for weekly discussion section. Uh, but it's a smaller class with Captain Roman.
Um, I- I would advise you that, um, if you feel ready for CS229 and CS230 to do those, uh, but, uh, CS229, you know, because of the... the math we do, this is a- this is a very heavy workload and pretty challenging class. And so if you're not sure if you're ready for C629, uh, C629A may be a good thing to- to- to take first.
Um, uh, and then- Um, CS239 and CS239A cover a broader range of machine learning algorithms, uh, and CS230 is more focused on deep learning algorithms specifically, right? Which is a much narrow set of algorithms but it is, you know, one of the hottest areas of deep learning. Uh, there is not that much- much overlap in content between the three classes.
So if you actually take all three, you learn relatively different things from all of them. In the past, we've had students simultaneously take 239 and 239A, and there is a little bit of overlap. You know, they, they do kind of cover related algorithms but from different points of view. So, so some people actually take multiple of these classes at the same time.
Um, uh, but 2.29a is more applied, a bit more, you know, practical know-how hands-on and so on and, and, and, uh, uh, much less mathematical. Uh, and, and CS230 is also less mathematical, more applied, more about kind of getting it to work, whereas CS230, um, we do much more mathematical derivations in CS230. Cool. Any other questions?
Yes. Uh, someone had a hand up. So, uh, uh, why don't you say what I would generally prefer students not do that in the interest of time, but what, what do you want? I just want to know how many people in your work role- Oh, I see. Sure, go for it.
Who's enrolled in 239 and 230? I- oh, not that many of you. Interesting.
Oh, that's actually really interesting. Cool. Yeah.
Thank you. Yeah. I- I just didn't want to set a presence of students using this as a forum to run surveys, but that was- that was a- that was a- that was an interesting question.
So thank you. Yeah. Um, cool. All right. And by the way, I think, you know, just one thing about Stanford is the AI world, the machine learning world, AI is bigger than machine learning, right?
And machine learning is bigger than deep learning. One of the great things about being a Stanford student is you can and I think should take multiple classes, right? I think that, you know, CS239 has for many years been the core of the machine learning world at Stanford.
Uh, but even beyond CS239 is worth your while to take multiple classes and gain- multiple perspectives. So, so if you want to, uh, be really effective, you know, after you graduate from Stanford, you do want to be an expert in machine learning. You do want to be an expert in deep learning, uh, and you probably want to know probably in statistics, maybe you want to know a bit of complex optimization, maybe you want to know a bit more about reinforcement learning, know a little bit about planning, know a bit about lots of things.
So, so I, I actually encourage you to, to take multiple classes, I guess. Cool. All right.
Good. Um, if there are no more questions, let's go on to talk a bit about, um, machine learning. Okay.
So, um, all right. So in the remainder of this class, what I'd like to do is, um, give a quick overview of, uh, you know, the major, uh, uh, areas of machine learning and also, um, and- and also give you a sort of overview of the things you learn, uh, in the next 10 weeks. So, you know, what is machine learning, right? It seems to be everywhere these days and is useful for so many places and- and I think that, um, uh, And, and, you know, and, and I feel like, but just to share with you my personal bias, right?
You read the news about these people making so much money building learning algorithms. I think that's great. I hope, I hope all of you go make a lot of money. But the thing I find even more exciting is, is the meaningful work we could do, right?
I think that, you know, I think that every time there's a major technological disruption, which there is now through machine learning, um, it gives us an opportunity to remake large parts of the world. And if, if we behave ethically in a principle way and use these superpowers of machine learning to do things that, you know, helps people's lives, right? Maybe we could, um, uh, maybe you can improve the healthcare system.
Maybe you can improve, give every child a personalized tutor. Maybe you can make our democracy run better rather than make it run worse. But I think that, um, the meaning I find in machine learning is that there's so many people that are so eager for us to go in and help them with these tools that, um, you know, if- if you become good at these tools, it gives you an opportunity to really remake some piece, some meaningful piece of the world, uh, hopefully in a way that helps other people and makes the world kind of- makes the world a better place is very cliched in Silicon Valley.
But- but I think, you know, with these tools, you actually have the power to do that. And if you go make a ton of money, that's great too. But I find a much greater meaning in the work we could do, um, uh, it gives us a unique opportunity to do these things. Okay.
But, um- despite all the excitement of machine learning, what is machine learning? So let me give you a couple, um, definitions of machine learning. Um, Arthur Samuel, whose claim to fame was, uh, building a checker's plane program, uh, defined it as follows.
So field study gives computer ability to learn about being explicitly programmed. Um, and, uh, you know, interesting, when- when Arthur Samuel many, many decades ago wrote a checker's plane program, uh, the debate of the day was, uh, can a computer- computer ever do something that it wasn't explicitly told to do. And Arthur Samuel, uh, wrote, um, uh, Checkers Playing Program that through self-play, learned what are the patterns of, uh, checkerboard that are more likely to lead to a win versus more likely to lead to a loss, and learned to be even better than Arthur Samuel, the author himself, at playing checkers. So back then, there was this remarkable result that a computer programmer, you know, that, uh, could write a piece of software to do something that the computer programmer himself could not do, right?
Because this program became better at also Samuel, um, at, uh, at- at- at the task of playing trackers. Um, and I think today we, um, are used to computers or machine learning algorithms outperforming humans on so many tasks. Uh, uh, but it turns out that when you choose a narrow task, like speech recognition on a certain type of task. You can maybe surpass human level performance. If you choose a narrow task like playing the game of Go, then by throwing really tons of computation power at it and self-play, uh, uh, you can have a computer, you know, become very good at, at these narrow tasks.
But this was maybe one of the first such examples in the history of computing. Um, Uh, and I think this is still one of the most widely cited, um, uh, definitions, right? Uh, gives computers ability to learn well-being explicitly program.
Um, my friend Tom Mitchell in his textbook, uh, defined this as a well-posed learning problem. Uh, program said to learn from experience E with respect to a task T on some performance measure P, if it's performed on T as measured by P improves on experience E. You know, and I- I asked Tom this. I asked Tom if, um, he wrote this definition just- just because he wanted it to rhyme and he did not say yes, but I don't know.
But in this definition, the experience E, for the case of playing checkers, the experience E would be the experience of having a checklist program play tons of games against itself. So, you know, computers lots of patients and sit there for days playing. games of checkers against itself. So that's the experience E. Uh, the task T is the task of playing checkers and performance measure P maybe, um, what's the chance of this program winning the next game of checkers it plays against the next opponent, right?
So- so we say that, uh, this- this is a well-posed learning problem, learning of Bay checkers. Now, within this, um, set of ideas of machine learning, there are many different tools we use in machine learning. And so in the next 10 weeks, you'll learn about a variety of these different tools. Um, and so the first of them, and the most widely used one is supervised learning. Um, let's see, I wanna switch to the whiteboard.
Do you guys know how to erase the screen? Uh, Oh, like that. Okay. So what I want to do today is really go over some of the major categories of machine learning tools and- and- and sort of what you learn in the next, um, uh, by the end of this quarter.
So the most widely used, Machine learning 2 is, um, today is supervised learning. Actually, let me check. How- how many of you know what supervised learning is?
Uh, like a two-thirds, half of you maybe? Okay, cool. Let me- let me just briefly define it.
Um, here's one example. Let's say, you have a database of, uh, housing prices. And so I'm gonna plot your dataset where on the horizontal axis, um, I'm gonna plot the size of the house in square feet, you know, and then the vertical axis will plot the price of the house, right?
And, um, maybe a dataset. Looks like that. Um, and so horizontal axis, I guess we'll call this x, and vertical axis, we'll call that y.
So, um, the supervised learning problem is, given the data set like this, to find the relationship mapping from x- to y, right? And so, um, for example, let's say you have- let's say you have a- let's say you're fortunate enough to own a house in Palo Alto, right? Uh, uh, and you're trying to sell it and you want to know how to price the house. So maybe your house- has a size, you know, of that amount on the horizontal axis.
I don't know, I mean, this is a, what, 500 square feet, 1,000 square feet, 1,500 square feet. So your house is, uh, 1,250 square feet, right? And you want to know, you know, how do you price this house?
So given this dataset, one thing you can do is, um, uh, fit a straight line to it, right? And then you could estimate or predict the price to be whatever value you read off on the, um, vertical axis. So in supervised learning, you are given a dataset with, uh, inputs x and labels y, and your goal is to learn a mapping from x to y, right?
Now, um, filling a straight line to data is maybe the simplest possible, maybe the simplest possible learning algorithm, maybe, uh, one of the simplest po- learning algorithms. Um, given the dataset like this, there are many possible ways. to learn the mapping, to learn the function mapping from the input size to the estimated price.
And so, um, maybe you want to fit a quadratic function instead. Maybe that actually fits the data a little bit better. And so how do you choose among different models will be, uh, either automatically or manual intervention will be, will be something we'll spend a lot of time talking about. Now, to give a little bit more, um, to define a few more things, this particular example is a problem called a regression problem.
And the term regression refers to that the, uh, value y you're trying to predict is continuous, right? Um, in contrast, here is a- here's a different type of problem. Um, so a problem that some of my friends were working on, and- and I'll simplify it, was- was a healthcare problem where, uh, they were looking at, uh, breast cancer, uh, breast- tumors and trying to decide if a tumor is benign or malignant, right?
So a tumor, you know, is of a lump in a- in a woman's breast, um, is, uh, uh, can be malign or cancerous, um, or benign, meaning, you know, roughly there's not that harmful. And so if on the horizontal axis, you plot the size of a tumor, um, and on the vertical axis, you plot, is it malignant? or not, so malignant means harmful, right?
Um, and some tumors are harmful, some are not. And so whether it's malignant or not, takes only two values, one or zero. And so you may have a dataset, um, like that.
Uh, and given this, can you learn a mapping from x to y so that if a new patient walks into your office, uh, walks into the doctor's office and the tumor size is, you know, say, this. Can the learning algorithm figure out from this data? there that, you know, is probably, well, based on this dataset, it looks like there's a- there's a high chance that that tumor is, um, malignant. Um, so, uh, so this is an example of a classification problem.
And the term classification refers to that y here takes on a discrete number of variables, right? So for a regression problem, y is a real number. I guess technically prices can be rounded off to the nearest dollar and cent.
So prices aren't really real numbers. uh, you know, because you probably not price of house at like pi times 1 million or whatever. Uh, but so- so- but- but for all practical purposes, prices are continuous, so we call them housing price prediction to be a regression problem, whereas if you have, uh, two values of possible output 0, 1, we call that classification problem. Um, if you have k discrete outputs, so, uh, if the tumor can be, um, malignant or if there are five types of cancer, right? So you have one of five possible outputs, then that's also a classification problem if the output is discrete.
Now, um, I want to find a different way to visualize this dataset, which is, um, let me draw a line on top. And I'm just gonna, you know, you know, map all this data on the horizontal axis upward onto a line. Um, but, oh, let me show you what I'm gonna do.
I'm going to use a symbol o to denote, um, I hope what I did was clear. So I took the two sets of examples, the positive and negative examples. Positive examples is one, negative examples is zero.
And I took all of these examples and, and kind of pushed them up onto a straight line. And I used two symbols. I used O's to denote negative examples and I used crosses to denote positive examples. Okay.
So this is just a different way of visualizing the same data, um, but drawing it on a line and using, you know, two symbols to denote. the two discrete values 0 and 1, right? So, um, it turns out that, uh, uh, in both of these examples, the input x was one-dimensional, it was a single row number. For most of the, um, machine learning applications you work with, the input x will be multi-dimensional. You won't be given just one number and also predict another number.
Instead, you often be given, uh, multiple features or multiple numbers to predict another number. So for example, instead of- of just using, uh, tumor size to predict, to estimate malignancy of malignant versus benign tumors, um, you may instead have two features where one is tumor size, and the second is age of the patient, and be given a dataset. Maybe I'll see. Thank you.
And be given a dataset that looks like that, right? Where now your task is, um, given two input features. So x is tumor size and age, you know, like a two-dimensional vector, um, and your task is given, uh, these two input. features, um, to predict whether a given tumor is malignant or benign. So if a new patient walks into the doctor's office and if the tumor size is here and the age is here, so that point there, then hopefully you can conclude that, you know, this patient's tumor is pro- probably benign, right, corresponding to O, the negative example.
Um, and so one thing, one thing you learn, uh, next week is a learning algorithm that can fit a straight line to the data as follows, kind of like that, to separate out the positive and negative examples, separate out the O's and the crosses. And so next week you'll learn about the logistic regression algorithm, which, um, which can do that, okay? So, um, one of the most interesting things you learn about is, uh, let's see. So in this example, I drew a dataset with two input features. Um, when I- so I have friends that actually worked on the breast cancer, uh, prediction problem and in practice, you usually have a lot more than one or two features.
And usually you have so many features you can't plot them on the board, right? And so for an actual breast cancer prediction problem, my friends are working on this. We- we- we're using many other features such as, don't worry about what these mean, I guess, clamp thickness. uh, you know, uniformity of cell size, uniformity of cell shape, right?
Um, uh, adhesion, how will the cells stick together? Don't worry about what this means, but, uh, if you're actually doing this in a, in a- medical application there's a good chance that you'll be using a lot more features than just two. And this means that you actually can't plot this data, right?
It's too high dimensional. You can't plot things higher than three-dimensional or maybe four-dimensional or something, right? And so we have a lot of features that's actually difficult to- plot this data.
I'll come back to this in a second in learning theory. Um, and, uh, one of the things you learn about, so as we develop learning algorithms, you learn how to build, um, regression algorithms or classification algorithms that can deal with these, uh, relatively larger number of features. One of the, uh, most fascinating results you learn is that, um, you also learn about an algorithm called the support vector machine, which uses not one or two or three or.
or 10 or 100 or a million input features, but uses an infinite number of input features, right? And so- so- so just be clear, if in this example, the state of a patient were represented as one number, you know, tumor size. In this example, you had two features. So the state of a patient were represented using two numbers, the tumor size and the age.
If you use this list of features, maybe a patient is represented with five or six numbers. Uh, but there's an algorithm called the support vector machine that allows you to to use an infinite dimensional vector, um, to represent a patient and, um, how do you deal with that and how can a computer even store an infinite dimensional vector, right? I mean, you know, computer memory, you could store one real number, two real numbers, but you can't store an infinite. the number of real numbers in the computer without running on memory or process speed or whatever.
So, so how do you do that? Uh, so when we talk about support vector machines, uh, and specifically the technical method called kernels, you learn how to build learning algorithms that work. with a sort of infinitely long list of features, infinitely long list of feature, uh, for- for which- which- and you can imagine that if you have an infinitely long list of numbers to represent the patient, that might give you a lot of information about that patient. And so that is one of the relatively effective learning algorithms for some problems. Okay.
Um, so that's supervised learning. And you know, let me just, um, uh, play a video, um, show you a fun slightly older example of supervised learning. I'll give you a sense of what this means.
But at the heart of supervised learning is the idea that during training, you are given inputs X together with the labels Y, and you give it both at the same time. And the job of your learning algorithm is to find a mapping, so that given a new X, you can map it to the most appropriate output Y. So this is a very old video made by Dean Pommelow, known for a long time as well, on using supervised learning for autonomous driving.
Uh, this is not state of the art for autonomous driving anymore, but it actually does remarkably well. Oh, and, uh, um, as you, uh, you hear a few technical terms like back propagation, you learn all those techniques in this class. Uh, and by the end of class, you really build a learning algorithm much more effective than what you see here.
But let's- let's- let's see this application. Could you turn up the volume if you have that? Are you guys getting volume? Volume.
Oh, I see. All right. I'll narrate this. So I'll be using artificial neural network to drive this vehicle that, uh, was built at Carnegie Mellon University, uh, many years ago.
And what happens is, uh, during training, it watches the human. um, drive the vehicle and I think 10 times per second, uh, it digitizes the image in front of the vehicle. And, um, so that's the picture taken by a front-facing camera.
Um, and what it does is- in order to collect label data, the car while the human is driving it, records both the image, such as you're seeing here, as well as the steering direction that was chosen by a human. So at the bottom here is the image turned to the grayscale and lower res. And, uh, on top, let me pause this for a second. Um, this is the driver direction. The font's kind of blurry, but this text says driver direction.
So this is the Y label, the label Y. that the human driver chose. Um, and so the position of this white bar, of this white blob shows how the human is choosing to steer the car. So in this, in this image, the white blob is a little bit to the left of center. So the human is, you know, steering just a little bit to the left.
Um, this second line here is the output of the neural network. And initially, the neural network doesn't know how to drive, and so it's just outputting this white smear everywhere. It's saying, you know, I don't know, do I?
I drive left, right, center, I don't know. So it's outputting this gray blur everywhere. Um, and as the algorithm learns using the back propagation learning algorithm or gradient descent, which you learn about, uh, you actually learn about gradient descent this Wednesday.
Um, you see that the neural networks outputs becomes less and less of this white smear, this white blur, but starts to become sharper, um, and starts to mimic more accurately the human selected. driving direction, right? So this, um, is an example of supervised learning because the human driver demonstrates inputs x and outputs y.
Uh, namely, if you see this in front of the car, steer like that, so that's x and y. And, uh, after the learning algorithm has learned, um, you can then, uh, well, he pushes the button, takes a hand off the steering wheel, um, and then it's using this neural network to drive itself. Right. Digitizing the image in front of the road, taking this image and passing it through the learning algorithm through the trained neural network, letting the neural network select the steering direction, and then using a little motor to turn the wheel. This is a slightly more advanced version which has trained two separate models, one for I think a two-lane road, one for a four-lane road.
So that's the, um, uh, So the second and third lines, this is for a two-lane road, this is a four-lane road, and the arbitrator is- is another algorithm that tries to decide whether the two-lane or the four-lane road model is the more- more appropriate one for a particular given situation. Um, and so as Alvin is drawing, excuse me, a one-lane road or a two-lane road. So it says driving from a one-lane road here, uh, toward an intersection, um, The, uh, the algorithm realizes it should switch over from, um, I think, I forget, I think the one-lane neural network to the, to the two-lane neural network, one of these, right. Um. Okay.
Oh, oh, all right, fine. We'll just see the final dramatic moment of switching from a one-lane road to a two-lane road. All right. Um, uh, and I think, you know, so this is just using supervised learning to take as input what's in front of your car, to decide on steering direction. This is not say the odds for how self-driving cars are built today.
But, you know, it could do some things in some limited contexts. Uh, uh, and I think, uh, in- in- in several weeks, you'll actually be able to build something that is more sophisticated than this. Um, so after supervised learning, uh, we will- will in this class suspend.
bit of time talking about machine learning strategy or some, or I think on the class notes we, um, annotate this as a learning theory. But what that means is, um, I wanna give you the tools to go out and apply learning algorithms effectively. And I think I've been fortunate to have, uh, you know, to know a lot of, uh, uh, uh, I think that, um, I've been fortunate to have, you know, over the years constantly visited lots of great tech companies, uh, uh, more than once, that I've been publicly associated with, right? But often, just to help friends out, I visit various tech companies of the sort whose products I'm sure are installed on your cell phone. But I often visit tech companies and, you know, talk to the machine learning teams and see what they're doing and see if I can help them out.
And what I see is that there's a huge difference in the effectiveness of how two different teams could apply the exact same learning algorithm, right? And I think that what I've seen... badly is that sometimes there will be a team, uh, even in some of the best tech companies, right? The, the, the, the, the elite AI companies, right? In, in, in multiple of them where you go talk to a team and they'll tell you about something they've been working on for six months.
And then you can quickly take a look at the data and, and hear that they're not, their algorithm isn't quite working and sometimes you could look at what they're doing and go, yeah, you know, I could have told you six months ago that this approach is never gonna work, right? Um, and, And, um, what I find is that the most skilled machine learning practitioners are very strategic by which I mean that your skill at deciding, um, when you work on a machine learning project, you all- you- you have a lot of decisions to make, right? Do you collect more data?
Do you try a different learning algorithm? Uh, do you rent faster GPUs to train your learning algorithm for longer? Or if you collect more data, what type of data do you collect?
Or for all of these architecture choices, using neural networks, vector machine, digital regression, which ones do you pick? But there are a lot of decisions you need to make when building these learning algorithms. So one thing that's quite unique to the way we teach is, we want to help you become more systematic in driving machine learning as a, as a systematic engineering discipline.
So that when one day you're working on machine learning project, you can efficiently figure out what to do next, right? And I sometimes make an analogy to how, to, to, to, to software. we're engineering.
Um, you know, I, many years ago, I had a friend, um, that would debug code by compiling it. And then, um, uh, this friend would look all these syntax errors, right? That, you know, C++ compiler outputs. And they thought that the best way to eliminate the errors is to delete all the lines of code with syntax errors and that was their first serious thing. So that did not go well, right?
Um, uh, uh, it took me a while to persuade them to start doing that. Uh, but, So it turns out that, um, when you run a learning algorithm, you know, it almost never works the first time, right? It's just like, uh, uh, and- and the way you go about debugging the learning algorithm will have a huge impact on your efficiency or on- on how quickly you can build effective learning systems. And I think until now, too much of the- of this process of, uh, making your learning algorithms work well has been a black magic kind of process where, you know, has worked on this for decades.
So when you run- something and don't know why it's not working, you go, hey, what did I do? And says, oh yeah, I do that. And then, and then because he's so experienced it works. But I think, um, what we're trying to do with the discipline of machine learning is to evolve it from a black magic, tribal knowledge experience-based thing to a systematic engineering process, right?
And so, um, later this quarter as we talk about machine learning strategy, I'll talk about learning theory, we'll try to systematically give you tools on how to, um, uh, go about strategizing, uh, so it can be very efficient in, um, how you, how you yourself or how you can lead a team to build an effective learning system. Because I, I don't want you to be one of those people that, you know, waste six months on some direction that maybe could have relatively quickly figured out what's not promising. Or maybe one last analogy, if you, um, if you're used to optimizing code, right? Making code run faster, not sure how many of you have done that, uh, uh, uh, less experienced software engineers will just dive in and optimize the code to try to make it run faster, right? Let's take the C++ and code an assembly or something.
But more experienced people will run a profiler to try to figure out what part of your code is actually a bottleneck, and then just focus on changing on that. So, uh, one of the things hope to do this quarter is, uh, uh, uh, convey to you some of these more systematic engineering principles, right? And, yeah.
Oh, and actually, if anyone interested, this is, uh, uh, uh, yeah. Actually, I've been, um, I've been writing- so actually, so how many of you have heard of machine learning and learning? Oh, just a few of you.
Interesting. So actually to- to- if any of you are interested, um, just in my, uh, spare time, uh, I've been writing a book, um, uh, to try to codify systematic- engineering principles for machine learning and so if you, uh, uh, and, and so, uh, if you want a, you know, free draft copy of the book, sign up for a mailing list here. I tend to just write stuff and put it on the Internet for free, right?
So if you want a free draft copy of the book, uh, uh, uh, you know, go to this website, uh, enter your email address and the website will send you a copy of the book. They'll talk a little bit about these engineering principles as well. Okay.
All right. So, uh, so first subject, machine learning. Second subject, learning theory. Um, and, uh, the third major subject we'll talk about is, uh, deep learning, right?
And so, you know, there are a lot of tools in machine learning and- many of them are worth learning about and I use many different tools in machine learning, you know, for many different applications. There's one subset of machine learning that's really hot right now because it's just advancing very rapidly which is deep learning. And so we'll spend a bit of time talking about. deep learning so that you can understand the basics of, uh, how to train a neural network as well, right?
Uh, but I think that, um, whereas 239 covers a much broader set of algorithms which are all useful, uh, CS230 more narrowly covers just deep learning, right? Um, so, uh, other than deep learning slash after, after deep learning slash neuro- neural networks, the other, the, the four, fourth of the five major topics we'll cover will be be, um, unsupervised learning. Um, so what is unsupervised learning? So you saw me draw a picture. Like this just now, right?
And this would be a classification problem, like the tumor-molecular benign problem. This is a classification problem. And that was a supervised learning problem because you have to learn a function mapping from x to y.
Um, unsupervised learning would- if I give you a dataset like this with no labels. So you're just given inputs x and no y, and you're asked to find me something interesting in this data, figure out, you know, interesting structure in this data. Um, And so in this dataset, it looks like there are two clusters and an unsupervised learning algorithm, which you learn about called k-means clustering, will discover this, um, this structure in the data.
Um, other examples of unsupervised learning, you know, if- if you- actually Google News is a- is a very interesting website. Sometimes I use it to look up, right, latest news. Uh, this is an old example. But Google News every day, uh, crawls or reads, uh, uh, uh, I don't know, uh, uh, many, many thousands or tens of thousands of tens of thousands of news articles on the Internet and groups them together, right?
For example, this is a set of articles on the BP Orwell spill, and it has, uh, taken a lot of the articles written by different reporters and grouped them together. So you can, you know, figure out that, uh, what, uh, BP, uh, Macondo Orwell, right? That this is a CNN article about the Orwell spill, this is a Guardian article about the Orwell spill, and this is an example of a clustering algorithm, whereas, uh, taking these different new sources and figuring out that these are all stories kind of about the same thing, right?
Um, and other examples of clustering, just getting data and figuring out what groups belong together. Um, a lot of work on, um, genetic data. This is a visualization of, uh, uh, uh, genetic microarray- array data where given data like this, you can group individuals into different types of . of individuals of different characteristics.
Or clustering algorithms grouping the separate data together is used to organize computing clusters, you know, figure out what machines workloads are more related to each other, and organize community clusters appropriately. So take a social network like, you know, LinkedIn or Facebook or other social networks and figure out which are the groups of friends or which are the cohesive communities within a social network. Or marker segmentation, uh, actually, many companies I've worked with look at the customer database and cluster the users together.
So you can say that looks like we have four types of users, you know, looks like that, um, there are the, uh, uh, young professionals looking to develop themselves. There are the, you know, soccer moms and soccer dads, there are the discality and discality. So you can then market to the different market segments, um, separately. Uh, and, and, and actually many years ago, my friend Andrew Moore, uh, uh, was using this type of data for astronomical data. analysis to group together galaxies.
Question? Oh, is all unsupervised learning clustering? Uh, no, it's not.
So unsupervised- unsupervised learning broadly is the concept of using unlabeled data. So just x and finding interesting things about it, right? Um, so, uh, for example, uh, actually here's- shoot. This won't work without audio.
We'll do this later in the cl- in the class I guess. Um, maybe we'll see if we can do this later. Cocktail party problem, um, uh, is another unsupervised learning problem.
Really need all- audio for this to explain this though. Um, let me think how to explain this. Uh, you know, cocktail party problems. I'll, I'll try to do the demo when we can get audio working on this laptop.
There's a problem where, um, uh, if you have a noisy room and you stick mul- multiple microphones in the room, and record overlapping voices, uh, so there are no labels, where you just have multiple microphones, a mic- an array of microphones in a room with lots of people talking, uh, how can you have the algorithm separate out the people's voices? So that's an unsurprised- learning problem because, um, there are no labels. You just stick microphones in the room and have it record different people's voices, overlapping voices, you have multiple people talking at the same time, and then have it try to separate out people's voices. And one of the programming exercises you do, uh, later is if we have, you know, five people talking.
So each microphone records five people's overlapping voices, right? Because, you know, each microphone hears five people at the same time. How can you have an algorithm separate out these voices so you get . clean recordings of just one voice at a time.
So that's called the cocktail party problem. And the algorithm we used to do this is called ICAO, Independent Components Analysis. And that's something you implement in one of the latest, uh, homework exercises. Great.
Um, and there are other examples of unsurprised learning as well. Uh, uh, the Internet has tons of unlabeled text data. You just suck down data from the Internet. Uh, there are no labels, uh, necessarily.
But can you learn interesting things about language? Figure out what, figure out the... Uh, I don't know, one of the best cited results recently was learning analogies, like, you know, a man as a woman, as king, as a queen, right? Um, uh, or, uh, what, uh, Tokyo as to Japan, as, uh, Washington DC as to United States, right?
To learn analogies like that. It turns out you can learn analogies like that from unlabeled data, just from text on the Internet. So that's also unsupervised learning.
Okay. Um, so after unsupervised learning, oh, and unsupervised learning, so, you know, machine learning is very useful today. Turns out that most of the recent wave of economic value created by machine learning is through supervised learning.
Uh, but there are important use cases for unsupervised learning as well. So I use them in my work occasionally. Uh, uh, and it's also a beating edge for a lot of exciting research.
And then the final topic, final of the five topics we'll cover. So we'll talk about super- supervised learning, um, machine learning strategy, deep learning, unsupervised learning, and then the fifth one is reinforcement learning, is this. Which is, um, let's say I give you the keys to Stanford Autonomous Helicopter. Uh, this helicopter is actually sitting in my office.
I'm trying to figure out how to get rid of it. Um, and- and I'll ask you to write a program to- to- to make it fly, right? So how do you do that? Um, so this is a video of a helicopter flying.
Uh, the audio is just of a lot of- of helicopter noise so that's not important. But, uh, we'll zoom out the video so you can see trees falling in the sky, right? But so, um, you can use learning algorithm. Yeah, that's kinda cool, right?
I- I was the- I was the cameraman that day. Um, but so you can use learning algorithms to get, you know, robots to do pretty interesting things like this. Um, and it turns out that a good way to do this is through reinforcement learning.
So- so what's reinforcement learning? Um, it turns out that no one knows what's the optimal way to fly a helicopter, right? If you're flying a helicopter, you have, uh, two control sticks that you're moving, um, but no one knows what's the optimal way to move the control stick. So the way you can get a helicopter to fly itself is, um, let the helicopter do whatever.
Think of it as training a dog, right? Like how- you can't teach a dog the optimal way to behave. But actually, how- how many of you have had a pet dog or pet cat before? Huh, not that many.
That's fascinating. Okay. So I had a pet dog when I was a kid, and my family made it my job. job to train the dog.
So how do you train the dog? You let the dog do whatever it wants and then whenever it behaves well, you go, oh, good dog. And when it misbehaves, you go, bad dog. Um, and then over time, the dog learns to do more of the good dog things and fear of the bad dog things. And so reinforcement learning is a bit like that, right?
I don't know what's the optimal way to fly a helicopter. So you let the helicopter do whatever it wants and then whenever it flies well, you know, does the maneuver you want or flies accurately without jittering around. around too much, you go, oh, good helicopter. And when it crashes, you go, bad helicopter.
And it's the job of the reinforcement learning algorithms to figure out how to control it over time so as to get more of the good helicopter things and fewer of the bad helicopter things. Um, And I think, um, well, just one more video. Um, oh, yeah, all right. And so again, given a robot like this, I- I actually don't know how to program a- I actually, you know, robot like this has a lot of joints, right?
So how do you get a robot like this to climb more obstacles? So well, this is actually a robot dog. So you can actually- actually say good dog or bad dog. Um, but by giving those signals called the reward signal, uh, you can have a learning algorithm figure out by itself how to optimize the reward and therefore climb over. these types of obstacles.
Um, and I think recently the most famous applications of reinforcement learning have been for game playing, uh, playing Atari games or playing, you know, game of Go, like in AlphaGo. Um, I think that, uh, I- I- I think that, uh, uh, game playing has made for some remarkable stunts, a remarkable PR, but I'm also, uh, equally excited or maybe even more excited about the inroads that reinforcement learning is making into robotics applications. So I think, uh, uh, I think, yeah. Reinforcement has been proven to be fantastic for playing games. It's also getting- making real traction in, um, optimizing robots and optimizing sort of, uh, the logistics system and things like that.
Um. So, you learn about all these things. Um, last thing for today, uh, I hope that you will start to, uh, talk- meet people in the class, make friends, form project partners and study groups.
Uh, and if you have any questions, you know, dive on the Piazza, ask your questions, let's help others answer the questions. So, let's break for today and I look forward to seeing you on Wednesday. Welcome to TubeTune.