So anyone who's been paying attention for the last few months has been seeing headlines like this, especially in education. The thesis has been students are going to be using chat GPT and other forms of AI to cheat, do their assignments, they're not going to learn, and it's going to completely undermine education as we know it. Now what I'm going to argue today is not only...
Are there ways to mitigate all of that? If we put the right guardrails, we do the right things, we can mitigate it. But I think we're at the cusp of using AI for probably the biggest positive transformation that education has ever seen.
And the way we're going to do that is by giving every student on the planet an artificially intelligent but amazing personal tutor, and we're going to give every teacher on the planet an amazing artificially intelligent teaching assistant. And just to appreciate how big of a deal it would be to give everyone a personal tutor, I show you this clip from Benjamin Bloom's 1984 Two Sigma study, or he called it the Two Sigma Problem. The two sigma comes from two standard deviations, sigma the symbol for standard deviation. And he had good data that showed that, look, a normal distribution, that's the one that you see in the traditional bell curve right in the middle, that's how the world kind of sorts itself out, that if you were to give personal one-to-one tutoring for students, that you could actually get a distribution that looks like that right.
It says tutorial one-to-one with the asterisk, like that right distribution, a two standard deviation improvement. Just to put that in plain language. That could take your average student and turn them into an exceptional student. It can take your below-average student and turn them into an above-average student.
Now, the reason why he framed it as a problem was he said, well, this is all good, but how do you actually scale group instruction this way? How do you actually give it to everyone in an economic way? What I'm about to show you is, I think, the first moves towards doing that. Obviously, we've been trying to approximate it in some way at Khan Academy for over a decade now, but I think we're at the cusp of accelerating it dramatically.
I'm going to show you the early stages of what RAI, which we call Khanmigo, what it can now do, and maybe a little bit of where it is actually going. So this right over here is a traditional exercise that you or many of your children might have seen on Khan Academy. But what's new is that little bot thing at the right. And we'll start by seeing one of the very important safeguards, which is the conversation is recorded and viewable by your teacher. It's moderated, actually, by a second AI.
And also, it does not tell you the answer. It is not a cheating tool. Notice, when the student says, tell me the answer, it says, I'm your tutor.
What do you think is the next step for solving the problem? Now, if the student makes a mistake, and this will surprise people who think large language models are not good at mathematics, notice not only does it notice the mistake, it asks the student to explain their reasoning. but it's actually doing what I would say not just even an average tutor would do, but an excellent tutor would do. It's actually able to divine what is probably the misconception in that student's mind, that they probably didn't use the distributive properly. Remember, we need to distribute the negative two to both the nine and the 2m inside of the parentheses.
This, to me, is a very, very, very big deal. And it's not just in math. This is a computer programming exercise on Khan Academy where the student needs to make the clouds part.
And so... We can see the student starts defining a variable, left x, minus, minus. It only made the left cloud part, but then they can ask a conmigo, what's going on?
Why is only the left cloud moving? And it understands the code. It knows all the context of what the student is doing, and it understands that those ellipses are there to draw clouds, which I think is kind of mind-blowing. And it says, to make the right cloud move as well, try adding a line of code inside the draw function that increments the right x variable by one pixel in each frame.
Now, this one is maybe even more amazing, because we have a lot of math teachers. We've all been trying to teach the world to code, but there aren't a lot of computing teachers out there. And what you just saw, even when I'm tutoring my kids when they're learning to code, I can't help them this well, this fast.
This is really going to be a super tutor. And it's not just exercises, it understands what you're watching, it understands the context of your video, it can answer the age-old question, why do I need to learn this? And it asks Socratically, well, what do you care about? And let's say the student says, I want to be a professional athlete, and it says, well... Learning about the size of cells, which is what this video is about, that could be really useful for understanding nutrition and how your body works, etc.
It can answer questions, it can quiz you, it can connect it to other ideas. You can now ask as many questions of a video as you could ever dream of. So, another big shortage out there.
I remember the high school I went to, the student-to-guidance counselor ratio was about 200 or 300 to 1. In a lot of the country, it's worse than that. We can use Conmigo to give every student a guidance counselor, academic coach, career coach, life coach, which is exactly what you see right over here. And this is all, you know, we launched this with the GPT-4 launch. We have a few thousand people on this. This isn't a fake demo.
This is really in action. And then there's... you know, things that I think it would have been even harder, it would have been a little science fiction to do with even a traditional tutor.
There's a student, we run an online high school with Arizona State University called Khan World School, and we have a student who attends that online school based in India, her name is Sanvi, and she was doing a report on the Great Gatsby. And when she was reading the Great Gatsby, Jay Gatsby keeps looking at the green light off into the distance. And she's like, well, why does he do that?
She did some web searches, and people have obviously studied this and commented about the symbolism of that, but none of it was really resonating with her. And then she realized that she had Conmigo and that she could talk to Jay Gatsby himself. And so... Ah, splendid choice, old sport. I am now Jay Gatsby, the enigmatic millionaire from Escott's Gerald's Fit Classic.
And so why do you keep staring at the green light? Ah, the green light, old sport. It's a symbol of my dreams and desires, you see.
It's situated at the end of Daisy Buchanan's dock, across the bay from my mansion. I gaze at it longingly as it represents my yearning for the past and my hope to reunite with Daisy, the love of my life. And what was cool is, Sanvi had said, I had this long conversation, she called him Mr. Gatsby, and at the end she actually apologized for taking his time, which I thought was very polite of her.
But you can imagine, this unlocks learning literature, learning, you can talk to historical figures, we're even probably going to add an activity, you can talk to like the Mississippi River. It brings things to life in ways that really were science fiction even six months or a year ago. Students can get into debates with the AI.
And we've got, here's the students debating whether we should cancel student debt. The student is against canceling student debt. And we've gotten very clear feedback.
We started running it at Con World School and our lab school that we have, Con Lab School. The students... the high school students especially, they're saying, this is amazing to be able to fine-tune my arguments without fearing judgment.
It makes me that much more confident to kind of go into the classroom and really participate. And we all know that Socratic dialogue debate is a great way to learn, but frankly, it's not out there for most students, but now it can be accessible to hopefully everyone. A lot of the narrative, we saw that in the headlines, has been, it's going to do the writing for kids, kids are not going to learn to write, but we are showing that there's ways that the AI doesn't write for you, it writes with you. So this is a little thing, and my eight-year-old is addicted to this, and he's not a kid that really liked writing before, but where, you know, you could say, I want to write a horror story, and it says, ooh, a horror story, how spine-tingling and thrilling.
Let's dive into the world of eerie shadows and chilling mysteries. And this is an activity where... the student will write two sentences, and then the AI will write two sentences.
And so they collaborate together on a story. The students write, Beatrice was a misunderstood ghost. She wanted to make friends but kept scaring them by accident. And the AI says, poor Beatrice, a lonely spirit yearning for companionship. One day she stumbled upon an old abandoned mansion, etc., etc.
I encourage you all to hopefully one day try this. This is surprisingly fun. Now, to even more directly hit this use case, and what I'm about to show you, everything I showed you so far is actually already part of Conmigo. What I'm about to show you, we haven't shown to anyone yet. This is a prototype.
We hope to be able to launch it in the next few months. But this is to directly use AI, use generative AI, to not undermine English and language arts, but to actually enhance it in ways that we couldn't have even conceived of even a year ago. This is reading comprehension. The students reading Steve Jobs'famous... speech at Stanford, and then as they get to certain points, they can click on that little question, and the AI will then Socratically, almost like an oral exam, ask the student about things.
And the AI can highlight parts of the passage. Why did the author use that word? What was their intent? Does it back up their argument?
They can start to do stuff that, once again, we never had the capability to give everyone a tutor, everyone a writing coach to actually dig into reading at this level. And you could go on the other side of it. We have a whole workflow that helps them write, helps them be a writing coach, draw an outline.
But once a student actually constructs a draft, and this is where they're constructing a draft, they can ask for feedback, once again, as you would expect from a good writing coach. In this case, the student will say, let's say, does my evidence support my claim? And then the AI not only is able to give feedback, but it's able to highlight certain parts of the passage and says, you know, on this passage, this doesn't quite support your claim, but once again, Socratically says, can you tell us why?
So it's pulling the student, it's making them a better writer, giving them far more feedback than they've ever been able to actually get before. And we think this is going to dramatically accelerate writing, not hurt it. Now, everything I've talked about so far is for the student, but we think this could be equally as powerful for the teacher to drive more personalized education and, frankly, save time and energy for themselves and for their students. So this is an American history exercise on Khan Academy. It's a question about the Spanish-American War.
And at first, it's in student mode. And if you say, tell me the answer, it's not going to tell the answer. It's going to go into tutoring mode.
But that little toggle which teachers have access to, they can turn student mode off, and then it goes into teacher mode. And what this does is, it turns into, you could view it as a teacher's guide on steroids. Not only can it explain the answer, it can explain how you might want to teach it. It can help prepare the teacher for that material. It can help them create lesson plans.
see doing right there. It'll eventually help them create progress reports and help them eventually grade. So once again, teachers spend about half their time with this type of activity, lesson planning.
All of that energy can go back to them or go back to human interactions with their actual students. So, you know, one point I want to make, these large language models are so powerful, there's a temptation to say, like, well, all these people are just going to slap them onto their websites and it kind of turns the applications themselves into commodities. And what I got to tell you is I kind of thought that that's one of the reasons why I didn't sleep for two weeks when I first had access to GPT-4 back in August. But we quickly realized that to actually make it magical, I think what you saw with Conmigo a little bit, it didn't interact with you the way that you see ChatGPT interacting.
It was a little bit more magical. It was more Socratic. It was clearly much better at math than what most people are used to thinking.
And the reason is there was a lot of work behind the scenes to make that happen. And I could go through the whole list of everything we've been working on, many, many people, for over six, seven months to make it feel magical. But perhaps the most intellectually interesting one is we realized that we were not only making it and this was an idea from an open AI researcher, that we could dramatically improve its ability in math and its ability in tutoring if we allowed the AI to think before it speaks. So if you're tutoring someone and you immediately just start talking before you assess their math, you might not get it right.
But if you construct thoughts for yourself, and what you see on the right there is an actual AI thought, something that it generates for itself, but it does not share with the student, then its accuracy went up dramatically and its ability to be a world-class tutor went up dramatically. And you can see it's talking to itself here. It says, the student got a different answer than I did, but do not tell them they made a mistake.
Instead, ask them to explain how they got to that step. So I'll just finish off. Hopefully, you know, what I've just shown you is just half of what we are working on, and we think this is just the very tip of the iceberg of where this can actually go.
And I'm pretty convinced, which I wouldn't have been even a year ago, that we, together, have a chance of addressing the two-sigma problem and turning it into a two-sigma opportunity, dramatically accelerating education as we know it. Now, just to take a step back at a meta level, obviously we heard a lot today, the debates on either side. There's folks who take a more pessimistic view of AI. They say, this is scary.
There's all these dystopian scenarios. We maybe want to slow down. We want to pause.
On the other side, there are the more... Optimistic folks say, well, we've gone through inflection points before, we've gone through the Industrial Revolution, it was scary, but it all kind of worked out. And what I'd argue right now is, I don't think this is like a flip of a coin, or this is something where we'll just have to wait and see which way it turns out. I think everyone here and beyond, we are active participants in this decision. I'm pretty convinced that the first line of reasoning is actually almost a self-fulfilling prophecy, that if we act with fear and if we say, hey, we just got to stop doing this stuff, what's really going to happen is the rule followers might pause, might slow down, but the rule breakers, as Alexander mentioned, the totalitarian governments, the criminal organizations, they're only going to accelerate.
And that leads to what I am pretty convinced is the dystopian state. Which is the good actors have worse AIs than the bad actors. But I'll also, you know, talk to the optimist a little bit.
I don't think that means that, oh yeah, then we should just relax and just hope for the best. That might not happen either. I think all of us together have to fight like hell to make sure that we put the guardrails, we put in, when the problems arise, reasonable regulations, but we fight like hell for the positive use cases.
Because, very close to my heart, and obviously there's many potential positive use cases, but perhaps the most powerful use case, and perhaps the most poetic use case, is if AI, artificial intelligence, can be used to enhance HI, human intelligence, human potential, and human purpose. Thank you.