Okay, welcome back everyone to my video podcast on the future of business and technology. Make sure you subscribe if you want to listen to many more fascinating conversations about the future of business and technology and the latest trends. Today's topic is artificially intelligent robots at Amazon. And my guest today is Ty Brady, who is the chief tech technologist at Amazon Robotics.
Welcome, Ty. Thank you, Bernard. Thank you for having me on. I appreciate it. Yeah, so this is a fascinating topic as robots have become more intelligent, especially when we combine our amazing progress we've made in AI and especially around generative AI and if we bring this into robotics and the physical world.
Before we go into some of those topics. I'd love you to give us an overview of robotics at Amazon today in order for us to better understand the scale of robots and how they're being used in Amazon today. Well, I'd be happy to.
Again, it's a pleasure to be here and looking forward to this great conversation. I always get super excited talking about not only the future, but how robotics and people can work together in order to enable a really bright future. So, yeah, so I'm Ty Brady. I'm the chief technologist for robotics at Amazon. My job is to incubate and help accelerate technology development for the benefit of our customers and most importantly for our employees.
And it's an amazing job. When we start talking about the future, there's no doubt that technology plays a big part of that. And our philosophy is that we use robotics and automation.
particularly fueled by AI to extend human capability, to allow people to do their jobs in a better manner, to allow them to be more efficient, to allow them to focus more on what matters and to really focus on higher level problems. So it's been really exciting. I've been in the field for a number of years.
It's super exciting because I feel like it's almost taken us 50 years even just to get here, right? The birth of the computer, of course. We have the age of the internet that has come across that helped connect all of us, our roboticists, and share our learnings. A lot of new sensors came on board, let's say, in the 2000s for sure.
Now we're all starting to kind of put these together in our different actuation systems as well with our sensing systems. And then in the tens, if we can call it the tens, I don't even know what we call the 2010s tens, we start to see AI kind of peek up and do some really interesting things for us in robotics, but it's very early stages for sure. And here we are now with generative AI, I think really changing the game when it comes to our fleet of robotics that we have inside of Fulfillment.
And it's quite a fleet. I mean, we have the world's largest fleet of industrial mobile robots out there, more than 750,000 drive units alone that we've deployed globally, helping our customers get it. better delivery at low cost. Very good. So the largest fleet of robots worldwide.
Industrial robots. Yeah. So maybe you can give us a flavor for what they do today.
Yeah. So we have what we call our Hercules Drive unit. And what that does is that moves pods on demand.
So when you go to amazon.com and you make an order, we've stored those goods in our what we call fulfillment centers. where we physically have the goods. And the revolution, if you will, is instead of a person going to the goods, like with carts and walking the long aisles and picking the goods and putting it into a cart and then picking another goods, putting it in the cart and then packing out that order, we turn that right on its head. And instead of people going to the goods, the goods actually come to the people, right? So we can now move these shelves of inventory on demand very efficiently and store them with a really great...
packing efficiency and bring those to a station where we have an employee actually pick those goods out automatically or pick those goods out with a lot of technology behind them to make sure that you're actually getting the right good just at the right time. And it's quite massive. You can see a 40% improvement just in density.
alone as compared to our traditional system where you have aisles because you can physically store more goods and it's much more timely as well. Or we went from two day delivery to, you know, some some cases, one hour delivery. Very good. And the big hype is around integrating AI effectively into robotics, especially generative AI, as you mentioned.
Maybe you can give us an overview of what you are seeing there in terms of the state of the art when it comes to AI in robotics. Yeah. Well, it may feel like hype, but at Amazon, we have been using AI for well over 25 years, believe it or not.
Imagine when you have so many goods to offer, 100 million plus different types of objects to offer to our customers, that just predicting where those goods need to be, predicting the inventory models, we've been doing that for 20 plus years. From the early days of Amazon, using machine learning systems in order to really source the right good at the right place. And we continue to evolve with our machine learning systems. Of course, AWS is, we use AWS inside our fulfillment stack, and it's pretty incredible with the tool set that is offered to us. But it's, there's an application layer of AWS.
There's also the middleware layer that doesn't get a lot of prime time, if you will, but it has to be very safe. It has to be very reliable, has to be very secure. AWS offers that as well. And even Below that, we actually have our own hardware and our own physical systems that enable for a better AI machine learning experience for us to use inside of fulfillment.
So it's a journey that is not an overnight journey for sure. It's multi-decades in the making. But we see it, boy, I'd say I'd be hard pressed not to find an example where we're not using machine learning and AI. For our mobile drive systems, like I... I mentioned with the 750,000 drive units alone, we have machine learning systems that help predict the paths or where each drive should be at any point in time.
It helps optimize the actual flow of material inside our fulfillment centers. In our manipulation systems where we're picking up large, heavy boxes, for example, we have systems that help perceive the boxes and what's the right angle to actually pick them up in what we call segmentation. We make sure that one box is one box and not two boxes.
How can we pick it up so it doesn't open up upon itself? Systems that are involved there. And one of our latest drives is called Proteus.
And this is a real game changer for us in that it is certified safe to be around people. So it's a collaborative robot. It can be wherever you walk.
This robot can carry goods as well. It is very perceptive in that it understands people, it understands what objects are, understands moving objects, and I think one of the coolest things that it does is that if there's a cluster of people and they're talking or whatever, let's imagine Proteus was at a cocktail party, right? What was state-of-the-art is that if you see a large group of people, this mobile robot would generally just stop and wait for people to kind of disperse until there's enough area for it to safely get through.
And how Proteus is different is it fuses together a whole bunch of sensors, and it just kind of slowly makes its way through the cocktail party. Of course, always very safely, but whenever there's a little opportunity to move, it'll just nudge itself forward and get through to get the job done. And that's really hard to do.
And it's one of the things that we're most proud of here is that can we build our robotics such that they extend human capability and also create a safer environment? I think we've done a great job. Yeah, that's fascinating because it's really a true game changer. In the past, we've had robotics that sit behind cages on assembly lines in factories. They were kept away from people because they had to be programmed to do whatever they needed to do.
And now we are bringing in technology like AI and generative AI to make them true core workers. Where do you see this going then in the future? And what are some of the challenges that need to be... overcome from your experience in order to make them truly effective co-workers in lots of different environments? Yeah, so the future, no doubt, is collaborative.
The future is where you allow people to have the best tool set possible in order to do their job. And that puts the burden not on our employees. or the people using the machines, but it puts the burden on us roboticists. We have to build the best robotics that bring out the best in people. right?
So it's a shared system of people and machines working together, not people versus machines, right? So anytime that we can make it more collaborative, more intuitive, more understandable, where you don't need this like, you know, 18 levels of degree in order to figure out how to use the machine, we want to make it very straightforward and understandable to have utility that allows people to do what they do best, which is problem solve, think at a higher level. and allow them to focus more on what matters inside fulfillment that's pretty easy for us right we want to have the ultimate in customer selection we want to bring that the the goods to at a low cost and we want to have the ultimate in customer convenience as well so allowing our folks whether it's in the first mile middle mile or last mile to have this robust tool set of robotics and automation um we see a great productivity gains right we and also we're really improving our safety as well by doing this. So the future in my mind is, I think it's going to be undeniable. Those that actually really embrace collaboration, embrace machines that help people do their jobs better.
Those are the ones that, you know, are going to accelerate. I agree. But whenever we talk about...
automation bringing in more capable AI-enabled robots people worry about their jobs and what this means what will automation mean for the jobs people do today how is this going to transform their job you you hinted to the fact that hopefully they will do slightly higher order jobs that really leverage some of the human capabilities that we have and pass some of the the more menial work and the repetitive work to our collaborative robots. How do you see this working out? And how do people prepare for this future?
Yeah, it's a great question. And I want to be super clear. I aim to eliminate the menial, the mundane, the repetitive. I want to automate all of that, right?
We want to create great jobs inside of our facilities. And there's plenty of great jobs. For example, since we...
really invested heavily in robotics about a decade ago. We've created 700 new job types alone related to robotics. So not only, so it's, it's, we're creating a new economy, a new workforce, and allowing our employees to be more productive in a safer environment.
And that has, that is not really to be understated because when you do that and you do that effectively. You as a company become more productive. And when you're more productive, then you have earned the right to invest in both people and in better robots.
You invest in both. And to go earlier in the conversation, we see the winning formula of people and machines working together. So if we can improve our machines, then that allows us to actually create new jobs and hire.
more jobs and that is exactly what the trend has been right so it's uh what we see is more robots more jobs when done right that's that's good to hear so what kind of jobs do you see being transformed or potentially even be eliminated and what exciting kind of jobs do you see emerging over the coming years that people might want to prepare themselves for? Yeah. So when it comes to jobs changing, absolutely they will.
Like I said, I aim to eliminate the menial, the mundane, and the repetitive. So if you are working in, you know, any one of those three, or hopefully not all three simultaneously, we want to automate that because we believe in the power of people. We believe that people have an amazing ability to understand problems, to think with common sense. to understand the big picture of what's going on. And we also live in a world where there's no such thing as 100% automation.
That's very clear. Nothing is going to work 100% of the time, all the time, especially at the scale of Amazon. So we need and we have amazing women and men that work frontline with our robotics, not in just directly with the robotics, but in what they do and what they achieve.
So we need things like... uh folks to make sure i'm sorry we need folks to make sure that the the machinery is working of course the robotics that are are working but also when you have more volume coming through and you can store more goods you're creating more jobs in the jobs that that you most definitely need, right? Flow control specialists, for example, understanding the flow of goods, making sure that trucks are coming in at the right time and trucks are leaving at the right time, talking to people that are on our front line and understanding where are the difficulties, how is the system working, and how can we improve?
And of course, I mean, I would be remiss if I didn't say the amazing, amazing staff that we have developing and truly pioneering robotics today. Robotics and automation is in our infancy, and we have an amazing high-tech team that works on this every day in order to make our robots even better. Very good. Another topic whenever we talk about intelligent robots is this whole idea of humanoid robots, robots that almost take the shape of people. They look like people.
And I'd love to... take or understand your take on all of this is this going to be successful will we see more humanoid robots in the future is this the place to be investing in i know that amazon was one of the first companies to employ humanoid robots so i'd love to know what you've learned from this i also know that you have made an investment in agility robotics so maybe you can take us through your point of when it comes to a humanoid robot Sure. Yeah, indeed.
So we are learning, right? We are interested in all things robotic. We are interested in manipulation systems, identification systems, sortation systems, and things that also move. And humanoid robots, particularly bipedal humanoid robots, we have an interest in that. It's interesting in the terrain that bipedals can navigate.
We have learned that... The wheels work as well, right? So it's not one size fits all for sure. Inside of our facilities, we have poured concrete floors and we understand what the floors are, for example. So again, anything in the mobility frame, we want to understand.
So with bipedals, we want to understand why is this unique and what can it do differently, right? Still learning. Whatever it is, it has to be dynamic, it has to be very reliable for us, and we know that we need to have even more exploration when it comes to mobile systems.
The second thing that is, I think, interesting is when you see the human form, the human form is beautiful. It's gorgeous. And there are expectations that come with the human form.
And I am a big proponent of utility. So robotics and automation needs utility first before form. It needs simplicity to eliminate complexity, right?
So if I see a humanoid robot, speaking, this is more speaking as a roboticist, when you see humanoid, I think expectations get to be pretty high pretty quick. Right. But now I am reminded of a great quote by Roy Amara, who is a futurist who says that we tend to overestimate technology in the short term, but underestimate technology in the long term. Paraphrasing it.
Right. So I get that. I mean, I totally get that. And we see that what we've done inside of Amazon with our robotics, we could not do with people alone. And we definitely could not do that with machines alone.
Right. So over the long term, we live in the applied realm of robotics and automation. Like it is not YouTube moments for sure. These are robots that are working with our employees every day on a scale that is almost unmanageable. It has to work.
And what the beauty, the beautiful thing about that is, is that that forces us to be better. And. It accelerates our technology development.
We do technology development with project context, and that project context allows us to solve real-world problems with our real-world machinery that allows us to do what many people think are just very simple things. I just need to pick up that box. I need to move this good from this part of the building to the other part of the building. But when you do it at scale and you have to do it extremely safe and you have to do it in a way that actually... adds to the equation of productivity, that's where we have reached the next level.
We have amazing folks that are doing that every day. You know, humanoids, we'll see. It's a work in progress. Yeah, and there's some people in robotics that believe very strongly that robots shouldn't look like humans for lots of different reasons, because it might freak out people, it blurs the boundaries between humans and robots, and others are big advocates of it.
Where do you sit in this debate? Well, if you look at our fleet, and particularly let's look at Proteus. For example, Proteus is our autonomous mobile robot that can work around people.
And it's moving carts today down in Nashville and in Houston as well. And in Louisiana, we're running that experiment and it's pretty amazing to see. They're moving carts of goods, carts of boxes on demand at the right time around people that are also moving them. And what we see is, first of all, our employees love our robots. They're cute.
They're adorable for sure. And that is a good thing because it means that they're comfortable being around Proteus. And Proteus has taillights and it has visual indicators.
It even has eyes. Okay. It doesn't have arms, but it has eyes so that if Proteus wants to go around a corner, corner it kind of will avert its gaze and look around the corner and a person can look at it go oh it looks like it wants to take a right turn or it puts its blinker on it looks like it wants to take a right turn so as people we get our our we get visual cues from each other if you and i were walking down a hallway and i had a big stack of books you would probably give me a little bit of extra leeway to make sure i don't you know tumble the books or if i have a big stack of books and i see a mom with a baby carriage, I'm going to naturally slow way down to make sure that there's, you know, there's no chance of a collision, right?
These are things that we do very naturally, but we're getting clues from whether it's body language or it's our faces about how to react in the human world. So where I stand is that we have to get better at human machine interfaces, right? Again, we don't put the burden on people. We put the burden on the roboticists developing the machines in order to give the clues back to a person to live in the human environment, right?
So sometimes I do believe that it is necessary to give human-like features to our machines so that people can better interact and understand those machines. Yeah, interesting. And you've talked about the fulfillment centers and the robots being used in there.
Beyond that, where else in Amazon do you see the potential to use robots in the future? Sure. Anywhere that involves our customer, right?
So we think about it as first mile. That's where we store our goods and get the goods out the door. Our middle mile where we're sorting the goods to get the goods to maybe it's the right state or the right region. And then our last mile where we actually have the goods and then deliver it right to the customer's door.
And that's right for automation all the way through. Our job, my job with our team is we're exclusively focused on Amazon. Like there is so much, it's such the early stages and our team is doing such an amazing job that we have plenty in front of us in order to help revolutionize each one of those three stages.
We've done a lot of work in our first mile under the fulfillment centers. We've done a lot of work in our sortation centers as well with drives taking packages and sorting packages automatically to the right location, with robotic arms picking packages up and placing them in different bins. For example, our Robin arm has sorted more than 3 billion packages, just to kind of put scale on this.
We have a containerized storage system inside our fulfillment centers called Sequoia. We just announced that last year. It's a containerized storage system, and we can reduce the time it takes to process an order by 25% alone just by using this containerized system. That really helps our shipping predictability.
That helps us get the right goods to the right time. And then also at our last mile, same story there. We have folks that need to use machines to help sort goods, to help identify goods, to help. Move goods to help manipulate those goods. And it's pretty ripe as well.
So, and that's just an e-commerce. If you really want, sure, agriculture, 100%, healthcare, 100%, transportation, of course, like it's. It's kind of fun. I will say it's fun. I learned to program in the late 70s as a kid through the goodness of one particular teacher.
I was hooked. I love Star Wars. I saw R2-D2 was definitely my robot for sure.
By the way, not in the humanoid form. I just wanted to point that out. I had C-3PO, but who's in the back of the X-Wing fighter? It was a C-3PO. It was R2-D2.
and and not only was r2d2 in the back but it's actually helping luke skywalker be more jedi right yeah no absolutely um how far away are we from the last mile automation this is an area i think that is posing lots of challenges do you see any anything anything interesting on the horizon well i don't know if that's the the end state that i'm hoping for I don't think I'm hoping for the last mile to be automated. Interesting. Right?
The face of our company is there interfacing to our customers. And we take that very, very seriously. I love an interaction where it's like, hi, Mrs. Jones.
Nice to see you. How's Tommy doing? Great.
Great. He's off to school. Oh, okay. Here's your package. Can I take a return for you?
I'd be happy to do that. It's connection. Right, so now having that delivery associate powered with amazing robotics to help the job, absolutely. Having that delivery associate in a machine that allows her or him to do their job, to drive the route more efficiently, we're doing that today. Having AI systems understand where traffic is and what's the optimal way to do the delivery path.
We're doing that today. That makes sense to me. I like that.
We want them to feel empowered with robotics. We want our delivery associates to feel great about being part of Amazon. And we want our customers, most importantly, to be excited for their delivery and to be connected to us in a way that is natural. Interesting.
And how far are we away from having robots in our homes? As someone that lives and breathes robotics every single day, what's the timeframe we're looking at here? Well, you can go on Amazon.com and get yourself an Astro right in your home right now. So what timeframe do you see and what sort of...
developments do you think are on the horizon to have a meaningful robotic assistant in our house? Yeah. Well, again, with the broad view of robotics, I actually think the majority of homes have an amazing robot in their house right now.
It's such a good robot that people don't even call it a robot. It is so good that it just blends into the background and you use the machine to your... advantage.
So what's a robot in my mind? A robot is the blend of sensing, computing, and actuation in a physical instantiation in the real world environment, right? So it's not a piece of software and it can't just be a piece of hardware.
It has to have sensing, computing, and actuation. And I think one of the best robots that we have out there is our dishwasher, right? You take your dishes, you put it in there, it cleans the dishes, you take the dishes out and great. Could you do it by hand? You could absolutely do those dishes by hand.
And what I think is interesting about that example is that if you were a startup in robotics and we didn't invent the dishwasher yet, you may say, well, how are we going to do this? Well, let's watch how people do it. They pick up a plate. They put it under running water. They put some soap on a sponge.
You grab a sponge and then you scrub the sponge over the top of the plate. You rinse it and then you put it over on the shelf. So I think there's a tendency for people to go, let's duplicate that. So I'm going to need... humanoid robot with arms, with sensing systems in order to duplicate the dishwashing experience.
And then, you know, technology is not quite there. You fail, you're out of business. Whereas I think if you open your mind up to say what's going to be beneficial in the people, machines, collaboration, how can we do this? I think the dishwasher does it really well. Dump your plates over there, let it do its job.
And I pull it open like magic, they're clean. It's great. So it's in there. Now I get your question.
When's Rosie the robot going to show up? We're a ways away. But AI is accelerating this.
I mean, that is undeniable. AI in all of our systems. It's undeniable. And actually, what I think is really cool, too, is it's allowing us to think about the problem differently.
If a generative AI solution proposes something that we haven't thought of, that knowledge is not lost. We don't have to do it for sure. But what it does is actually informs us about something that is... interesting in the design space that we didn't consider yet.
So where I'm super bullish about AI, it actually allows us people to be more intelligent and to be more capable, teaching us to be more human. And I love that. Yeah, me too.
That's the vision. That would be great if we can achieve that. And when you look into the future, what trends do you see in robotics that you are excited about?
And what do you see on the horizon? Maybe you can give us an overview of some of the more immediate trends the next five to 10 years you're seeing, and then maybe even looking further ahead to 2050. Where do you see all of this going? Yeah, so I think offloading a lot of computational power into the cloud is a really good trend.
Because if you have to carry everything around and you're not connected, those systems become heavy, they can become power hungry, and they become unwieldy. So I think the right balance of edge computing, making your robotics just felt enough to have enough edge computing to make sure that you always maintain safety, but offloading the majority of it into the cloud so that other robotic agents can not only help you compute what you need to compute, but you also learn from other robotic agents in the field and allow you to update your own behavior. I'm super bullish on that.
I really like that. The second part is that our AI systems allow us to perceive the world better, right? To allow our machines to understand The human world in a way that allows it to move safely around people to do tasks that extend human capability, I'm very bullish on, right?
Our perception systems have just really improved, vastly improved over the last five years. Just vast, vast improvement. And that makes sense because at the heart of AI is pattern matching, right?
It's lots of patterns that what's this environment look like? How can I match the patterns? And if I see a trend in that pattern, could that tell me something?
And I think that there's a lot of analogy of how we do that. So definitely there. The down the road, I, okay, I'm willing to, I'm willing to make this, this, this statement. Down the road. I think it's not going to be the one robot that does it all.
On the contrary, I think it's going to be a suite of robotics that works in a coordinated fashion, each with their own utility, allowing people to do more things, whether it's at your home, or it's at your office, or it's at your fulfillment center. I think at the core of it, there's... Some fundamental capabilities like movement, like manipulation, like identification, like sortation, like storage, and for us we have packing, that when we master those core capabilities, we also call them primitives, when we master those, then we can put those primitives together in different manners to achieve new utility. But When we have robotic system A with robotic system B and C, we're connected to the cloud, they can learn from each other, they can react better to the environment as well, and then most importantly, they can share their learnings with people so that we can now have human supervisory control on top of our machines.
So in my world, the shortest way I can say this is that... Human supervisory control is a field that will completely change the world when done right with our machines. Right, so we have to build the machine so that we can put our intuition into machines and we have to build the machines so that they can understand the environment better.
But how to allow that, I'd say the theory, the philosophy of people and machines working together, of people of extending human capability, of collaborative robotics. of really embracing what we're good at, like really understanding what we're really good at and allowing that menial and the mundane and the repetitive to be automated. That's the stuff, right? And I think it's really exciting.
And we've had this proof point now for the last 10 years that Not only can you do that, and we're doing it for our customers, and we're customer obsessed, and we're always going to do it for our customers, but not only can you do that in a way that allows to be more productive, but you can do it in a way that enables more safety. You can do it in a way that allows us to, I think, have the world's best technology company out there, because it's all applied. It's real. Very good, wonderful. Tai, thank you so much for sharing all your amazing insights.
I very much share your hope that applying AI and robotics, that this will hopefully make us more human, because at the moment our amazing capabilities are not really leveraged because we do have to do stuff that we shouldn't really be wasting our time on. If we get to that point where we can become even more human, that will be an amazing future. So thank you very much.
My pleasure. And for anyone who ever wants to re-watch or re-listen to this conversation, simply head to any podcast platform or to YouTube, where you can find this conversation and hundreds of others like it. Thank you very much. Thanks, Bernard.
Thank you.