[Music] [Applause] he [Applause] [Music] [Music] [Music] [Music] [Applause] [Applause] [Music] [Applause] [Music] the [Music] [Applause] [Music] a [Music] [Music] I [Applause] [Music] [Applause] [Music] oh [Music] [Applause] [Applause] [Music] [Music] [Music] [Applause] [Applause] [Music] [Music] [Music] [Music] [Applause] [Music] [Music] [Music] [Applause] [Music] [Applause] [Music] [Music] [Music] [Music] [Music] [Applause] [Music] [Applause] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Applause] [Music] he [Music] oh [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] this is how intelligence is made a new kind of factory generator of tokens the building blocks of AI tokens have opened a new frontier the first step into an extraordinary world where endless possibilities are born tokens transform images into scientific data charting alien atmospheres and guiding the explorers of tomorrow they turn raw data into foresight so next time we'll be ready tokens decode the laws of physics to get us there [Music] faster and take us [Music] further token C disease before it takes hold they help us unravel the language of [Music] life and learn what makes us [Music] tick tokens connect the dots so we can protect our most noble creatures they turn potential into plenty and help us Harvest our Bounty tokens don't just teach robots how to move but to bring joy to lend us a hand [Applause] and put life within [Music] reach together we take the Next Great Leap to bravely go where no one has gone before [Music] [Music] and here is where it all [Music] begins welcome to the stage Nvidia founder and CEO Jensen [Music] Wong welcome to GTC what an amazing year we wanted to do this at Nvidia so through the magic of artificial intelligence we're going to bring you to nvidia's headquarters I think I'm bringing you to Nvidia headquarters what do you think this is this is where we work this is where we work what an amazing year it was and we have a lot of incredible things to talk about and I just want you to know that I'm up here without a net there are no scripts there's no teleprompter and I've got a lot of things to cover so let's get started first of all I want to thank all of the sponsors all the amazing people who are part of this conference just about every single industry is represented Healthcare is here Transportation retail gosh the computer industry everybody in the computer industry is here and so it's really really terrific to see all of you and thank you for sponsoring it GTC started with GeForce it all started with GeForce and today I have here a GeForce 5090 and 5090 unbelievably 25 years later 25 years after we started working on GeForce GeForce is sold out all over the world this is the 90 the Blackwell generation and compar bring it to the 49 you look how it's 30% smaller in volume it's 30% better at dissipating energy and incredible performance hard to even compare and the reason for that is because of artificial intelligence GeForce brought Cuda to the world Cuda enabled Ai and AI has now come back to revolutionize computer Graphics what you're looking at is real time computer Graphics 100% path traced for every pixel that's rendered artificial intelligence predicts the other 15 think about this for a second for every pixel that we mathematically rendered artificial intelligence inferred the other 15 and it has to do so with so much Precision that the image looks right and it's temporarily accurate meaning that from frame to frame to frame going forward or backwards because it's computer Graphics it has to stay temporarily stable incredible artificial intelligence has made extraordinary progress it has only been 10 years now we've been talking about AI for a little longer than that but AI really came into the world's Consciousness about a decade ago started with perception AI computer vision speech recognition then generative the last 5 years we've largely focused on generative AI teaching an AI how to translate from one modality to another another modality text to image image to text text to video amino acids to proteins properties to chemicals all kinds of different ways that we can use AI to generate generate content generative AI fundamentally changed how Computing is done from a retrieval Computing model we now have a generative Computing model whereas almost everything that we did in the past was about creating content in advance storing multiple versions of it and fetching whatever version we think is appropriate at the moment of use now ai understands the context understands what we're asking understands the meaning of our request and generates what it knows if it needs it'll retrieve information augments its understanding and generate answer for us rather than retrieving data it now generates answers fundamentally changed how Computing is done every single layer of computing has been transformed the last several years the last couple two three years major breakthrough happened fundamental advance in artificial intelligence we call it agentic ai agentic ai basically means that you have an AI that has agency it can perceive and understand the context of the circumstance it can reason very importantly it can reason about how to answer or how to solve a problem and it can plan an action it can plan and take action it can use tools because it now understands multimodality information it can go to a website and look at the format of the website words and videos maybe even play a video learns from what it learns from that website understands it and come back and use that information use that new found knowledge to do its job agentic AI at the foundation of agentic AI of course something that's very new reasoning and then of course the next wave is already happening we're going to talk a lot about that today robotics which has been enabled by physical ai ai that understands the physical world it understands things like friction and inertia cause and effect object permanence when some Corner doesn't mean has disappeared from this universe it's still there just not seeable and so that ability to understand the physical world the three-dimensional world is what's going to enable a new era of AI we called physical Ai and it's going to enable robotics each one of these phases each one of these waves opens up New Market opportunities for all of us it brings more and new partners to GTC as a result GTC is now jam-packed the only way to hold more people at GTC is we're going to have to grow San Jose and and we're working on it we got a lot of land to work with we got to grow San Jose so that we can make GTC I just just you know as I'm standing here I wish all of you could see what I see and we're we're in the middle of a stadium um and La last year was the first year back that we did this live and it was it was like a rock concert and it was described GTC is was described as the Woodstock of AI and this year it's described as the Super Bowl of AI the only difference is everybody wins at this Super Bowl everybody's a winner and so every single year more people come because AI is able to solve more interesting problems for more Industries and more companies and this year we're going to talk a lot about agentic Ai and physical AI at its core what enables each wave and each phase of of AI three fundamental matters are involved the first is how do you solve the data problem and the reason why that's important is because AI is a datadriven computer science approach it needs data to learn from it needs digital experience to learn from to gain its dig to learn knowledge and to gain digital experience how do you solve the data problem the second is how do you solve the training problem without human in the loop the reason why human in the loop is fundamentally challenging is because we only have so much time and we would like an AI to be able to learn at super human rates at Super realtime rates and to be able to learn at a scale that no humans can keep up with and so the second question is how do you train the model and the third is how do you scale how do do you create how do you find an algorithm whereby the more resource you provide whatever the resource is the smarter the AI becomes the scaling law well this last year this is where almost the entire world got it wrong the computation requirement the scaling law of AI is more resilient and in fact hyper accelerated the amount of computation we need at this point as a result of agentic AI as a result of reasoning is easily a hundred times more than we thought we need it this time last year and let's reason about why that's true the first part is let's just go from what the AI can do let me work backwards agentic AI as I mentioned at this Foundation is reasoning we now have AIS that can reason which is fundamentally about breaking a problem down step by step maybe it approaches a problem in a few different ways and selects the best answer maybe it solves the problems the same problem in a variety of ways and and sure it has the best the same answer consistency checking or maybe after it's done deriving the answer it plugs it back into the equation maybe a quadratic equation to confirm that in fact that's the right answer instead of just one shot bluring it out remember two years ago when we started working with chat GPT a miracle as it was many complicated questions and many simple questions it simply can't get right and it's understandably so it took a one shot whatever it learned by studying pre-trained data whatever it saw from other experiences pre-trained data it does a one shot blurps it out like a saon now we have AIS that can reason step by step by step using a technology called Chain of Thought best of end consistency checking a variety of different path planning a a variety of different techniques we now have AIS that can reason break a problem down reason step by step by step well you could imagine as a result the number of tokens we generate and the fundamental technology of AI is still the same generate the next token predict the next token it's just that the next token now makes up step one then the next token after that after it generates step one that step one has gone into the input of the AI again as a generate step two and step three and step four so instead of just generating one token or one word after next it generate a sequence of words that represents a step of reasoning the amount of tokens that's generated as a result is substantially higher and I'll show you in a second easily a 100 times more now 100 times more what does that mean well it could generate 100 times more tokens and you can see that happening as I explained previously or the model is more complex it generates 10 times more tokens and in order for us to keep the model responsive interactive so that we don't lose our patients waiting for it to think we now have to compute 10 times faster and so 10 times tokens 10 times faster the amount of computation we have to do is 10 100 times more easily and so you're going to see this in the rest of the presentation the amount of computation we have to do for inference is dramatically higher than it used to be well the question then becomes how do we teach an AI how to do what I just described how to execute this chain of thigh well one method is you have to teach the AI how to reason and as I mentioned earlier in training there are two fundamental problems we have to we have to solve where does the data come from where does the data come from and how do we not have it be limited by human in the loop there's only so much data and so much human demonstration we can perform and so this is the big breakthrough in the last couple years reinforcement learning verifiable results basically reinforcement learning of an AI as it appro as it attacks or tries to engage solving a problem step by step well we have many problems that have been solved in the history of humanity where we know the answer we know the equation of a quadratic equation how to solve that we know how to solve a Pythagorean theorem um the the rules of a right triangle we know many many rules of math and geometry and logic and science we have puzzle games that we could give it constraints constraint constrainted um uh uh type of problems like Sudoko um those kind of problems on and on and on we have hundreds of these problem spaces where we can generate millions of different examples and give the AI hundreds of T hundreds of chances to solve it step by step by step as we use reinforcement learning to reward it as it does a better and better job so as a result you take hundreds of different topics millions of different examples hundreds of different tries each one of the tries generating tens of thousands of Tok tokens you put that all together we're talking about trillions and trillions of tokens in order to train that model and now with reinforcement learning we have the ability to generate an enormous amount of tokens synthetic data generation basically using a robotic approach to teach an AI the combination of these two things has put an enormous enormous challenge of computing in front of the industry and you can see that the industry is responding this is what I'm about to show you is Hopper shipments of the top four csps the the top four csps they're the the ones with the public clouds uh Amazon Azure gcp and oci the top four C top four csps not the AI companies that's not included not all the startups not included not Enterprise not included a whole bunch of things not included just those four just give you a sense of comparing the peak year of Hopper and the first year of Blackwell okay the peak year of Hopper and the first Weir of blackw so you can kind of see that in fact AI is going through an inflection point it has become more useful because it's smarter it can reason it is more used you can tell it's more used because whenever you go to chat GPT these days the the seems like you have to wait longer and longer and longer which is a good thing it says a lot of people are using it with great effect and the amount of computation necessary to train those models and to inflence those models has grown tremendously so in just one year and blackw has just started shipping in just one year you could see the incredible growth in AI infrastructure well that's been reflected in Computing across the board we're now seen and this is the purple is the forecast of uh of an of analysts uh about the the next uh the increase of capital expense of the world's data centers including csps and Enterprise and so on um the world's data centers uh through uh the end through the end of the decade so 2030 um I've said before that I expect data center build out to reach a trillion dollars and I am fairly certain we're going to reach that very soon two Dynamics is happening at the same time the first Dynamic is that the vast majority of that growth is likely to be accelerated meaning we've known for some time that general purpose Computing is run out of course run its course and that we need a new Computing approach and the world is going through a platform shift from hand-coded software running on general purpose computers to machine learning software running on accelerators and gpus this way of doing computation is at this point past this Tipping Point and we are now seeing the inflection point happening the inflection happening in the world's data center build outs so the first thing is a transition in the way we do Computing second is an increase in recognition that the future of software requires capital investment now this is a very big idea whereas in the past we wrote the software and we ran it on computers in the future the computer's going to generate the tokens for the software and so the computer has become a generator of tokens not a retrieval of files from retrieval based Computing to generative based Computing from the old way of doing data centers to a new way of building these infrastructure and I call them AI factories they're AI factories because it has one job and one job only generating these incredible tokens that we then reconstitute into music into words into videos into Research into chemicals or proteins we reconstitute it into all kinds of information of different types so the world is going through a transition in not just the amount of data centers that will be built but also how it's built well everything in the data center will be accelerated not all of its Ai and I want to say a few words about this you know this slide this slide this slide is is uh genuinely my favorite and the reason for that is because for all of you who've been coming to GTC uh all of these years you've been listening to me talk about these libraries uh this whole time this this is in fact what TC is all about this one slide and in fact a long time ago 20 years ago this is the only only slide we had one library after another library after another Library you can't just accelerate software just as we needed an AI framework in order to create AIS and we accelerate the AI Frameworks you need Frameworks for physics and biology and multiphysics and you know all kinds of different quantum physics you need all kinds of libraries and Frameworks we call them Cuda X libraries acceleration Frameworks for each one of these fields of Science and so this first one is incredible this is C cpai numeric uh numpy is the number one most downloaded python Library most used python library in the world downloaded 400 million times this last year uh K litho is computate and coupon numeric is a um a zero change drop in acceleration for numpy so if any of you are using numpy out there give Cai numeric a try you're going to love it a kitho a computational lithography Library over the course of four years we've now taken the entire process of processing lithography computational lithography which is the second Factory in a Fab there's the factory that manufactures the Wafers and then there's the factory that manufactures the information to manufacture the wafers every industry every company that has factories will have two factories in the future the factory for what they build and the factory for the mathematics the factory for the AI Factory for cars Factory for AIS for the cars Factory for smart speakers and factories for AI for the smart speakers and so kitho is our computational orography tsmc Samsung asml our partners synopsis mentor incredible support all over I think that this is now at its Tipping Point in in another 5 years time every mask every single lithography will be processed on Nvidia Cuda Ariel is our library for 5G turning a GPU into a 5G radio why not signal processing is something we do incredibly well once we do that we can layer on top of it ai ai for ran or what we call AI ran the next generation of of uh of uh Radio Radio Networks uh will be will have ai deeply inserted into it why is it that we're limited by the limits of information Theory um because there's only so much information Spectrum we can get not if we add AI to it uh coopt numerical or mathematical optimization almost every single industry uses this when you plan uh seats and uh flights uh inventory and customers um uh workers and plants uh uh drivers and Riders uh so on so forth where we have multiple constraints multiple constraints um a whole bunch of variables and you're optimizing for time uh profit uh quality of service um usage of resource whatever it happens to be Nvidia uses it for our Supply Chain management uh kuop is an incredible Library it takes What It Takes what would take hours and hours and it turns into into seconds the reason why that's a big deal is so that we can now explore much larger space we announced uh that we are going to open source kuop the almost everybody is using either guui uh goobi or um IBM clex uh or FICO uh we're working with all three of them the industry is so excited excited we're about to accelerate The Living Daylights out of the industry uh pair bricks for uh Gene sequencing and Gene uh analysis Moni is the world's leading Medical Imaging Library Earth 2 multif physics for pre for uh predicting in very high resolution uh local weather uh C Quantum and Cuda Q we're going to have our first Quantum day here at GTC we're working with just about everybody in the ecosystem either helping them research on Quantum architectures Quantum algorithms or in building a uh classical accelerated Quantum uh heterogeneous architecture and so really exciting work there uh C equivariance and censor for tensor contraction quantum chemistry of course this stack is world famous people think that there's one piece of software called cuda but in fact on top of Cuda is a whole bunch of liaries that's integrated into all different parts of the ecosystem and software and infrastructure in order to make AI possible uh I've got a new one here to announce today uh cdss uh our Spar solvers really important for CAE this is one of the biggest things that has happened in the last year working with cadence and synopsis and ansis and the so and um and and well all all of the uh the the systems companies we've now made possible just about every important Eda and CAE library to be accelerated what's amazing is until recently Nvidia has been using general purpose computers running software super slowly to design accelerated computers for everybody else and the reason for that is because we never had that software that body of software optimized for Auda until recently and so now our entire industry is going to get supercharged as we move to accelerated Computing uh qdf a data frame for structure data we now have a drop in acceleration for spark and drop in acceleration for pandas incredible and then we have warp a library for physics that runs in py a python library for physics for Cuda we have a big announcement there I'll save in just a second this is just a sampling of the libraries that make possible accelerated Computing it's not just Cuda we're so proud of Cuda but if not for Cuda in the fact that we have such a large installed base none of these libraries would be useful for any of the developers who use them for all the developers that use them you use it because one it's going to give you incredible speed up it's going to give you incredible scale up and two because the install base of Cuda is now everywhere it's in every cloud it's in every data center it's available from every computer company in the world it's every literally everywhere and therefore by using one of these libraries your software your amazing software can reach everyone and so we've now reached the Tipping Point of accelerated Computing Cuda has made it possible and all of you this is what GTC is about the ecosystem all of you made this possible and so we made a little short video for you thank you to the creators the Pioneers the Builders of the future Cuda was made for you since 2006 6 million developers in over 200 countries have used Cuda and transformed Computing with over 900 Cuda X libraries and AI models you're accelerating science reshaping Industries and giving machines the power to see learn and reason now Nvidia Blackwell is 50,000 times faster than the first Cuda GPU these orders of magnitude gains in speed and scale are closing the gap between simulation and realtime digital Twins and for you this is still just the beginning we can't wait to see what you do next I love what we do I love even more what you do with it and one of the things that that most touched me in in my 33 years doing this one scientist said to me Jensen because of the work because of your work I can do my life's work in my lifetime and boy if that doesn't if that doesn't touch you well you got to be a corpse so this is all about you guys thank you all right so we're going to talk about AI but you know AI started in a cloud it started in a cloud for a good reason because it turns out that AI needs infrastructure it's machine learning if the science says machine learning then you need a machine to do the science and so machine learning requires infrastructure and the cloud data centers had infrastructure they also have extraordinary computer science extraordinary research the perfect circumstance for AI to take off in the cloud and the csps but that's not where AI is limited to AI will go everywhere and we're going to talk about AI in a lot of different ways and the cloud service providers of course they they they like our Leading Edge technology they like the fact that we have full stack because accelerated Computing as you know as I was explaining earlier is not about the chip it's not even just the chip in the library the programming model is the chip the programming model and a whole bunch of software that goes on top of it that entire stack is incredibly complex each one of those layers each one of those libraries is essentially like SQL SQL as you know is called in storage Computing it was the big revolution of computation by IBM SQL is one Library just imagine I just showed you a whole bunch of them and in the case of AI there's a whole bunch more so the stack is complicated they also love the fact that csps love that Nvidia Cuda developers are CSP customers because in the final analysis are building infrastructure for the world to use and so the rich developer ecosystem is really valued and really really uh deeply appreciated well now that we're going to take AI out to the rest of the world the rest of the world has different system configurations operating environment differences domain specific Library differences usage differences and so AI as it translates to Enterprise it as it translates to manufacturing as it translates to robotics or self-driving cars or even companies that are starting GPU clouds there's a whole bunch of companies maybe 20 of them who started during the Nvidia time and what they do is just one thing they host gpus they call themselves GPU clouds and one of our one of our great Partners cor weave is in the process of going public and we're super proud of them and so GPU clouds they have their own requirements but one of the areas that I'm super excited about is Edge and today we announced we announced today that Cisco Nvidia T-Mobile the largest telecommunications company in the world cerebrus ODC are going to build a full stack for Radio Networks here in United States and that's going to be the second stack so that this current stack this current stack we're announcing today will put AI into the edge remember a hundred billion dollar of the world's Capital Investments each year is in the Radio Networks and all of the data centers provisioning for communication in the future there is no question in my mind that's going to be accelerated Computing infused with ai ai will do a far far better job adapting the radio signals the massive MOS to the changing environments and the traffic conditions of course it would of course we would use reinforcement learning to do that of course myo is essentially one giant radio robot of course it is and so we will of course provide for those capabilities of course AI could revolutionize Comm Communications you know when I call home you don't have to say but that that few words because my wife knows where I work what that condition's like conversation Carries On from yesterday she kind of remembers what I like don't like and often times just a few words you communicated a whole bunch the reason for that is because of context and human priors prior knowledge well combining those capabilities could revolutionize commun Communications look what it's doing for video processing looks look what I just described earlier in 3D graphics and so of course we're going to do the same for Edge so I'm super excited about the announcement that we made today T-Mobile Cisco Nvidia cus ODC are going to build a full stack [Applause] well AI is going to go into every industry that's just one one of the earliest industries that AI went into was autonomous vehicles The Moment I Saw Alex net and we've been working on computer vision for a long time The Moment I Saw Alex net was such an ex inspiring moment such an exciting moment it caused us to decide to go all in on building self-driving cars so we've been working on self-driving cars now for over a decade we build technology that almost every single self-driving car company uses it could be either in the data center for example Tesla uses Nvidia lots of Nvidia gpus in the data center it could be in the data center or the car wayo and wave uses Nvidia computers in data centers as well as the car it could be just in the car it's very rare but sometimes it's just in the car or they use all of our software in addition we work with the car industry however the car industry would like us to work with them we build all three computers the training computer the simulation computer and the robotics computer the self-driving car computer all the software stack that sits on top of it models and algorithms just as we do with all of the other industries that I've demonstrated and so today I'm super excited to announce that GM has selected Nvidia to partner with them to build their future self-driving car Fleet the time for autonomous vehicles has arrived and we're work looking forward to building with GM AI in all three areas AI for manufacturing so they could revolutionize the way they manufacture AI for Enterprise so they could re uiz the way they work design cars and simulate cars and and then also AI for in the car so AI infrastructure for GM partnering with GM and building with GM their AI so I'm super excited about that one of the areas that I'm deeply proud of and it rarely gets any attention is safety Automotive Safety it's called halos in our company it's called Halos safety requires technology from Silicon to systems to system software the algorithms the methodologies everything from diversity to ensuring diversity monitoring and transparency explainability all of these different philosophies have to be deeply ingrained into every Single part of how you develop the system and the software we're the first company in the world I believe to have every line of code safety assessed 7 million lines of code safety assessed our chip our system our system software and our algorithms are safety assessed by Third parties that crawl through every line of code to ensure that it is designed to ensure diversity transparency and explainability we also have followed over a thousand patents and during this GTC and I really encourage you to do so is to go spend time in the Halos Workshop so that you could see all of the different things that comes together to ensure that cars of the future are going to be safe as well as autonomous and so this is something I'm very proud of it barely it rarely gets any attention and so I I thought I would spend the extra time this time to talk about that okay Nvidia Halos all of you have seen cars drive by themselves um the wayo robo taxis are incredible but we made a video to share with you some of the technology we use to solve the problems of data and training and diversity so that we could use the magic of AI to go create AI let's take a look Nvidia is accelerating AI development for AVS with Omniverse and [Music] Cosmos Cosmos prediction and reasoning capabilities support AI first AV systems that are endtoend trainable with new methods of development model distillation Clos Loop training and synthetic data generation first model distillation adapted as a policy model cosmos's driving knowledge transfers from a slower intelligent teacher to a smaller faster student inferenced in the car the teacher's policy model demonstrates the optimal trajectory followed by the student model learning through iterations until it performs at nearly the same level as the teacher the distillation process bootstraps a policy model but complex scenarios require further tuning Clos Loop training enables fine-tuning of policy models log data is turned into 3D scenes for driving closed loop in physics based simulation using Omniverse neural reconstruction variations of these scenes are created to test the model's trajectory generation [Music] capabilities Cosmos Behavior evaluator can then score the generated driving behavior to measure model performance newly generated scenarios and their evaluation create a large data set for Clos Loop training helping AVS navigate complex scenarios more robustly last 3D synthetic data generation enhances av's adaptability to diverse environments from log data Omniverse builds detailed 4D driving environments by fusing maps and images and generates a digital twin of the real world including segmentation to guide Cosmos by classifying each pixel Cosmos then scales the training data by generating accurate and diverse scenarios closing the Sim to real Gap Omniverse and Cosmos enable AVS to learn adapt and drive intelligently advancing safer Mobility [Music] Nvidia is the perfect company to do that gosh that's our destiny use AI to recreate AI the technology that we showed you there uh is very similar to the technology that you're enjoying um uh to uh take you to a digital twin we call Nvidia all right let's talk about data centers that's not bad huh Gan Splats just in case Gan Splats well let's talk about data centers uh Blackwell is in full production and this is what it looks like it's an incredible incredible you know for for people for us this is a sight of beauty would you agree this is how how is this not beautiful how is this not beautiful well this is a big deal because we made a fundamental transition in computer architecture I just want you to know that in fact I've shown you a version of this uh about 3 years ago it was called uh Grace Hopper and the system was called ranger the ranger system uh is about uh maybe about half of the width of the screen and it was the world's first EnV link 32 3 years ago we showed Ranger working and it was way too large but it was exactly the right idea we were trying to solve scale up distributed computing is about using a whole lot of different computers working together to solve a very large problem but there's no replacement for scaling up before you scale out both are important but you want to scale up first before you scale out well scaling up is incredibly hard there is no simple answer for it you're not going to scale it up you're not going to scale it out like Hadoop take a whole bunch of commodity uh computers hook it up into a large Network and do in storage Computing using Hadoop Hadoop was a revolutionary idea as we know it enabled hyperscale data centers to solve problems of gigantic sizes and uh of using off the-shelf computers however the problem we're trying to solve is so complex that scaling in that way would have simply cost way too much power way too much energy it would have never deep learning would have never happened and so the thing that we had to do was scale up first well this is the way we scaled up I'm not going to lift this this is this is 70 lbs this is the the the last generation system architecture it's hgx this revolutionize Computing As We Know It This revolutionize artificial intelligence this is 8 gpus eight gpus each one of them is kind of like this okay this this is two gpus two Blackwell gpus in one blackwall package two blackwall gpus in one black black blackwall package and um there are eight of these underneath this okay and this connects into what we call MV link 8 this then connects to a CPU shelf like that so there's dual CPUs and that sits on top and we connect it over PCI Express and then many of these get connected with infiniband which turns into uh what is an AI supercomputer this is the way it was in the past this is the way this is how we started well this is as far as we scaled up before we scaled out but we wanted to scale up even further and I I told you that Ranger took this system and scaled it out scaled it up by another factor of four and so we have MV link 32 but the system was way too large and so we had to do something quite remarkable re re-engineer how MV length worked and how scaleup worked and so the first thing that we did was we said listen the mvlink switches are in this system embedded on the motherboard we need we need to disaggregate the MV link system and take it out so this is the mvy link system okay this is an mvy link switch this is the most this is the highest performance switch the world's ever made and this makes it possible for every GPU to talk to every GPU at exactly the same time at full bandwidth okay so this is the MV link switch we disaggregated it we took out and we put it in the center of the chassis so there's all the there 18 of these switches in nine different racks nine different switch trays we call them and then the switches are disaggregated the compute is now sitting in here this is equivalent to these two things in compute what's amazing is this is completely liquid cooled and by liquid cooling it we can compress all of these compute nodes into one rack this is the big change of the entire industry all of you in the audience I know how many of you are here I want to thank thank you for making this fundamental shift from integrated MV link to disaggregated MV link from air cooled to liquid cooled from 60 ,000 components per computer or so to 600,000 components per rack 120 kilow fully loc cooled and as a result we have a one Exel flops computer in one rack isn't it incredible so this is the compute node this is the compute node okay and that now fits in one of these now we 3,000 lb 5,000 cables about 2 miles worth just an incredible Electronics 600,000 Parts I think that's like 20 20 cars 20 cars worth of parts and integrates into one supercomputer well our goal is to do this our goal is to do scale up and this is what it now looks like we essentially wanted to build this chip it's just that no retical limits can do this no processed technology can do this it's 130 trillion transistors 20 trillion of it is used for computing so it's not like you you could you can't reasonably build this anytime soon and so the way to solve this problem is to disaggregate it as I've described into the grace Blackwell MV link 72 rack but as a result we have done the ultimate scale up this is the most extreme scale up the world has ever done the amount of computation that's possible here the memory bandwidth 570 terabytes per second everything is everything in this machine is now in T's everything's a trillion and you have uh an exit flops which is a million trillion floating Point operations per second well the reason why we wanted to do this is to solve an extreme problem and that extreme problem a lot of people misunderstood to be easy and in fact it is the ultimate extreme Computing problem and it's called inference and the reason for that is very simple inference is token Generation by a factory and a factory is revenue and profit generating or lack of and so this Factory has to be built with extreme efficiency with Extreme Performance because everything about this Factory directly affects your quality of service your revenues and your profitability let me show you how to read this chart because we're want to come back to this a few more times basically you have two axes on the x-axis is the tokens per second whenever you chat when you uh put a prompt into chat gbt what comes out is tokens those to tokens are reformulated into words you know it's more than a token per word okay and they'll tokenize things like th could be used for the it could be used for them it could be used for Theory it could be used for theatrics it could be used for all kinds of okay and so th is a Tok an example of a token they reformulate these tokens to turn into words well we've already established that if you want your AI to be smarter you want to generate a whole bunch of tokens those tokens are reasoning tokens consistency checking tokens coming up with a whole bunch of ideas so they can select the best of those ideas tokens and so those tokens might it might be second guessing itself it might be this this the best work you could do and so it ask it it talks to itself just like we talk to ourselves and so the more tokens you generate the smarter your AI but if you take too long to answer question the customer is not going to come back this is no different than web search there is a real limit to how long it can take before it comes back with a smart answer and so you have these two Dimensions that you're fighting against you're trying to generate a whole bunch of tokens but you're trying to do it as quickly as possible Therefore your token rate matters so you want your tokens per second for that one user to be as fast as possible however in Computer Sciences and factories there's a fundamental tension between latency response time and throughput and the reason is very simple if you're in the large high volume business you batch up it's called batching you batch up a lot of customer demand and you manufacture a certain version of it for everybody to consume later however from the moment that they batched up and manufacture whatever they did to the time that you consumed it could take a long time so no different for computer science no different than no to no different for AI factories that are generating tokens and so you have these two fundamental tensions on the one hand you would like the customer's quality of service to be as good as possible smart AIS that are super fast on the other hand you're trying to get your data center to produce tokens for as many people as possible so you can maximize your revenues the perfect answer is to the upper right ideally the shape of that curve is a square that you could generate very fast tokens per person up until the limits of the factory but no Factory can do that and so it's probably some curve and your goal is to maximize the area under the curve okay the product of X and Y and the further you push out more likely it means the better of a factory that you're building well it turns out that in tokens per second for the whole Factory and tokens per second response time one of them requires enormous amount of computation flops and then the other dimension requires an enormous amount of bandwidth and flops and so this is a very difficult problem to solve the the good answer is that you should have lots of flops and lots of bandwidth and lots of memory lots of everything that's the best answer to start which is the reason why this is such a great computer you start with the most flops you can the most memory you can the most bandwidth you can of course the best architecture you can the most Energy Efficiency you can and you have to have a programming model that allows you to run software across all of this insanely hard so that you could do this now let's just take a look at this one demo to give you a tactical feeling of what I'm talking about please play it traditional llms capture foundational knowledge while reasoning models help solve complex problems with thinking tokens here a prompt asks to seek people around a wedding table while adhering to constraints like traditions photogenic angles and feuding family members traditional llm answers quickly with under 500 tokens it makes mistakes in seating the guests while the reasoning model thinks with over 8,000 tokens to come up with the correct answer it takes a pastor to keep the peace okay as as as as all of you know as all of you know if you have a wedding party of 300 and you're trying to find the perfect well the optimal seating for everyone that's a problem that only AI can solve or a mother-in-law can solve and so that's one of those problems that that Coop cannot solve okay so what you see here is that that uh we gave it a problem that requires reasoning and you saw uh R1 goes off often reasons about it tries all these different scenarios and it comes back and it tests his own answer it asks it asks itself whether it did it right meanwhile the last generation language model does a one shot so the one shot is 439 tokens it was fast it was effective but it was wrong so it was 439 waste of tokens on the other hand in order for you to reason about this problem and this is just a that was actually a very simple problem you know we just give it a few more un a few more difficult variables and it becomes very difficult to reason through and it took 8,000 almost 9,000 tokens and it took a lot more computation because the model is more complex okay so that's one dimension before I show you some results let me just show let me explain something else so the answer if you look at if you look at um Blackwell you look at the the Blackwell system and it's now the scaled up MV link 72 the first thing that we have to do do is we have to take this model and this model is not small it's you know in the case of R1 people think R1 is small but it's 680 billion parameters Next Generation models could be trillions of parameters and the way that you solve that problem is you take these trillions and trillions of parameters in this model and you uh distribute the workload across the whole system of gpus you can use uh tensor parallel you can take one layer of the model and and run it across multiple gpus you you could take um uh a slice of the pipeline and call that pipeline parallel and put that on multiple gpus you could take different experts and put it across different gpus we call it expert parallel the the combination of pipeline parallelism and tensor parallelism and expert parallelism the number of combinations is insane and depending on the model depending on the workload depending on the conf the circumstance how you configure that computer has to change so that you can get the maximum throughput out of it you also sometimes optimize for very low Lany sometimes you try to optimize for throughput and so you have to do some inflight batching a lot of different techniques for batching and and aggregating work and so the the software the operating system for these AI factories is insanely complicated well one of the observations and this is this is a really terrific terrific iic thing about having a homogeneous architecture like mvlink 72 is that every single GPU could do all the things that I just described and we observe that these reasoning models are doing a couple of phases of computing one of the phases of computing is thinking when you're thinking you're not producing a lot of tokens you're producing tokens that you're maybe consuming yourself you're thinking maybe you're reading you're digesting information that information could be a PDF that information could be a website you could literally be watching a video ingesting all of that at Super linear rates and you take all of that information and you then formulate the answer formulate a planed answer and so that digestion of information context processing is very flops intensive on the other hand during the next phase is called decode so the first part we call prefill the next phase of decode requires floating Point operations but it requires an enormous amount of bandwidth and it's fairly easy to calculate you know if you have a model and it's a few trillion parameters well it takes a few terabytes per second notice I was mentioning 576 terabytes per second it takes terabytes per second to just pull the mod model in from hbm memory and to generate literally one token and the reason it generates one token is because remember that these large language models are predicting the next token that's why they say the next token it's not predicting every single token it's predicting the next token now we have all kinds of new techniques speculative decoding and all kinds of new techniques for doing that faster but in the final analysis you're predicting the next token okay and so that you ingest pull in the entire model and the context we call it a KV cache and then we produce one token and then we take that one token we put it back into our brain we produce the next token every single one every single time we do that we take trillions of parameters in we produce one token trillions of parameters in produce another token trillions of parameters in produce another token and notice that demo we produced 8,600 tokens so trillions of btes of information trillions of btes of information have been taken into our gpus and produce one token at a time which is fundamentally the reason why you want mvy link MV link gives us the ability to take all of those gpus and turn them into one massive GPU the ultimate scale up and the second thing is that now that everything is on mvy link I can disaggregate the prefill from from the decode and I could decide I want to use more gpus for prefill less for decode because I'm thinking a lot I'm doing it's agentic I'm reading a lot of information I'm doing deep research notice during deep research you know and and earlier I was listening to Michael and Michael was talking about his his him doing research and I do the same thing and we go off and we write these really long research projects for our Ai and I love doing that because you know I already paid for it and I just love making our gpus work and nothing gives me more joy so so so I I write up and then it goes off and it does all this research and it went off to like 94 different websites and it read all this and I'm reading all this information and it formulates an answer and writes the report it's incredible okay during that entire time prefill is super busy and it's not really generating that many tokens on the other hand when you're chatting with the chatbot and millions of us are doing the same thing it is very token generation heavy it's very decode heavy okay and so um depending on the workload we might decide to put more gpus into decode dep depending on the workload put more gpus into prefill well this Dynamic operation is really complicate complicated so I've just now described pipeline pipeline parallel tensor parallel um expert parallel pre uh inflight batching disaggregated inferencing workload management and then I've got to take this thing called a KV cache I got to Route it to the right GPU I've got to manage it through all the memory hierarchies that piece of software is insanely complicated and so today we're announcing the Nvidia Dynamo and Nvidia Dynamo does all that it is essentially the operating system of an AI Factory whereas in the past in the way that we ran data centers our operating system would be something like VMware and we would orchestrate and we still do um you know we're a big user orchestrate a whole bunch of different Enterprise applications running on top of our Enterprise it but in the future the application is not Enterprise it it's agents and the operating system is not something like VMware it's something like Dynamo and this operating system is running on top of not a data center but on top of an AI Factory now we call it Dynamo for a good reason as you know the Dynamo was the first instrument that started the last Industrial Revolution the industrial revolution of energy water comes in electricity comes out it's pretty fantastic you know water comes in you light it on fire turn to steam and it what comes out is this invisible thing that's incredibly valuable it took another 80 years to go to alternate and current but Dynamo Dynamo is the where it all started okay so we decided to call this operating system this piece of software insanely complicated software the Nvidia Dynamo it's open source it's open source and we're so happy that so many of our partners are working with us on it and one of one of my favorite favorite partners I just love them so much because of the Revolutionary work that they do and also because Aran is such a great guy but perplexity as a great partner of ours in working through this okay so anyhow uh really really great okay so now we're going to have to wait until we scale up all these infrastructure but in the meantime we've done a whole bunch of very indepth simulation we have supercomputers doing simulation of our supercomputers which makes sense and and I'm now going to show you the benefit of everything that I've just said and remember the factory diagram on the x-axis on the x-axis is tokens per second throughput excuse me the y- axis tokens per second throughput of the factory and the x- axis tokens per second of the user experience and you want super smart AIS and you want to produce a whole bunch of them this is Hopper okay so this is Hopper and it can produce it can produce for one user about for each user about 100 tokens per second 100 this is eight gpus and it's connected with infiniband and the um um I'm normalizing it to tokens per second per megawatt so it's a one megawatt data center which is not a very large AI Factory but anyhow one megawatt okay and so it can produce for each user 100 tokens per second and it can produce at this at this level whatever that happens to be 100,000 tokens per second for that one megawatt data center or it can produce about 2 and A2 million tokens per second 2 and a half million tokens per second for that AI Factory if it was super batched up and the customer is willing to wait a very long time okay does that make sense all right so nod all right because this is this is where you know every GTC there's there's the price for entry you guys know and it's like you get tortured with math okay this is the only only only at Nvidia do you get tortured with math all right so Hopper you get two and a half now what's that 2 and a half million what's it what's how do you translate that 2 and a half million remember um chbt is like $10 per million tokens right $10 per million tokens let's pretend for a second that that that's I I I think the 10 million $10 per million tokens is probably down here okay I i' probably say it's down here but let me pretend it's up there because 25 million um 10 so $25 million per second does that make sense that's that's how you think through or on the other hand if it's way down here then the question is you know so it's 100,000 100,000 just divide that by 10 okay $250,000 per Factory per second and then as was 31 million 30 million seconds in a year and that translates into revenues for that 1 million that one megawatt Data Center and so that's your goal on the one hand you would like your your token rate to be as fast as possible so that you can make really smart AIS and if you have Smart AIS people pay you more money for it on the other hand the smarter the AI the less you can make in volume very sensible tradeoff and this is the curve we're trying to bend now what I'm just showing you right now is the fastest computer in the world Hopper it's the computer that revolutionized everything and so how do we make that better so the first thing that we do is we come up with Blackwell with MV link 8 same same Blackwell that one same one same compute and that one compute node with MV link 8 using fp8 and so Blackwell is just faster faster bigger more transistors more everything but we like to do more than that and so we introduce a new Precision it's not quite as simple as 4bit floating point but using 4bit floating point we can quantize the model use less energy use less energy to do the same and as a result when you use less energy to do the same you could do more because remember one big idea is that every single data center in the future will be power limited your revenues are power limited you could figure out what your revenues are going to be based on the power you have to work with this is no different than you know like many other Industries and so we are now a power limited industry our revenues will associated with that well based on that you want to make sure you have the most energy efficient compute architecture you can possibly get the next then we scale up with MV link 72 does that make sense look at the difference between that MV link 72 fp4 and then because our architecture is so tightly integrated and now we add Dynamo to it Dynamo can extend that even further are you following me so Dynamo also helps Hopper but Dynamo helps blackwall incredibly now yep only at GTC do you get an Applause for that and and so so now notice what I put those two shiny parts that's kind of where your max Q is you know that's likely where you'll run your factory operations you're trying to find that balance between maximum throughput and maximum quality of AI smartest AI the most of it those two that XY intercept is really what you're optimizing for and that's what it looks like if you look underneath those two squares black well is way way better than Hopper and remember this is not ISO chips this is ISO power this is Ultimate Moors law this is what Moors law was always about in the past and now here we are 25x in one generation as ISO power there's not ISO chips it's not ISO transistors it's not ISO anything ISO power the ultimate the ultimate limiter there's only so much energy we can get into a Data Center and So within ISO power Blackwell is 25 times now here's the that rainbow that's incredible that's the fun part look all the different config every underneath the Paro the frontier par we call it the frontier Paro under under the frontier paredo are millions of points we could have configured the data center to do we could have paralyzed and split the work and sharded the work in a whole lot of different ways and we found the most optimal answer which is the Paro the frontier Paro okay the Paro Frontier and each one of them because of the color shows you it's a different configuration which is the reason why this image says very very clearly you want a programmable architecture that is as homogeneously fungible as fungible as possible because the workload changes so dramatically across the entire Frontier and look we got on the top EXP parallel 8 batch of 3,000 disaggregation off Dynamo off in the middle expert parallel 64 with with uh uh oh the P the 26% of 26% is used for context so so Dynamo is turned on 26% context the other 64% is 74% is not batch of 64 and expert parallel of 64 on one expert parallel four on the other and then down here all the way to the the bottom you got you got tensor parallel 16 with expert parallel 4 batch of two 1% context the configuration of the computer is changing across that entire spectrum and then this is what happen so this is with input sequence length this is a kind of a commodity test case this this is a test case that you can Benchmark relatively easily um the input is 1,000 tokens the output is 2,000 notice earlier we just showed you a demo where the output is very simply 9,000 right 8,000 okay and so obviously this is not representative of just that one chat now this one is more representative and this is what you know the goal is to build these next Generation computers for Next Generation workloads and so here's example of a reasoning model and in a reasoning model Blackwell is 40 times 40 times the performance of Hopper straight up pretty amazing you know I've said before somebody actually asked you know why would I say that but I said before that when Blackwell starts shipping in volume you couldn't give Hoppers away and this is what I mean and this makes sense if anybody if you're still looking to buy a hopper don't be afraid I'm I'm it's okay but I'm the chief re Revenue Destroyer my sales guys are going oh no don't say that there are circumstances where Hopper is fine that's the best thing I could say about Hopper there are circumstances where you're fine not many if I have to take a swing and so that's kind of my point um when the technology is moving this fast uh you you and because the workload is so intense and you're building these things they are factories you we really we really like you to to um uh uh to invest in the right the right versions okay just to put it in perspective this is what a 100 megawatt Factory looks like this 100 megawatt Factory you have based on Hoppers you have 45,000 dieses 1,400 racks and It produced is 300 million tokens per second okay and then this is what it looks like with Blackwell you have 8 yeah I [Applause] know that doesn't make any sense okay so so we're not trying to sell you less okay our sales guys are going Jensen you're selling them less this is is better okay and so so anyways um the more you buy the more you save it's even better than that now the more you buy the more you make you know and so so anyhow uh remember everything is in the context everything now in the context of AI factories and and although we talk about the chips you always start from scale up we talk about the chips but you always start from scale up the full scale up what can you scale up to the to the maximum um I want to show you now what an AI Factory looks like but AI factories are so complicated I just gave you an example of one rack it has 600,000 Parts you know it's 3,000 lb now you've got to take that and connect it with a whole bunch of others and so we are starting to build what we call the digital twin of every data center before you build a data center you got to build a digital twin let's take a look at this this is just incredibly beautiful the world is racing to build state-of-the-art large scale AI factories bringing up an AI gigafactory is an extraordinary feat of engineering requiring tens of thousands of workers from suppliers Architects contractors and Eng Engineers to build ship and assemble nearly 5 billion components and over 200,000 Mi of fiber nearly the distance from the Earth to the Moon the Nvidia Omniverse blueprint for AI Factory digital twins enables us to design and optimize these AI factories long before physical construction Starts Here Nvidia Engineers use the blueprint to plan a 1 gwatt AI Factory integrating 3D and layout data of the latest Nvidia dgx super pods and advanced power and cooling systems from verv and Schneider Electric and optimized topology from Nvidia air a framework for simulating Network logic layout and protocols this work is traditionally done in silos the Omniverse blueprint lets our engineering teams work in parallel and collaboratively letting us explore various configurations to maximizing TCO and power usage Effectiveness Nvidia uses Cadence reality digital twin accelerated by Cuda and Omniverse libraries to simulate air and liquid cooling systems and Schneider Electric with EAP an application to simulate power block efficiency and reliability realtime simulation lets us iterate and run large scale whatif scenarios in seconds versus hours we use the digital twin to communicate instructions to the large body of teams and suppliers reducing execution errors and accelerating time to bring up and when planning for retrofits or upgrades we can easily test and simulate cost and downtime ensuring a futureproof AI [Music] Factory this is the first time anybody who builds data oh that's so beautiful all right I got a race here because I'm turns out I got a lot to tell you and and so if I if I go a little too fast it's not because I don't care about you it's just I got a lot of information to go through all right so so uh first our road map uh we're at we're now in full production of Blackwell uh computer companies all over the world are ramping these incredible machines at scale and uh uh I'm just so so uh pleased and and so grateful that all of you worked hard on uh transitioning into this new architecture and now uh in the second half of this year will uh easily transition into the upgrade so we have the Blackwell Ultra mbink 72 uh you know it's a one and a half times more flabs it's you know it's got a new instruction for attention it's one and a half times more memory all that memory is useful for uh things like KV cache it's you know two times more bandwidth okay for networking bandwidth and so so you're going to now that we have the same architecture we'll just kind of gracefully uh glide into that and uh that's called Blackwell Ultra okay so that's coming second half of this year now there's a reason why we we uh this is the only product announcement in any company where everybody's going yeah next yeah and in fact that's exactly the response I was hoping to get and and here's why look we're building AI factories and AI infrastructure it's going to take years of planning this isn't this isn't like buying a laptop you know this isn't a this isn't discretionary spend this is spend that we have to go plan on and so we have to plan on having of course the land and the power and and we have to get get our our capex ready and we get engineering teams and and we have to lay it out a couple two three years in advance which is the reason why I show you our road map a couple 2 3 years in advance so that you we don't surprise you in May you know hi you know in another month we're going to go to this incredible new system I'll show you an example in a second and so we plann this out in multiple years the next the next click one year out is named after an astronomer and her uh her grandkids are here her name is Vera Ruben she discovered Dark Matter okay it's yep Vera Vera Ruben is incredible because the CPU is new it's twice the performance of Grace had more memory more bandwidth and yet just a little tiny 50 watt CPU is really quite incredible okay and Reuben brand new GPU CX9 brand new networking smart Nick MV link 6 brand new MV link brand new memories hbm for basically everything is brand new except for the chassis and this way we could take a whole lot of risk in One Direction and not risk a whole bunch of other things uh related to the infrastructure and so Vera ruin mvlink 144 is the second half of next year now one one of the things that I made a mistake on and so I just need you to make this pivot we're going to do this one time Blackwell is really two gpus in one Blackwell chip we call that one chip a GPU and that was wrong and the reason for that is it it screws up all the MV link nomenclature and things like that so going forward without going back to Blackwell to fix it going forward when I say MV link 144 it just means that it's connected to 144 gpus and each one of those gpus is a GPU die and it could be assembled in some package how it's assembled could change from time to time okay and so each GPU dies a GPU each MV Link's connected to the to uh to the GPU and so ver Ruben MV link 144 and then this now sets the stage for the second half of the year the following year we call Reuben Ultra okay so ver Ruben Ultra I know this one is where you should you go all right so so this is Vera Ruben Reuben Ultra second half of 27 it's MV link 576 extreme scale up each rack is 600 KW 2 and5 million Parts okay and obviously a whole lot of gpus and uh everything is X factored more so 14 times more uh more flops 15 exop flops instead of one exop flop as you me as I mentioned earlier is now 15 exop flops scaled up exop flops okay and it's 300 what 4.6 pedit so 4,600 terabytes per second scale up bandwidth I don't mean aggregate I mean scale up bandwidth and of course lots a brand new MV link switch and CX9 okay and so notice um 16 sites four gpus and one package extremely large MV link I just put that in perspective this is what it looks like okay now this is this is this is this is going to be fun so this you are just literally ramping up Grace Blackwell at the moment and I I don't mean to make it look like a laptop but here we go okay so this is what Grace Blackwell looks like and this is what Reuben looks like IO ISO Dimension and so this is another way of saying before you scale out you have to scale up does that make sense before you scale up scale out you scale up and then after that you scale scale out with amazing technology that I'll show you in just a second all right so first you scale up and then now that gives you a sense of the pace at which we're moving this is the amount of scale up flops this is scale up flops Hopper is 1x black W is 68 x Reuben is 900x scale up flops and then if I turn it into essentially your TCO which is power on top power per and the underneath is the is the area underneath the curve that I was talking to you about the square underneath the curve which is basically flops times bandwidth okay so the the way you think about a very easy gut feel gut check on whether your AI factories are making progress is Watts divided by those numbers and you can see that Ruben it's going to drive the cost down tremendously okay so that's very quickly nvidia's road map once a year once a year like like clock ticks once a year okay how do we scale up well we introduced we were prep preparing to scale out that will scale up was MV link our scale out network is infin band and Spectrum X most were quite surprised that we came into the ethernet world and the reason why we decided to do ethernet is if we could help ethernet become like infiniband have the qualities of infiniband then the network itself would be a lot easier for everybody to use and manage and so we uh decided to invest in Spectrum we call it Spectrum X and we brought to it the properties of of uh congestion control and and um uh very low latency and uh and amount of software that's part of our Computing Fabric and as a result we made Spectrum X incredibly high performance uh we scaled up the largest single GPU cluster ever as one giant cluster with Spectrum X right and that was Colossus and so there are many other examples of it spectral Max is is unquestionably a huge home run for us one of the areas that I'm very excited about is largest Enterprise networking company to take Spectrum X and integrate it into their product line so that they could help the world's Enterprises become AI companies we're at 100,000 um with cx8 CX7 now cx8 is coming cx-9's coming and during Ruben's time frame we would like to scale out the number of gpus to many hundreds of thousands now the challenge with scaling up gpus to many hundreds of thousands is the connection of the scale out on the connection on scale up is copper we should use copper as far as we can and that's you know call it a meter or two and that's incredibly good connectivity very low very high reliability very good Energy Efficiency very low cost and so we use copper as much as we can on scale up but on scale out where the data centers are now the size of the stadium we're going to need something um much uh longdistance running and this is where silicon photonics comes in the challenge of silicon photonics has been that the transceivers consume a lot of energy to go from electrical to photonic has to go through a CIS go through a transceiver and a ceris a several CIS and so each one of these each one of these each one of these am I am I alone is anybody what happened to my uh my networking guys what can I have this up here yeah yeah let's bring it up so so I can show people what I'm talking about okay so first of all we're announcing nvidia's first co- packaged option silicon photonic system it is the world's first 1.6 terabit per second CPO we're going to it is based on a technology called micro ring resonator modulator and it is completely built with this incredible process technology at tsmc that we've been working with for some time and and we partnered with just a giant ecosystem of Technology providers to invent what I'm about to show you this is really crazy technology crazy crazy technology now the reason why we decided to invest in mrm is so that we could prepare ourselves using mrm's incredible density and power better density and power compared to moander which is used for telecommunications when you when you um uh drive from one data center to another data center uh in telecommunications or even in the transceivers that we use we use Mo Xander because the density requirement is not very high until now and so if you look at look at um these transceivers this is an example of a transceiver they did a very good job tangling this up for me oh well thank [Music] you oh Mother of God okay this is where you got to turn reasoning on it's not as easy as you think these are squirly little things all right so this this one right here this is 30 Watts just so keep you remember this 30 watts and and if you get it on if you buy in high volume it's $1,000 this is a plug on this side on this side is electrical on this side is is Optical okay so Optics come in through the the yellow you plug this into a switch it's electrical on this side there's uh transceivers lasers um uh it's a technology called moander and uh uh incredible and so we use this to go from the GPU to the switch to the next switch and then the next switch down and the next switch down to the GPU for example and so each one of these if we had a 100,000 gpus we would have 100,000 of this side and then another you know 100,000 which connects the the switch to the switch and then on the other side uh tribute that to the other to the other Nick if we had 250,000 we add another l of switches and so each GPU every GPU 250,000 every GPU would have six transceivers every GPU would have six of these plugs and these six plugs would add 180 watts per GPU 180 watts per GPU and $6,000 per GPU okay and so the question is how do we scale up now to millions of gpus because if we had a million gpus multiply by six right it would be a million 6 million transceivers times 30 Watts 180 megaw of transceivers they didn't do any math they just move signals around and and so the question is how do we how could we afford and as I mentioned earlier energy is our most important commodity everything is related ultimate to energy so this is going to limit our revenues the our customer revenues by subtracting out 180 megaw of power and so this is the this is the amazing thing that we did we invented the world's first mrm micro mirror and this is what it looks like there's a little uh wave guide you see that on that wave guide goes to a ring that ring resonates and it controls the amount of reflectivity of the wave guide as it goes around and limits and modulates the uh energy the amount of light that goes through and shuts It Off by absorbing it or pass it on okay turns the light this direct continuous laser beam into ones and zeros and that's the miracle and that technology is then uh the photonic IC is stacked with the electronic IC which is then stacked with a whole bunch of micro lenses which is stacked with this thing called fiber array these things are all manufactured using this technology at tsmc called they call it Coupe and um packag using a 3D Coos technology working with all of these technology providers a whole bunch of them the names I just showed you earlier and it turns it into this incredible machine so let's take a look at the video of it [Applause] [Music] [Music] [Music] [Music] just a technology Marvel and they turn into these switches are infin band switch the Silicon is is working fantastically second half of this year we will ship the the Silicon fonic switch uh in the second half of this year and the second half of next year will ship the Spectrum X because of the mrm choice because of the incredible technology risks that over the last 5 years that we did and filed hundreds of patents and we've licensed it to our partners so that we can all build them now we're in a position to put silic photonics with co- package options no transceivers fiber direct fiber in into our switches with a Radix of 512 this is the this is the 512 ports this would just simply not be possible any other way and so this is this now set our set us up to be able to scale up to these multi 100,000 gpus and multi-million gpus and the benefit just so you you you you imagine this this incredible in a data center we could we could save tens of megawatts tens of megawatts let's say 10 megaw well let's let's say 60 megaw 60 what 6 megaw is 10 Reuben Ultra racks 6 megaw is 10 Reuben Ultra racks right and 60 that's a lot 100 rub Ultra racks of power that we can now deploy into ruin all right so this is our road map once a year once a year an architecture every every uh two years a new product line every single year X factors up and we try to take silicon risk or networking risk or system chassis risk um in in pieces so that we can move the industry forward as we pursue these incredible technology uh Vera Rubin and uh I really appreciate that the uh the grandkids for being here uh this is our opportunity to recognize her and and to honor her for the incredible work that she did our next generation will be named after Fineman okay nvidia's road map let me talk to you about Enterprise Computing this is really important in order for us to bring AI to the world's Enterprise first we have to go to a different part of Nvidia the beauty of gaan Splats okay in order in order for us to take AI to Enterprise take a step back for a second and remind yourself this remember Ai and machine learning has reinvented the entire Computing stack the processor is different the operating system is different the applications on top are different the way the applications are different the way you orchestrate it are different and the way you run them are different let me give you one example um the way you access data will be fundamentally different than the past instead of retrieving precisely the data that you want and you read it to try to understand it in the future we will do what we do with perplexity instead of doing doing retrieval that way I'll just ask perplexity what I want ask it a question and it will tell you the answer this is the way Enterprise it will work in the future as well we'll have ai agents which are part of our digital Workforce there's a billion knowledge workers in the world there probably going to be 10 billion digital workers working with us side by side 100% of software engineers in the future there are 30 million of them around the world 100% of them are going to be AI assisted I'm certain of that 100% of Nvidia software Engineers will be AI assisted by the end of this year and so AI agents will be everywhere how they run what what Enterprises run and how we run it will be fundamentally different and so we need a new line of computers and this this is what a PC should look like 20 pedop flops unbelievable 72 CPU cores chipto chip interface hbm memory and just just in case some PCI Express slots for your GeForce okay so so this uh is called djx station djx spark and djx station are going to be available by all of the OEM HP Dell Lenovo assus uh it's going to be manufactured uh for data scientists and researchers all over the world this is the computer of the age of AI this is what computers should look like and this is what computers will run in the future and we have a whole lineup for Enterprise now from little tiny one to workstation ones the server ones to uh supercomputer ones and these will be available uh by all of our partners we will also revolutionize the rest of the Computing stack remember Computing has three pillars there's Computing you're looking at it there's networking as I mentioned earlier Spectrum X going to the world's Enterprise an AI Network and the third is Storage storage has to be completely reinvented rather than a retrieval based storage system is going to be a semantics based retrieval system a semantics based storage system and so the storage system has to be continuously embedding information in the background taking raw data embedding it into knowledge and then later when you access it you don't retrieve retrieve it you just talk to it you ask it questions you give it problems and one of the one of the examples I wish we had a video of it um but Aaron at box even put one up in the cloud worked with us to put it up in the cloud and it's basically you know a super smart storage system and in the future you're going to have something like that in every single Enterprise that is the Enterprise storage of the future and working with the entire storage industry really fantastic Partners uh DD and Dell and HP Enterprise and Hitachi and IBM and net apppp and neonics and Pure Storage and vast and W basically the entire world storage industry will be offering this this stack for the very first time your storage system will be GPU accelerated and so somebody thought I was I didn't have enough slides and so Michael thought I didn't have enough slides so he he said Jensen just in case you don't have enough slides can I just put this in there and so this is Michael slides but but this is this he sent this to me he goes just in case you don't have any slides and I I got too many slides but this is such a great slide and and let me tell you why in one single slide he's explaining that Dell is going to be offering a whole line of Nvidia Enterprise it AI infrastructure systems and and all this software that runs on top of it okay so you can see that we're in the process of revolutionizing the world's Enterprise we're also announcing today this incredible model that everybody can run and so I showed you earlier R1 a reasoning model I showed you versus llama 3 a non- reasoning model and obviously R1 is much smarter um but we can do it even better than that and we can make it possible to be Enterprise ready for any company and it's now completely open source is part of our system we call Nims and you can download it you can run it anywhere you can run it on djx spark you can run it on dgx station you can run on any of the servers that the the oems make you can run it in the cloud you can integrate into any of your agentic AI Frameworks and we're working with companies all over the world and I'm going to flip through these so watch very carefully I've got some great Partners in the audience want to recognize Accenture Julie SED and her team are building their AI Factory and their AI framework uh AMD dos the world's largest telecommunication software company uh AT&T John Stanky and his team U building an AT&T AI system agentic system Larry think and uh Black Rock team building theirs uh Annie Roode uh in the future not only will we hire ASC designers we're going to hire a whole bunch of digital ASC designers from anude Cadence that will help us design our chips and so Cadence is building their uh AI framework and as you can see in every single one of them there Nvidia models Nvidia Nims and viia libraries integrated throughout so that you can run it on Prem in the cloud any Cloud uh Capital One one of the most advanced financial services companies and using technology has Nvidia all over it uh deoe Jason and his team uh enany Janet and his team NASDAQ and Adena and her team uh integrating Nvidia technology into their AI Frameworks and then Christian and his team at sap bill mcder and his team at service now that was pretty good huh first this is one of those Keynotes where the first slide took 30 minutes and then all the other slide took 30 minutes all right so so next let's go somewhere else let's go talk about robotics shall [Music] we let's talk about robots well well the time has come the time have has come for robots uh robots have the benefit the benefit of being able to interact with the physical world and do things that otherwise digital information cannot uh we know very clearly that the world is has severe shortage of of human labors human workers by the end of this decade the world is going to be at least 50 million workers short we'd be more than delighted to pay them each $50,000 to come to work work probably going to have to pay robots $50,000 a year to come to work and so this is going to be a very very large industry there are all kinds of robotic systems your infrastructure would be robotic billions of cameras and warehouses and factories 10 20 million factories around the world every car is already a robot as I mentioned earlier and then now we're building General robots let me show you how we're doing [Music] that everything that moves will be autonomous physical AI will embody robots of every kind in every industry three computers built by Nvidia enable a continuous loop of robot AI simulation training testing and Real World Experience training robots requires huge volumes of data Internet scale data provides common sense and reasoning but robots need action and Control Data which is expensive to capture with blueprints built on Nvidia Omniverse and Cosmos developers can generate massive amounts of diverse synthetic data for training robot policies first in Omniverse developers aggregate real world sensor or demonstration data according to their different domains robots and tasks then use Omniverse to condition Cosmos multiplying the original captures into large volumes of photoreal diverse data developers use Isaac lab to post-train the robot policies with the augmented data set and let the robots learn new skills by cloning behaviors through imitation learning or through trial and error with reinforcement learning AI feedback practicing in a lab is different than the real world new policies need to be field tested developers use Omniverse for software and Hardware in the loop testing simulating the policies in a digital twin with real world environmental Dynamics with domain randomization physics feedback and High Fidelity sensor simulation real world operations require multiple robots to work together Mega an Omniverse blueprint lets developers test fleets of post-train policies at scale here foxc contests heterogeneous robots in a virtual Nvidia Blackwell production facility as the robot brains execute their missions they perceive the results of their actions through sensor simulation then plan their next action Mega lets developers test many robot policies enabling the robots to work as a system whether for spatial reasoning navigation Mobility or dexterity amazing things are born in simulation today we're introducing Nvidia Isaac Groot N1 Groot N1 is a generalist Foundation model for humanoid robots it's built on the foundations of synthetic data generation and learning in simulation Groot N1 features a dual system architecture for things fast and slow inspired by principles of human cognitive processing the slow thinking system lets the robot perceive and reason about its environment and instructions and plan the right actions to take the fast thinking system translates the plan into precise and continuous robot actions Groot n1's generalization lets robots manipulate common objects with ease and execute multi-step sequences collaboratively and with this entire pipeline of synthetic data generation and robot learning humanoid robot developers can post-train Gro N1 across multiple embodiments and tasks across many environments around the world in every industry developers are using nvidia's 3 computers to build the next generation of embodied AI [Music] physical Ai and Robotics are moving so fast everybody pay attention to this space this could very well likely be the largest industry of all at its core we have the same challenges as I mentioned before there are three that we focus on they are rather systematic one how do you solve the data problem how where do you create the data necessary to train the AI two what's the model architecture and then three what's the scaling loss how can we scale either the data the compute or both so that we can make AIS smarter and smarter and smarter how do we scale and those two those fundamental problems exist in robotics as well in robotics we created a system called Omniverse it's our operating system physical AIS you've heard me talk about Omniverse for a long time we added two technologies to it today I'm going to show you two things one of them is so that we could scale AI with generative capabilities and generative model that understand the physical world we call it Cosmos using Omniverse to condition Cosmos and using Cosmos to generate an infinite number of environments allows us to create data that is grounded grounded controlled by us and yet be systematically infinite at the same time okay so you see Omniverse we use candy colors to give you an example of us controlling the robot in the scenario perfectly and yet o Cosmos can create all these virtual environments the second thing just as we were talking about earlier one of the incredible scaling capabilities of language models today is reinforcement learning verifiable rewards the question is what's the verifiable rewards in robotics and as we know very well is the laws of physics verifiable physics Rewards and so we need an incredible physics engine well most physics engines have been designed for a variety of reasons they could be designed because we wanted to use it for large machineries or uh maybe we design it for uh virtual worlds video games and such but we need a physics engine that is designed for very fine grain rigid and soft bodies designed for being able to train tactile feedback and fine motor skills and actuator controls we needed to be GPU accelerated so that we these Virtual Worlds could live in super linear time super real time and train these AI models incredibly fast and we needed to be integrated harmoniously into a framework that is used by roboticist all over the world Moko and so today we're announcing something really really special it is a partnership of three companies Deep Mind Disney research and Nvidia and we call it Newton let's let's take a look at Newton [Music] [Music] [Applause] thank you all right let's start that over shall we let's not ruin it for them hang on a second somebody talk to me I need feedback what happened who I just need a human to talk to come on that's a good joke give me a human to talk to Janine I know it's not your fault but talk to me we got we just got a two minutes left I'm right here they're reing it they're rcking it I don't even know what that means okay [Music] w [Music] [Applause] [Music] tell me that wasn't amazing hey blue how are you doing how do you like how do you like your new physics engine you like it huh yeah I bet I know tacticle feedback rigid body soft body simulation super real time can you imagine just now what you were looking at as complete real time simulation this is how we're going to train robots in a future uh just so you know blue has uh two computers two Nvidia computers inside look how smart you are yes you're smart okay all right hey blue listen how about let's take them home let's finish this keynote it's lunchtime are you ready ready let's finish it up we have another announcement to you're good you're good just stand right here stand right here stand right here all right good right there that's good all right Stan okay we have another amazing news I told you the progress of our robotics has been making enormous progress and today we're announcing that Groot N1 is open sourced I want to thank all of you to come let's wrap up I want to thank all of you for coming to GT PC we talked about several things one Blackwell is in full production and the ramp is incredible customer demand is incredible and for good reason because there's an inflection point in AI the amount of computation we have to do in AI is so much greater as a result of reasoning Ai and the training of reasoning AI systems and agent agentic Systems Second Blackwell MV link 72 with Dynamo is 40 times the performance AI Factory performance of Hopper and inference is going to be one of the most important workloads in the next decade as we scale out AI third we have an annual annual rhythm of road maps that has been laid out for you so that you could plan your AI infrastructure and then we have two we have three AI infrastructures we're building AI infrastructure for the cloud AI infrastructure for Enterprise and AI infrastructure for robots we have one more treat for you play it [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] thank you everybody thank you for all the partners that made this video possible thank you everybody that made this video possible have a great GTC thank you hey blue let's go home good job good little [Music] man thank you I love you too thank you [Music]