[Music] hi everybody welcome to in good company today really exciting we have Dario amod the CEO and co-founder of entropic visiting now Dario he is a superstar in the AI world and together with his team has developed the clae language model one of the best out there and they are backed by Amazon and Google now you are a leading figure Dario on AI safety and ethics and you even interrupted your holiday to come here and uh and to talk to us so big thanks for coming thank you for having me now what are the latest breakthrough in AI yes um so a few things I could talk about um one is you know I think the scaling trends of AI are continuing so I think we're going to see over the over the next year you know much bigger and more powerful models that are able to do greater tasks in fact by the time this podcast airs a new model will be out from anthropic that will probably be the most intelligent and Powerful model in the world um but one area I'm particularly excited about that we're developing being in parallel with that is interpretability of models the ability to see inside our AI models and see why they make the decisions they make that area has been mainly a research area for the last few years and it's just at the beginning of starting to have practical applications so that's one area I'm very excited about why is that so important if you look at what what AI models do today often you won't understand why an AI model does what it does um you know I was just uh I was just talking to someone uh at at lunch uh let's say you want uh you know consider your industry let's say you want an AI model to be trained on some data to be able to predict you know what what happened did uh you know what you know uh in with particular set of financial data one problem you have with training a model to work on that is that if you train it on data in the past the model might have memorized it because it was trained on uh you know it it basically knows what happens it it knows the future in that case interpretability might allow you to tell difference is the model deducing the answer to the question or is it memorizing the answer to the question similarly if a model acts in a way that say shows prejudice against a particular group or or appears to do so can we look at the reasoning of the model you know is it really being driven by Prejudice there are also a number of legal requirements right in the EU you know there's a right to explanation and so interpretability being able to see inside the model could help us to understand why the model do and say the things that they do and say and even to intervene in them and change what they do and say so so a while back you stated that we still don't know how the advanced AI models work does this mean that this will solve this problem you know I wouldn't say solve I would say we're at the beginning maybe we now like understand 3% of how they work really um uh you know we're we're at the level where we can look inside the model and we can find features inside it that correspond to very complex Concepts like one feature might represent the concept of hene or hesitating the the a particular genre of music a particular type of metaphorical situation that a character could be in or the idea of you know again Prejudice for for against various groups um so we have all of these features but we think we've only found a small fraction of what there is and what we still don't understand is we don't understand how all of these things interact to give us the behaviors we see models every day so you know it's a little like the brain right we can do brain scans we can say a little about the human brain but you know we we don't we we don't have a spec sheet for it we can't go and say well this is this is why that that person did exactly what they did so will we ever understand fly how they work I don't know about you know fully down to the Last Detail but I think progress is happening fast and I'm I'm optimistic about getting but is pro is progress happening faster than than complexity of the new models that is a great question and that is the thing contending with so we are putting a lot of resources behind interpretability of language models to try and keep Pace with the rate at which the complexity of the models is increasing I think this is one of the biggest challenges in the field the field is moving so fast including by our own efforts that we want to make sure that our understanding keeps Pace with our our abilities our capabilities to produce powerful models what's so good about your model um so this is a CLA model right this is CL models yeah so to give some context we recently released um a set of Claude 3 models um uh they're called Opus Sonet and Hau um they're different tradeoffs between power and intelligence and uh uh you know speed and low cost while still being intelligent at the time that Opus was released it was actually the best allaround model in the world but I think one thing that particularly made it good is that we put a lot of engineering into its character and we recently put out a post about know how do we design claude's character people have generally found the Claude models are warmer more human they enjoy interacting with them more some of the other models sound more uh robotic more uninspired um we're continuing to innovate quickly and as I said by the time this podcast comes out we'll probably have at least part of a new generation of models out tell me about the new one um uh so uh I can't say I can't say too much about it but uh if you had to say a bit um but if I if I had to say a bit you know I would say that uh we're pushing the frontier right now there's a trade-off between speed of model speed and um uh low cost of models and quality so you can imagine that as a trade-off curve right a frontier there's going to be a new generation of models that pushes that Frontier outward and so you know you're going to see by by the time this podcast is out you know we'll have we'll have we'll have a name for it um uh for at least at least some of those models and we'll see that things that you needed the most powerful model model to be a to be able to do you'll be able to do with some of the some of the mid-tier or low tier models that are faster cheaper and even more capable than the than the past than the the past generation the previous model so Dario what's going to be the wow factor here so when I get these model you know what is what what is it going to do to me yeah um you know you're going to see uh models that are better at say things like code math uh better at reasoning one of my favorite is uh you know uh biology and medicine that's one of the sets of applications I'm most excited about for for for for the new models um so uh you know the models we have today they're kind of like early undergrads in their knowledge of many things or like interns and I think we're starting to push that boundary towards Advanced undergrads or even graduate level knowledge and so when we think of use of models for drug development or you know in your own industry use of the models for thinking about investing or or even trading um uh I think the models are just going to get a good deal more sophisticated at those tasks um and you know we're hoping that every few months we can release a new model that pushes those boundaries further and further now one of the things which is accelerated lately is just how we um how we kind of weave AI into everything we do yes and uh you know with recently the announcement from from Apple uh and open AI um what is how do do you look at this you know anthropic thinks of itself uh you know more as providing services to Enterprises than it does on the consumer side um and so we're thinking a lot about how to integrate AI in work settings um so if you think about you know today's models today's chatbots you know it's it's a bit like if I use them in an Enterprise setting it's like if I took some you know random person on the street but who was very smart but who knew nothing about your company and I brought them in and I asked them for advice what I really like is someone that acts more like an AI model that acts more like uh someone that's been you know trained with knowledge of your company for many years and so we're working on connecting our AI models to knowledge databases having them site work having them be able to use internal Enterprise tools and really integrate with the Enterprise as you know sort of a virtual assistant to an employee uh so that's that's one way I think about you know really driving the integration if if you look at the long-term goal of entropic what what is the long-term goal yeah you know if I think about our longterm go you're only like three years old right yeah we're we're only three and a half years old yeah we're by far the you know the newest player in the space that's been able to build models on the frontier you know we're a public benefit Corporation and I think our long-term goal is to make sure all of this goes well um and you know that's being done obviously through the vehicle of a company but if you think about our long-term strategy what we're really trying to do is create what we call a race to the top so you know race to the bottom is this well-known thing where everyone you know fights to cut Corners because the market competition is so intense we think that there's a way to have the reverse effect which is that if you're able to produce higher standards innovate in ways that make the technology more ethical um then others will follow suit they'll either be inspired by it or they'll be kind of you know bullied into it by their own employees or public sentiment or ultimately the law will go in that direction and so we're hoping to kind of provide an example of how to do AI right and pull the rest of the industry along along with us um that's a lot of the work behind our interpretability work behind our safety work behind how we think about responsible scaling we have something called a responsible scaling policy so I think our our overall goal is is kind of to try and try and help the whole industry be better so you kind of pitch yourself as a good guys I mean you know I wouldn't I wouldn't say I wouldn't say anything that grandiose right I it's more like I want to you know I think more in terms of incentives and structures more than I think of good guys and bad guys I want to help change the incentives so that everyone can be the good guys do you think we will care who which model we interact with or are we going to have just like one agent who picks the model which is the best for you know for that purpose that that was kind of what Bill gay said when he was on the on the podcast I you know I think I think it really depends on the setting um a few points on this one is I think um we are increasingly going in the direction where models are good at different things so for example I was just talking about claude's character right Claude is more warm and friendly to interact with for a lot of applications and use cases that's very desirable for other applications in use cases a model which focuses on different things might be helpful some people are going in the direction of Agents some people are going in the direction of models that are good as code Claude for example is another thing it's good at is creative writing and so I think we're going to have an ecosystem where people use different models for different purposes now in practice does that mean you're kind of there's something that's choosing models for you I think in some consumer context that will be the case I think in other context someone will say oh yeah no you know the the job I'm doing or the kind of person that I am I want to use this particular model all the time but what makes a warm model is I mean how can you make a model friendly yeah is he more humoristic or more polite or just like putting some red hearts in between or and we actually try and avoid too many emojis because it gets annoying but um I don't know if you go on Twitter and see some of the comments when people when people interact with Claude um it you know it it just kind of I don't know how to describe it it just kind of sounds more like a human right I think a lot of these Bots have certain ticks right like you know models will you know there are certain phras es I apologize but as an AI language model I can't do X Y and Z right that's kind of like a common phrase and we've helped the model to to vary their thinking more to sound more like a human that that kind of thing when you launched new models you got a pretty pretty good uh predictions on how accurate it will be right it's a function of number of parameters and so on now to get to AGI what how far out are we there that this is the general intelligence so are more more intelligent than this said this a few times but you know back in 10 years ago when all of this was kind of Science Fiction I used to talk about AGI a lot I I now have a different perspective where I don't think of it as one point in time I just think we're on this smooth exponential the models are getting better and better over time um there's no one point where it's like oh the models weren't generally intelligent and now and now they are I just think you know like like a human Child Learning and developing they're getting better and better smarter and smarter more and more knowledgeable and I don't think there will be any single point of note but I think there's a phenomenon happening where over time these models are getting better better and better than even the best humans um I do think that if we continue to increase the scale the amount of funding for the models if it goes to say 10 billion so now a model would cost what 100 million uh right now 100 million there are models in training today that are more like a billion um I think if we go to 10 or 100 billion and I think that will happen in 2025 2026 maybe 2027 um and the algorithmic improvements continue a pace and the chip improvements continue a pace then I think there there is in my mind a good chance that by that time we'll be able to get models that are better than most humans at most things so 10 billion you think a model will be next year I think that the training of aor 10 billion doll model yeah could start sometime in 2025 not many people can participate in that race no no no no and you know I you know of course I think there's going to be a vibrant Downstream ecosystem and there's going to be an ecosystem for small models you don't have that much money uh I mean we have of order of order that um we've raised uh I believe it's a little over8 billion to date right um so generally generally of ordered that and of you know of course we're we're you know we're we're we're always uh we're always interested in getting to the next level of scale now this is of course also a function of of the chips and we just learned that Nvidia is har the time now between launches right so in the past every other year now is more like every year so what what are the implications of this yeah um you know I think that is a you know I can't speak for NVIDIA but I think that is a natural consequence of the recognition that uh chips are going to be super important right and also facing competition um Google is building their own own you know chips as we know Amazon is building their own chips uh you know anthropic is collaborating with with both to uh to uh you know to work with those chips and you know without getting specific what I can say is that the you know the chip industry is getting very competitive and there are some very strong offerings how far behind with Google and Amazon being in the chip development I you know that's not something I could say and it's not but just some kind of indication again you know I would just I would just repeat that I think there are now strong offerings from from multiple players that have been useful to us and will be useful to us in different ways okay so it's not only about Nvidia anymore is I I don't think it's only about Nvidia anymore um but you know of course you look at their stock Val you look at their stock price which you know you certainly certainly aware of and you know it's it's it's an indicator I think both about them and the industry you mentioned that you were more on the Enterprise side and not necessarily on the consumer side but um just lately there has been more talk about uh having chips in in phones and we talk about uh AI PC and so on how do you look at this yeah no I think that's going to be an important development and again if we go back to the curve I talked about right the trade-off curve between powerful smart but you know relatively expensive in slow models and models that are super cheap super fast but very smart for how fast fast and cheap they are as that curves shifts outward we are going to have models that are very fast and cheap that are smarter than the best models of today even though the best models then will be even smarter than that um and and I think uh we'll be able to put those models on phones and on mobile chips and uh you know they'll they'll pass some threshold where the things that you need to call to a cloud or the server for today you can do there and so I'm I'm very excited about the implications of that um you know I'm of course even more excited about pushing the frontier of where things will go but this curve shifts outward an implication is that both things will happen we he from mrr the French uh competitor that they have developed some really efficient uh kind of low costs or lower cost models what what do you how do you view that I can't comment on uh you know what's going on at other companies but I think we are seeing this this kind of General moving of the curve and so it is definitely true we're seeing efficient lowcost models but I think of it less as like things are leveling out you know costs are going down and more as the curve is Shifting right we can do more with less but we can also do even more with more resources yeah so I think both both Trends coexist dar changing tab bit here uh your background you kicked off in physics yes I was an undergrad in physics and then uh uh did grad school in Neuroscience yeah so how come you ended up in AI yeah so you know when I finished my Physics degree I you know I wanted to do something that you know would would have an impact on you know Humanity in the world um and I felt that you know an important component of that would be understanding intelligence you know that that that's one of the things that's obviously shaped our world and that was back in the mid 2000s and in those days I wasn't particularly to be honest that excited about the AI of the day uh and so I felt like the best way to study intelligence in those days was to study the human brain so I went into Neuroscience for grad school computational Neuroscience that Ed some of my physics background and you know studied kind of collective properties of neurons but by the end of that by you know by the end of grad school after that I did a short post stock um AI was really starting to work we really saw you know the Deep learning Revolution I saw the work of you know ilos suit gavver back back then uh and so I decided based on that to go into the AI field uh and I worked you know different places I was at Buu for a bit I was at Google for a year I worked at open AI for five years and and and you were instrumentally in developing Chad CH 2 and three right yes yes I led the development of both of those why did you leave you know we had reached around the end of 2020 um we had kind of reached a point the set of us who worked in this these on these projects in these areas where we kind of had our own vision for how to do things uh so uh you know again you know we had this picture that I I think I've already kind of implicitly laid out of one you know real belief in this scaling hypothesis and two in the importance of safety and interpretability so it was a safety side which made you leave I think you know we just we just had our own vision of things there were a set of us who were co-founders who really felt like we were on the same page really felt like we trusted each other really felt like you know we just wanted to do something together but you were a bit more AI doomsday before than you are now I you know I wouldn't say that my view has always been that uh there there are important risks and there are benefits and that is the technology goes on its exponential the risks become greater and the benefits become greater um and so we are you know including that anthropic very interested in these questions of catastrophic risk right we have this thing called responsible scaling policy and that's basically about measuring models at each step for catastrophic risk what what is catastrophic risk so this would be um I would put it in two categories um one is misus of the models which could include things in the realm of biology or cyber or kind of um you know election operations at scale um things that are really disruptive to society um to that misuse would be one bucket and then the other bucket would be autonomous unintended behavior of the model so you know today it might be just you know the model doing something unexpected but increasingly as models act in the world we have to worry about them behaving in ways that you wouldn't expect and what was it that you saw exactly with chip um well three I guess then which made you particularly concerned about this yeah it wasn't about any particular model um you know if we go all the way back to 2016 you know before I even worked at openai when I was at Google I wrote a paper called with some colleagues some of whom are now anthropic co-founders concrete problems in AI safety um and concrete problems in AI safety laid out this concern that you know we have these powerful AI models neural Nets but they're fundamentally statistical systems and so that's going to create all these problems about predictability and uncertainty and if you combine that with the scaling hypothesis and I really came to believe in the scaling hypothesis as I worked on gpt2 and gpt3 those two things together told me okay we're going to have something powerful and it's not going to be trivial to control it m and so we put those two things together and and that makes me think oh this is an important problem that we have to solve how do you solve um the two catastrophic risk problems yes in a Tropic one of the biggest tools for this is our RSP our responsible scaling policy and so the way that works is every time we have a new model that you you know that that represents a significant leap a certain amount of compute above an old model we measure it for both the misuse risks and the autonomous self-replication risks and how do you do that so we have a set of evaluations that we run um we've in fact worked with for the misuse risks folks in the National Security community so for example we've worked with this company called Griffin biosciences that um contracts with the US government that does biocurity work and they're they're the experts on responding to biological risk and so they say what is the stuff that's not on the internet that if the model knew it would be concerning and they run their test and you know give Maxis the new model they run their tests and every time so far they've said well it's better at the task than it was before but you know it's not yet at the level where it's a serious concern so a misuse test would be if for instance if I put in hey can you come up with a virus which is just going to wipe out the Earth the people is that that's that's an example right I conceptually yes although it's less about answering one question it's more about can the model go through a whole workflow like could some could some bad actor over the period of weeks use this model to to do as they were doing something nefarious in the real world could the model give them hints on how to help them could the model help them through the task over a long period of time okay so what you were saying is that uh the AI models so far cannot uh do this they know individual isolated things which are concerning right um and they get better at it every time we release a new model but they haven't reached this point yet about the other one the autonomous yeah so how far are we away from that we we test the models there for things like ability to train their own models ability to provision Cloud compute accounts and you know take actions on those accounts ability to like you know simultane you know sign up for accounts and engage in financial transactions just some of the measures of things that would kind of unbind the model and enable them to take actions how far are we away from that do you think um I think it's kind of the same story as with misuse they're getting better and better at individual pieces of the task there's a clear Trend towards ability to do that but we're not there yet um I I again point to the 2025 2026 maybe 2027 window just as I think a lot of the the extreme positive economic applications of AI are are going to arrive sometime around then um I think some of the negative concerns may may start to arise then as well but you know I I'm not a crystal B 25 26 around that you know I mean what do you do then do you build in like a kill switch or what do you yeah well I mean there's a number of things um I think on the autonomous Behavior a lot of our work on interpretability a lot of our work on um you know we haven't discussed constitutional AI but that's another way we provide kind of values and principles for the AI system on the autonomous risk what we really want to do is understand what's going on inside the model and make sure that we design it and can iterate on it so that it doesn't do these dangerous things we don't want it to do um on misuse risk again it's it's there it's more about putting safeguards into the model so that people can't ask it to do dangerous things and we can monitor when people try to use it to do dangerous things so generally speaking I mean there's been a lot of talk about this but how can one regulate AI can companies self-regulate you know one way I think about it is uh the RSP ke the responsible scaling policy that I was describing is maybe the beginning of a process right that represents voluntary self-regulation and you know I mentioned this concept of race to the top last September we put in place our RSP since then other companies like Google open AI have put in place similar Frameworks they've given them different names but they operate in roughly the same way and now we've heard you know Amazon Microsoft even meta reportedly public reporting are are at least considering similar Frameworks and so I would like it if that process continues right where we have some time for companies to experiment with different ways of voluntarily self-regulating some kind of consensus emerges from some mixture of public pressure experimentation with what what is unnecessary versus what is really needed um and then I I I I would imagine the real way for things to go is once there's some consensus once there's industry best practices probably the role for legislation is to is to look in and say hey there's this thing that 80% of the companies are already doing that's a consensus for how to make it safe the job of legislation is just to enforce Force those 20% who aren't doing it force that companies are telling the truth about what they're doing I don't think regulation is good at coming up with a bunch of new Concepts that people should follow so so how do you view the EU AI act the eui act I should say first of all and and the California safety bill as well right yeah yeah you know I I should say I should say the eui Act you know kind of it's still being it's still being um you know the the kind of details of it are still be even though the Act was passed the you know many details are still being worked out so you know I think a lot of this dep a lot of this depends on the details um you know the the the the the the California bill you know I would say that um it is very you know it has some structures in it that are very much like kind of the RSP you know I think something that resembles that structure at at some point could be a good thing if I have a concern though I think it's that we're very early in the process right I describ this process that's like you know first first one company has an RSP then many have rsps then these these kind of industry consensus comes into place my only question would be are we are we too early in that process to in regulation maybe regulation should be the last step of a series of steps yeah what's what's the danger of regulating too early I don't know one one um one thing I could say is that I'll look at our own experience with rsps um so if I look at what we've done with RSP you know we rode an RSP in September um and you know since then we've deployed one model we're soon going to deploy another you you see so many things that not that it was too strict or not strict enough but you just didn't anticipate them in the RSP right like you know there are various kinds of like AB tests you can run on your models that are even informative about safety and our RSP didn't speak one way or another about like when those are okay and when there's not and so we're updating our RSP to say hey how should we handle this issue we've never we've never even thought of and and so I think in the early days that flexibility is easy if you don't have that flexibility if your RSP was written by a third party um and you didn't have the ability to change it in the process for changing it was very complicated I think it it could create a version of the RSP that doesn't protect against the risks but also is very honorous and then people could say oh man all this regulation stuff all this catastrophic it's all nonsense it's all a pain so I I'm not I'm not against it you just have to do it delicately and in the right order but we build AI into you know into the race between the superpowers right we building into the weapons the cars the medical research into everything how can you how can you regulate when it's part of the the Power Balance in the world yeah yeah so I think there's different there's different questions right one question is uh you know how do you regulate the use domestically and you know there I think there's there's a history of it right um you know I think an analogy I would make is like you know I don't know the way cars and airplanes are regulated right I think that's been a reasonable story I don't know that much about Europe but like in the US I think that's been a reasonable story everyone understands there's huge economic value everyone understands that these things are dangerous and they can kill people and you know everyone understands yes you have to do this kind of basic safety testing um and you know that's that's evolved over years I think that's generally um you know gone reasonably well it hasn't been perfect um so so I think for domestic regulation you know that's that's what we should aim to things are moving fast but you know we should try to go through all the steps to get there from an international point of view I mean I think that's a completely different question that's less about regulation and more about there's an International race to the to the bottom and how do you how do you handle how do you handle that race to the bottom I mean I think it's an inherently difficult question because on one hand we don't want to you know you know we don't want to just recklessly build as fast as we can particularly on the weapon side you know on on the other side I think looking you know as as a citizen of the US here I here I am in Norway another democracy um I'm I'm very worried about if autocratic regimes were to lead lead in this technology I think that's very very dangerous how far behind are they now or are they behind it's hard to say I would say that with some of the restrictions that have put been put in place uh on for example you know shipment of chips and equipment to Russia and China I think if the US government plays its CS right then uh you know those countries could be kept behind I don't know maybe two or three years right um that doesn't give us much margin talking about democracies will AI impact the US election yes I am concerned about that you know anthropic actually just put out a post about what we're doing to you know to counter election interference how could it interfere if we look back at say the 2016 election something that happened in that election was that there were large numbers of people who were being create you know who were being uh paid to provide content I don't know how effective that was in the end it's very hard to measure um but a lot of the things that you know you had you had you know Farms of of people being paid to do could could now be done by AI I think it's less that like you know you could make content that people necessarily believe it's it's more that you could kind of flood the information ecosystem with a bunch of very low quality content that would make it hard for people to believe things that really are true did that happen for instance in India in the European election I mean is it is it really happening this year we don't have particular evidence of the use of our models we've banned their use for lection earing and we monitor use of the models um you know occasionally we shut things down but I don't think we've ever seen a super large scale operation I can only speak for use of our models but I don't I don't think we've ever seen a super large scale operation there changing um topic slightly you mentioned that you thought we were going to see some extreme positive effects of AI in 25 26 what are these extremely positive things yeah so again if we go back to the analogy of like today's models are like undergraduates um if we get to the point where the models are I suspect you were better underground than me though I can kind of feel it couldn't speak to it but um uh but if uh you know let's say those models get to the point where you know they're kind of you know graduate level or strong professional level think of biology and Drug Discovery think of um a model that is as strong as you know a Nobel prizewinning scientist or you know the head of the you know the head of head of drug Discovery at a major pharmaceutical company I look at all the things that have been invented you know if I look back at biology you know crisper the ability to like edit genes if I look at um you know C therapies which have have cured certain kinds of cancers there's probably dozens of discoveries like that lying around and if we had a million copies of an AI system that are as knowledgeable and as creative about the field as all those scientists that invented those things then I think the rate of of those discoveries could really proliferate and you know some of our really really longstanding diseases uh you know could be could be addressed or even cured now I don't think all of will come to fruition in 2025 2026 at most I think that the caliber of AI That's that's capable of starting the process of addressing all those things could be ready then it's another question of like applying it all putting it through the regulatory system sure but what could you do to productivity in society uh you know I think of again virtual assistance like uh you know uh like a chief of staff for everyone right I have a chief of staff but not everyone has a Chief of Staff uh you know could could everyone have a chief of staff who helps them uh you know just deal with every deal with everything that lands on their desk if everybody had that kind of thing what would you do could you put a number on productivity gain uh you know I'm not an economist I couldn't tell you you know X I couldn't tell you x%c um but if we look at kind of the exponential right if we look at like you know revenues for AI companies like it seems like they've been growing roughly 10x a year and so you could imagine getting you can imagine getting to the hundreds of billions in you know two to three years and getting even to the trillions which per year which no company has reached but I'm say that's revenue for the company revenue for the comp right and then what about what about productiv productivity in society right so that depends on how much is this replacing something that was already being done versus doing new things I think with things like biology we're probably going to be doing new things so I don't know if let's say you end people's you know let's say you know you extend people's productive ability to work by 10 years right that could be you know 16 of the whole economy do you think that's a realistic Target I mean again like I know some biology I know something about how the AA models are going to happen I wouldn't be able to tell you exactly what would happen but like I can tell a story where it's possible so so 15% and when will we so when could we have added the equivalent of 10 years to I mean how what what's the time frame again like you know this involves so many unknowns right if I if I try and give an exact number it's just going to sound like hype but like a thing I could a thing I could imagine is like I don't know like two to three years from now we have like AI systems that are like capable of making that kind of Discovery like five years from now like you know those those discoveries are actually being made and 5 years after that it's all gone through the regulatory apparatus and and really so you know we're talking about more we're talking about you know a little over a decade but really I'm just pulling things out of my hat here like I don't know that much about drug Discovery I don't know that much about biology and frankly although although I invented AI scaling I don't know that much about that either I can't predict it I think you know more about these things than than most of us yet and and and yet it is also hard to predict absolutely have you thought about what he could do to inflation yeah um so again I'm not I'm not in a eist um if we look at uh if we look at inflation I mean again using my limited economic reasoning I think if we had very large real productivity gains um that would tend to be deflationary rather than inflationary right Abol like you would be able to do you'd be able to do more with less the dollar would go further um so directionally at least that suggests disinflation but to totally but what what kind of magnitude have what kind what kind of magn I mean I I that you more the expert on than I am maybe I should ask you to predict that how do you work with the hyperscalers like you know you some of your shareholders like Google and and and Amazon yeah so you know I think um so just get it straight these are called hyperscalers because why I actually don't know the reason for the name but you know they're they're they're hyperap companies in terms of their valuation but but also they make very large you know very large AI data centers okay I assume refers to the second one but uh absolutely how do you work with them you know I would say that the relationship with these companies makes sense in the sense that we have complimentary inputs right that they provide the chips in the cloud and then we provide the model and and then that model is something that again can be sold to customers on the cloud so there's kind of a layered cake where we provide some layers and they provide the other layers so this partnership these Partnerships make sense on on multiple grounds you know at the same time we've always been very careful right we have our own kind of Val as values as a company our own way of doing things and so we try to stay as independent as possible and one of the things we've done is of course we have relationships with multiple of these Cloud providers right we work with both Google and Amazon and and that has allowed us some flexibility in in in our ability to make sure that there isn't too much um uh exclusivity and that we're kind of uh you know free to deploy our models on multiple surfaces the fact that the companies are becoming so incredibly powerful what kind of systemic risk does that pose I would say that and this is maybe broader than AI it maybe relates to just kind of the era that we're that we're living in there are certain eras in history where you know there's a powerful there's a powerful technology or there's an economic force that kind of tends to concentrate resources you know I I think you know probably the same thing happened in the 19th century and so I think it I think it actually is important to make sure that the benefits are shared by all um so one thing that's often on my mind is there's been for example very little penetration of AI and language models in some parts of the developing world right in like subsaharan Africa and so how do we how do we bring these models to those areas how do we even help with challenges in those areas like health or education um so I I I definitely agree we're we're you know living in an era of more con concentrated wealth and that's an area of concern and an area that you know we should we should we should you know we should do what we can to find counterveiling force what's the risk in in that these companies are now becoming more powerful than than countries and governments yeah I mean you know this is this is kind of you know what I said on in terms of in terms of Regulation like you know I think that uh AI is a very powerful technology um and you know our governments our democratic governments do need to step in and set some basic rules of the road right it needs to be done in the right order it can't be stifling but I think it I think it does it does need to be done uh because you know because because as you said like we're getting to a point where um the amount of concentration of power can be you know can be greater than that of national economy you know n National governments and you know we don't want that to happen at the end of the day all the people of the country and all entities including companies that that work in it you know they ultimately have to be accountable to democratic processes right there's there's no other there's no other way would will AI increase or decrease the difference between rich and poor countries that I think depends on what we choose to do with it the way you look at the path forward just now I would say that we are looking for ways um for it not to make sure but is that happening it's too early to say with like how you know with how the technology is being deployed um I would definitely say I I do see something related to it that's a little worrying to me and that we're trying to counter which is that uh if you look at the natural applications of the technology the you know the the things that customers are are you know the the most eager customers that come to us I think because we're a Silicon Valley company often the most eager customers are other kind of technologically forward Silicon Valley companies that kind of also use the technology and so I think there's this danger of that what you might call like a a kind of closed loop where it's like an AI company you know supplies a like AI legal company which supplies an AI productivity company which supplies a you know some other company in Silicon Valley and you know is it all a closed ecosystem where it's and it's all being used by the most highly educated people exactly and and so how do we break out of that Loop and so we thought about a number of ways to break out of that Loop one of the reasons I talk about biology and health is that I think biology and health can be used to help us to break out of that Loop Innovations in health assuming we distribute them well can apply to everyone I think things like education uh can help here um another area that I'm very excited about is use of AI for provision of everyday government services you know I don't know what the names of these services are in Norway you know in the US every time you interact with like the DMV the IRS various Social Services people almost always have a bad experience and it it drives cynicism about the role of government um and I would I would love it if we can modernize government services that everyone use so that they can actually deliver what people across the world need I have to say that I think in this country we are we are fortunate in that we are not so many people and we are we got you know is heavily digitalized and you are probably much better than we are at this I'm reacting to my experience in the United States which which you know I think could be better so net net what do you think will in 10 years time will the gap between rich and poor be bigger or smaller I just have to say like if we handle this the right way I hear what you say the right way we can narrow the Gap I hear what you saying what do you think what do you think will happen I I don't know what I think will happen I know that if we are not ex extremely thoughtful about this if we're not extremely deliberate about it then yes it will increase the Gap okay okay who will make the most money on on AI will it be the chip manufacturers or will it be uh you guys or the scalers or all the consumers or companies yeah my boring answer is that I think it's going to be distributed among all of them and that the pie is going to be so large um that in some ways it may not even matter like certainly right now the you know the the chip companies are making the most money I think that's because training of models comes before deployment of models comes before Revenue so I think the way I think about it is the valuation of the chip companies is a leading indicator um the um valuation the AI companies is maybe a present indicator and the valuation of lots of things Downstream is a lagging indicator but but the wave is going to reach everyone so when you look at the the market cap of uh Nvidia for instance you that's an indicator I mean what do you multiply that by to find uh the potential impact of AI what I think that and you know obviously I can't give stock advice on on podcast about chips so so that's three trillion dollar right yeah but but three trillion so so why is that which is nearly twice the size of U this fund which is the largest Sovereign wealth Fund in the world yes um you know if I think about that again like speaking very abstractly and conceptually um what's that driven by probably that's driven by like anticipated demand like people are building very large AI clusters those clusters involve lots of revenue for NVIDIA presumably companies like us are paying for those clusters because they think that the models they build with them will generate lots of Revenue but that revenue is not present yet and so what we're seeing so far is just man people want to buy a lot of chips and of course it's possible it's consistent with the whole picture that all of this will be a bust the models don't turn out to be you know that powerful like companies like anthropic and the other companies in the space don't do as well as we expected because the models don't keep getting better that always could happen that's not my bet that's not what I think is going to happen what I think is going to happen is that these models are going to produce a great deal of Revenue and then there's going to be even more demand for chips nvidia's value will go up the AI company's value will go up all these Downstream company you know that's the op that's the bullish scenario that I'm that I'm I'm betting on by by Leading this company um but I'm I'm not sure it could go the other way I don't think anyone knows where is the uh biggest constraint just now I mean is it in is it in chips uh Talent uh algorithms electricity my big bottleneck we're dealing with is data um but as I've said elsewhere uh we we and other companies are working very hard on synthetic data um and I think that bottleneck is going to be lifted so data just to get a straight that's just information you feed into your models to based information that's that's fed into the models but we're getting increasingly good at synthesizing the data tell me what is synthetic data the example I like to give is is uh seven years ago uh you know deep mind uh as part of Google uh produced the alphago model which was able to beat the world champion in go and there was no that version or there was a version of it called alphao zero that was not trained on any humans playing go all it did was the model played go against itself for for a long time basically for forever um uh and and so basically with just the little tiny rules of and the models playing against each other pushing against each other using that rule they were able to get better and better to the level where they were better than any human and so you can think of those models as having been trained on synthetic data that are created by other models with the help of these this kind of logical structure of the rules of Go and so I think there are things analogous to that that can be done for language models how do you think AI will affect geopolitics no I think that's a big one um um uh my my view is that again if we get to the level of AI systems that are better than the best professionals at a wide range of tasks um you know then tasks like military and intelligence are going to be among those tasks uh and uh you know we shouldn't be naive everyone is going to try to deploy those um I think we should try to create cooperation and restraints where we can um but that uh you know in many cases that won't be possible and when it isn't possible you know I'm I'm on the side of democracies in the Free World um I I want to make sure that the future is democratic that as much as possible of the world is democratic and that democracies have um a lead and an advantage on the world stage um the idea of powerful AI plus autocracies terrifies me and I don't want it to happen should each country have its own language model yeah um should no way build a language Model 5 million people it kind of really depends on what you're aiming to do right um uh you know it it may make sense from a national security perspective for every country to have language models I think an idea that might work uh that you know like another direction we could go in is imagining some kind of democratic Coalition or cooperation in which Democratic countries you know work together to provide for their Mutual Security to protect each other to protect the Integrity of their Democratic processes maybe it makes sense for them all to pull their resources and make a very small number of very large language models but then there you know there may also be value in decentralization I don't have a strong opinion on which of those is better is it a national security issue that us controls AI should Europe be worried about this each country has to kind of worry about its own Security even separately from its allies um you know I think that's that's more of a question for for kind of for kind of for kind of individual governments I mean you know I would think of it probably this is a provocative analogy but a little like nuclear weapons right some countries even though they're allies feel the need to have their own nuclear weapons for example France um other countries say no we trust that we're being you know protected by the US and the UK and France I think it may be somewhat similar with these more powerful models and I think it's less important how many of them exist within the Democratic world as that the Democratic world is in a strong position relative to relative to autocracies you talk about um cooperation and partners and so on do you guys in in II actually like each other we've done a number of collaborations um so you know uh I think uh very early on when I was at open AI you know I drove the original RL from Human feedback paper was considered safe work and this ended up being a collaboration between Deep Mind and open Ai and we've worked together um you know in organizations like the Frontier Model Forum uh to collaborate with each other that said I mean you know I'll be honest I don't think every company in this space takes issues of safety and responsibility equally seriously from from from all the other companies but you know instead of pointing fingers is that the kind of things that make you you not being so keen on other companies is it their view on Safety and Security no I mean it's one of the few Industries where you where you even consider you know having a cage fight between you know yeah so I'm I'm I'm not a fan I'm not a fan of the cage fights I'm not a fan of the I mean you pretty fit so I'm sure you do well but know even though I suspect it won't be your strength I no I fighting in cage fights is not not my not my forte but the thing I was going to say is look instead of instead of pointing fingers instead of having feuds and saying this guy is the bad guy this guy is the good guy let's think systemically right going back to like the race to the top idea right the idea that it's like let's set standards instead of pointing fingers at people doing something bad let's do something good and and then a lot of the time people just follow along we invent an interpretability idea you know just a few weeks ago we we put out you know I was was talking about it a few minutes ago this innovation in interpretability being able to see inside the Model A few weeks later we got similar things from open AI we've seen internally other companies increase their prioritization on it so a lot of the time you can just do something good and you can Inspire others to do something good now if you've done a lot of that if you set these standards if they're industry standards and then there's someone who's not complying with them there's something that's really wrong then you can talk about pointing fingers let's spend a few minutes talking about culture how many people are you in The Firm we are uh we are about uh 600 uh as of as of a couple weeks ago I've been on vacation so it may be even higher now What's the culture like I would describe a few elements of the culture um uh one one element of the culture is is what I describe as do the stupid simple thing that works um uh a number of folks at anthropic are uh ex physicists uh because you know I myself had that background and a couple of my co-founders had that background including one person who was actually a professor of physics before uh before he co-founded anthropic and you know physicists look for simple explanations of things so one of the elements of our culture is you know don't do something over complicated right a lot of academic ml research tends to over complicate things we go for the simplest thing possible that works we have the same view in engineering and again we have the same view even on things like safety and ethics on interpretability on you know our constitutional AI methods they're all incredibly simple ideas that we just try and push as far as we can even this race the top thing right you can say it in a sentence or two right it's not complicated like you don't need a 100 page paper to talk about it it's a simple strategy do good things and try and encourage others to follow when you hire 600 people in 3 years how can you be confident that they are good I think candidly one challenge of the AI industry right is how fast everything moves so uh you know in a normal startup things you know might grow 1.5x or 2x a year um we recognize that in this field things move so fast that faster growth is required in order to meet the needs of the market and that ends up entailing um faster growth than than usual I I was actually worried about this at the beginning of the I said oh my God we have this dilemma how do we deal with it I have generally been positively surprised at how well we've been able to handle it so far right how how good we've been able to scale hiring processes how much I feel everyone is both technically talented knowledgeable and just just generally kind and compassionate people which I think are equally important as hiring technically Talent so what do you look for so here I'm sitting um you are interviewing me now for that position what do you look for yeah I mean again you know we look for willingness to do the simple thing that works uh you know we look for talent um it generally you know we don't necessarily look at years of experience in the AI field like a number of folks we hire are physicists or other natural scientists who you know have have maybe only been doing doing AI for a month or so right have only have only been doing a project on their own and so we look for ability to learn we look for curiosity ability to quickly get to the heart of the matter and then in terms of values you know we we just look for things thinking in terms of the public benefit right like it's it's less that we have particular opinions on what the right policies for anthropic is or what the right things to do in the world it's it's more we we want to carry a spirit as we as we scale the company and it gets increasingly hard as the company gets bigger and bigger um because you know how do you how do you find how do you find all these people uh but we we want people who who carry some amount of public Spirit who understand on one hand that anthropic needs to be a commercial entity to be to be close enough to the center of this to have an impact but when you hire but to understand that that at that in the long run we're we're aiming to you know we're aiming for this this public benefit this societal impact when you hire do you feel you have a limited amount of money you know I think compute is almost all of our expenses um I won't give an exact number but you know I I think you know can be publicly backed out that it's more than 80% so salaries doesn't matter really yeah in terms of in terms of paying people we think more about what is fair right we want to do something that's you know that's fair that meets the market that treats people well it's it's less of a consideration of like you know how much how much money are we spending because compute is is the biggest expenditure it's more how can we how can we create a place where everyone feels they're treated fairly and people who do equal work get equal pay now you work with all these Brilliant Minds and kind of geniuses and and perhaps even some some Pradas how what's the best way to manage them or lead them yeah you know I think I guess they can't be managed so you need to lead right one of the most important principles is is just the thing you said which is letting creativity happen um uh you know if things are too top down then it's it's hard for people to be fully creative uh it uh you know if you look at a lot of the big Innovations in the ml field over the last 10 years like the invention of the Transformer you know no one at Google kind of ordered you know oh you know here's the project here's what we're trying to produce it was it was just kind of you know it was it was a decentralized effort at the same time you have to make a product and everyone has to work together to make a single thing and I think that creative tension between we need new ideas but we need everyone to kind of contribute to one thing I think that creative tension is where is where the magic is finding finding the right combination so that you can get the best of both worlds you run this company together with your sister right yes yes how is that we we both worked at uh open Ai and then we both uh founded anthropic together um it's really great so um you know the real division of labor is you know she she does most of the things you would describe as running the company day-to-day managing people figuring out the structure of the company um you know making sure we have you know a CFO a chief product officer um you know making sure comp is set up in a reasonable way making sure the culture is good I I think more in terms of uh kind of ideas and strategy um every couple weeks I'll give a talk to the company basically a vision talk where I say here's some things we're thinking about strategically these aren't decisions this is kind of a a picture of what leadership is thinking about what do we think is going to be big in the next year where do we think things are going both on the commercial side the research side the public benefit side is she younger or older than you uh she is four years younger than me is she clever than you uh we we we are both extremely skilled in different ways what did your parents do um so my my father is deceased he was he was previously a a Craftsman um my mother's uh my mother retired uh she was uh she was a project uh project manager for public libraries how were you raised uh how was I raised um you know I there really was I think a big focus on social responsibility and helping the world like that was that was I think a big thing for my for my parents you know they really thought about how do you how do you you know how do you make things better how do you how do people who have been born in a for forunate position um you know reflect their responsibilities and you know deliver their responsibilities to those who are who are less fortunate and you know you can kind of see that in the public benefit orientation of the company so like the 14-year-old Dario what was he up to um I mean I was really into you know math and science like you know I did like math competitions and all of that but you know I was I was just also thinking about like uh you know what you know how could I how could I apply those skills to you know invent something that would that would help people did you have any friends did I have any friends um uh you know less than less than I would have less than I would have liked you know I was I was I was a little bit uh I was a little bit introverted but um you know that there were there were people who who you know who I knew back then who I still who I still know now so is entropic like revenge of the nerd uh you know I wouldn't really put it I wouldn't really I think that's a good thing I love that kind of stuff yeah I wouldn't really put it in those terms if only because you know I'm I'm kind of reluctant to like set different groups against off each other you know like people are different kinds of people are good at different things you know like we have a whole sales team like they're good at a whole different set of things than I am like you know of course I'm the CEO so I have to learn how to do some sales as well but there are just very different skills and one of the things you think about a you realize in a company is that different kinds of people with very different kind of skills just you know like you you recognize the value of a very wide range of skills including ones that you know that that you're that that that you have no ability in yourself right so what but what's what drives you now I think we're in a very special time in kind of like the AI world like you know this is you know these these things I've said about you know how crazy things could be in 2025 or 2026 um you know I think it's important to get that right and uh you know running anthropic I'm you know that's only one small piece of that right there are there are other companies you know some of them are uh are bigger or better known than we are and so uh on one hand you know we have only one small part to play but uh you know I think given the importance of what's happening for you know for the economy for for for for for Humanity I I think we have an important opportunity to you know make sure that this things go well there's a lot of variant in how things could go um and I think we have the ability to affect that you know of course dayto day we have to we have to grow the business we have to hire people we have to sell products and you know I think that's important and it's you know it's it's important to to do that well so that the company is relevant but but I think in the long run the thing that that drives me or that at least I hope drives me is is you know is is is the desire to to capture some of that variance and push things in a good direction how do you relax how do I relax um so you know I'm in Norway now um uh yeah but I just not relaxing this is not relaxing but I I came here from my vacation in Italy so you know every every year I take uh I take a few weeks uh I take a few weeks off to kind of relax and think about the deeper Concepts um I go swimming every day um actually uh me and my sister uh still play video games we used to do this since high school um and uh you know now now uh you know I'm I'm over 40 and she's she's like you know what kind what kind of games do you play uh well we recently got the new uh the new Final Fantasy game so we played Final Fantasy in high school it was like you know a game made in the 9s um and they recently made a remake of it um and and so so you know we recently started playing like you know the new version with all the like fancy Graphics from like you know 20 years of progress in well actually gpus uh and you know we were ourselves noticing it was like wow you know this is like we used to do this when we were in high school now we're like running this company well I'm glad to hear that some people never grow up uh I I I I don't think we've gr certain way I don't think we've grown up hopefully we have in others uh talking of which we always finish off these podcasts with um with um with a question what kind of advice do you have to young people gain familiarity with these new AI Technologies um you know I'm not I'm not going to offer some kind of bromide about you know I know exactly which jobs are going to you know be be big and which which aren't um I think we don't know that and and also um you know we don't we don't know that AI won't won't touch every area um but I think it's safe to say that there will all you know there there's going to be a role for humans in kind of using these Technologies and working alongside them at the very least at the very least understanding them in the public debate that's going to come from them I guess the other thing I would say and this is already important advice but I think it's going to get really more important is just the the faculties about skepticism about information as AI generates more and more information and content um being Discerning about that information is going to become more and more important and more and more necessary I hope that we'll have ai systems that help us sift through everything that you know that help us understand the world so that we're kind of less vulnerable to these kind of you know to these kind of attacks um but but at the end of the day it has to come from you you have to have some basic desire some basic curiosity some basic discernment and so I think develop knows is important well that's a a really great great advice well big thanks this has been a a true blast and I uh wish you all the best uh get back to Italy get some more rest and do some more deep conceptual thinking thank you so much for having me on the podcast thank you