Azeem’s Picks: Sam Altman on How GPTs Are Shaping Our AI Future

AZEEM AZHAR: Hello, I’m Azeem Azhar. Eight years ago I started Exponential View to explore the underlying forces driving the development of exponential technologies. I’m a child of the exponential age, born the year after Intel released the 4004, the world’s first single chip microprocessor. I got my first computer in 1981. I’m holding it as I speak. And as I moved into adulthood, the internet was growing exponentially, transforming from a tool of academia to the worldwide utility it is today. I at least did not grow exponentially with it. Now, back in 2015 when I started writing Exponential View, I’d noticed something curious was going on. You could feel the pace picking up. New technologies were clearly on some kind of accelerating trend. One of those is artificial intelligence. And so, three years ago on this very show, I had an amazing conversation with Sam Altman, the CEO of OpenAI. Now, you might not have known OpenAI back then, but surely you know it now. They’ve been pushing the boundaries of AI development with their GPT models, most recently GPT-4, which is just quite the most incredible software product that I’ve ever used. Now Sam and I spoke at the time of GPT-3’s release. And during that conversation, he shared his views on the potential of AI, its societal implications and the challenges of trying to manage this, not just from a risk and governance standpoint, but really from the perspective of what it is to be human. He spoke very candidly about the development of OpenAI’s software and the profound risks associated with its use. We dived into the societal milestones necessary to ensure safe and ethical use of AI, and the importance of expanding the number of tasks evaluated by AI systems. He also shared his insights in the differences between the research and startup culture, and how OpenAI manages to balance both. Our conversation tackled some of the most pressing questions facing the development of AI, including the cost of development and the need for accountability. Sam’s thought-provoking perspectives on the philosophical, economic, and social implications of AI will leave you with a new perspective on the world-changing potential of this technology. So please enjoy a rerun of this essential conversation. Sam, welcome to Exponential View.

SAM ALTMAN: Thanks for having me.

AZEEM AZHAR: Now, do you still believe a few years on from setting up OpenAI that the AI revolution is going to be more significant than the agricultural revolution which created civilization itself?

SAM ALTMAN: On a long enough time horizon, I definitely do. I think that technologies get sort of to stand on the shoulders of the technologies that come before. And we needed civilization itself to be able to have the industrial revolution and then the computer revolution. Certainly we needed those to be able to build AI. But everything just compounds at this phenomenal rate. And so at this next level of emergent power, I do think is going to transform everything.

AZEEM AZHAR: So, this summer we had GPT-3, which was a really powerful milestone in a number of ways. How would you characterize GPT-3 as a technology on that continuous curve?

SAM ALTMAN: On the curve of AI, I think it is a baby, baby step. I think that it’s still quite weak in important ways, but it is a preview of what is to come. It is definitely not AGI in the sense that people mean when they use that word, but it is a model that has general capabilities and some degree of what feels like intelligence, or at least what can be used in an intelligent way for lots of interesting applications. And I think it’s also sort of a preview of how these technologies are going to be, as you mentioned, both good and bad, or they have at least the potential to be significantly helpful or harmful. And I think as important as the technological milestone is the sort of societal milestone of saying, okay, this is going to happen. How do we as a society decide what the rules should be in how we’re going to use this? I don’t think Silicon Valley has a perfect track record here, and so hopefully we can do it a little bit differently this time.

AZEEM AZHAR: So, what was the dissonance that you were experiencing between what you thought of GPT-3 and what you were sort of reading in the frenzied excitement around it?

SAM ALTMAN : Yeah, I think what you just said is exactly right. I think it just got sort of frenzied. I think it is, if you’re not looking at this every day, and sort of watching the exponential curve, I see why it looked like a discontinuity. I can understand that. Because people have been talking about AI for so long, and there’s been no publicly available system that feels really general. And so the first one of those is in some sense a big deal. But I think people got ahead of themselves, and this often happens with the new technology, and then you use it for a while and you get used to it. So what we were trying to do there is just remind people that, look, this is a moment in time. This is an interesting checkpoint. This is not AGI, this is not mean AGI is around the corner. But it does mean that something powerful in this direction is going to happen, and we should have a conversation about how that’s going to go.

AZEEM AZHAR: Let’s talk a little bit about GPT-3 and what it actually does. I mean if you are explaining it to somebody who’s not involved in the field, what does this tool that you’ve built to actually do?

SAM ALTMAN: I would say that it is a general purpose language tool. You can have it respond to text in any format you’d like. And it’s pretty good in many cases, not all, at understanding what should come next. But to do that well, it has to sort of understand what a human would think would come next. It’s trained on looking at lots of texts and predicting what would come next. And to the surprise of a lot of other people, it can do that. It can use that same model to have a chat and answer questions, write code, summarize documents, translate between languages. And I think that’s what’s powerful about it is that this one model, just by predicting the words that should come next, can do so many powerful applications.

AZEEM AZHAR: Right. The point being that when you think about where we were say three years ago with text-based systems, you’d have had to have one system that could do customer service chat support for a mortgage company, another to do customer service for an insurance company, a completely different one to help you with writing software code, and a fourth model to translate between Mandarin and French. These would all be distinct, separate, narrow tools. And what we’ve started to see across the industry in the last few years, and GPT-3 is a very good example of this, is ones that are more general, that can be used for many, many different applications.

SAM ALTMAN : Yeah. And that’s what I think is really exciting about this little stepping stone on the path is this the first really general purpose AI that we’re aware of that has been released. When these general purpose things come along, like the iPhone spawned a huge number of new companies. Before that, the internet, the computer, the combustion engine, they spawned a lot of new derivative technologies and companies. And why I think this matters is we are now at the beginning of what I think will be fairly rapid development of sufficiently general AI that it counts as one of these general purpose technologies. And obviously I’m biased here, but my guess is that this will be the next technological platform that we’ve all sort of been waiting for in the industry for some time to enable a lot of new incredible services that would even be difficult to imagine today.

AZEEM AZHAR: As a dilettante sort of observing this from afar, I can say, oh, this thing looks quite general, but do you have a formal way of saying this particular model is actually general? I mean, what is the test for something to be general?

SAM ALTMAN: The best way is to just evaluate it across lots of different sorts of tasks and lots of different metrics. I’m a big believer in you make what you measure. And so if you measure some narrow thing, you’ll do well at that, not everything else. And we really try to hold ourselves to evaluating its performance across many, many different tasks. And we kind of know the areas where it’s weak, and then we know the areas where it’s strong, and some it’s really strong. But we’re going to just keep expanding the number of tasks we evaluate these systems on and try to do better on all of them simultaneously. So there’s mathematical definitions that I think we’re kind of reaching the edge of the utility of. And at this point our evaluation metric will just be how useful is it to the world.

AZEEM AZHAR: And you’ll be able to measure that by the amount of use people make of it in a number of different applications.

SAM ALTMAN: One of the things that we learned at YC was that if you just set a growth goal and a very aggressive growth goal for yourself, like 10% a week or something on your key metric, you can’t hide from that very long. And it’s either a good enough product that people really love it, and base their company off of it, and tell their friends it’s amazing, or it’s not. And if you just sort of hold yourself to a growth metric, that will eventually be an acid test of whether what you’re doing is good enough or not.

AZEEM AZHAR: You keep referring to GPT-3 as a small step on this curve. When I look at it, I think of it as one of the most complex machines that has been built in the world. It has 175 billion parameters, which if you’re not a computer scientist, think of a parameter as a dial that you can turn, like a volume button or the base or the treble, or it’s a slider on a graphic equalizer. That’s a lot of parameters.

And the previous baby step that you took, which was GPT-2, which was a year and a half ago, was 1.5 billion parameters. So it was 100 times less. And the one before that was a 10th the size of GPT-2. So for you as insiders, you see these as sort of incremental steps that you’re taking along the curve. For people observing from the outside, these look really big steps. I mean the kind of order of magnitudes is increasing each time.

SAM ALTMAN: Sure. But if you think on a log scale, it’s like…

AZEEM AZHAR: Right. So do you think, on a log scale, is that how it’s done within OpenAI?

SAM ALTMAN: I do. Yeah, I think that’s basically been the story of the field for some time now.

AZEEM AZHAR: So in a product like GPT-3, which is quite complicated, what have you had to learn in order to build something like GPT-3? So why is it May 2020, not May 2017, to bring this out?

SAM ALTMAN: One of the things about OpenAI that is unusual is that we are a combination of a research lab, of a very good system scaling startup engineering group, and sort of a policy and safety think tank. And the thing that we’ve learned is you have to execute as well as anybody in each of those three areas at the same time. So there’s a lot of complicated research. There’s very complicated system scaling. The policy issues around how we allow access to this technology, who gets to use it for what, how we make sure people are not misusing it, are very complicated. How you have the model itself help with that. And we’ve had to learn how to do well in all of those axes. And I think the thing that is tricky is they’re very different cultures. And so getting one group of people to execute is challenging, but that’s why we’ve been, to whatever degree we’ve been successful so far, it’s not that we just scaled the system up, it’s not that we just did research, it’s not that we just figured out the policy rules here. It’s doing all of them at the same time.

AZEEM AZHAR: So, what would you say is culturally different about the way that OpenAI runs, relative to the many hundreds of startups that you will have seen and experienced through your time of running Y Combinator?

SAM ALTMAN: It actually feels quite different. I had a steep learning curve here, and I’m thankful for people’s indulgence with me. But I think research culture is quite different than startup culture. And even what the company values, how you set the goals, how you measure progress, what motivates people, it’s a very different world. An example of something that was hard for me to understand is how important publication is to people. I just didn’t really have a mental framework for that, and it took me a while to understand. Now I think I get it, but how you get that to coexist with people who are like we’re going to measure ourselves off of a product growth goal and people who say we’re going to measure ourself off of the number of citations we get. It sounds like those should be complimentary. And I’d say it turns out you have to do more work than you’d think to get those to coexist. Getting the company to exist happily in the middle of these three sort of clans or cultures has been an interesting challenge. I think we’re doing pretty well on it now though.

AZEEM AZHAR: But you’ve also discussed this idea that you have to now start to measure OpenAI’s value in terms of the usage, right?

SAM ALTMAN: I certainly don’t think that’s the only way to evaluate our progress. In addition to the academic benchmarks, we now have a new one, which is the real world. We will continue to measure ourselves on lots of metrics. I think that’s a key to getting something general. But what we really care about, or at least what I think most people doing this kind of work should really care about, is do you do something useful? Do you make people’s lives better? And now that we can directly measure sort of impact in the world, I think that’s a wonderful new avenue that’s exposed itself to us.

AZEEM AZHAR: As this entity that is part research and development, and also putting products out there, it seems like the way in which you need to make decisions about what projects to go forward with and what avenues you don’t is perhaps different to what it might be in a traditional product-based company. So what is the framework that you might use to internally to decide whether to initiate an internal project and whether to progress it?

SAM ALTMAN: So, one thing that helps with that is it’s a difficult mission, but we have a very clear mission, which is to build safe AGI, and maximally share the benefits with humanity. And we basically evaluate everything we do on that question. If we should do a piece of research at all, if we should scale it up, if we should release it. And we have a small enough company where we can, even though we’re doing something new and hard and with this kind of quirky structure, and as far as I know, no other organization in the world runs quite like this, but when we have to make these hard decisions, we have a small enough team that we can get everybody in the room, or now on the Zoom call or the Google Hangouts call, and talk through it. And I think that’s a real advantage of being small. We also realize that we don’t have enough input from broader society.

AZEEM AZHAR: How do you know that you don’t have the right external input?

SAM ALTMAN: I think there’s like 150 people that work at OpenAI. There’s no way we have enough input from the concerns of the world as a whole.

AZEEM AZHAR: Okay, fair enough. Then I’m curious about one of the things that you learnt with GPT-2, which was a precursor to the latest model, I think there was a particular statement around not wanting to release this into the wild, into the open, because you felt it was too risky. There were a whole set of risks.

SAM ALTMAN: We said it might be too risky.

AZEEM AZHAR: That might be too risky. And when you got to GPT-3, which was a much more sophisticated, more powerful technology, you did open it up. There’s an access program with a big long queue of people where researchers and developers can access this through an API and build their own applications. So I’m curious about what you learned such that the decision was different for the more powerful technology.

SAM ALTMAN: So, one thing that we learned is that you just need to stick with your convictions, even if you get panned on Twitter and completely ignore it. Which everyone knows, but it’s good to be reminded. When we did that for GPT-2, we were just sort of continuously and ruthlessly mocked saying like, “Oh, this is so silly, this is never going to be dangerous.” Blah, blah, blah. And then GPT-3 comes along, and those same people are like, “I can’t believe OpenAI’s releasing this at all. It’s so dangerous.” Blah, blah, blah. So it was a good moment for the company to really stick with the courage of our convictions and I think the general mistakes Silicon Valley has made has been be cautious too late and we’d rather err on being cautious too early. So that was a good overall learning. One specific learning is that if you just release the model weights, like we did eventually with GPT-2 on the stage process, it’s out there, and that’s that. You can’t do anything about it. And if you instead release things via an API, which we did with GPT-3, you can turn people off, you can turn the whole thing off, you can change the model, you can improve it to continually do less bad things. You can rate limit it, you can do a lot of things. So, this idea that we’re going to have to have some access control to this technologies seems very clear, and this current method may not be the best, but it’s a start. This is a way where we can enforce some usage rules and continue to improve the model so that it does more of the good and less of the bad. And I think something like that is going to be a framework that people want as these technologies get really powerful. I mean again, I think we’re at the baby, baby stage.

AZEEM AZHAR: I think what we’ve seen with GPT-3 of course is that the first thing that happens with any new piece of software that is programmatic is people try to push it to its limits, right? You get access to the Twitter API in 2009, 2010, and the first thing you try to do is what can I break? And so you must have seen people trying to break GPT-3 in different ways. To what extent is that helpful for you, and are there surprising ways in which people have tried to stress the system?

SAM ALTMAN: It’s totally helpful. Look, at some level you could say that it’s unfair that most of the negative stuff you see on Twitter is people turning off the content toxicity filter, really trying hard to bait it into giving offensive outputs. But then on another level, that’s the kind of stuff that can happen with technology like this. And it is important for us to get that input and that feedback so that we can build models that do much less of that or someday just don’t do it. And I think it’s important with a technology like this to get enough exposure to the real world that you find some of the misuse cases you wouldn’t have thought of so that you can build better tools. There’s a whole set of hard questions about if someone really deliberately wants to use the model to say offensive things in a context where that might be okay or even good, should that be allowed? And my personal belief is yes, but it’s a tricky question. But I think what we can all agree on is people, developers, and users should have controls on these models such that they behave in the way that the actual user using them wants. And I think seeing where that doesn’t happen and getting the feedback is what lets us get there.

AZEEM AZHAR: I think there’s a challenging set of questions here though that relate in general to the question of governance of powerful technologies. One of the things I think that’s tricky with AI and software technologies in general is that they get progressively easier to develop and deliver. It’s very different to nuclear power, which is still a state level entity. And you can be a big country like Iran and still struggle militarizing those technologies. But with software every two years it gets 10 times cheaper and the breakthroughs that you’ve made this year will be quotidian within three or four years. So the sort of march of Huang’s law and Moore’s Law. So, how do you think about whether we need some kind of broader governance framework for these technologies? I mean, I think Silicon Valley has been brilliant at many things, but it hasn’t been brilliant around the questions of ought there be some sort of governance around the technologies in the way in which they get used? How have you thought through that problem with AI in general?

SAM ALTMAN: I think we do need new governance. I think that, more generally, I would say our existing governance models did not account for the idea that technology would eventually become so powerful and such that companies like OpenAI could eventually do what only nation state actors could do. If we go back and think about your example of, let’s say, nuclear weapons first and then power, or something like the Apollo program, those required an amount of capital and just resources that a private company just wasn’t going to do it. I forget the exact number for the amount of electricity in 1943 that went to enriching uranium, but it was something like 23% of all of the US’s electricity. Just took massive resources. And so I think we need new governance models and we also need new economic models. And part of what I think OpenAI is good about is that we are able, because of our unique structure and this kept profit model that we came up with, to advocate loudly for those.

AZEEM AZHAR: I think your economic model is super interesting, and I would love us to spend a bit of time explaining it. Let’s talk about the governance question though, because I think that seems to me to be a really difficult question. So the way that I look at technology governance at the moment is that especially internet technologies have not really had a governance framework. They emerged from a West Coast of the free addressable, end-to-end internet, which in a way is a governance model of its own. And recently, as you rightly point out, these technologies have got more and more powerful. And the biggest companies like the Microsofts and the Googles have state-like capabilities in certain areas. When there’s a cyber attack, the local military can’t do anything about it. And then the trouble with this governance model is that these large companies don’t have any accountability to citizens. And we are ultimately at the whim of the senior executives and the board of directors to make good decisions. But the chain of accountability has been broken. So I’m curious about what you think global compact might look like in the next few years where we feel we can get these technologies working for humanity, but in a way that is well governed.

SAM ALTMAN: It’s a super important question. We can study history and see how that goes. When there is no public oversight, when you have these companies that are as powerful or more powerful than most nations, and are sort of governed by unelected, and to be perfectly honest, fairly unaccountable leaders, I think that should give all of us pause as these technologies get more and more powerful. And I think we need to figure out some way, not just that economics get shared and that we deal with an equality problem that society is facing, but that governance about these very big decisions. What do we want our future to look like with very powerful AI? That that is not a decision that 150 people sitting in San Francisco make collectively. That’s not going to be good.

AZEEM AZHAR: I haven’t found good models right now that do this. I mean I think that people talk about the idea of mini-lateralism, which is that if you can just bring enough people to talk often enough, perhaps you create the kernel of a framework that other people can buy into and then people copy that.

SAM ALTMAN : Yeah, I think that is helpful for sure. But then there’s always a question of how much teeth does it really have? Let’s say we really do create AGI, and right now, if I’m doing a bad job with that, our board of directors, which is also not publicly accountable can fire me and say let’s try somebody else. But it feels to me at that point, at some level of power, the world as a whole should be able to say, hey, maybe we need a new leader of this company, just like we are able to vote on the person who runs a country. And so this question of what democratic process for companies that have sufficient impact looks like, I am very far from an expert here, and much smarter people than me have been thinking about it. But it occurs to me that we’ve got these models and we can look at the different ones around the world and talk about some countries where it works better and where it doesn’t work as well.

AZEEM AZHAR: Is there one that springs to mind for you?

SAM ALTMAN :We are working on this actively right now. I wouldn’t say we’re at a point where we’re like, okay, this is definitely the right model for us or for the industry as a whole, but it’s something that we’re trying to turn our focus to.

AZEEM AZHAR: The other piece that’s interesting is that historically technologies have always created inequality. That’s why we see the arrival of status and hierarchy and private property 10, 12, 15,000 years ago. This technology is going to be no different. But it’s clearly been something on your mind as you’ve gone through to design with your colleagues OpenAI. What are the particular things you’ve built within OpenAI that start to address that?

SAM ALTMAN: So, I think the biggest thing that we did is this idea of a capped profit structure. So we said, look, we started as a nonprofit. We were hoping that we could make it work there. Making AGI looks like it’s just going to take so much capital that we thought we couldn’t do that as a nonprofit, but we didn’t want to be a full for-profit either because of the potential to drive truly massive inequality. And so we said is there some new structure where we can have the benefits of capitalism and all of the wonderful aligning power that produces, attract the capital and talent we need and fairly reward them, but then figure out a way to mostly share the benefits here. And I think as technology keeps going, these companies get bigger and bigger and more powerful and have more leverage, we will need something new as a more general structure. And I think somehow either corporations make less money because they pay people more, or somehow there’s got to be some degree of societal ownership of corporations, or a direct sharing in the equity value. Otherwise inequality will just run away. And OpenAI is one way of sharing the equity value.

AZEEM AZHAR: I think just to maybe put some historical framing on some of those observations, I mean the labor share of income in most developed countries has declined by about 10 points over the last 40 years, with most of that gain going back to companies themselves, in fact, not even to shareholders. And the other thing is that within an AI kind of product, you have a rich get richer phenomenon because you’ve got the data network effect that is making your model better than the next person’s. And every query that comes into GPT-3 is a new form of data that you can use to improve the next model. So you have these dynamics. And I think it’s fantastic that you’ve identified that this is one of the challenges. You talked about there being a problem with the inequality that would cause. So do you have an internal mental model about how much inequality is too much and how much inequality is just enough to keep people motivated?

SAM ALTMAN : That is such a hard question. Some degree of inequality is the price of capitalism. I still think capitalism has these wonderful benefits that have been replicated by no other system. And if you stop rewarding people for innovating or even for efficiently allocating capital, that’d be really bad. So perfect equality is wrong, I think. But the current level of inequality we have is way too much. And in terms of the multiple of how much should the average person have relative to the richest or whatever else, I don’t really know. What I will say is that I think people are most sensitive to just being on a very steep curve of their lives improving. And the fundamental thing that has gone most wrong with inequality in the US is not the spread between the richest person and the median person, but it is that all of the gains, all of the relative progress, has gone to the people at the top. And that I think is the most toxic of all. So rather than a spread, I would say everybody needs to participate in a massive updraft year over year. I do think AGI will do that.

AZEEM AZHAR: So, let’s talk about the mechanism by which AGI then helps address the inequality problem.

SAM ALTMAN: The cost of the most critical goods and services should, in some sort of real dollar sense, just go down and down and down as technology can do more and more of what human labor has done, which drives the cost of most goods and services. And that means that what you really want is not a certain number of dollars, but a certain amount of wealth. And as we’re saying, an increasing amount of wealth. And as AGI can drive the cost of goods and services down dramatically, if we even hopefully just redistribute a little bit of money, I hope we do more than that, everybody should be able to afford what they need for a really good life and more of it every year.

AZEEM AZHAR: One of the problems that we’ve seen in the US in particular is that the price of things that I suppose are kind of consumer oriented, maybe a little bit trivial, whether it’s televisions or games and so on, has gone down. But the things that you actually need to have a fulfilling, flourishing, safe life, health and education, has shot through the roof.

SAM ALTMAN: It’s really disappointing to see. I think this is a huge problem and not well understood. But basically where technology’s been able to do its thing, and where people have had to have a competitive market, it’s been great. And then where you’ve had these problems where there’s not very good market competition, and particularly where services are sort of in some way or other government subsidized or government protected, but not government run, you get these wild inefficiencies. So I think it clearly works in other countries besides the US where the government just provides the healthcare, but they also pay for it and they make the decisions about it. But what we have in the US is an absolute disaster. And if you look at like the three horsemen of the cost apocalypse, housing, healthcare, and higher education, for slightly different reasons in all three cases, we are just facing an absolute catastrophe. And that, technology can help those somewhat for sure. But I think those are policy failures more than anything else. And until we address those, you can do a basic income, you can build AGI, you can do a lot of other things, it’s still going to be bad. And I think those are so critical to, as you said, a good life, a fair life, and the American dream, that if we can’t fix those basically as a society, there’ll be very limited impact on everything else that we can do.

AZEEM AZHAR: So, you talk about being a nonprofit, and the cost of doing AGI was clearly going to be high. I mean, to the average listener, I think your initial funding was around a billion dollars is what’s been put in the press. So how much will AGI actually cost?

SAM ALTMAN: A lot more than that. I don’t know the exact number. We’re going to spend as responsibly as we can and as efficiently as we can, but we’ll also spend whatever it takes. So I don’t know what that number is, but it’s going to be vastly more than a billion dollars.

AZEEM AZHAR: So, somebody with a cheeky spreadsheet might try and run some numbers. They might say, well, you’ve got 150 headcount. We know that GPT-3 cost X million dollars of computer time to train the model, and there are some numbers that are floating out there. We know that the price per compute cycle is declining at this rate because of the great work of NVIDIA or Graphcore, whoever’s doing it. So they’re going to put that in very quickly and say, well, if it’s more than a billion dollars, and we’ve got these declining kind of price curves, we have a sense of the amount of compute that is going to be needed in order to build AGI. So I feel like I have to ask you this question, which is how much compute is needed?

SAM ALTMAN : If an oracle could tell me exactly how much compute we’re going to need, it would save me a lot of handwringing, and I’d love to know. So if you figure that out, please tell me. It would be really useful information. First of all, the estimates that I’ve seen for how much it costs to train GPT-3 were all way under, but I’m sure I haven’t seen all of them. None of this stuff is cheap. When people have run those spreadsheets, they come up with wildly different numbers. I have seen people say 10 billion. I have seen people say 100 billion. I have seen people say a trillion. I have seen people say 100 million. We just need the right algorithm. And the true answer of course is no one knows, and everyone’s spreadsheet is wrong in some important way. I think we do some of the best work in the world trying to project this, and we ourselves have huge error bars. But the more we learn, the more confident we are, then we can go off and raise more specific amounts of capital and make more specific computer orders. But all I can say with confidence is it’s going to be a lot.

AZEEM AZHAR: Yeah. But I think the important thing as well is it’s not just a dollar sum. There’s something that you said earlier in our conversation about you have these different teams who have competing interests and competing constraints, and essentially the capabilities of each one have to grow like an ecosystem with the other teams. So it’s not as if you could get someone to write you a huge check today and that would solve the problem tomorrow. It’s more that there’s some undiscovered capabilities that need to be learned.

SAM ALTMAN: For sure. One thing that I certainly believe is that you could throw a trillion dollars at the problem without any new ideas and you still wouldn’t get to where we’d like to get. I’m very confident that we’re making fast progress on the new ideas, and certainly on the policy and safety work. The question that we get a lot is OpenAI is smaller and has less compute than some of these other organizations and labs. How are you outperforming them or why are you outperforming them? And I think one of the answers that sounds a little bit silly but is somehow true is it’s not in spite of that, it’s because of it. And because we can stay so small and focused and only do a few things at a time, and really have to align a lot of people with these different cultures into thinking about where are we going to bet our very extremely limited resources and talent, that’s actually the best way to make progress towards these problems.

AZEEM AZHAR: That approach – I mean it sounds to me like it’s the startup approach. So it’s come out of your experience of not having been a researcher but having nurtured 1,000 startups or more.

SAM ALTMAN: Absolutely.

AZEEM AZHAR: It’s a very distinctly different way than a lab would work. But then I’m curious also about whether you can do basic science in that way. Sometimes basic science just requires someone to be able to ask the question, and just spend their time scratching their forehead and throwing things up on a whiteboard.

SAM ALTMAN: Well, it seems like we’re doing okay at that. We really try to build an organization that supports that. And I view a big part of my job is not letting all of the pressures that come with success and offering a product to get in the way of what got us here in the first place, which is someone scratching their head in front of a whiteboard. I still think we have a lot of that that we need to do, obviously some of the most talented people in the world to do it. And we’ve got to make sure that we keep a culture where that continues to happen.

AZEEM AZHAR: I want to zoom back out a little bit. We talked about the importance of AI and AGI as heralding a new epochal revolution in human development. And when we look at these revolutions, one of the things that’s interesting is I think of it as you can’t tell a goldfish about the water that they’re in because they’re in the water. And a new technology comes out and takes a goldfish out of the water and say, “Hey, look, you’re in water and there’s a bowl.” And the goldfish has to learn about that. These sort of emergent properties that come out of society. So, whenever we see these breakthrough general purpose technologies, we see that change happening. And it’s not just an industrial change, it is a kind of political economic change. It is a values change. So, it took Galileo 350 years to get his pardon from the Catholic church, and yet here we are with a new set of ways of thinking about the world that you can’t disentangle them from the technologies that went before and the potentials that they created. So it’s a really hard question for me to say to you, when you build AGI, which essentially is a five dimensional telescope, what are we going to see in the fifth dimension? But when you ponder that question and you think, where are the fracture lines in our existing world that will look so primitive when we are living in a society that has as its bedrock technology-

SAM ALTMAN: I think it’s going to call into question all of the deep philosophical issues that have been on humanity’s mind for millennia, and they will be newly relevant. What it really means to be conscious, and what it means to sort of be a separate entity. If we really try to look through that five dimensional telescope about what this is going to tell us about our place in the universe, and what intelligence and consciousness and awareness and all of that mean, that’s hard to think about. That’s a big, big thing. And that’s the one that I would say, when I wake up at 3:00 in the morning is the thing that’s on my mind. It’s like what’s going to happen? When thing gets kicked off, and these intelligent entities that are much smarter than any humans, and sort of self-aware in some way, but maybe in a very alien way, maybe we build a very alien intelligence, what’s that going to mean for the next 10 to 100 years of the universe? On a more pedantic level for society, I think it will be a big shift because we have for so long, we humanity have identified and gotten our identity based off of being the smartest things on the planet. And when that’s no longer true, how will we define our success? How will we get our intrinsic motivation and happiness? What will we do with our days? And also how will this system working in concert with us help us govern ourselves in ways better than we could ever think of ourselves?

AZEEM AZHAR: How do you avoid the challenge of taking a kind of anthropocentric view of what that intelligence is? I mean, I think you hinted at this where you said this intelligence may be more alien-like. That maybe that the tools you end up building have as much in common with our intelligence as a helicopter has with an eagle or a hawk. If that’s the case, do these things become tools? Do they ever have agency that we could recognize as agency?

SAM ALTMAN: I suspect they will on a long enough time horizon, but that might be a very long time horizon, and it might be in a different way than we think now. But I suspect that they will. I still sort of think that some version of a merge is what’s most likely to happen, and we won’t be talking about these things and us, but it’ll all sort of co-evolve in one very powerful direction. And I think that’s honestly the future that seems the safest and the most exciting to me.

AZEEM AZHAR: Well, one of my favorite examples of the merge is what happened to our low intestines as we started to use flints, which as I’m sure you know, we started to externalize our digestion. And so our gut proportions are very different to hominids who have to digest those fibers internally. So we’re used to this kind of [inaudible 00:41:21].

SAM ALTMAN: We’ve been merging with technology for a long time, and it doesn’t have to happen with plugging electrodes into our brain. There’s a lot of more pleasant ways than that to happen. And I expect it will. But I don’t think it’s going to seem like this super differentiated bright line thing that’s us versus them. I hope it doesn’t. I think that’ll be bad.

AZEEM AZHAR: I think every year, is it right, that you and the team get together and discuss forecasts for when you get to the milestone that we’ve been discussing, which is AGI? So where is the current forecast consensus view, and how do those discussions take place? I mean, take us to that room.

SAM ALTMAN: Sure. So this is the year where we realized it was a bad question. And I think we’ve all, as part of just seeing GPT-3 out in the world, not all of us, but most of us have updated towards thinking that instead of this moment in time, it’s going to be this continual exponential curve, like everything is. And this question of when we get AGI is somehow framed wrong because it’s going to be a many year, I think, smooth, exponential curve of progress. And so then the question is, do you want to talk about when that curve starts, when it ends, what does it even mean to end? Maybe it never ends, maybe it just keeps going. And certainly for me, this was the year where I said, I can’t give a number because it’s going to be this thing that’s already started, and it’s going to go on for a long time. And I can’t say we’re done because it’s just going to keep getting more powerful. But the way that it happens is we do an annual offsite every year. We give talks, we have discussions, we drink around a fire, whatever. And then at some point we do a company poll. And it’s very unscientific. You don’t have to respond. Not everybody does. And the question is AGI as want to define it, how many years? And it was the number of years away was going down faster than one year per year. So again, some sort of exponential curve.

AZEEM AZHAR: Yeah. And some optimism.

SAM ALTMAN: And optimism. Could be the wrong kind of optimism, could be like we just sort of hired people who were too optimistic. So I don’t want to make too much of that. But this year many people would say things like, “It’s really hard to define because it’s not a point in time.” And that’s definitely what I feel. So I think next year we’ll figure out how to ask the question in a very different way.

AZEEM AZHAR: It sounds like you and OpenAI are always learning, Sam.

SAM ALTMAN: We are trying. I think we do a lot of things badly, but I think one thing we do well is get data and adapt.

AZEEM AZHAR: That has been evolution’s answer for the miracle of survival and thriving, which is get data and adapt.

SAM ALTMAN: It will keep working, I’m pretty sure.

AZEEM AZHAR: Thank you very much, Sam Altman.

SAM ALTMAN: Thank you.

AZEEM AZHAR: Well, thanks for listening. If you enjoyed this conversation, be sure to subscribe because we have some of the most powerful figures in the AI and tech world lined up for you later in this season. In the meantime, you should check out some of my previous discussions with many of the world’s top AI researchers, including Fei-Fei Li, Gary Marcus, and Jürgen Schmidhuber. I would encourage you to listen to those three in particular as they provide some perspectives on some of the issues that I discussed with Sam. To stay in touch, subscribe to my podcast or my newsletter at exponentialview.co. This podcast was produced by Marija Gavrilov and Fred Casella. Ilan Goodman is our researcher, Bojan Sabioncello, the sound editor. Special thanks to Exponential View members, Elisabeth Ling, Paola Bonomo, and Gianni Giacomelli for help with this episode. Exponential View is a production of E to the Pi I Plus One, Limited.