How to Build Artificial Superintelligence | Jonathan Siddharth, Founder & CEO of Turing

Speaker A: White collar digital knowledge work will be automated in about three years, including the job of the CEO. If I look at my job, it involves looking at dashboards, analyzing data rather poorly compared to like a state of the art LLM. Like I'm token constrained, I can't work in parallel. Like I cannot consume tokens from Jira and Salesforce and these dashboards and Slack and Zoom. I'm constrained by space and time. Right. But the agentic version of me will not be right. And I think we are about to make the shift from the copilot era to the agentic era. In the Copilot era, we used to think of these LLMs as tools that will work alongside you and help you be 10% more productive, 20% more productive. That era, I think, is going to go away. We're going to enter the agentic era where these tools are just going to be compute and data constrained. Once they are trained well, uh, they'll go and amplify you 100x.

Speaker B: My guest today is Jonathan Siddharth, the co founder and CEO of Turing. Jonathan incubated Turing in Foundation Capital's Palo Alto office in 2018. Since then, it's grown into a multi billion dollar company that helps leading AI labs build their most advanced models. If you've seen a recent breakthrough in how AI reasons or codes, odds are Turing had a hand in it. In our conversation, he unpacks what it will take to achieve superintelligence, why it's such an important goal, and how the world might change once we get there. We also touched on the present, how his leadership style has evolved over seven years of running Turing, his take on founder mode, and how to cultivate the speed and intensity needed to compete at, ah, AI's frontier. Here's Jonathan. Super excited to have you back on the podcast for the second time. I mean, we did the last recording, what, three years ago. And so much has changed in the world of AI and so much has changed for you and Turing. So, uh, I'm sure we'll end up with a very fun conversation.

Speaker A: Thank you, Ashu, for having me back. And we'll stick to the tradition of maybe like Star wars, where the second movie is better.

Speaker B: Well, when you and I first met, I think it was 2015, mid or late 2015. You'd already been an entrepreneur once and gone through the journey, uh, right out of Stanford and having founded Rover. So maybe walk us through that story about what was it that spoke to you about becoming a founder?

Speaker A: So, uh, when I came to Stanford, Ashu, um, I came with this idea, um, that I would do my PhD and become a research scientist. Uh, so I came with that goal. Um, and at Stanford there is this uh, radioactive spider that crawls through the Gates building and bites you and you become a startup founder. And I think what Stanford did was injected me with a, uh, dose of confidence, um, that I was ready to start a company. And prior to coming to Stanford I had this view that, okay, you have to go to, you know, you go to grad school, maybe you get an mba, maybe you need some work experience and then you start a company. That's how you become a, uh, successful, uh, founder, CEO, uh, and Stanford injects you. And now I realize it's really like overconfidence, it's really false. Starting a company is so hard and I absolutely did not have all the skills I needed when I was in school.

Speaker B: But you never do. I mean even if you had done all those things, you had done a master's degree, you had done an mba, you had done check the right three boxes, you still don't have the right skills, you might as well start young.

Speaker A: That's right, you're never ready and you always have to be editing yourself. So I came with that mindset and like, of just doing research. I published a bunch of papers at Stanford. I received the best master's thesis award in computer science, uh, for my work on neo duplicate document detection on the web. So one part of Stanford was this dose of confidence. And you see these founders of YouTube who are your classmates and you hear YouTube sort of getting um, uh, acquired for 1.6 billion. You're like, that's crazy. I can do this too.

Speaker B: Exactly, yeah, I know that guy and he's not that good.

Speaker A: Uh, uh, Stanford does this really good job where when you're in the fourth floor, when I was there, they had the offices of Larry and Sergey still there with their name tag. So you walk around like the, oh, this is uh, it's very real. You see Google as this big company and here are some grad students in the same floor and they went and started it. They basically dropped out and did it. What could, uh, you know, maybe um, you believe that you have um, the same knowledge, the same sort of, um, the skills that you need to start. So that's part of it. And the second part of it is the talent density. And uh, so I met my co founder Vijay in one of the first classes we took together, which was on building a web search engine. And VK and I, his name is Vijay Krishnan, he goes by vk. We did like a class project together where you had to build, um, you. You had to build like different elements of a search engine together. And all these late nights coding, uh, doing stuff, and you kind of see, um, you see other people with like a lot of drive and other people who also want to start companies. Like, Vijay was also like, uh, like was also came with the idea of like getting into research, but then sees like, okay, let's go on the entrepreneurial journey. Um, so that was how I got started. And at the time, when I think about it, like this was like 2007 and the ecosystem was actually not that built out.

Speaker B: Right.

Speaker A: The, um, Y, uh, Combinator didn't exist. Uh, there weren't even like a lot of Indian VCs. I kind of remember at the time, like there were maybe there was, um,

Speaker B: uh, only a couple. Yeah. When I joined Venture, there are very few Indian Road. I mean, Vinod was obviously a very successful VC even then. Uh, but there was a very short list after Vinod.

Speaker A: Yeah.

Speaker B: Yeah.

Speaker A: I remember like being on a panel, attending a panel somewhere and was like Ravi Matre, Naveen Chadha, and um, um, I think Rajiv Motwani was there. Like, he was again like a legend, like connecting people.

Speaker B: Absolute legend. Yeah.

Speaker A: Yeah. And it's funny, at the time, like, none of these playbooks that we take for granted existed. Like the founder community didn't quite exist. Like it was. There was not a lot of. It was a lot harder to like start a company. And um, so we started Rover, uh, fresh out of Stanford, uh, had to figure out the whole Visa thing. Like, how can you even start a company on an, on an F1 visa or an H1B visa? We had to figure all of that out. Um, and that was such an, that was such a, um, incredibly, um, useful, uh, journey. Like it was a nine year journey. Um, we went through ups and downs. I learned a lot from that experience. Rover basically was a personalized search and discovery engine that understood what a user's interests were based on content that they looked at on the web and recommended stuff to them. Uh, we had acquisition offers from Google, Twitter that we turned down and then kept going. And Rover really tested me on my, I would say, um, it increased my stress ceiling, like the stress envelope. Um, and your persistence. And you kind of discover stuff about yourself when you're on the edge. The thing I discovered about myself is that I probably hated failing a lot more than I wanted to win. Like, I really wanted to win and build a Big company, but I was absolutely not going to fail.

Speaker B: Uh, sometimes that's the thing that keeps you going. The winning and the upside is somewhat theoretical, but failing is real in the moment.

Speaker A: That's right.

Speaker B: Yeah. No, I totally get that.

Speaker A: And we got battle tested in the ups and downs of the startup roller coaster. And eventually the company had a good acquisition. Uh, but it was a lot of useful founder lessons learned. And one business lesson, which was the seed for Turing, which was the idea, like one of the lessons from Rover was look for Silicon Valley caliber engineering talent in places where nobody's looking. And one of our competitive edges we had was we had a rockstar engineering team, but that was not based in the Bay Area. It was distributed and it helped us keep our burn really low while still moving really fast.

Speaker B: And it enabled you to survive from tough tires, Correct?

Speaker A: Right. That's right.

Speaker B: And so when you landed that plane, you know, at, ah, Rover, I remember very quickly, you were bouncing back and Both you and VK were clear. I remember when we met in 2015 that, yes, we're going to do another company. We don't know what that is, but we're going to do another company. We're going to do it together. And that's sort of how our dialogue started. And, you know, you started spending more time with us in the foundation office initially, I remember, in Menlo park, and then ultimately in Palo Alto. You became eirs with us and Turing was born. Maybe talk through the Turing story a little bit, uh, briefly, and then we can talk about how it's evolved over time.

Speaker A: Uh, so after selling Rover, I took some time off to recharge and figure out what I wanted to do. Um, for a brief period of time, I wondered if I wanted to become a vc. There was a very brief period of time. Um, but then there was this urge to keep building again. Um, uh, it's hard to explain. It's almost like being in love. Right. It's like, you know that that's what you wanted to do. And I had all these accumulated lessons from Rover. Like, think of it like a training set of like, okay, this was good. I did this. That was a positive outcome. That was a negative outcome. Probably shouldn't have done that. So there were all these accumulated lessons and I wanted to take another shot.

Speaker B: All these RL traces, man. Yes, in the modern vernacular.

Speaker A: That's right. That's right. Like, uh, experiential learning.

Speaker B: Right.

Speaker A: And, um, so it was very clear I wanted to do that. And I'd heard these lessons from the PayPal founders, like when they started, like how PayPal was such a grueling journey that it was so grueling and so brutal that the people that came out of it like they wanted to do something, they had all these learnings, uh, wasn't as big of a hit as they had wanted it to be, but they knew they could do bigger. And then we got LinkedIn out of it, SpaceX out of it, Tesla out of it, and all of these YouTube and all these companies. My thinking was similar. There are all these accumulated lessons from Rover. What could we take to company number two? So after a few months of taking a break, um, I got together with vk, uh, again and we started brainstorming what we should do next. And it was at that time, uh, Ashu, that we met. I attended one of your iit, uh, dinners and those were really good too because it was again like being at Stanford where you're surrounded by other founders, other people building. And my juices were again stimulated. And it was also when I got to know you better, I was very impressed by your insights, thoughts on company building and how you were um, guiding those uh, seed stage founders. And um, one of the things I realized from company number one was making sure you're surrounding yourselves with the right team, the right ecosystem to help you win. And one of that is of course having really high quality investors. And then, um, I remember Ashu, uh, we came to you and said, we uh, have one of two ideas. And uh, uh, Ashu was like, um, whatever it is, I'm in, let me invest. Told you, Ashu, like, please, please wait. Like let's, we want to figure we want to reduce entropy and then go with the thing, uh, that's more promising. And one thing that Foundation Capital did and you did and your partners did that was super helpful, was you connected us with a lot of potential prospects and advisors to quickly vet both ideas. And in those conversations, which was very clear, okay, this has market pool. This is going to be a red ocean. Uh, and that was super insightful. And one of my lessons from company number one, which we took, uh, to Turing, was be in a big market and be against weak competition. Ideally it's analog competition. Like you're not competing with tech companies, you're competing with some brick and mortar thing.

Speaker B: There are many companies that have competed with previous generation of technology companies in one. And I would never say that that's not an interesting market. I've been in many of those. But when you can find the equivalent of the digital to analog competition I mean, as Netflix did for Blockbuster and then over time, movie studios, or in some ways as Google did for newspapers. In terms of classified ads, you completely change the dynamics of a market.

Speaker A: That is exactly right. And like Uber is an example, a taxi company is never going to compete with you. Airbnb, who's their competitor? Some random collection of hotels? Like it wouldn't. So competing with non tech companies was definitely a, uh, lesson that I took away. It's also interesting when you're evaluating ideas, there are some ideas you fall in love with and some ideas you don't. And, um, I was, um, uh, dating someone at the time who asked me, hey, what are you working on? And I'd shared these two things. Hey, it's going to be one of these two things. And she told me, like, Jonathan, you were like 10 times more excited about this one than this other one. And I couldn't even tell that I was. I thought they were both like, equal in my mind.

Speaker B: Well, you got something out of that date. So anyway, you ended up starting Turing and obviously, you know, now what, eight years in, uh, you know, it's been a huge success. Uh, maybe. Talk about sort of the arc of the curve for Turing over this time period. What was the original idea and where the company is today?

Speaker A: So we started Turing with this idea that it's really hard to find high quality engineers, software engineers, especially if you're constraining yourself to a talent radius that's just the Bay Area. So we built a platform where we would find the world's smartest software engineers at scale. Imagine Silicon Valley caliber engineering talent, but at a fraction of the cost because you tapped into the global talent, uh, pool. So we built that platform where you could push a button, get, uh, an engineer or get an engineering team. And startups were, um, using us like crazy. And even now, like, if you're building a startup and if you need engineers to build your applications or products, you should check out turing.com. so we started with that, uh, and then when Covid hit, we just accelerated and took off. Because the world knew now that, uh, you could work with talent wherever they're located. So we grew really, really fast. So we had this pool. We had the world's largest developer cloud, like, um, about 4 million software engineers. So we were perfectly positioned for the AGI wave that was about to hit. Um, when we got that fateful call from OpenAI where OpenAI was getting ready to teach GPT3 to code and to do function calling and tool use, they were like, okay, what's the company to go to with the best engineers. So they came to Turing and we worked with them to improve the coding abilities of uh, the model behind ChatGPT. As it launched. The world obviously saw, uh, saw that and were amazed at how well LLMs could code. As people from OpenAI left and started other companies, joined other labs, they kept taking Turing with them. So we expanded to work with almost Every Frontier Lab, OpenAI, Anthropic, Meta, Google, Microsoft, Nvidia, uh, literally anyone that's building a Frontier model, if it's coding. Well, like statistically it's a Turing customer. In fact Ashu, I'm not going to name the companies, but numbers 1, 2 and 3 on Swebench, which is the benchmark for coding, are all Turing clients like gold, silver, bronze? Actually let's think of it like a set, OpenAI, Anthropic, Gemini. They're all Turing clients for coding.

Speaker B: It's been amazing, which you truly did. I think when the history of these models is written, I think Turing will have played such a pivotal role in what is the hardest part about pushing the upper boundary, which is really coding. Because in a sense coding has become the way you build better models. I mean you can process the world's web data and uh, you're building the memory bank. But the reasoning has come from coding.

Speaker A: That's exactly right. Ashu. Coding has been known to be foundational to reasoning. When the models get better at coding, they get better at a wide variety of out of domain tasks that have nothing to do with writing code. It was fascinating. So we, so we expanded like that. And the thing that set us apart is that we are not a data vendor or a data labeling company. We're not a data factory or a recruiting uh, or like just a mere talent platform. We become a research accelerator where we help all these Frontier Labs improve their models in coding, multimodality, tool use, reasoning, all these advanced AGI, uh, capabilities. Um, and full credit Ashu, we give all the credit in the world to the labs, to these researchers working at these Frontier labs and we are proud to partner with them to help them and the US win the race to um, I guess now we should call it Artificial Superintelligence. It's not AGI anymore, it's asi, um, so full credit to them. And you're right that we've played a very important role in helping the world get to AGI and hopefully ASI soon. And we kept expanding. We were this fine tuning partner to all of the Frontier Labs. We helped Them build models and agents on top of the models, including some of the well known agents from the top labs that you've heard of. So our expertise in fine tuning models and building agents, we've now started taking that to the enterprise. So enterprises also need custom models fine tuned on proprietary data. You want to distill proprietary knowledge trapped in the minds of humans that work in these companies and distill that into LLMs. You want to teach the models how to automate proprietary workflows. So we were fine tuned, uh, as we continue to fine tune models and agents for the frontier labs we've started also fine tuning models and building agents for the enterprise, which I think is an even bigger market. And all of that has as its foundation the Turing that we started. Find the best software engineers in the world, which we've now expanded to find the smartest humans in the world, including people at the pinnacle of math, physics, chemistry, biology, doctors, healthcare professionals, finance professionals, marketing professionals. So in some sense I think, um, ASI is energy constrained. When you have enough energy, then you get to compute, then you become compute constrained. But they're also fundamentally intelligence constrained. They're constrained by intelligent tokens that have to be mined from the minds of smart humans at scale. Uh, so I think there's a lot more to come.

Speaker B: Absolutely there is. And I think as you rightly said, while there's one underlying platform, there are three distinct go to markets, working closely with all of the model providers and model developers. And that number will continue to expand. It's not an infinite list, but it'll continue to expand. Uh, but there's almost an infinite list of enterprises, uh, that will fine tune, that will apply reinforcement, learning will do customizations of all kinds to these models to build their own systems of agents. And then of course companies of all types, including startups, will always want access to the unique talent platform uh, that you have built. Uh, you know, as you look through that journey and especially in the last few years, I mean it seems like things just keep getting faster and faster. The pace of change has increased and the velocity of the business has continued to grow. Uh, tell me a little bit about sort of some of your lessons learned.

Speaker A: So we are living in a world that moves incredibly fast. And especially if you are touching AGI as we are where our customers are, these frontier labs that are moving super fast, um, you have to move even faster to keep in touch. And I feel like one of the most important things is to make sure the learning rate of the organization is super high so that you can react very quickly to changes. Um, I feel like these concepts of product market fit were all defined in a different, um, era where the world was relatively more static. Uh, there is a problem and a solution and you kind of keep doing the same thing and you're going to be okay. Now it's like you have to fight for position every quarter or, um, some cases every month. You kind of have to build an organization that's capable of iterating really fast and moving really fast. When Brian Chesky and Paul Graham sort of had their famous sort of essay on founder mode, like the it, um, I think it spoke to a lot of founders because, like, Brian's journey, like with Airbnb, like, really resonated with people at a personal level because many founders have gone on that exact same, exact same arc. What's needed is to drive a company unapologetically forward with intensity and speed, knowing that you'll piss people off. Uh, you'll, you'll not. There'll be some type of people who will not want to work at this type of company. Um, and that's okay.

Speaker B: It's an explicit choice that you make and that employees have to make. And there's no right or wrong. But it's not for everyone.

Speaker A: It's not for everyone.

Speaker B: Right.

Speaker A: And like, the, um, that I would say, like, one thing I've changed in my sort of working style is, um. Uh, I would say like in the last couple of years, I've just realized that running a startup is like, so different from running like a big tech company that when you hire from like, bigger tech companies, you have to be very clear with the folks you're hiring on how this is totally different. And you shouldn't necessarily try to adapt to their styles, but instead, like, educate execs coming from big tech on how like, a startup is so different. Folks have to be okay with moving really fast, like working nights and weekends. Like, um, for example, I had to like, create a culture at Turing where literally like repeating to people that it's okay to call somebody nights and weekends. Like, what would normally happen is that people would be like, oh, this person is out of office. And I'm like, there's like a big client here. They could be out of office, but you can call them.

Speaker B: It's okay. And it's life at a startup.

Speaker A: Yes, Izaiah. And I think that that that culture is like, really important. Like, I've realized that when you're hiring people, um, like the. That is like, there's almost like this two types of people. There is one type of people who say they prioritize work life balance. And then there's another bucket where I would say, like, even this work life balance is like a false dichotomy. There is this giant thing called life. And, like, work is such an important, meaningful part of what we do, right? It gives us purpose, and we are excited about creating something amazing and useful. So we now like to steer, uh, into people who care deeply about their work and will do whatever it takes

Speaker B: to

Speaker A: serve clients, serve our customers, work really hard. We are switching to a structure where the company is relatively flat. Not too many layers of management. Um, when something, for example, um, is read, like, we do a daily standup where we literally work on it every day to fix it. And there are some execs who don't like it. They're like, okay, Jatan, I got this. This is a, you know, why do

Speaker B: we do one short?

Speaker A: Let's do it once a week or once every two weeks. But the clock pulse slows down, like, magically, whenever we do a daily standup, stuff seems to improve. So my execs hate it, but I still do it. And the earlier version of me would have assumed that, okay, they know something that I don't. They've done these. They've run these large teams at scale. But now I realize, like, running a startup at speed, this is like a Formula one team. Like, that's like a large car company. And, like, here, like, speed is of the essence. And, uh, Frank, in your podcast had, like, something, uh, very interesting to say if folks who haven't heard Ashu's podcast with Frank Slootmanshoot should watch it and read Frank's book, amp it up. Frank said, um, you have to bring a certain confrontational energy to meetings, and that's okay. So I've had to role model it for our team. It's okay to disagree with people. It's okay to push teams hard. Generally having this culture that achieving, um, extraordinary outcomes requires extraordinary effort, um, is something that needs to be, um, emphasized. The flip side of it is like, somebody who might say, it's not about working hard, it's about working smart. Right? And to them, I would say you've got to do both if you just smart.

Speaker B: But the thing you can control is working hard. Yes.

Speaker A: And you need both. Right? Like, if you just work smart, somebody that's working smart. And working hard will run circles around you. So I hate these false dichotomies. Like the. And that was my whole thing with work life balance, too. Like they're all part of like one bucket.

Speaker B: And you know, ultimately, you know, it is a choice. It is in as leaders of organizations, I mean I think about the same thing even for my own little team at foundation and for all of my portfolio companies, which is, you know, working in the companies we do is a choice. It's a privilege. And to the extent that you want to build something extraordinary, you have to opt in to do what it takes. And there are days when it feels effortless and there are days when it feels like you're paying a real price. Uh, but that's a choice and you don't have to opt in for life and you don't have to opt in at all. But if you opt in, then for as long as you're there, you're part of a Formula one team and you are driving as fast as you can, as close to the edge as you can and doing what it takes, no questions asked. I absolutely think so. Did it mean some folks in the team sort of decided it was not for them? And how did you deal with that?

Speaker A: You're right, Ashu, that it is a choice. And you're exactly right, it is a choice. And there's nothing wrong with choosing the non startup path. Right? Like there are plenty of good opportunities at big tech companies where you can have work life balance the way you think of it and you could have a relatively more relaxed sort of uh, work experience but you're not going to do the best work of your lives, you're not going to change the world, you're not going to do something as um, impactful. Um, so how did it land? We had churn. Like there are some people for whom it's clearly not the right, the right choice. I expect we'll continue to have churn in some areas like where people realize this is not for them and it's just a different, it's just a different job. And it's totally fine for people to prioritize different things at different stages in their life. Uh, we are at the stage where uh, if somebody said that their number one priority was work life balance, Turing would not be the right place for them.

Speaker B: Right.

Speaker A: It would be probably like another company. But if their priority was I want to do the best work of my life, have a big impact on the world, I want to be at the bleeding edge of AGI, I want to prioritize my own learning rate and I'm willing to move heaven on earth to do it.

Speaker B: And it's the best place in the world to Be absolutely. Uh, Jonathan, thank you so much for sharing this. I mean it is, it, it's a conscious choice that you have made. It sounds like with a lot of self reflection. Uh, and as you've communicated that Turing have been part of it has become a different company. It feels like even as we've gotten bigger, we're moving faster and we're moving with much more of us. It feels like more of a startup than it did even three years ago. Uh, despite the fact that we're so many times larger than we were three years ago. Yeah.

Speaker A: Thank you Ashu. I mean it's been a journey and we are continuing to get even faster. It's exciting also when you have so much market pull. Like our clients are moving super fast. ASI is so important to all the Frontier labs. Uh, I see our clients like working super hard. The people who work at OpenAI, Anthropic, Google Meta in all these topics, foundation model companies, they work nights and weekends too.

Speaker B: They're in founder mode too. Yeah. And let's talk about what these labs are doing because you know, you positioned yourself very uniquely as the Switzerland, uh, for all of these labs and at a time when their needs are changing pretty dramatically. So maybe talk a little bit about what's going on in the world of AI that's driving that.

Speaker A: What's happened is as these models have gotten smarter and smarter, the data that's needed to improve them has increasingly become harder to generate. The floor is going up. The models are advancing along depth, encoding, stem reasoning, et cetera. They're advancing along breadth in multimodality, multilinguality, multi industry, etc. And the models are becoming agentic, meaning they can actually go and do tasks, sometimes multi step tasks. In a world where this is happening, what's needed from a platform to generate data to improve these models is you need your platform to have the smartest humans in the world working as a team to break the models and then generate data to improve the model. You have to find gaps in the models and you have to improve them. I used to say a year ago you needed like Iron man to improve the models. Now you need the Avengers, uh, for example, like in physics, Ashu, like for example, to test whether the model does not, um, model has a gap in some area. In physics you would need a PhD in physics to test the model. Maybe you want to test some theory. Uh, you might need to daisy chain a software engineer who can code up a simulation to test that theory. You might need a data scientist to analyze the results of that simulation to test whether the theory, um, is correct or not. So to break the model, you have this interdisciplinary team that's working together to generate data, um, which is exciting. Uh, and as the labs are starting to use more and more reinforcement learning, you kind of have to. You need a platform that can offer rl gyms to train these models. You need a platform that can spin up and down, uh, because the needs of the labs change quarter over quarter. Um, and you also need a platform that can optimally divide up work between humans and AI because, um, it'll be very costly if you just let humans do all of the work. Um, and the humans would also get bored doing the more mundane stuff that an AI can do better. So you have to do all of this. Uh, so the shift that has happened is, and this shift became stark after the reasoning models came out. O1 being, uh, the most. The famous one. So it used to be before O1, people wanted like a data factory or like a recruiting platform, send me lots of humans or throw data over a wall. Relatively simple data. Now what the labs need is a strategic research partner that can work hand in hand with them to identify what are the gaps in the models, how do we create custom tools to generate data to address those specific gaps, um, then evaluate whether the model improved in those areas and then iterate. So they need a strategic research partner. Which, uh, is why we say the era of data vendors is dead and it's the era of research accelerators. And Turing has emerged as the leading research accelerator working with all of these labs. And we have in house experts in coding, stem, multimodality, et cetera, that work on top of our platform, which with expert humans in software engineering, stem, et cetera, working on our platform to generate data to move these models forward.

Speaker B: The macro shift, and I'm going to maybe grossly oversimplify it, is it feels like the last three or four years have been a little bit. I used the Daniel Kahn manual, know System one, System two analogy sometimes where it was really about building System one. Like the more data the models they had, the more intuition they were able to build. And it's not done. There's still, you know, pockets of data. You're helping fill it out. There's other people helping fill it out. But the era of sort of improving system one is coming to an end. And it really has become about, you know, how do you have more intentional, deliberate thinking and reasoning? And that just requires a very different worldview. I mean, it requires different data and it requires different skills from folks like Turing, but it's also changing the way these models interact with the rest of the world. And maybe you can talk a little bit about, as you look out in the future, where do you think the world of AI is headed in the next six to 12 months?

Speaker A: So AI is on an exciting trajectory for the next six to 12 months. Um, the four pillars of ASI, I think are multimodality, reasoning tool use and coding. Multimodality, because we, I mean as humans you have to be able to process audio, video, image, um, of these, um, different modalities. Reasoning Ashu, like, for the reason you mentioned, is absolutely important. You need systems that can think step by step and take um, actions in line ah, with an eventual goal. Tool use is incredibly important. It's how we as humans climb the intelligence ladder. So if you can teach the model to use different software tools, it's going to have superpowers. And coding is the ultimate meta skill where if the model doesn't have a tool it can use, it can code it up itself. Right. So if you solve these four pillars, multimodality, reasoning, tool use and coding, we will get to artificial superintelligence. And what that means for the enterprise is, um, so I'm going to, for the software engineers in the audience, maybe this will resonate a little more. Think of like four nested loops for every industry in the world. Retail, healthcare, life sciences, etc. Imagine every function in that industry, software engineering, sales, marketing, finance, etc. Imagine every role a human place in that function. In marketing, maybe there is a director of performance marketing, there is an SEO analyst, there's a brand marketer, et cetera. And the fourth loop is imagine a workflow that a human in that loop goes through. Right. A director of performance marketing might have a specific workflow to assess the marketing performance of different channels. What does that work involve it exactly? It involves looking at a screen. Maybe you're looking at, okay, how are my ads performing on Facebook's uh, performance Marketing, uh, platform, LinkedIn's performance marketing platform. You're probably doing some data analysis to figure out, okay, where is the CAC better? Where is the yield better? You might be connecting it with data from Salesforce to see how the leads that you acquired here, how did they actually perform? If you break it down, it's a combination of multimodality. You have to be able to understand an image screen, et cetera. It's a element of reasoning where you have to think about um, how you're going to solve the eventual end goal, which is um, demand gen or something like that, it involves tool use because you're in Facebook's campaign management tool, you're in Salesforce. You might even connect it with NetSuite to see how much revenue did you actually get from that lead that you acquired six months ago. Um, and then of course coding, where if you want to change something in your website, maybe now you can actually write code to change the funnel directly. The four pillars that I mentioned, now every cell in that four dimensional matrix that I just went through or four for loops could be automated this way. These are all agentic workflows and I think all types of white collar digital knowledge work will be automated in about three years, including the job of the CEO. If I look at my job, it involves looking at dashboards, analyzing data rather poorly compared to like a state of the art LLM. Like I'm token constrained, I can't work in parallel. Like I cannot consume tokens from JIRA and Salesforce and these dashboards and Slack and Zoom and I can only, I'm constrained by space and time. Right. But the agentic version of me will not be right. And I think we are about to make the shift from the Copilot era to the agentic era. In the Copilot era, we used to think of these LLMs as tools that will work alongside you and help you be 10% more productive, 20% more productive. That era I think is going to go away. We're going to enter the agentic era where these tools are just going to be compute and data constrained. Like once they are trained well, they'll go and amplify you 100x like you're not you. You may, you may want to be the human in the loop holding your hands on the wheel, but uh, but

Speaker B: they're so much faster. You know, I think, look, I would love to see a world where, you know, we, we could stamp out more Jonathan's with ASI or more ashu's and, and more Courtneys. Uh, and I think it will happen. Uh, I think what to me is exciting is as we see an explosion of capability across the four dimensions that you talked about, uh, I think we'll start to see real superpowers. Not one of those it can do 5% of my job or 10% or it's a good trigger to initiate ideas, but really the ability for people to go away and come back. It's like oh my God, things that I would have done or I would have had people in my team do have been done. Uh, uh, I think the real challenge for society is going to be we're now constrained by the adoption of these technologies as against the ability to build them. Uh, how do you think about that? I mean you have to. In some ways while you're building AI and building asi, you're also having to deal with adoption in your own organization.

Speaker A: Yeah, that's exactly right Ashu. And we are sort of in these. Adoption is really key and our big focus at Turing is to accelerate AGI advancement and deployment where people are actually seeing value. And it's been a tale of two cities where on the consumer side things are great in that. You know, everybody's seen the magic of ChatGPT. My mom uses ChatGPT. Like it's, it's going to be, it's like very, very um, uh, people are seeing the magic of these systems. But with enterprise, I think we haven't had the chatgpt moment yet, um, in its best form. And it's kind of constrained by the fact that oftentimes in enterprise you need custom models that are fine tuned on proprietary data, um, that distill proprietary knowledge into LLMs. They that automate proprietary workflows that know how to use uh, business tools well and we are not there yet. And these are constrained by both data as well as uh, in house expertise in many of these um, enterprises. Uh, on the data side what's needed is data that's complex. Like one of the things that you need to um, improve these models is um, have one of the areas of active research is to have these models be able to do complex multi step workflows in realistic business settings. Um, so the three dimensions are complexity, diversity, uh, of use cases and um, realism. Uh, like you wanted to mirror real world use. At Turing, one of the things we focus on is generating data that is complex, real world and diverse to capture a lot of these different enterprise workflows to make sure that the models can become more agentic. When we work with enterprises, we help them um, build custom models that are fine tuned on their proprietary data and help them build these custom agentic solutions to solve their uh, business problems. Um, and these enterprises need a partner that has done this before. And because we are the fine tuning partner to these frontier labs, like we're able to bring our talent, tools and expertise to help enterprises do the same um, inside their companies. We don't build our own models but that's a good thing because we can be Switzerland where we pick the right model for the task, um, across all of our um, uh, AGI partners, we uh, are partners with OpenAI, Anthropic, Google, um, Nvidia. We pick the right model, we fine tune it, we build an end to end system. Uh, and the enterprises also need a partner that knows what's coming. And I would humbly submit that maybe we have the visibility for maybe like six months. It's hard to really predict what's coming beyond that. Um, but it's inevitable that all types of knowledge work and digital work will be automated to a huge degree. Uh, and we are only constrained by compute and data. I think algorithms like we're almost there, like we are there like the current setup, uh, particularly, I mean basically deep learning and reinforcement learning work. So you scale that up. Uh, we are only compute and uh, data constrained. Uh, we're going to take care of the data and we have companies like Nvidia, Groq, Cerebras, et cetera, doing really cool stuff on training compute and test time, compute. Um, so it's going to be great.

Speaker B: I think that's well said. Two follow up questions on the algorithms part. One of the things that's interesting is while reinforcement learning is a very well understood discipline and the algorithms are well understood, most of reinforcement learning was historically designed for a world where outcomes were verified. Games is where a lot of reinforcement learning was really built out. Uh, as we're entering a world where a lot of the outcomes are not verifiable or are not even objective in nature, what does success look like in some of the, as you go through the uh, the four sort of dimensions that you articulated is not black and white? Uh, don't you think we'll see a lot more innovation on the algorithms themselves or possibly in uh, eval techniques, uh, in ways to build these RL gyms as well. Do you see scope for innovation in all of the above? Yeah.

Speaker A: So let me qualify my statement a bit. Um, I'm certain that a couple of years from now we'll probably have new algorithmic innovations on top of transformers, on top of reinforcement, uh, learning. There might be different neural network architectures that we come up with that are different from a transformer that work really well. Um, my point is that a lot of the researchers I spend time with believe that by scaling up compute and data with the current setup that is sufficient, but that is sufficient to get to large scale automation of a lot of these workflows and to get to asi. Um, but think of all the amazing work researchers at OpenAI, anthropic apple meta Google, all of these labs are doing, I'd be very surprised with the billions of dollars pouring in. We don't have algorithmic breakthroughs also. We will have that and that'll be a multiplier. But it's kind of crazy to see that the current, the scaling laws are holding meaning more data, more compute, bigger model mean the models smoothly keep improving.

Speaker B: Got it, got it. Makes sense. The other thing is you talked at length about how for large enterprises, uh, there is both a need and an opportunity to build on top of what the model providers offer. That's obviously an opportunity for Turing, but that's what they need in order to deploy these models successfully. Uh, at the same time as the model providers are uh, building out data sets in large part with your help across these four dimensions. The line between what a model provider does and, and what an AI application startup does is starting to blur. Pre Operator, there were a whole bunch of startups that sort of did what operator does and now that's just a feature in OpenAI. Uh, and I think that will happen to many, many more startups over time. How do you think about that evolution and what advice do you have for startup founders building application companies in this new paradigm?

Speaker A: Yeah, so you're exactly right Ashu. There are these two ends of the spectrum where on one hand we have the models becoming more agentic, where the models are having these general capabilities to do lots of different tasks, uh, themselves and then you have these point solutions with different startups pursuing um, these point solutions where you're automating one workflow for a nurse or you're automating one workflow for an investment analyst. Like there are all these uh, agentic startups with these very narrow, well defined use cases. Um, uh, the advice I would give for a startup that's super focused is um, um, master the data flywheel would be my advice. Um, if you look at um, Google search, uh, during the search wars, Google's self improvement rate for their algorithms was always a lot higher than the other search engines. Even though the algorithms were kind of commodity. Everybody knew PageRank, everybody knew everybody were using the same algorithms. But Google had like a lot more users using their system where they had clickstream data, other data to personalize stuff with, which becomes a moat. Um, so if you have ah, and when I think of some of these coding copilots like Cursor, Windsurf, um, Replit companies like that, I think the opportunity is if you um, it's like can one commoditize the other, right? Can the application um, layer commoditize the models, can the models kind of do what the applications uh, are doing themselves? And I think there's an opportunity where if you own the user relationship and you are just acquiring more and more data about this particular user to solve this particular model. And if you can find, if you are fine tuning, you have a data flywheel where you are getting proprietary data that you're using to fine tune your own models. I uh, think that could be a, could be a moat over time. Um, and I do think there is like wisdom in the sort of startup advice of focus, right? And the startup that is like laser focused on solving the problems of um, a nurse doing this particular workflow in a hospital will probably be better than a generic computer use agent that's doing that task for a very long time. And there's probably an opportunity. I think in a world where compute is commodity, algorithms are kind of, kind of a commodity in that like people move around and the information gets diffused. I think the only moat will be proprietary data.

Speaker B: I think that's well said. And you know, the thing that I've been thinking a lot about is in some ways the application companies for the last 20 years have been built by people who are workflow and UX experts. Application companies today actually have to be built by real technologists again because there's a data flywheel. But there's also a question of how do you extract value from the data. Some of the more interesting startups that are building applications are building intermediate data abstractions. They're taking the raw data, for example, if you're in sales, sales tax, uh, there's tons of raw data, whether it's zoom recordings or it's data in various systems. But the ability to take this unstructured data and connect the dots for a customer, for a vertical, for a use case that you're trying to sell and in different ways create slices of the same data. But you're creating intermediate data layers and intermediate data abstractions based on the unique data sets you have for an individual customer, but also across customers. And both data abstractions have to be created because some data customers will share and others they would. So I think. But that's way more technical than a traditional applications company has ever done. Like you have to think of data in a very different way. And then secondly, as you is, as you rightly alluded to, just like you are using reinforcement learning for large enterprises, every startup in this space will have to say, look, how can I take an off the shelf model and given the specific use cases that I'm sort of seeking to automate, how do I do post training for my set of use cases and how do I generate enough data in traces and possibly in some cases even uh, push the limits on the algorithms? Because a lot of these use cases have nonverifiable outcomes and you're dealing with the complexity of fine tuning uh, uh, for that environment.

Speaker A: That is exactly right, Ashu. And the other side of data is evals. And today really good benchmarks don't exist for a variety of different enterprise workflows. They just don't exist.

Speaker B: Right.

Speaker A: If you think of um, a benchmark as simple as um, extracting um, fields from a document accurately from a contract, or think of a task like um, analyzing um, unstructured medical information for underwriting, where you categorize this medical information as this person is high risk, medium risk, low risk. What is the benchmark for that? There's no sweep bench for that. There's no MMMU Pro for that. That's not what people publish in NeurIPS and ICML and ICLR on. So a focused startup that picks like a uh, real enterprise pain point, creates like a really good set of evals to optimize around. I think we'll win because the labs are not really aiming for these benchmarks, these niche enterprise benchmarks because they just don't exist. A lot of the data is proprietary. If you ask an insurance company and we work with some of them in many cases. Like the biggest fear that also holds back enterprise adoption is the enterprise companies don't want to give data back that could help their competitors in some way. They also realize that their model is um, their data is their moat. So you want like a custom solution where you've harnessed the power of LLMs, probably a smaller model with really tailored evals for their use case for their customers. And you're in a self improvement loop there where the model gets better as real humans interact with the model in a real use case. You see the, and the way I think of it, Ashu is like a triad. This is how at Turing we do when we work with clients. I'll pick underwriting as an example. Like imagine um, a human underwriter has a set of instructions they follow in insurance to figure out, okay, I've looked at this patient's medical history based on this. This patient is high risk, medium risk, low risk, it's a cardiovascular disease or um, it's a renal disorder. You can pick a disease. Um, so what we do is we train an agent to follow those same instructions. We have a human doing the task and we have an agent doing the task. In parallel we have that underwriter's manager seeing the outputs of both when the agent and the human disagree. If the agent is right and the human is wrong, it's an opportunity to train the human. If the human is right and the agent is wrong, you've basically created a benchmark, you've created a data set that's going to fine tune the next iteration of the agent. So you're on a self improvement loop where over time the agent is getting smarter and smarter, the agreement rate is quite high and the manager is accepting all the samples. Now the manager will have a Choice. I've gotten 100x more productive human or I could operate with 100 the headcount. That's the difficult sort of choice that that manager would have. Right.

Speaker B: And in different situations people will make different choices. Yes. Depending on their trade offs.

Speaker A: Yes, that's right. And so I think there's an opportunity for focused startups and Ashu. I want to get back also to your point on is reinforcement learning only for these super verifiable things like coding and math. Like how does it generalize for enterprise agents? Um, um. What we are seeing Ashu is like the rise of RL gyms where uh, enterprises ask uh us for these RL gyms, um, where we are creating this environment with prompts, verifiers, um, where we create either a GUI clone, where we've created these clones of websites like Dota, Uber Eats, NetSuite, Salesforce, um, all these different, we've literally created like hundreds of pieces of SaaS software, right?

Speaker B: Yeah.

Speaker A: So we've created these environments and they could either be designed for training a computer use agent. Meaning like it's a GUI clone.

Speaker B: Yeah.

Speaker A: Or it could be an environment for a machine to use for like function calling and tool use. And we have these expert humans who create different tasks and workflows uh, for how they would use these different tools to accomplish a task. And we create these verifiers and we dockerize it so that we can ship it to a client to run. And now these uh, just like humans train in a gym, these RL agents are training in these gymnasiums where they are trying different trajectories to come up with a sequence of tool calls or application um, uses that results in a verifiable output. And even though you may think that a work and you can imagine like if you have to plan your vacation, you can imagine like um, you can verify different steps in the flow. Did you log into Kayak? Did you access the right records from your calendar? So we kind of track system state the data model state in this virtual environment. It's basically a world model for a business. Uh, I think it's going to be crazy to see what these agents will be able to accomplish. And there's something really neat about self play. It's just really cool when you, it's kind of like the same techniques that DeepMind used for AlphaZero, um, that uh, we are about to use. We're about to see more and more for enterprise.

Speaker B: Jonathan, as we start to wrap up, I had a couple of sort of maybe closing comments. You know, from your perspective, what is there's so much hype in AI so that I hesitate to say this, but is there something that you think is still under hyped despite all the hype?

Speaker A: The fact that every white collar knowledge work. If your job involves looking at a computer, operating a keyboard and a mouse, if your job involves looking at a computer, analyzing some data, using some tools to take action and take a decision, it will be automated in three years. Like if we let that sit in, that's like $30 trillion of knowledge work that's about to be automated. I actually think it's going to be great for humanity because with every human being 100x more productive, uh, you might ask me, Jonathan, what would you do when you've automated yourself? Maybe I'd run 100 companies in parallel. And um, I think it's really hard and difficult to sit with the fact that everything that we thought was complex knowledge work would be automated. And people say it, but I don't know if they truly get it. Um, and it's uncomfortable, right, because there will be job transitions that involve people getting upskilled, reskilled. I think every company, and I'd love Ashu at some point. Maybe we should do a project where we analyze this data. My hunch is companies today are a lot smaller for their stage. We could look at companies that reached a million in ARR or 10 million in ARR. How big is the team? My hunch is the teams are a lot smaller.

Speaker B: I think you're spot on. While there's so much hype about AI, I think the impact is underappreciated. And I see that around me every day, uh, including at foundation and at our portfolio companies. When you talk to the, the marketing team of an AI company, they're still not using enough AI for their marketing. You talk to the finance team of an AI company, They're still not using enough AI in their finance. And I think every individual, no matter what your job is, if you, if you have a white collar job of any kind, you have to ask yourself, how do I prepare for a world where my productivity will have to be 2x at a minimum, and for some people, 10x. But let's just start with 2x in the next three years. If you're not twice as productive in three years, like, there's going to be a problem. And I think that's a great thing for society. But it's also. You have. People have to let that sink in. And I don't think that's actually sunk in. In society.

Speaker A: It's not sunk in. I think the relationships humans have with work is going to change, which is why I'd say if you want to work hard, come join Turing. Time is running out. It's like the swan song for like, let's build something awesome. Uh, but I'm joking. I do think it's going to happen. You're right, Ashu. The impact is kind of underestimated. And these models are only constrained right now by compute and data and every type of advanced knowledge work you can think of. You can imagine at Turing, we are hiring humans like that to train the models to do that. I think an interesting thing that'll happen is all of us will be training models in our spare time. I think it'll also be something that will happen. It'll become economically viable to do so. Absolutely.

Speaker B: And it'll become technically feasible. People will download a model and say, I have some proprietary data. I'm going to fine tune this model. I mean, my 14 year old is doing it, so hopefully everyone on the planet is doing it in a couple of years.

Speaker A: The positive side of it, Ashu, if I think of what's the test for artificial superintelligence, or, uh, asi, I would say it's the ability of these models to discover new science or make discoveries. And it'll be an incredible world we live in where when we've scaled intelligence to a point where we can cure diseases, speed up drug discovery, understand the universe better, maybe we'll solve interstellar travel. Like we'll do other funky crazy things. If you solve intelligence as an API, it's the ultimate meta problem to solve. And huge kudos to Sam Altman, Greg Brockman and the OpenAI team were embarking on this quest that kind of set off the entire industry. It's the grandest challenge of all.

Speaker B: Thank you so much. Jonathan, this was a phenomenal conversation, and it's been so cool to see how even your own philosophy around, um, leadership has evolved so much in the last few years, uh, and how much the company has responded to that in terms of rising to the opportunity.

Speaker A: Thank you. And it's like good imitation, learning from other CEOs, also doing a great job.

Speaker B: Thank you.