The Moving Bottleneck: Networking, Power, Memory, and the Race to Win AI

Speaker A: and i think we will need almost an infinite amount of compute in the

Speaker B: years ahead silicon is back in silicon valley for a period it was called software valley and now everybody only talks

Speaker A: about semiconductors right my view on this is that incumbents become very successful and become big and then what happens is that when they become big it becomes hard for no fault of theirs to target smaller size markets you know some

Speaker B: of these large hyperscalers they don't want to bet the farm on nvidia even though that's worked out very well for them they don't want to be in the nvidia jail hi everyone this is the techsurge deep tech podcast presented by celesta capital each episode we spotlight issues and voices at uh the intersection of emerging technologies company building and venture investment i am sriram viswanathan um founding managing partner at celesta capital if you enjoy tech surge now is a perfect time to hit the like and subscribe button and while you're at it you can leave us a review on your favorite podcast platform if you're just discovering us visit techsurge podcast dot com to sign up for our newsletter and check out the archive of some very interesting past

Speaker C: episodes at celesta capital we back emerging tech companies and stay close to them long after we invest our team is constantly coordinating work across the portfolio and agents increasingly support us with the recent launch of custom agents notion became the collaborative ai workspace where teams and agents work side by side now their new developer platform is turning that workspace into infrastructure that developers can build on the developer platform ships with new primitives to sync any data source into notion build custom tools for your notion agents and orchestrate any agent inside the workspace with the external agent api we bring our own in house agents into notion as native participants with their own triggers tools and permissions they can track portfolio metrics and milestones flag what needs attention and we review and collaborate in our normal workflows unlike other platforms notion is built for teams with shared context and permissions from day one learn more about notion's developer platform today at uh notion dot com notion tech surge that's all lowercase letters notion dot com techsurge to try notion's developer platform today and when you use our link you're supporting our show our guest today is an entrepreneur who has spent the last thirty years aligning his career to every new mega trend in technology rajiv kamani was one of the founding executives of security company cavium that went public in two thousand seven and sold to marvel a decade later for six billion dollars he co founded innovium which built switching silicon for the cloud giants today he leads low power compute company valora ai and is also co founder and executive chairman of upscale ai a networking company that has raised five hundred million dollars in under a year from prominent financial and strategic investors we discuss how rajiv identifies the choke point in each technology wave why a semiconductor startup now requires ten times the capital it did even a few years ago and the hardest calls to make as a founder including knowing when to walk away his insights are valuable to any young entrepreneur or investor operating in this incredibly fast moving space today welcome

Speaker B: rajeev nice to have you here on

Speaker A: this podcast thank you shiram it's a pleasure to join you today so let's

Speaker B: start with your entrepreneurial uh journey so this is an interesting uh area to sort of uh delve into because i think in every one of the companies that you've gotten involved in they've almost always preceded by a big industry trend uh in the case of netboost uh it was the time when networking and enterprise networking became a big deal and cisco was really pushing that very hard and then comes the cloud and the likes of juniper and arista maybe uh arista was much later but juniper and the guys that were really pushing cloud infrastructure were really sort of running into the same sort of networking issues and then of course subsequently when you started innovium and the other companies that we'll talk about in the ai wave there's lots of uh ai trends and the thing that strikes me is that you have been able to identify the constraint that exists in each of these mega trends so talk about that how do you come up with uh it's almost like theory of constraints finding what is that little hard problem to solve even though the trend is very large you managed to hone in on that so

Speaker A: talk about that yeah and you get better at this uh with time certainly right and especially as you look backward you see the learnings uh that happen right so i'll give you a few examples on how this happened so at cavium as an example um we had some in all of these stories by the way there are amazing people on the team that work with you it's always a team um effort rather than an individual but we had um a team that built security acceleration that was actually twenty x better than the next best option and in the early days we said look the world doesn't need ssl as much uh you typically go and encrypt your credit card only when you're buying something for rest of the pages you really don't need to encrypt it right and so normally most people would say yeah that's true and when we came up with this twenty x um there was initially this fear that companies have that the tam is going to shrink because you need very few chips to do it um but in the computer industry time and again we've found that that's not true people tend to use up uh the technology that becomes available the speed that becomes available so here what happened is that i remember distinctly going to f five in the early days and i said what if you made every web page https through your front end application delivery this

Speaker B: is the highest level of security and encryption that you could offer at the webpage regardless of whether it is the underlying content uh is financial transactions or

Speaker A: not or not exactly so we basically tried uh to make everything secure on the web traffic for a variety of reasons there's privacy reasons and so forth nobody should know what web pages you are traversing as an example because that's your private information and uh so uh f five said my god this is too expensive we don't want to put it on every box but we said what if we gave you a price that made it possible for you to make it a default option um and we had a number m of discussions and they said okay if you give give us a better price we'll put

Speaker B: it on a larger sorry who's this

Speaker A: this is f five network f five networks a leading provider in the internet and even today a leading company in this space providing application delivery sitting in front of the web pages and web servers and we changed the world from being a few percentage of ssl traffic to now ninety plus percent being uh and so the thing is that in order to do that what you do need to do is you need to experiment you need to know you need to try and then things happen um more recently uh in the ai wave for example um actually before you go

Speaker B: to the ai wave let's just linger on this uh ssl encryption phase so again going back to the big industry waves that happened so the compute wave as you said you sort of missed that even though you were part of intel at that time as part of netboost and everything else that compute wave was on a tear with intel and amd then comes cisco in that same period uh in the networking space and they were actually working quite a bit on security and the whole ios strategy that cisco had had a lot of things related to security and they didn't really take advantage of it and then comes cavium and f five and others so in a way in that period um you still saw incumbents sort of slipping and some of these new innovative startups uh there was something unique about those startups that were able to do talk about why is it that the big guys missed it because you know you had rsa you had you know uh other players in that period and they all missed what you built during the cavium days yeah i think the

Speaker A: reason they uh missed it is that incumbents in my view on this is that incumbents become very successful and become big and then what happens is that when they become big it becomes hard for no fault of theirs to target smaller size markets and typically all markets that are emerging and small and that's where i think startups have an opportunity and incumbents frankly in many cases are okay with that because they have the opportunity to acquire those companies as happened

Speaker B: in the case of cavium which got

Speaker A: acquired by marvell marvell and so i think that the interesting opportunity for startups is to uh have a perspective on what could happen and why it could happen and try start down the path but at the end of the day you have to be honest with yourself and you have to be able to adapt and pivot along the way which is very critical uh and that's what we did as an example so many examples of how we did it but the idea is that you ought to have a perspective a perspective that a big incumbent really won't jump uh on uh you have to assemble the team and the capital to make it happen and then you go down that path but along the way be flexible to

Speaker B: adapt yeah i think uh identifying the market uh you almost have to pick a market which at uh start uh uh almost paradoxically shouldn't be too big because if it is then the big guys are going to focus on it it has to be niche enough that you can actually establish your beachhead and expand into it then at the right point in time the big guys as in the case of cisco or marvell and others they jump in and it turns out to be a great outcome just in terms of uh numbers cavium uh at the time when you took it public uh i mean just for full disclosure you were the chief business officer chief operating officer operating officer one of the early employees you took it public at the time when you took it public the company was north of one hundred million dollars in revenues actually

Speaker A: when we took it public believe it or not it was less than one hundred it was growing very fast so some people told us maybe we were doing an ipo too soon but we were less than one hundred million dollars run rate but we were trading at that time um twenty thirty x of revenues um but what we had was what cavium uniquely had was a very broad set of design wins across a bunch of secular high growth areas and we had visibility because these design wins had a long life cycle had revenue and growth visibility a lot of stickiness

Speaker B: in that revenue because once you're designed in it's very difficult for you to be designed out so that happened uh in was it two thousand seven six seven yeah and uh you took it public and the stock went up ran up quite a bit over six seven

Speaker A: billion dollars right uh eventually got to six plus billion dollars but along the way as you know we were not immune to to the two thousand eight financial crisis we went down during that

Speaker B: period as well and then marvell acquires

Speaker A: it marvell acquires it so marvell acquired

Speaker B: it for just product enhancement roadmap enhancement at that point in time they were not in the security business they were

Speaker A: not in the security or the infrastructure business i mean infrastructure was a very small portion of marvell's business they were mainly consumer and hard disk uh type business and matt uh murphy who is uh marvel's ceo is a visionary and he basically made the bet to transform the company to be more of an infrastructure company and cavium was one of his first ah um bets in that

Speaker B: area so now fast forward and then comes uh before the true ai wave even you're still sticking around in the whole networking switching routing space because you end up starting another company at that time uh innovium uh which is much later but talk about innovium and why what the opportunity was at that point

Speaker A: yeah so i mean the innovium story was actually again very simple story which is that as we were moving into the cloud wave it became very obvious to us that the first generation of cloud was really built by uh components semiconductor components that were really not designed for the cloud they were designed for enterprise or for telco or other applications uh and that by the way applies to not just chips but software and systems and everything right and so the cloud companies um use this to create and they put it in a warehouse data center and a cloud but it became clear that the requirements for this use case were different than the requirements for enterprise or for telco you didn't need to add many features the scale was very different uh the uptime requirements were very different uh frankly what's interesting is telco had very high uptime requirements but the cloud could tolerate for each individual server to have lower uptime because they could replace it and build a redundant system so the requirements were very different for cloud and we wanted to build a purpose built product for that and we obviously um the other thing is you leverage what you know and the talents that you can accumulate and the problem statement where you are best suited to solve it um and we had a team amazing team um my two co founders from broadcom and we said let's build a purpose built solution for the cloud and that's how we

Speaker B: got started yeah but at that time there's another big industry wave that was happening because prior to that prior to the whole cloud explosion when the hyperscalers ended up becoming uh some of the largest oem or white box uh customers you only had to go through dell hp ibm and others to actually bring in either networking or server related products but now you had facebook and amazon and google and others so how much of the market dynamics uh really played in your thinking of targeting a uh switching routing solution because now you had a plethora of customers in the previous generation there were only very few only a handful how did that play into

Speaker A: your yeah i think it's good and i would say uh having gone through this it's good and bad okay so the good part about it is that the hyperscalers really have become so big that each hyperscaler is a market segment by themselves um so if you have the right connections you can talk to them you can get a lot of clarity on the product requirements and if you execute uh you can actually ramp up your revenues much faster so that's the good part about it i mean

Speaker B: in this case you were really not really selling into the dells or the hps of the world you were going directly into the amazons of the world yes and they were specking what the product was going to be and you may have had another uh customer like an hp but you started paying um more attention to the end user which happens to be the hyperscalers and they were dictating product requirements much more than

Speaker A: the traditional oems yes so that's absolutely right so that's a good part about it it's amazing and that's wonderful obviously the not so good part about this is that if you're building a company uh committing to a lot of capital and you only have five customers and these customers never make commitments until you deliver the product and so you could end up spending a lot of time and capital and if you don't get one of these five then that's tough

Speaker B: luck but the other problem is one of these five guys pick you it's very unlikely the another guy picks you at the same time is that a

Speaker A: fair statement i would say that um no you could have multiple i mean historically it's been the case that you can't win all five for sure because these guys have such demanding requirements not just for from product but from supply and support and things like that so you could win a couple and couple would be just amazing um customers to have um but uh yes but that's what it is that's what the cloud era did um and i think now we are moving a little bit to an era where things are getting a little bit more right but before we

Speaker B: go to that era given the fact that you've actually worked with the traditional oems uh the hps of the world cisco's of the world cisco's of the world uh versus the cloud guys i know that you are now in the business of selling to some of them so i want to be careful about what you are able to comment on how they are as a customer but in that period what was helpful to you to sell to the hyperscalers versus the traditional guys was there any difference in how the sales process worked and how you approached them and how you actually were able to get design wins with them was that one was easier than the other and if so why

Speaker A: i mean i would say that when we were building i mean there's a lot of other factors right so first of all we were building an amazing product and um that product actually fit one hyperscaler better than some others you know so if you're if you're doing some of these switching things there's clearly bandwidth right what performance do you want to put on top of rack which is the you know the switching uh networking product in iraq that uh connects all the servers uh so the requirements are different right and so clearly the product we were building was better fit for some folks um there were also obviously business and market factors right so people as we all know right when we started the company broadcom was and is a ah eight hundred pound gorilla and does their own strong business uh methodologies and customers want choice they don't want to be um committed they were

Speaker B: your backend provider right at that time

Speaker A: so they were a backend provider but in the market they were a competitor and so it was like our customers wanted a dual source strategy so that they're not reliant on one uh company so there was a factor in us being selected um and of course it was also a situation where some of these bigger companies do bundling to win the customers and becomes challenge for a new startup to enter so there were a lot of factors that went into play in that market a lot of learnings by the way that we are applying into future companies um so this

Speaker B: period was a very stressful period for you one side you are trying to deal with the eight hundred pound gorilla uh aka broadcom on bunch of the backend issues and you're dealing with them on some ip issues and then you were also dealing with the hyperscalers and in parallel you were also trying to raise capital and that was probably one of the stressful periods in your life i've sort of interacted with you but the reason i mentioned that is that you did something very profound which i find very unique in entrepreneurs and we can touch on that more because there are m many similar instances you have run into we're going to talk about in the ai world and that is to know when to fold you uh were on a tear with one of these hyperscalers who had a huge order book with you and suddenly that order book changes dramatically and you were facing an existential issue so let's talk about

Speaker A: that yeah well i would say we had and that continues to be a great product in the market with an amazing product we also had an incredible order pipeline that was not going away but we were in the middle of

Speaker B: this we're talking about innovium innovium and you had a hyperscaler and i think we can mention who it is it

Speaker A: was one of the biggest hyperscalerscalers one of the biggest hyperscalers in the world one of the top two three biggest hyperscalers and so we had lots of orders we had a great product but the challenge is because of a variety of supply issues and you know uh other players in the market we got into a supply related limitation yeah and it was so bad uh shiram that you know we used every help we could you know i i reached out to jensen i reached out to pat gelsinger to qualcomm to microchip to cadence and yeah yeah every major company to be to beg for supply and there

Speaker B: wasn't any you were asking for supply for some in some cases it was ip that you were trying to fix in serdes and others in some cases it was other for the product we

Speaker A: were shipping there was no serdes issue there were other issues on follow on chips we had serdes issues but i think that in that we had a supply issue and on the supply issue um what we were able to do was that again uh look the the help that we get through the investors and the board is just invaluable and you gotta respect that and be grateful for that but we were able to get an alternative supply source that we were able to build however our big customer uh and their supply folks got a little concerned jittery jittery so to speak and they started to look for alternatives and whatnot so that was one and we were working on a follow on roadmap product relying on serdes from another vendor and that serdes was we were the first ones to use that serdes and the serdes had issues so we were hit with two big issues at the same time one is the inability to supply an agitri customer and then um you know serdes issues for our roadmap and at that time um you know that's when we decided that it made sense for our company to be part of another big company and and again our customer helped in that uh process and we were able to get a very good outcome for everybody

Speaker B: uh and you sold it to marvell for a good billion plus i want to just hone in on that point that you made where you were dealing with supply issues you were dealing with ip issues you were dealing with customer concentration which at that time this was one of the largest hyperscalers as you mentioned they had a one hundred million dollars sort of a run rate for you uh and you sensed that this was going to be a problem and you came into the board and uh i recall this distinctly when you said you know what we got to sell uh some other entrepreneurs would have said you know what let's go find another hyperscaler or let's go do something else what was the thing that actually drove you to make that decision to say let's fold it and we'll win another day and win another battle another day you did that yeah i would say

Speaker A: that um the uncertainty of what twelve or eighteen months out the world would look like for us was a big factor and we were kind of trying to look ahead and see what were the possible scenarios for the company and so we felt like the best scenario again by the way looking back in hindsight if we had been able to convince uh people and power through that with the benefit of hindsight i think we would have been a big company and the ai wave would have turned us into a big uh successful company but at that time when we had to make that decision we looked at what the uh possible scenarios were twelve eighteen months out and what are the possible probabilities associated with those and we said what's the best decision knowing that and that's how we made that decision

Speaker B: and you had a good group of investors that supported you and all of that we have actually gone into a similar situation in some of the other companies that you're involved with and there's a learning to take from that so you end up in marvell for some period when marvell acquires innovium and somewhere in that period you start thinking you're branch predicting and you're saying well i can just retire and do my own thing or i can do a bunch of other things uh in the ai landscape and that's when we meet again but when did you decide that you had another you had another bite at the apple as being an entrepreneur when

Speaker A: did you decide that um i had decided that i was going to spend at least um one or two or few years after that helping other entrepreneurs um it was incredible learning which i benefit from every day today um and i connected with barun kaur who was on the founding team of palo alto networks and he was very interested in attacking what was at that time the web three space and because uh the cloud was well into its um many years into the cloud era we were trying to see what's coming next for that window of time there was this big uh um hype and buzz about the web three and uh how blockchain technology was going to change uh how we did a lot of things on the internet he reached out with an idea of building a chip uh in the blockchain area we had a customer that was interested in being a lead customer for that product and we were looking to get this company started and that's when we met you sriram right so uh my original thinking was that i was just going to be maybe an advisor or a board member in the company and help them out but uh as you know very well one thing led to another and i ended

Speaker B: up this is an interesting story because i end up talking to you about this and you were reluctant to actually get your hands dirty as an operating exec you you wanted to be an investor you wanted to be uh an advisor and barun was very successful he was one of the founding members of palo alto networks which was a one hundred plus billion dollar company but i had not known barun and you had introduced me to him so blockchain was the big idea but then you leveraged uh the hype that was going on in bitcoin so somewhere in that period you decided uh there's an opportunity in crypto so explain that and what changed

Speaker A: since then yeah so originally you know when we jumped into this uh crypto bitcoin space uh what we found was that doing these bitcoin chips even though the logic and the algorithm is very easy for doing this bitcoin which is sha two hundred fifty six a mathematical uh algorithm what uh was really uh needed was uh technology that people refer to as near threshold voltage which means that you build semiconductor chips that run at much lower voltage than the rest of the chips that uh are used in the world and by reducing the voltage you get a substantial benefit in power because power is related to the square of the voltage you get a benefit but what happens is when you reduce the voltage you get hit with yield with reliability with uh reduced frequency

Speaker B: which reduces performance performance essentially it takes

Speaker A: a hit on performance takes a hit on performance and you have to then do innovation to recover all of that back so as you bring the voltage down you have to do separate innovation to recover the yield the reliability the performance uh of this stuff and very different and it requires very close partnership with the foundry because it requires in many cases to work with the foundry and understand the characteristics of the uh semiconductor manufacturing equipment and work very closely with that and so that's what we set out to do and we got that started in twenty twenty two and we did multiple iterations of that um and we perfected that technology over multiple chips we've shipped about thirty five million of those chips deployed in the field amazing technology we have one hundred patents plus and so forth but that was vector one and we were in addition looking at other things in the web three point zero but the chatgpt moment happened and that happened in late twenty two early twenty twenty three very quickly it became clear to us that my god this is going to be um an amazing change in the infrastructure and it became very obvious several months later people started talking about it being bigger than the internet or the cloud era and we said where is it that we can make an impact and where is it that our team has the talent where is it that we can build a perspective given what we know of the industry that's when we started into moving into the ai with security being the first area we focused on

Speaker B: actually before we talk about that so just to get our audience grounded on the companies that we're talking about obviously we discussed cavium which was in the switching routing space in the networking security space and then we talked about innovium which was really more ethernet switching uh for again the cloud era and then you got into uh autodyne which is now called velora and that was more in the space of blockchain and you looked at bitcoin uh and really drove power uh to be lower and be able to do pretty significant amount of uh crypto mining infrastructure out of that came the understanding of the value of low power and security um but before we talk about companies that resulted out of that wave in the bitcoin space linger on the bitcoin space where do you think the bitcoin space is today because one of the key value propositions you had was built in america for this new infrastructure which up until then was still dependent on china so how much has that changed and what is the lay of land right now in

Speaker A: the bitcoin space yeah so bitcoin still is quite heavily dependent on chinese uh suppliers uh and our goal was to change that but uh what has happened with bitcoin as you know bitcoin is a phenomenal technology and a phenomenal protocol and uh you know i would say that in the four years since we started more people have become aware of it being a technology and not just a um ponzi scheme as some people believed it to be um and they recognize the technological um marvel of it being totally decentralized there's no bank that controls bitcoin it's completely decentralized um and uh the compute provides the capability for adding new transactions in a very decentralized fashion so that's like the other interesting thing is that since its inception which was about uh you know ten fifteen years ago it has not been hacked so it's been very secure it's proven to be extremely secure it's an amazing technology now having said that the predictions of bitcoin price were a lot higher than where it is today i mean there was this uh sense with the trump administration that the crypto was going to be much higher value than it is today there was going to be a strong push by the us government to get crypto hardware being built in the us now i think what ended up happening is ai came and took everybody's mindshare away and so um both from an investor point of view uh people are investing much more in ai than they are in bitcoin so bitcoin exists it will continue to exist but price uh has come down bitcoin uses energy to secure the the network that it has and the protocol it has uh but what has also happened is ai has an insatiable need for energy so now we have ai competing against that bitcoin energy and the bitcoin data center companies trying to transition to becoming ai data center companies i think i'd

Speaker B: love to hear your perspective on this because i sort of see this similar to what you're saying which is bitcoin and blockchain as a innovative technology for a decentralized infrastructure is a natural evolution of what web three point zero is attempting to do so those are all industry trends where aligned in the perfect way for companies like autodyne and others to actually leverage in addition to that you had the benefit of the chinese reliance which was a uh big no no who knows where the current administration is in that context putting that aside the bigger point is what you mentioned which is the dependence on energy became the constraint where when ai took over uh or the wave started with ai they were competing against this scant resource which was energy so inherently even though bitcoin by itself may not have lost its shine it was a headwind when you are comparing it to ai which is also competing for their energy do you see that changing now that many more gigawatt plus uh uh energy uh infrastructure companies are getting started do you see a way for bitcoin infrastructure or i should say blockchain infrastructure to really have a more get a stronger position relative to ai in the near term or is that something that it's a

Speaker A: lost cause um you know there are obviously companies first of all it's very hard to predict bitcoin prices right um right you know if you could predict bitcoin prices i'll give you an exact answer to that question yeah but regardless

Speaker B: of bitcoin prices the energy part of it is independent of the bitcoin prices even though the viability of the underlying business is dependent on the bitcoin price you know the cost per kilowatt of energy that i'm going to use that is independent of the bitcoin price no

Speaker A: so the way i'm looking at it is the following which is that both bitcoin and ai take energy and convert it to some economic value both of them do that and what is happening is that today at this point in time given bitcoin's price and so forth if i have the ability at any given point in time to pick a or b today ai gives me a better return on that energy if i look forward into the future let's say bitcoin does very well ai gets commoditized there could be conceivably a period in the future where bitcoin then becomes more attractive again um where you take that unit of energy and put it into bitcoin now in the case of ai it may not be as simple to dial it down bitcoin you can dial it up and down uh instantaneously ai you can't dial it up and down instantaneously because people are dependent on that ai right but but on the margin if bitcoin prices go up there could be a better economic return on bitcoin uh mining okay but that's bitcoin let's talk about blockchain um separately than bitcoin blockchain is a technology which if you look at ethereum and the various other uh blockchain protocols that actually are not um proof of work based not compute hardware based but that can run in a cloud environment i think are ah poised for massive growth and uh the thing that is going to drive that is tokenized assets i mean we have not even started that uh wave yet where we believe that the timing is hard to predict but if you look forward five ten years down the road every asset will be tokenized and we will be able to transact in a way that we haven't been since the beginning of time and blockchain will enable

Speaker B: that so this is interesting because i had in one of our earlier podcasts we had sec commissioner hester uh peirce uh on the podcast she's the commissioner that is uh in charge of regulating stablecoins and tokenized assets and all that and she had quite a few things

Speaker A: to talk about this some people want tokens per second as another metric which is why you're seeing some segmentation of the ai compute market um and i do think that there will be further segmentation of of the ai compute market it will be heterogeneous will require open standard networks and we'll come to that in a second but i think that in the case of bitcoin of course it's dollar per terahash but there are sorry uh joules per terahash is energy efficiency is one important metric another metric which gets into this equation is how much is the global compute being applied to that and so um that gets into play also in terms of how much revenue you can generate what is

Speaker B: the overall hashing available in aggregate for mining uh for bitcoin that's what you're

Speaker A: referring to that's what i'm referring to and what happens is that bitcoin you can only generate a certain number of bitcoin every ten minutes and that gets spread across all of this compute so as a result the more the compute that is deployed you get less yeah right and in some ways by the way i think that's a very attractive aspect of the bitcoin protocol because it self polices the amount of compute that's applied to this um problem so what

Speaker B: i'm really getting to is in the bitcoin space there is a aggregate amount of network hashing that is there and that grows at certain level but in the ai space there's no such thing you can throw as much compute as possible that doesn't seem to be a compute constraint uh and there's just an unbounded uh appetite for more compute so is that the right way to think about it and can you comment on

Speaker A: that it is it is i think that um in ai we just haven't we're just starting right we are in the early days and we know that people talk about agentic ai uh which will be a huge driver of uh more and more tokens because today what happens is tokens are um used or generated when we type a prompt and humans can only do it as fast as you can type but these agents can be up twenty four seven doing things and that will be a massive um source of the need for tokens and i think we will need almost an infinite amount of compute in the

Speaker C: years ahead at celesta capital we back emerging tech companies and stay close to them long after we invest our team is constantly coordinating work across the portfolio and agents increasingly support us with the recent launch of custom agents notion became the collaborative ai workspace where teams and agents work side by side now their new developer platform is turning that workspace into into infrastructure that developers can build on the developer platform ships with new primitives to sync any data source into notion build custom tools for your notion agents and orchestrate any agent inside the workspace with the external agent api we bring our own in house agents into notion as native participants with their own triggers tools and permissions they can track portfolio metrics and milestones flag what needs attention and we review and collaborate in our normal workflows unlike other platforms notion is built for teams with shared context and permissions from day one learn more about notion's developer platform today at uh notion dot com techsurge that's all lowercase letters notion dot com techsurge to try notion's developer platform today and when you use our link you're supporting our show the four biggest cloud companies have committed to more than seven hundred billion dollars of capital spending in twenty twenty six up roughly seventy seven percent in a single year most of it buys accelerators but every accelerator is only as good as the network that feeds it and that layer has become a market of its own del oro expects ai back end network switching to blow past one hundred billion dollars in cumulative sales by twenty thirty and called scale up networking possibly the largest new market the chip industry has ever seen today the dominant fabrics are proprietary but support for open source alternatives is building quickly ua link published its first spec last april ultra ethernet followed in june and in october twelve companies amd broadcom cisco meta microsoft and nvidia among them launched an effort called isun to push ethernet into scale up one hundred seventy five companies have joined since while promising this wave seems to be very much in the early

Speaker B: stages so let's talk about the ai data center because this is the next big wave of infrastructure build out uh and as a consequence uh there are lots of companies that are trying to solve the compute problem they're trying to solve the energy problem and they're also trying to solve the networking problem and you've actually you're leading a company that got spun out of velora which is called upscale and you had a particular problem statement that you were going to go solve which is in the scale up architecture so define what the scale up is and what is the problem statement that you're trying to solve yeah

Speaker A: uh so you know what has happened uh is that um we are in this world of ai where ai gets trained on the world's entire data the entire data that's in the internet right and we all know that that's a lot of data it's hard to fit this data on one chip so these models that try to um get trained on all this data are big you know we are talking trillion parameters or m multiple trillions of parameters and it's very hard to fit it on one chip but what in order to really make you know a very large computer you need many of these chips connected to each other in a network all operating as one giant computer operating on a very large model okay so that's the uh basic problem that we are trying to solve and these these models then give you the answers to the prompts that you ask so scale up is a uh technology that connects multiple gpu's or ai accelerator chips together into one domain and that domain could be one rack people are trying to make it go to multiple racks and get to larger scale uh nvidia for example has an nvl seventy two where seventy two chips are connected to each other through a switching network inside the rack through a ah switch that they refer to as nv switch and that's the scale up domain uh yeah so your

Speaker B: premise um here is actually twofold one you're obviously betting on the fact that nvidia is not like winner takes all nvidia is not going to be the only gpu provider or a rack provider and there are going to be multiple uh participants in the industry providing compute that's first premise and the second premise is somebody else who's building a rack is going to need a networking connectivity solution and you're betting that that's going to be an open sort of architecture versus some proprietary each vendor is not going to build their own networking stack and you're going to build a standardized product and sanitize capability for connecting and that's those are the two premises that

Speaker A: you're making yeah correct i mean if you look at the general purpose compute world right and if you look back in time in the general purpose compute world there were different types of networks right ethernet was not the dominant uh always there was atm there was fiber channel you know there was token ring

Speaker B: x twenty five so there were a

Speaker A: lot of different networks but what happened is that the world converged on ethernet as a single standard so when you build uh a cloud data center you don't think about oh for me to deploy intel or amd or arm i need a very specific network for that you just build a network and you can go and plug in compute into that network and you can grow over time okay we think the aha moment was that we think that the ai world will look similar and the idea is that you know nvidia does products that are amazing products and by the way they're a good partner and investor in our company but so they do amazing products of certain types you know they are the king when it comes to training they do certain types of inference very well um so um yeah so nvidia builds certain types of ai chips and they are the best in the world for that training and certain types of inference but we do know that there is room in the world for chips that are purpose built for specific applications so the hyperscalers are building their own chips there are a variety of companies building chips uh in this era and you saw nvidia's acquisition of grok really validated that where they look at groq as a solution to speed up tokens per second and so we think that the world is going to be heterogeneous ai compute and to have like a common scale up network is of benefit to everybody i think i

Speaker B: buy that argument but in your comparison in the prior examples that you gave token ring or ethernet or uh atm or any of these things most of those things were industry led consortium this is probably the first time a compute provider builds its own networking architecture in the form of nvidia nbl seventy two and defines that uh as the main stay uh of networking that's different the fact that they own ninety five percent of the market segment you uh don't see a scenario where some open version of nvl becomes a standard but instead there'll be an ethernet derivative for networking

Speaker A: i think uh nvidia had to do it right because there was no open standard before that right so they built their own so for them it was a must have requirement but they could

Speaker B: have used ethernet at that time no

Speaker A: the thing is it's a very fundamentally different type of network it's a completely different set of requirements just like cloud has different requirements than enterprise the ai network has very different requirements and they build it purpose built for that use case and i think that the rest of the world doesn't have that option so so the choice that every compute vendor now invests heavily into a network of their own doesn't make sense i mean it makes sense for a corner uh we also see that today nvidia is a dominant player the hyperscalers like google and amazon and others are building their own xpus and they need a network to connect it in some fashion and some of them are going towards open standards some of them have their own proprietary uh networks but i'm saying if you look forward far enough uh several years out into the future there will be an open standard the general view is that people are trying to do a variant of ethernet which they call esun ethernet scale up network so that they can leverage the management capabilities and the familiarity of ethernet but fundamentally put different attributes in that switch to make it interesting for ai networking so

Speaker B: just to define this scale up architectures is really about connecting within the rack the different compute subsystems that exist people

Speaker A: are trying to go outside the rack

Speaker B: also and the scale out is outside the rack into multiple racks is there a reason why these two network architectures

Speaker A: have to be different i mean in theory if you could build a very large scale up um you could combine and these two things could um converge but the reality is that uh uh today or at least in the coming future what we are finding is that um you will have small clusters and when i say small could go as high as hundreds of gpu's or maybe even one thousand gpu in one cluster or a pod and then to connect it to hundreds of thousands of pods of pods you need a different network

Speaker B: right so that today is infiniband that

Speaker A: today's infiniband or actually also ethernet both so that is infiniband or ethernet and that ethernet is also slightly different than the front end network it's a scale out ethernet doesn't have all the features that you need for the ethernet or for uh the enterprise or the cloud ah but the characteristics of the scale up network which are very critical um and which is why you can't have very one large scale up network is that these are all memory semantic transactions not packets latency matters quite a bit tail latency is critical so in the regular ethernet network you can have tail latencies and tcp protocol adapts to it here you don't have that luxury um the kinds of operations you're doing some of them can be offloaded into the network through some in network computation capabilities so the characteristics of this network are very different and to have like one very large scale up network um you have to sacrifice things like latency and

Speaker B: things like that yeah so as we move to more on this um inference workloads where a bunch of the workload is actually ending up becoming memory intensive um on the inference how does that factor in the architectural implementation of scale uh up architecture or scale out where memory becomes an important parameter in this thing to do that you actually need on the same blade from the gpu your going offline to memory and you don't really need to be thinking about the networking infrastructure is that the right

Speaker A: way to think about it no i think see today what people do is they try to put as much memory as they can on one xpu or gpu right and they use or at least close to it yeah they use hbms or they use ddr memory which is dram memory and they put as much as they can within one chip but that's not enough and so what you do is you today what you do is you shard it or you spread that across lots of chips then what happens is any gpu can read and write from any other gpu and

Speaker B: you really have to solve the coherence problem because you need to make sure that everything is in line and all

Speaker A: that right i mean there's some interesting things where in your traditional ethernet in order traffic packet uh makes sense here you don't really need in order because you're all addressing different memory locations and the other gpu's uh a memory ah domain um so you can do that now what has also happened is that we are getting into a world where memory is very limited and the uh memory companies have become trillion dollar companies there's a shortage of memory going forward so people are trying to say what can you do to share memory across and this is where people are looking at innovative solutions to plug it into the scale up network and have all these gpu's and xpus share that memory and we'll see products in that area in the coming uh year or two

Speaker B: clearly nvidia just crossed five trillion dollars uh yesterday micron uh crossed a trillion dollars earlier in the week so it's obvious that the market is becoming aware of uh it's almost like theory of constraints compute was the constraint value got created and nvidia is uh just killing it in that and memory is the next is it sort of natural to think about networking and connectivity as the next opportunity for value creation and if so uh who are the incumbents that are well positioned and obviously startups like upscale that you've created and nextop and others that are there but is there an obvious winner uh that could play as an incumbent in the networking space

Speaker A: you know clearly there's existing players like the aristas and cisco's that will and are doing things in the ai networking uh era but i mean there's something to be said about purpose built grounds up solutions right and arista became very successful being a purpose built grounds up cloud networking provider we are trying to be a purpose built uh grounds up ai networking provider and when you do that you can actually build it in a way where you're not bolting on legacy things you're just being very purpose built and that gives you advantages in cost power latency um quality security and people like that in these networks the penalty for downtime is a lot they typically want to keep using that network right and so that aspect makes it

Speaker B: a little more uh uh what's the next big milestone for upscale yeah on

Speaker A: upscale uh what we are focused on is both scale up and scale out on the scale up we are building our own chip which we refer to as sky hammer on the scale out we are building systems we are partnering with nvidia spectrum uh family of ethernet switches um we are putting a uh grounds up ai networking software stack that we are building uh we are actually partnering with optics company so we are really building this as a full stack ai networking company where we could go to a new cloud and say we'll build you your entire network we are going to be releasing products in both of these areas and announcing them later this year and then we are uh working very closely with a few hyperscalers on the scale up uh switching where they are uh interested in using our scale up networking with their uh ai xpus so we expect to bring that to market sometime in the next year and get that deployed that's fantastic so

Speaker B: uh it almost feels quaint to think about the times when people raised ten twenty million dollars as the first round but you went and raised close to five hundred million dollars in a matter of less than nine months uh does it make you pause and ask yourself what's going on it's like has the industry changed pretty dramatically or do you think uh this is just a period when a lot of capital is available in this place and it might reverse to the mean at some point in

Speaker A: time so my view is that the ai infrastructure as we just spoke about earlier is a massive opportunity it's an opportunity that's ten x or one hundred x of the previous opportunities that we had right and so capital flows into opportunities i think where the markets can be very very big and i think this is an opportunity where you can drive a lot of revenue and you can drive a lot of value and so people are willing to bet on

Speaker B: this infrastructure but the thing that worries me is that when you have three large companies likely to go to the market raising probably close to three hundred dollars four hundred billion dollars i'm talking about spacex openai anthropic uh and a few others do you worry that there is this giant sucking sound of capital both in private markets and in public markets that's going into few of these companies there's not that much free capital that's going to be available at some point the music is going to stop uh does that feature in your calculus of how much capital is needed for companies like upscale and valora and others

Speaker A: yeah i think uh like i said nobody has a crystal ball and we could have blips along the way whether it's six months out or two years out hard to tell um but i think what is for sure the thing that we know for sure is that the infrastructure that we are deploying collectively is being used right in the internet bubble that was not the case people were putting uh network bandwidths ten x one hundred x of what was needed when people came to the realization that that bandwidth that they've uh you know the fiber that they've laid out and the bandwidth that they've deployed is not really needed that was kind of one of the reasons why the bubble burst in the ai era thus far we know that the need for tokens is massive so clearly we could have some breakthroughs in software that would reduce that i think that could be uh um a reason for people to pause uh investments or infrastructure but i think that uh even if that happens by the way that will drive a much faster adoption of ai because your tokens per dollar will drop dramatically through these things

Speaker B: but my question is a little different because you take your own example of cavium in cavium when it was private before it went public how uh much was the aggregate capital that was raised

Speaker A: very low very low right exactly double

Speaker B: digit millions double digit millions and you delivered one hundred plus million after you went public also you delivered significant amount of top line growth and you're profitable and you had a great exit i'm just trying to understand what has changed that has necessitated these startups upscale next hop and a variety of others raising three hundred four hundred five hundred million dollars in private capital uh is the market opportunity for you to be able to deliver one hundred two hundred million dollars way bigger now it is it is or is the time to revenue

Speaker A: no i think there are a few factors first is that when we were cavium or in those days mellanox netlogic those were the companies that were peer companies of ours in the public market

Speaker B: by the way is the one that arguably made nvidia uh where they are

Speaker A: today super successful yes uh but the peer group that we had in the period that we were in the market we were all trading in single digit billion valuations with lots of customers and whatnot and the reason for that was that the markets we were going after at that time were not that big enterprise uh telco markets cloud made it bigger more recently if you look at the role models or the companies that inspire us as startups in the semiconductor space are uh companies like cerebras like um uh samanova like credo like astera labs these companies that have built successful businesses with tens of billions of dollars of valuation so you can see a ten x value there okay right so that is the opportunity part of the equation on the cost part of the equation what has happened is that as we used to do chips and tape it out we would do twenty thirty one or two million dollars now we are talking thirty million dollars uh for a tape out right and that's because

Speaker B: you're actually as you go into the leading edge uh five four three two nanometer the cost of tape out has exploded and the ip and some of the back end providers they've been pretty significant in the cost side of cost

Speaker A: side of equation so cost of bringing

Speaker B: chips have gone explosively higher exactly and

Speaker A: then on top of that we are in a world which is supply constrained and the thing is that you have to make commitments that uh are long term commitments for these components and when you go to these vendors and if you give them a one year or two year po they won't accept that po unless they look at your balance sheet and feel comfortable that you can buy this all those factors necessitate having a balance sheet a watch list of balance sheet balance sheet that's very different than that was that is a very

Speaker B: very interesting point maybe let me give you a structure to comment uh on if you were to think about those days when obviously we've had similar journeys from different uh vantage points capital being one of the things that would keep startup founders awake sequential financing and are we going to get the next round put together that's one sort of worry if you will uh attracting the right team becomes another constraint uh are there going to be enough customers it's going to be another uh issue and then execution even though you have the right team can we deliver what we promise if you took those four buckets what has changed now in those four buckets

Speaker A: i think that um first of all all those four buckets are very important capital team execution those are all markets are very very important issues um when i was doing the first startup that i founded in ovium the world had moved away from semiconductors it was all saas and it was challenging to raise capital for semiconductors and uh what has changed is certainly capital for semiconductors has gone through the roof uh in recent

Speaker B: years someone was telling me that silicon is back in silicon valley for a period it was called software valley and now everybody only talks about semiconductors right

Speaker A: exactly and hardware is back in fashion and hardware was moving to um asia and manufacturing and so forth was getting out of here so i think that actually has become much better uh it's funny because in some of our funding rounds we have companies that were historically saas only investors that want to invest in us so that certainly is a big change talent uh is always hard because silicon valley um different companies compete for the best talent so you have to uh it really helps to have people that are like minded potentially have worked with you in the past so that becomes an important aspect of the equation you have to have an interesting vision and um big picture so that you are able to attract but do

Speaker B: you see the risk of churn or do you see in the market uh at least in the valley today uh it's hard to hire best talent uh are you worried about losing talent or being able to attract the right talent

Speaker A: i think we continue to attract good talent uh uh because we are solving interesting problems the other thing that's happening as we all know is that ai is making engineering work easier and i think we'll continue to do so right so we keep thinking of with the people that we have can we do more right and so we are constantly uh thinking and i think in the next one or two years that will be a massive massive change so this scarcity of talent will become a little better i think it will never go away because good people are uh few but i think that ai will be uh a big help in that and

Speaker B: as the customer concentration with four or five or less than ten customers who are insanely powerful the hyperscalers and others does uh that worry you i mean

Speaker A: it's always the case the hyperscalers um are a very big portion of the market now in today's world uh we know that roughly half the spend is neo clouds and sovereign clouds uh hyperscalers used to be like ninety percent now they're probably closer to fifty sixty percent because of these new clouds and sovereign clouds although what we are finding that in today's world um some of the frontier labs are sucking up all of that capacity as well so there is some level of concentration um even right now i think it would be better if it was much more distributed and decentralized for uh companies like us um but you have to adjust to the

Speaker B: way the world is in that context i think uh one of the criticisms that a lot of people on wall street would say is that there's a lot of cyclical uh dealings that are going on openai buys nvidia stuff nvidia gives them cash at some level it feels like uh investment capital dollars get construed as revenues uh and so on and so forth does that worry you

Speaker A: i don't think so i mean uh i think we are still at a point where that infrastructure is being used right the fear would be if that infrastructure that is being deployed using these models that you talk about is not used then it's a big red flag okay um but today we are at a point where this infrastructure is being

Speaker B: consumed right yeah um well uh rajiv ah this is just uh such an enriching conversation i'm really very pleased with all the things that you're involved in and one of the things that you do is ah you're uh big into vipassana and meditation uh and you have found a right uh balance of actually being on the cutting edge on a lot of these things technologies while keeping your sanity about it and there's a certain stillness about you i mean can you elaborate and is there a magic to the rajiv formula of being able to do multiple of these companies and have a semblance of sanity around you

Speaker A: yeah i think the magic formula is uh i think the single biggest magic formula is the team and basically working with people who are your partners in these journeys right and so it's not like i'm doing it we as a team are making these things happen and we are putting our collective uh brain power to uh identifying problems solving problems creating innovation so i think that is the number one biggest thing um and you know you're not in it alone right that's the biggest thing the second biggest thing is that you know what i found is that uh when you run into these problems and if you can think somewhat clearly without getting too excited or depressed that you are able to figure out some solution and you can think more creatively and more innovatively

Speaker B: uh that's where your meditation comes into

Speaker A: play comes into play how often do you do it you know i do it every day uh in the evening and i think that's why when you ask me what keeps me awake at night nothing really because i sleep well and uh that's an important aspect of these things where you every day you encounter some problems some great news some not so great news and you have to kind of stay calm um in each of those instances so ten years

Speaker B: from now do you continue to hope to be involved in many companies like

Speaker A: this in a few companies for sure right so i would love to do that over time you know i'll obviously get into more of a coaching type role as we go forward um because we you know as we develop more and more people within these companies but certainly we'll be involved in uh tech

Speaker B: in some fashion fantastic you know thank you for this uh friendship and journey and kudos to all the great things that you've done thanks rajeev thank you

Speaker C: for being on it in every technology wave the value accretes at the bottleneck and that bottleneck is forever moving for the first years of the ai boom the constraint was the gpu this spring it was memory micron crossed a trillion dollar market cap in may with its high bandwidth memory sold out for the year the network is shaping up as the next fight with the industry's biggest names now writing open standards for it and underneath everything sits power delivery the iea expects data centers to use more than twice the electricity they do today by twenty thirty more than all of japan consumes that's the real map of ai infrastructure not a single bottleneck a moving set of interwoven pieces rajiv has spent decades positioning for exactly that motion wherever the constraint lands next power delivery memory optics physical ai data centers in space or something nobody's watching yet that's where the next generation of important companies will be built

Speaker B: thank you for tuning in to the tech surge podcast from celesta capital if you enjoy this episode feel free to share it subscribe or leave us a review on your favorite

Speaker C: podcast platform we'll be back every two

Speaker B: weeks with more insights and discussions of

Speaker C: all things deep tech bye for now