Are our networks ready for AI?

SAM JARRELL Good morning, Michael. MICHAEL BIRD Good morning, Sam. Now Sam, have you ever tried to do something with the wrong equipment? Like for me, often I will need to unscrew something and I'm too lazy to go downstairs.

So I use whatever I have in front of me. car keys, maybe a credit card. none of them work. And then I have to then be like, fine, I'll go downstairs and find a screwdriver.

I feel like you would get the right tool for the right job every time SAM JARRELL Usually I try. what immediately came to mind was, last Discover Barcelona. I brought our editor, some food very late at night and I also hadn't eaten, but I forgot to bring utensils. And do with what was around.

And those were, wooden coffee stirs. So we ate trying to use them. Kind of similar to chopsticks. It was not very effective, but we did at least eat.

MICHAEL BIRD I've definitely done that before and it's now why I have wooden cutlery in the glove box of my car, Anyway, as you know, sometimes something new comes along and suddenly everything that came before it has to change to keep up with the times. And as per this episode, even the way that we design our technological infrastructure, it'll all make sense, I promise I’m Michael Bird SAM JARRELL I'm Sam Jarrell MICHAEL BIRD And welcome to Technology Now from HPE. MICHAEL BIRD Now, Sam AI hasn't just changed how we interact with our technology.

AI itself interacts with our current technology infrastructure in a completely novel and different way. SAM JARRELL And it’s literally forcing us to rething how everything is designed right? MICHAEL BIRD yeah, yeah, exactly. AI has a set of quite specific requirements, which means that the network architecture required to support it has to be completely different to what came before.

AI doesn't sleep, and the data needs to be kept live to give accurate results, SAM JARRELL Oh yeah, those have come up several times in recent memory. So I guess with all of these new requirements and the explosion in AI usage, we are having to respond rapidly and create new ways of designing networks. MICHAEL BIRD Exactly Sam. And these changes come with a level of urgency for organizations because AI infrastructure isn't cheap.

And if it's not being used to the best of its ability, it can then be a huge waste of money. So to find out more about how our network architecture is adapting to the age of AI, I spoke with AE Natarajan, SVP, general Manager for Routing infrastructure solutions, HPE networking. And first things, first, I wanted to get an overview of the topic and define what network architecture actually is. AE NATARAJAN essentially is the way you build networks to interconnect.

And there are different parts of the network that are core networks which interconnect globally. there are edge networks. That is access networks where you bring in customers. There's aggregation networks that you have, and then there is networks in the customer premise, there is networks in the data center.

So architectures vary with each one of those things. Uh, some of them have similar architectures. Most of these networks are built for scale, connectivity, growth and ability to adapt to new things that come about. MICHAEL BIRD can you just sort of paint me a picture of what the sort of traditional pre AI network architecture looked like and, and maybe a, a bit of a sense of like how long it's been like that for?

AE NATARAJAN It traces back in history if you really want to go through it. towards the late nineties, early two thousands, we called it the internet boom. And the whole idea behind that internet boom was to try and get everybody online, It's not just people. it's things, people, iot, all of those devices.

And then we embarked into a different era in networking, which was cloud. Previously we used to have client server based applications. We moved over to cloud where the applications actually got pushed into a cloud because the applications demanded. dynamic compute and growth and cloud was an easy way to actually push that over to the cloud so that your applications can actually grow to the dynamic demands and growth.

And then. We said that was not enough. Not just you've moved the applications around in the network and globally, I'm gonna move the users around. And then we brought mobility into place so users were actually moving around.

So network architectures had to adapt to all of these things over the periods of time. And then last but not the least, we brought in ai. And this AI brought some interesting elements into the network, which is what I think we should be discussing MICHAEL BIRD when we talk about sort of traditional network architecture, we talk about how, there's different peaks at different times in the day. Can you sort of talk through that, like in a sort of typical organization?

AE NATARAJAN let's assume that, you were in the age of the internet or the cloud. And you had a SaaS service or a streaming service and you wanted to do something about it. You wanted to watch a movie, you wanted to watch some recording, you wanted to watch something, pull down from there, you would send a small pile of bits over to the server side or to the cloud application, and it'll stream a huge amount of data to you. So the data was asymmetric pretty much, and the network architectures catered to that.

Your. Upload speeds from the consumer was always less than the download speeds that you would get from consumers, and they would rate themselves based on that. They would always say, Hey, you got. 10 megs of download or one gig of download or something like that, but they never talk about the upload speeds because it was always meager.

Then the second aspect of it is we use these networks to multiplex. Not everything was on at the same time, so you would actually time multiplex it, so you would use your network resources effectively. To redistribute the load where it's needed and shut off load if you don't need it completely to save time, energy, and money and whatever else that you can. So we built network architectures that did all of these things, but that used to be in the cloud and mobility era.

The main challenges of the cloud and mobility is now unlike the client server, you could no longer predict. Where data is coming and where it is going. It was still in terms of upload being small download being high, but with mobile users, they were traveling and they were moving around, so they would trigger from different places and downloads had to cater from different places. Applications would move with the sun, so you would have to change where the applications were getting loaded or microservices or SaaS page services were getting loaded.

So you would do all of those things in a cloud environment. MICHAEL BIRD okay, so let's talk now the sort of post AI world, which we live in now. and I suppose you'd probably call it an AI native network. What does an AI native network look like and how does that differ to a traditionally architected network?

AE NATARAJAN In the case of ai. Whether you're doing ingest for training purposes or you're doing inferencing, you are sending rich content upwards. And you were downloading rich content because you could be using, AI based classes or virtual, gaming machines and things like that. You were sending rich content upstream, and you were delivering rich content downstream for what you were doing.

So the AI started super imposing, symmetric data patterns and traffics. The second aspect, what AI did was. It really created, a notion of agentic AI where the person is no longer there and agentic AI is running all the time, every time, which means the data never has peaks and valleys and you can't multiplex, you can't actually load balance across different things, and you have to actually have a network that is provisioned to handle this load constantly, every time. The last thing that AI did, which we didn't talk about in the streaming service, is when you stream the service, if it was coming from a distant place in the network, it would say buffering and you would get really irritated So we started building caches towards the edge.

But with the AI and AI data, you no longer can build it with caches because AI data gets obsoleted the moment you try to cache it. MICHAEL BIRD Right AE NATARAJAN And so the network architectures have to rethink the way we do stuff in AI network is symmetric. The network is always on and the network is not cashable. MICHAEL BIRD Yeah.

caching in a traditional network is Quite a crucial way of saving bandwidth, But we're saying in the sort of, post AI world can't really cache things because every request is essentially unique. AE NATARAJAN Correct MICHAEL BIRD so these networks run 24/7. how can you ensure they're being used efficiently? because presumably if you are running your network 24 7, or if you are architecting a network, you're running 24 7, that's potentially like more investment in terms of equipment, in terms of power, energy.

So how do you make sure actually that it's being used effectively? AE NATARAJAN So, uh, you hit upon something very interesting. So, AI is automatically a distributed element. we all know that a single GPU does not carry the AI workload, so you have to build clusters of GPUs to do ai, Now with power and space constraint, you can't put the clusters geographically located.

It has to be distributed. And we talk about AI build outs that scale up within a rack of a data center scale out within the data center, interconnecting all these GPUs within the data center. And now we are talking the third dimension of scale across geographies where. The network elements and the network pieces have to actually tie in and build the scale across.

And you bring a very interesting point here about scale across where when you're scaling across, or you're scaling up or scaling out in the in, in the data centers, you need to make sure that the GPUs are constantly being used and delivered, their. Ability to compute. If they are waiting on data or they're waiting on the network to give data, then you're losing cycles on the GPU and those are the most expensive things you need the ROI for it. MICHAEL BIRD So, so AI lends itself quite nicely to, a distributed network architecture and it performs best, in fact, with a distributed network.

Architecture AE NATARAJAN Absolutely. It performs extremely well in distributed architectures, and AI by nature is very distributed and it renders itself for the distributed architecture MICHAEL BIRD So it's about every step along the way, really thinking about, how that piece of the network is designed, how it's connected together, what hardware's there. Because you don't wanna put bottlenecks into the system. AE NATARAJAN Yes, absolutely.

So you have to really architect the system so that you have to build this in such a way that it is efficient at every stage of the network so that you get the maximum benefit. And the maximum ROI of the large investment that people are doing with AI today. MICHAEL BIRD with that in mind, why don't we just move all AI infrastructure to the edge? AE NATARAJAN Oh, that's interesting.

I just talked about the factory, right? Yeah. So, you still have to do training. You still have to do certain things in the factory to build from raw materials to goods.

So you have the data coming in and you have models to train. You have to do all of that stuff that goes into the center of the factory and generations of it. And then you have to also scale it up so that the edge becomes intelligent, the edge architectures become intelligent so that the end product gets delivered. In a very quick fashion to the end user.

So the consumers, when they look at it, when they're consuming ai, which is where the biggest thing is, the user experience really matters. So you want the edge to be there with the capabilities to deliver the lowest latency, predictable response times for anything and everything that you do, and that is important. As well as the rest of it in the core for building it and doing it is also important. You cannot move everything to the edge one shot.

MICHAEL BIRD So, as AI continues to grow, what's the most important piece of advice that you can think for organizations, service providers who are adopting this sort of, AI native architecture, into their systems and infrastructure? AE NATARAJAN when you look at it as a service provider and you look at these products and solutions, there are. Multiple aspects of this ai. First and foremost is the products and the solutions and the architecture that you build is most efficient in handling inter GPU communications of the data center interconnect so that the GPUs can work really, really efficiently.

Second on the edge, your architecture is defined in such a way that, the consumers have a user experience that is really good and have an excellent experience. Last but not the least, it is constantly growing. You're gonna get the next generation of GPUs and the next generation of applications and the next generation of agent AI that is going to impose more and more demands and throughputs of air. Can you build a system that can seamlessly migrate from one step to another and provide you that longevity?

Good. We actually announced those products that today, which are 800 gig, can easily move over to one point 60 and above, and that is huge. So when you are an operator, whether you are a service provider who's transforming the edge to deliver it to your enterprises and customers, or you're somebody else who's building models and training models, or you're providing that service to your enterprises, you need to think about the network architecture, not just for today also for tomorrow.

And think about what AI imposes to you. It is always on. Both the upstream and downstream is going to be, symmetric or huge loads, and you cannot cache it. You cannot cheat the system anymore.

MICHAEL BIRD AE thank you so much for joining us on technology now. AE NATARAJAN Cheers MICHAEL BIRD I really enjoyed, chatting to AE and I always find it fascinating talking about topics where you come away thinking, I never really thought about that in that way before. SAM JARRELL I appreciated the walkthrough on the evolution, from Internet cloud mobility. 'cause it kind of does show how they've always adapted and changed over time.

I kind of figured it was all relatively the same just with improvements. But I found it really fascinating that when you were talking to him about why not just move everything to the edge, right. That he said, we can't move it all to the edge in one shot. He didn't say though, that it couldn't be something that happens in the future.

And I wonder if maybe in the future if, if that will be the direction that things go. Because when he was discussing about it, he was talking about the user experience and that users are consuming ai and so then they need to have it be like a, a positive user experience. But I wonder if someday it could be an agentic AI at the edge and then it's interacting with another agentic ai and there's no need for that user experience. MICHAEL BIRD Yeah, that's really true.

and just touching on what he said at the top of the interview about how the way that we think about our networks has changed. what I took away from that is it has always been changing our networks have had to react to, the internet boom, the rise of cloud mobility, I remember during the pandemic, like the way that we thought about our networks had to change essentially overnight. And I wonder if we're now at that stage with AI But I thing that I thought was really interesting was caching.

If you're accessing a website, stuff gets cashed. so the end user isn't pulling that down from a network every single time. but the fact that you can't do any caching with AI because essentially the data that you are accessing is brand new every time. 'cause you're putting a new request in every time, like the load on your network and the way that you have to think about your networks.

I just found that so fascinating that actually Yeah, you really have to think about this in a different way. SAM JARRELL Yeah, you're right. the caching has been around for decades and it was sort of a clever way to save bandwidth. but with every request being like.

Unique and instantly obsolete. I'm wondering if that sort of explains why AI workloads feel so expensive so quickly. AI wants a naturally more distributed architecture, and don't work alone. so it makes sense.

The workloads have to scale up and out across all the geographies. This entire conversation kind of brought me back to, um, prior episodes where they've talked about, in regards to networks, how, uh, up doesn't necessarily equal good. Um. I feel like that maybe applies here.

MICHAEL BIRD Yeah. And, and like an end user just expects a network to work, don't they? and you know, I, I guess we talked about how our organizations are having to adapt to AI native architecture for their networks, but for the, I guess, everyday person, this is something that affects us too. So the final thing I wanted to know from AE was actually how will we, as consumers experience this change?

AE NATARAJAN Yeah. You are holding a tablet. Uh, I have a smartphone. We are all using all sorts of devices and we are going to be consumers of ai and we're be going to use ai, um, in our, in our work, in how we play and how we entertain ourselves and how we do things.

And we could be doing things. Training models, we could be using AI to do our work and getting answers. There has to be a consistent way by which AI responds to us and gives us predictability in terms of how it operates so that the user experience is enhanced. Which goes back to all of the things that we talked about in terms of network architecture, transformation in the core, transformation in the DCI transformation in the edge, so that you can deliver AI in the best possible user experience.

And we need to think about it, and we need to also think about the growth when this explodes and inference is taking on, it is going to grow exponentially. So we need to think about how these networks are going to just multifold grow. In the next few years, people even talk about five x growth in the next three to five years. So that is what is upon us.

SAM JARRELL Okay that brings us to the end of Technology Now for this week. Thank you to our guest, AE Natarajan And of course, to our listeners. Thank you so much for joining us. MICHAEL BIRD Yes.

And if you've enjoyed this episode, please do let us know. Rate and review us wherever you listen to episodes. don't forget to subscribe so you can listen first every week. And if you wanna get in contact with us, send us an email to technologynow@HPE.

com. Subject line. Any ideas, Sam? SAM JARRELL Can’t cashe MICHAEL BIRD Can’t cashe no cashe.

Technology Now is hosted by Sam Jarrell and myself, Michael Bird This episode was produced by Harry Lampert, Izzie Clarke and Eva Higginbotham with production support from Alysha Kempson-Taylor, Zoe Revis, Beckie Bird, Alissa Mitry, and Jenessa Ayache. Our theme music was composed by Greg Hooper. SAM JARRELL Our social editorial team is Rebecca Wissinger, Judy-Anne Goldman and Jacqueline Green and our social media designers are Alejandra Garcia, and Ambar Maldonado. MICHAEL BIRD Technology Now is a Fresh Air Production for Hewlett Packard Enterprise.

(and) we’ll see you next week. Cheers!