NAN126: Fine-Tuning Open Source LLMs for Network Engineering

Speaker A: We're sponsored today by Curveium, an industry leading system integrator that offers strategic IT consulting, professional engagements, automation and AI. Curvium takes the time to understand your infrastructure needs and how to best support your business objectives. Curviam skilled professionals put your needs first. From rapid designs to full scale architectural planning and from short term project completion to multi year support, Curvium provides the know how and vendor connections you need to succeed. Find out more@curviam.com While you're there, check out their secure campus network architecture blueprint. That's curviam.com.

Speaker B: Hello, welcome to the Network Automation Nerds podcast where we explore the latest in network automation from a practitioner's perspective. I'm your host Eric Cho, a network engineer who loves everything about network automation. Our guest today is Eduard Duharu, a veteran network architect and the founder of CTO of Vexpert AI. With over 25 years of experience ranging from military radar system to enterprise networks, Edward now specializes in building autonomous AI powered NOC and SoC systems. He's joining us to share his insights on how Agentec AI, open source, LLMs and digital twins are revolutionizing network operations. I can't wait to get started. So welcome to the show, Ed.

Speaker C: Thank you very much for having me.

Speaker B: First of all, can you tell us a little bit about yourself? Uh, maybe your background, especially your name. I know I didn't say it correctly, so let's, let's correct that right on the spot here.

Speaker C: Yeah, it's okay. So my name is Eduardo Haro. I graduated military university. I worked um, six years for NATO as engineer for missiles. And then uh, I moved entirely to it where I worked 20 years and the last 15 years as a network and security architect. And I decided to go deep in AI four years ago, uh, when I discovered the transformer architecture. And um, last year I decided to go all in AI and to build an AI startup and to follow my dream.

Speaker B: Yeah, that's amazing story because I uh, think a lot of us are secretly have this dream about oh my God, this is like such BS in the corporate world. This is, you know, not my, my true passion was. But we always lack the, the courage to just go forward. And if I, if I remember correctly you, you mentioned you, you're, you were just about to take your third CCIE and then you decided like okay, instead of spending time on this, I found this other thing that was uh, more exciting, more uh, worthwhile of my time. So what was that? I'm wondering at these pivotal Moments on, you know, where you decide to go, go for this new thing that you're so convinced versus like you decided to quit and go. I mean, what are the, what are those moments that made you just go for it?

Speaker C: In 2022, I failed, uh, my second CCIE attempt in Brussels. I ran out all the time. I was never very fast in, uh, typing and troubleshooting.

Speaker B: That's part of it too, right? Like, CCIE is not just about the knowledge, it's also about how fast to that solution. That's a good point.

Speaker C: Exactly. And at the moment chargpt, uh, appeared in the format, we know it today. And I was curious to see what powers with, uh, charpt, uh, product.

Speaker B: Right.

Speaker C: And then I discovered the attention is all you need paper, which was, oh,

Speaker B: the actual paper itself, not just, just a tool that, that derived from that. Yeah.

Speaker C: And I read that paper. Uh, actually the paper I think was published in 2017, so it was a relatively old paper. Yeah. And I discovered within that paper, um, the attention mechanism. And uh, it struck me because, uh, I realized a lot of similarities with how people think, how people pay attention to words within a sentence to understand, uh, how people can extract meaning now is replicated within that paper. And that was for me, uh, aha. Moment when I realized that this architecture, this transformer architecture will change the world as we know it. And I need to take the decision, should I go for the third attempt or should I start to understand and become a friend with a technology which will be soon better than me?

Speaker B: That's uh. I don't make that leap though. So you got to go a little bit deeper on making that connection. Right. Because I read that paper as well. And um, so maybe first of all you could tell us a little bit about what you mean by attention is all you need on like how people could pick up certain, certain meanings without like listening to handsome sentence or, or something like that and then making that leap on why do you think just from that that it's eventually going to be because. Be better than human? Because back then it was very. Still very primitive, right? Like GPT, what 2.0 is still very primitive. So maybe those two points are not clicking for me. Maybe you could tell me a little bit more about that.

Speaker C: One of my, um, hero is Jordan Peterson. It's uh, a famous psychologist from Toronto.

Speaker A: Okay.

Speaker C: And one of his first books is, uh, Maps of Meaning.

Speaker B: Okay.

Speaker C: He wrote this book, uh, when he taught at, uh, Harvard, I think. And I read his book, I tried to read his book. Very, very tough to Understand, that's me

Speaker B: for like half of the book. I try to read. Yeah, I try to understand them.

Speaker C: But sometimes you just are very tough. Very well written and very uh, dense uh, book. And uh, because I, I was somehow fascinated by this idea of meaning in general. And I hope that reading Jordan's book, uh, Maps of Meaning, uh, I'll be able to find an answer to my question about the meaning.

Speaker B: Sure.

Speaker C: So for me the idea uh, of a meaning was very deep, uh, instilled within my own research, finding my own way. And uh, that's why when I see attention, when I see this attention mechanism which replicates so well, I should say maybe even better how uh, the meaning can be extracted out of a text. I realized that um, technology now can reason and can do deep uh, kind of thinking, which was not possible by that moment. By that moment. The AI limitation was um, um, because of the sequential um, processing of data, which is what neural networks did. Uh, processing data in a sequential manner and transformer showed that uh, they invented the safest parallelism. The way in which a um, model can attend to every word in a sentence and to give a weight or a word to every word in a sentence. And in this way you can uh, identify using, let's say even within a longer paragraph, which are basically the keywords which carry the meaning of that paragraph.

Speaker A: Right.

Speaker B: Or the majority of it. Just from those words you could get like 80% of what this paragraph is trying to show and a lot simplify as well as parallelizing that process. That's what you're saying.

Speaker C: Exactly.

Speaker B: Okay.

Speaker C: Yeah. So the model was able to attend in the same time to multiple words. And um, this is what makes transform so powerful. Which means that um, all the previous uh, limitations of the uh, recurrent neural networks or uh, LSTM or neural networks were basically um, disappeared. So um, I've seen the potential of such mechanisms.

Speaker A: Ah,

Speaker C: and I uh, realized that with all the hallucinations, if you remember back in 2022 of the hallucination of ChatGPT, I realized that it's a matter of time until this stuff will refine and reiterate and become um, I don't know if smarter is the best word, but more knowledgeable than us. Yeah. So this was my mental projection of AI MHM in 2022 and I compared against my future CCIE.

Speaker B: So I realized that because who needs CCIE when you have like general intelligence on the way.

Speaker C: Yeah. So that was ah, a decision, a big decision moment in my life when I said okay, um, I need to go deeper in this stuff.

Speaker B: Yeah, I mean I think that makes more sense for me now because I think it's couple your prior knowledge and then with um, this new thing where you have a deeper understanding than for example Erica 2022 then uh, you're like okay, if I squint a little bit, I could see even though it's hallucinating now, that limit is going to be just solved really really quickly. Right. Okay, perfect. Then you made that decision because you are way smarter than, I don't know, at least me. And so you decided to go deeper into this and uh, tell us about that path between. Okay, now you're going forward and what is that um, kind of thing that you're building? We mentioned briefly in the intro like um, should I just say agentixock knock or so like how would you describe this product that you're trying to achieve?

Speaker C: So this is a learning process. And um, maybe I had the advantage that I started and I learned and developed with uh, every single technology as they appeared. And then um, every new improvement or innovation was basically solving a problem from the previous technology.

Speaker A: Yeah.

Speaker C: And um, so when you hit the technology to its limits and find obviously its limitations and failure modes, especially uh, you. It's easier to consume um, or to understand the next innovation which comes into the field which solves some of the previous problems which, and also introduces new problems which will be solved by the next uh, iteration. I can make a parallel with networking if you want. So.

Speaker A: Sure.

Speaker B: Um, in fact if you don't, I'm going to ask for it. Yeah, make a parallelism for uh, networking.

Speaker C: So we had SP protocol uh, which solved a very clear problem within networking, redundancy and broadcast etc.

Speaker B: Sure.

Speaker C: And then uh, um. So it solved a problem in the first place and then it came with its own limitations. And we had all the instances of spanning tree, rabbit, et cetera, uh, multiple spanner. And then um, all these uh, um instances or um, spanning tree variants uh brought their own set of problems which were solved by VPC channel and fabri pass and then uh, uh, VXLAN evpn. So basically it's a uh, logical sequence of innovation which uh, uh adds one on top of the other trying to solve the previous problems. And the same happened with um, and the same happened with the uh, process of learning AI starting with prompting. If you remember back then prompting was a big deal. How you structure.

Speaker B: Yeah, I mean it's still a big deal.

Speaker C: It's still a big deal. But it was uh, uh the only way to interact with the model at that moment. And we wanted to optimize these interactions.

Speaker B: You got it. All the contacts went in your prom which makes it uh, such a big deal. But maybe not as much so because we. Other instruments. Yeah, yeah.

Speaker C: And then uh, uh, because these models were trained with some data which uh, is already outdated or not up to date in any case.

Speaker B: 2023, whatever. Yeah.

Speaker C: And uh, the behavior of models showed with um, probabilistic nature which was instantiated in uh, hallucination. And the solution to this problem was rug was to have your own database with your up to date documents and to perform a retrieval from your database. So the quality of the retrieval depends on the quality of your data. So it's your problem now and is within your control. And then this content from your database, from this rack system will represent the um, relevant context for the model. So the model will then provide a uh, better answer than simply prompting and answer which is actually rooted in. Grounded in your data within your RAL database.

Speaker B: I see. So, so what you're saying is the thing that you're trying to build is because of your previous experience and we should talk about that a little bit. Um, because of your previous experience you know that eventually what you need is really your local uh, insights.

Speaker A: Right.

Speaker B: Like because whatever the motto was is they stop at whatever year and it's not, it's definitely not going to be. Yeah it's going to be like two years at best, one year at best. But it's not going to be today and therefore you won't get the most up to date. And then uh, what you want is what uh, you say rag. So I think that also brought up a good point is that you know all of these just like the moment you decide to go low in is because of your private experience. It's all additive. As Scott, our you uh, know, co founder for for Nav would say, it's all additive. So because of your prior experience, you know this was going to be the technology even though it's mature enough and because of your prior experience at IBM consulting and everything else, you know this even if it's not, you know doesn't, doesn't have the contacts window or whatnot. The rag is what's going to be. So maybe we could also dive into your previous networking experience then, then people could kind of piece together the puzzles on how did he. It's not just overnight. You woke up and you say I'm going to start this new business and this is what I'm going to do.

Speaker A: Right.

Speaker B: Ragging is the way to go.

Speaker C: So.

Speaker B: Yeah, tell us a little bit.

Speaker C: Yeah. So to continue so with evolution, um, diagram let's say so after rag, uh RAG also came with its own limitations, a lot of limitations actually. And then the next level was to fine tune and to fine tune a model. To fine tune an open source model to learn specific data. Ah, uh, which is uh, completely within your control in order to shape the behavior of the model in the direction you desire. Yeah. So and then uh, uh, you, you have a brain but you need some hands for that brain to take some action. So here anthropic came with uh, mcp.

Speaker B: Yes.

Speaker C: Which basically give hands to the brain, to your uh, educated brain, let's say artificial brain. And then um, uh you can reason and you can perform analysis on uh very specific content which in our case is the output of the various uh devices show commands, routing, etc. So and now comes to the agentic part which is this ide, uh which is somehow the top of the technology stack which is basically creates um, in my case a deterministic workflow which um, connects all these pieces together in order to um, to avoid it, in order to avoid downtime and at the end of the day to um, make people make engineers life better and easier. Um, so once somebody understands there are probably maximum 10 AI fundamental concepts. Once m somebody understands these fundamental concepts then it will be a very simple job to build whatever system you want based on these uh, pieces and uh, to solve your specific use cases. Cases. So you don't need maybe agents or you don't need maybe fine tuning or rag for solving a specific use case. But the condition is like in networking where we learn the fundamentals, uh how um, uh, spanning tree works or uh, waste PA or BGP or so. And we think in layers because we have to sync in layers uh within networking. But the point is that once you understand and uh, practice this is super important, uh these fundamentals then you will be able to build your own systems to do almost everything. Not only in networking.

Speaker B: Yeah, I think we're almost talking about two parallel tracks. Just like a lot of times the uh, network engineers do. Right. So I think for your background, one is your own path about learning AI, knowing, projecting what a next step is. And you talk about prompting rag and then agents and all that natural progression. But we're also talking about networking where you're talking about the domain specific knowledge is networking. You talk in layers and so on. So I think we've gone through a good number of you know, what you think about the natural progression for AI. But how did you, what what was your networking background like? What, how did you learn about spending tree? How did you um, you know, learn about how to talk to customers that sort of said? Because I, I know you spent a, a good chunk of time either as a, as an individual contributor as well as a leader.

Speaker C: Right. So my first years ah in CCNA where I think 20 plus years ago when I discovered networking as uh a. It was a relatively new discipline in Europe at least.

Speaker B: Really.

Speaker C: Okay. Um, 25 years ago.

Speaker B: Yeah.

Speaker C: And um, I was somehow fascinated by the um, layers of abstractions which were required to develop in order to understand how a network works.

Speaker B: Right.

Speaker C: And um, uh I just uh, um learned and uh, took a lot of courses and ccna, ccmp, et cetera, classes to understand better how the network can be uh operated first and then went uh, up to the um, design level to understand how to uh, design it. And here a big role played in let's say in my career, uh, Russ White. M. Uh which is Dr. Russ White

Speaker B: who's also a philosopher and networking genius

Speaker C: and a lot of other smarter guys. And uh, in networking which um, um basically opened my mind to go to the root causes and to understand the principles behind how things are built and to focus um, on fundamentals. I think this was something important which I did to read rfcs, to read books which were vendor uh, uh agnostic, to understand the deepest level possible, how uh, a given protocol works and how the protocols interact on which the other um. So mastering fundamentals in networking and um, also now I realize that also is one of the most important things.

Speaker B: Yeah. Like the first principle. Right. So not just about ospr, but why does it exist then? Once you understand that you don't, you don't really care whether it's ISIS as Dr. White would say.

Speaker C: Right.

Speaker B: You know it didn't really matter what the protocol was. You know, just they basically do you know, four or five different fundamental things go to the first principle. So now I kind of see your path converging. Right. So you have this networking path. You try to do the uh, root, it goes the root of these understanding. And you saw the uh, rise of AI, the rise of ChatGPT. You've read the attention, so you need paper and now they're converging and finally you made the call to say I'm going to build this. Uh, and you're building it according to the contacts rack and agents. So if you were to. I have this interesting question for you. So if you were to launch a network specific agent today. Right. How do you choose between say Llama, Mistro or Quaint? Is there any specific networking tasks Right. Like bgp, uh troubleshooting or config generation where one model is better than the other. What do you think?

Speaker C: Uh yes of course one models are different.

Speaker B: Mhm.

Speaker C: And um, what is important especially for the open source models which have publicly. Which are publicly documented.

Speaker B: Yeah.

Speaker C: Is to understand as much as possible the um type of data used for pre training. Okay. For example code llama from meta was used. They used for training uh programming languages. So which means that natively this pre trained model, this baseline model uh knows um programming.

Speaker B: Wait, how do you know the data that they would train with? Are these public? Because I was under the impression that they don't tell you what data they train with.

Speaker C: Data itself is not public. I didn't see any vendor to make it public. But the type of data used for pre training sometimes you uh, can.

Speaker B: Oh I see, I see what you're saying. Okay.

Speaker C: You know that uh base model has a ah predilection or can perform better for um code or uh. A model like Quin for example is pre trained uh in a way in which is um better reasoning.

Speaker B: Uh, uh.

Speaker C: Which for example uh uh llama llama 2 for example was not a reasoning model. Uh a model which I fine tuned two or three years ago. Uh it was a normal language model M. Uh and um. Its performance after fine tuning were not so great compared with Quin which was a uh reasoning model. So basically um.

Speaker B: So tell us the difference though. Like what's the difference between a reasoning model versus a general model?

Speaker C: A reasoning model um is trained in a way in which uh is incentivized to identify solutions to problems.

Speaker B: Okay. Okay.

Speaker C: So there is a reward function uh um which is in place which uh incentivize a model to um be as creative in the good sense as possible to find a solution until reasoning becomes the only strategy for the model to solve the problem. Okay. This was the deep sick moment in Last year ago 2025 when um the uh researchers from China uh took a different approach than uh researchers from United States.

Speaker B: I remember that it was making a big fuzz. Yeah.

Speaker C: And the idea was that uh it was a philosop actually. Uh while the OpenAI researchers uh use the reinforcement learning with human feedback which means that all the data set used for training was validated by experts which obviously is not a scalable way to.

Speaker A: Right.

Speaker B: You only have so much human hours, human experts and then you're slowing down the process.

Speaker C: You're slowing down the process and you also hit some biases which are inherent to anybody.

Speaker B: Sure.

Speaker C: On the other hand the um, Chinese researchers they uh, used a different approach. They said we don't need to seal the model capabilities and we need to um, let him find the best answer. And they developed a method uh, called reinforcement learning with verifiable results. And they applied this method for domains where the results can be verified, which is math and code. And uh, they uh, trained this model using this method, reinforcement with verifiable results. And they uh, had this aha moment or uh, breakthrough when the model starting to reason against its own decisions. M Because the model was uh, incentivized to find uh, um, to find the best solution to a problem with a given result, verifiable result, uh the model started to uh, identify uh, the most efficient reasoning. So the reasoning became basically the only strategy for the model to find to take the incentive. And this was the biggest uh, let's say philosophical difference between these two approaches which led to this explosion in um, um, um reasoning models inspired from this architecture.

Speaker A: Let's pause for a message from sponsor Curvium As a systems integrator in var, Curveium doesn't just sell to you. They consult with you to pick the best technology to accelerate success, always putting trust in your organization's unique needs first. Whether you need a quick design critique or full scale architectural planning, curveian's experts provide the oversight and vendor connections you need to. Succeedium doesn't just plan, they execute. They use powerful industry leading automation tools to handle everything from initial rack and stack to the final operational cutover. They even train your team to work more efficiently along the way. And Curvium could help you enter the age of AI ops with Curvium's Curve os, a centralized AI powered platform to manage all your infrastructure from a single point. It's not just an afterthought, it's a stress reducing, sleep saving control layer designed to give you total visibility and peace of mind. Find out how Curvium can help you@craveium.com that's C-U-R V I U M M dot com. Thanks Curvem for being a sponsor. And now back to the podcast.

Speaker B: Yeah, I remember that. And so what you're saying is essentially you know, you use AI to speed up the instead of having human in a loop, right. Use AI to train AI and that just becomes this, you know, uh, potentially at the time it wasn't proven but now we know it's better. Yeah. So what you're saying is like between these three, uh, Llama, Mistral and Quinn, that uh, the reasoning models are more uh, superior and uh, I get that and thank you for the explanation. And you also talk about fine tuning. You mentioned the fine tuning.

Speaker C: Right.

Speaker B: So tell us a little bit. What I think that was part of your argument is just that yes, it's better, but better doesn't mean it's workable and fitted for your situation. You still need to fine tune it. So tell us a little bit about that.

Speaker C: Yeah, so in order to define something better we need a baseline. Sure. And um, the baseline, uh, in this example or the baseline in case of fine tuning is the base model itself or a frontier model or end frontier model. So that means that uh, on the same unseen test, the new finer model should be able to perform better than the base model and better than how, uh, let's say small, like um, openai40 mini would answer.

Speaker B: Right.

Speaker C: So, and that constitutes the definition of better. Okay. I tested and it happened uh, several times to fine tune and not to get these results. Which means that I was not able to say that I achieved a um, better model, um, with regards to the test reference which was uh, never seen by um, base model or fine tune or the uh, uh, public model. So once this fine tuning process is successful, we can have enough confidence that our new local model can help us in tasks for which it was trained. The idea of training basically means to feed it enough uh, data in various uh, formats and uh, from various angles for the same topic in order for the model to be, to do what a model does best, which is pattern recognition, maybe you could.

Speaker B: So even though you mentioned it did not uh, materialistically made you uh, say that your fine tune model is better, but maybe you could walk us through the fine tune process. What do you do when you say I'm fine tuning it? Um, because I think that's still in my mind a confusing part as well.

Speaker C: So first we start with a problem to solve. Okay, let's say we want to build um, a virtual uh, architect which can reason against all vendors and all technologies. I don't know if somebody in this world is able to claim that has this capability. All the technologies and all the vendors,

Speaker B: um, they could claim it. Do we trust them?

Speaker C: Yeah, they proved that. Yeah, it's a different story.

Speaker B: Yeah.

Speaker C: Um, okay, so this uh, is the level of uh, let's say of um, the level of how we define a problem. I mean to do something useful and meaningful and to uh, be able to be useful for my, for my job instead of spending I don't know, hours researching I don't know which um, um, particular design or uh. Then I just interact with this model. So here we talk about simple chat. It's not let's say part of any system, just a uh, model which autonomously or independently responds uh, to our requests.

Speaker B: Sure.

Speaker C: So first is defining the problem. Mhm. Second is to second step is to uh, collect high quality data to uh, use uh, for the fine tuning of that model, which is a very tedious process, which is probably the most boring process actually. To curate data it must be human verified. Because if let's say uh, error or um, uh a concept is misrepresented within this data, then the model we will say it hallucinates. But in reality it's the data.

Speaker B: It was the wrong data. Ah yeah.

Speaker C: Then we need to identify a GPU and there are uh, free resources, there are paid resources. And um. For example if we take a model which has uh, um. 8 billion parameters we can uh. Every parameter is represented in 16 uh, floating points. Which means that we need 2 bytes for every parameter. Which means that for 8 uh, billion parameters we need 16 gigabytes GPU only for the weights. Mhm. For the weights of the model we need to factor in also the KV cache which is the um, key value cache which is uh, also a significant part of the, the GPU requirements which is I don't know, couple which require a couple of gigabytes.

Speaker B: I'm just nodding and agreeing without checking your math but I trust you.

Speaker C: Uh, and uh, another part for the inference. So for eight uh, uh, billion parameters model open source which is uh, it's a good one. I mean it has a lot of data. Ah uh, within this 8 billion parameters uh, we need let's say to be safe 24 gigabytes GPU which is available for free for example in Google Colab. Okay. So it's available for everybody to test

Speaker B: it and to yeah at least reasonably priced. And availability for the colab. Yeah.

Speaker C: So uh, we have also the hardware. Then we need to define the um, fine tuning algorithm. And here are involved multiple um data science, um characteristics like learning rate, um, size beige etc. Which are empirically determined by the engineer, by us when we do this um fine tuning. And the objective is to. The first let's say data science objective is to minimize the loss function during the fine tune process which is we can visualize it, there are tools so we can visualize uh, the loss. So we want to minimize the loss.

Speaker B: Yeah.

Speaker C: Uh, and ultimately to uh, test the uh, to test the um, merged model against our own questions. So this is the ultimate test to see if the model answers correctly to according to our predefined test set.

Speaker B: Yeah.

Speaker C: So and then deploy uh, it on your GPU or in hugging face or um, in cloud. I use AWS uh for that to store my models as private, but it can be stored everywhere and then you can make inference using normal API calls.

Speaker B: Got it. So essentially when you say you fine tune, you find high quality model with appropriate hardware and your algorithm to be able to influence the answer into a better answer that uh, than the base model. And how is that different from retraining? Right? Like how is that different? Is it just on a larger scale or is there something I'm m missing? Like retrain your model basically. Which is probably cost prohibitive for all of us.

Speaker C: Uh, yes it is cost prohibitive for most of us uh because the size of GPUs now increases exponentially. So um, we need a lot of GPUs, we need a lot of high quality data which is a super big task. Ah to collect uh, uh tons of uh, tokens to use for pre training just to build a baseline model. Um,

Speaker B: so do you think it's even feasible for us to think about it because. Or what we could do reasonably is just to fine tune the model, the whatever model that we think is better as a base.

Speaker C: Uh, I did actually both. So um. Um.

Speaker B: You did both. You rob a bank to retrain or.

Speaker C: I used. No.

Speaker B: Okay, we're still here so obviously you didn't.

Speaker C: Yeah, yeah, still here. Uh, so there is a very um, smart uh, uh researcher at mit, Sebastian Raschka, which wrote uh.

Speaker B: Yeah, it was like one of the classic book.

Speaker C: Yes, he wrote a book, uh, how to build a large language model from scratch. Uh or two years ago. And uh, I um, was part of uh, a discord channel where a group of data scientists and practitioners, um, we studied together and we build this model.

Speaker B: Oh, nice.

Speaker C: From scratch. And every. Or it was a voluntary basis. Uh, we offer to um, present every weekend a chapter and with mathematics behind, with code etc.

Speaker B: Oh, you're a part of that? I read about it but I'm obviously I'm not smart enough to read it or to understand it. But yeah, it's awesome to actually met somebody who was part of that project.

Speaker C: Yeah. So. And uh, I build a small um, it's a contradictory terms from scratch. So I know how to build it. So I know it is a very tedious process and um, the effort. If we need to compare which are the pros and cons for these two options, I'll say that it was, it was. It is better to spend time, money and energy in uh, shaping and adjusting the behavior and uh, training a model using fine tuning and then using uh DPO for example than to start it from scratch. Unless you are a research or uh, organization or um, um. So if you have a limited budget and you need to decide, I would say that uh, Lora or what is it Lora Qlora which are fine tuning techniques among others, uh, is the best investment of time and money to have your own assistant which knows uh, um, whatever you.

Speaker B: I would have to say I am part of that Cam that would limited budget. So yeah fine tuning it is. And instead of retraining the whole model. So I have two more questions for you. Ed is uh. So first one is how does open source play into the picture? Because I've heard of obviously you will host a model that you have zero control over with OpenAI and Anthropic, that sort of stuff. And then I've heard of open weight models. Um, is there any true. Which to my understanding is you saw the mathematical algorithm but you don't get the data, you don't get a source code you'll never be able to reproduce. Exactly. On your local machine. For open weight. Is there any true open source models out there that in theory we talk about cost and tokens and all that, but in theory that we could reproduce it locally?

Speaker C: I don't think so.

Speaker B: Okay.

Speaker C: Um, weapon weights is it was a big step uh towards um, let's say the definition of the word open.

Speaker B: Yeah.

Speaker C: Data using training. I'm not aware of any open source model to share it.

Speaker B: Yeah, yeah, yeah. I mean even then. So even for most open source projects I think it's just this um. Should we say hallucination that uh, you're able to contribute. Most of the time we just use open like Python. Right. It's open source. I never look at the source code but I know it's just, you know, deep down psychologically that I have this hope that one day if I ever wanted to I could go to the root of the problem. But. But I hear what you're saying, you know like there's no true open source model is open way with the big step forward. And um. Yeah we're just, we're just kind of leave it at that. We don't know what the Future will hold. So I have one more question for you is you're writing a book, right? Your upcoming book, Gen AI for Network Engineers. Tell us a little bit about that.

Speaker C: Um, I will share the book with the uh, people which attended to the workshop which I delivered yesterday. Um, and um, did you say you shared the book? I will share. Oh, you will share by the end of this week. I promised. Okay. Um, so the idea of this book is to encapsulate all my uh, hard learned lessons and the uh, progression of learning starting with fundamentals and then uh, like prompt engineering reg and then ending with agents A to A protocols, uh, MCP and fine tuning. So I want to transmit, I want to encapsulate as I said, uh, my experiences and my tests and my failures because I also have a lot of failures.

Speaker B: Those are the best learning parts, right? Yeah.

Speaker C: Within that book. And um, I want, I hope to help engineers uh, to people like me

Speaker B: who are willing to learn but just don't have the resources. Yeah.

Speaker C: To update our uh, tools, tool set skills and to you know, to, to make the life easier and better. This is the goal at the end of the day. And um. Yeah.

Speaker B: When do you think you have that book ready? And I'm so looking forward to this. Like when do you think you're ready?

Speaker C: Will be ready by Sunday.

Speaker A: Oh. Oh, Sunday.

Speaker B: Oh, interesting, interesting.

Speaker C: I promise that I will. I delivered yesterday, but.

Speaker B: Okay.

Speaker C: I found a bug in my uh, in my um, in my uh, application.

Speaker B: Yeah.

Speaker C: Um, and uh, I was focused to solve this bug and.

Speaker B: Oh, that's why you were uh, like I, I saw, I saw this dude sitting there and he was just you know, kind of like needing five more hours of sleep and that's why.

Speaker C: Yeah, yes, yes, it was. I had only a couple of hours and off sleep and um, and then

Speaker B: just came right here to oticon.

Speaker C: Yeah. Yeah. Wow.

Speaker B: I, I didn't realize it was that fast. I thought it was still you know, maybe weeks or months away. But congrats. I mean I, I'll be the first, I don't know if you had the first but you know, the first on I guess Wednesday today that we're recording from to congratulate you on the book and uh, tell us where can people find you to follow you for more work or maybe you want to, want to reach out for uh, some of your consulting or you know, take a look at your products. Where can people find you?

Speaker C: Um, I'm on LinkedIn. LinkedIn and on my website for my uh, company vexpertai.com where people can book calls for um, understanding if AI makes sense to define maybe a strategy for a given, um, use case for a particular, uh, vertical or um, particular situation. So. Yeah.

Speaker A: Yeah.

Speaker B: Well, you know, I want to thank you for being here. Um, you talk very slow, but, uh, I think it's very thoughtful responses and very solid, uh, knowledge. Even though, even if I the question I throw at you that it's not something we discuss. You answer with ease and you answer with the depth of knowledge that is high quality. So I thank you again for being here.

Speaker C: Thank you very much. Look forward. Here we are.

Speaker B: Thank you for listening to this episode of Network Animation Nerds. Do you have any feedback for this this episode? Our guest ed today? Please do send us some follow ups@packbushers.net followup. We do want to hear from you, our wonderful listeners. If you enjoyed this episode and found it helpful, I really appreciate it. If you could take a moment to leave us a review.

Speaker A: Wherever you get your podcast, your review

Speaker B: helps other network engineers discover the show and keep us motivated to bring you more great content like this one. We thank you, the wonderful listeners, for your support. Last but not least, remember that too much network automation will never be enough.