Hidden Layers · 2026-01-22 · 30 min
Key moments - from our scoring
Substance score
58 / 100
Five dimensions, 20 points each
Domain-specific AI models - systems designed to excel at one task using proprietary data - consistently outperform expensive, general-purpose large language models in enterprise settings, yet 90% of AI initiatives fail to reach production. Ron Greene, Dr. Z.Z. Tsai, and Michael Wharton from Kung Fu AI break down why narrow focus wins: a financial services model that predicts invoice delinquency using document understanding, an industrial pump failure detector using time series sensors (predicting breakdowns 24+ hours early to avoid $1M/week losses), and a tax form extraction system demonstrate superhuman performance in constrained domains. The trio outline the decision framework for when domain-specific models matter - high accuracy requirements where hallucination is unacceptable, compliance-sensitive data unsuitable for third-party APIs, latency constraints at scale, and scenarios with proprietary or out-of-distribution data (satellite imagery, fisheye cameras, industrial signals). They address common pitfalls: companies lacking sufficient historical data, uncertainty in confidence estimation, and the spectrum between pure single-use classifiers and T-shaped models that handle a few related tasks. This conversation matters for executives deploying billions in AI spend - showing where focused, data-driven approaches generate real competitive differentiation versus costly, unreliable generalist systems.
A domain-specific model is designed to do one thing exceptionally well, trained on proprietary data, and optimized end-to-end for a single task. Unlike general-purpose models like ChatGPT that handle many tasks but introduce unnecessary complexity and uncertainty, domain-specific models are faster, more precise, more reliable, and avoid wasting computational capability on irrelevant functions - a classic vision example uses a tiny Alexnet-style model instead of billion-parameter VLMs to classify objects at a fraction of the cost and latency.
The model combines document AI to extract information from submitted invoices with a decisioning engine trained on Triumph Financial's proprietary historical risk data. It evaluates whether invoices are likely to become delinquent (unpaid by debtors) and processes hundreds of invoices daily that internal teams couldn't manually review, leveraging domain-specific data unavailable to general competitors.
The model predicted pump failures more than 24 hours in advance with over 80% accuracy by analyzing time series signals (temperature, pressure, vibration) leading to failure. This prevented catastrophic breakdowns that cost approximately $1 million per week in lost revenue and weeks of downtime in remote locations, since pumps only needed preventive servicing instead of emergency restoration.
Latency and scale (cost at inference time), compliance and data sensitivity (whether you can use third-party APIs), and accuracy requirements in your specific domain where hallucination is unacceptable - such as self-driving car-level performance or detecting rare events like fraud or clinical misdiagnoses where you have proprietary data.
Fine-tuning may improve performance on specific tasks but doesn't eliminate fundamental limitations like poor uncertainty estimation (language models are bad at confidence calibration) or the inability to handle certain data modalities like time series signals that can't be meaningfully processed through text prompts.
Our reviewer’s read on each dimension, with quotes from the episode.
The episode makes several non-obvious operational points - orders-of-magnitude cost differences, uncertainty estimation weaknesses of LLMs, data retention killing projects, latency thresholds for real-time bidding - but much of the runtime repeats the core thesis that domain-specific models beat general ones.
you're talking about at least four orders of magnitude size difference between the two
they're really, really bad at uncertainty estimation because, I mean they're famously bad at that because a lot of the benchmarks reward confident guessing
The 'smaller specialized models beat frontier LLMs for narrow tasks' argument is somewhat contrarian to current hype but is itself a well-circulated take; the ASIC analogy, the spectrum framing, and the retrospective 'would you rebuild it the same way' discussion add modest fresh angles.
think about them like those um, like asics, those application specific integrated circuits
when I think about the projects that I worked on, I would say most of the project I will still not use the large language model
Guests are genuine practitioners - a distinguished engineer and VP of Engineering at an AI consulting firm with hands-on delivery experience - but they are the host's own colleagues, making this effectively an in-house discussion rather than independent senior operators.
My co founder and distinguished engineer, Dr. Z.Z. tsai, and our VP of Engineering, Michael Wharton
I come from a computer vision background. I know that ZZ does as well
Several concrete cases (Triumph Financial invoices, submersible pumps with 80% failure prediction, breast cancer risk model, 10ms ad bidding) with some numbers, but many figures are vague or host-supplied and softly confirmed rather than precise.
we built this model that I think it ended up being more than 80% of the time could predict when a pump would fail even more than a day in advance
wasn't it something like a million dollars a week in lost revenue
The host asks reasonable framing questions and there is one genuine live disagreement about whether coding agents are domain-specific, but overall it's a friendly internal chat with agreement-seeking and self-promotion rather than probing challenge.
Do you count coding agents as domain specific or not?
Here's maybe the litmus test, would you go grab one of those models and use it for um, some other language task
Computed from the transcript - who did the talking, and the words that came up most.
What if the most powerful AI in your organization isn’t the biggest model you can buy, but the one trained on data only you own? In this episode of Hidden Layers, Ron Green is joined by Dr. ZZ Si and Michael Wharton to break down why domain-specific AI models consistently outperform general-purpose systems in real enterprise environments. They explore how narrowly scoped models deliver higher accuracy, lower costs, better reliability, and stronger governance, especially when built on proprietary data. Through real-world examples spanning finance, industrial systems, healthcare, and document understanding, the conversation tackles when to build custom models, when to rely on APIs, and how to identify AI initiatives that actually make it into production. The takeaway is clear: focus beats scale, and specificity is often the fastest path to durable competitive advantage. Chapters 00:00:00 What Is Domain-Specific AI 00:01:15 General Models vs.
Transcribed and scored by The B2B Podcast Index.
Speaker A: Welcome to Hidden Layers, where we explore the people and technology behind artificial intelligence. I'm your host, Ron Greene. So here's the question for today. What if the most powerful AI in your company is not the largest model you can buy, but the one trained on data that only you own? That's the idea behind domain specific AI. These are systems designed to do one thing and do it exceptionally well. Over the past decade, we've seen a clear and repeatable pattern. When problems are tightly defined, when models reflect real operational constraints, and when they are trained on proprietary data, they outperform general purpose systems in enterprise environments and create durable competitive advantage. That kind of focus changes what performance looks like. These systems are faster, more precise and more reliable because they are built, uh, to perform a single task and optimized end to end. Today we're going to break down this and talk about what it actually takes to get AI into production. And to do that, I'm joined by two of the sharpest minds in the industry. My co founder and distinguished engineer, Dr. Z.Z. tsai, and our VP of Engineering, Michael Wharton. All right, Michael Zizi. Happy new year. Happy 2026.
Speaker B: Happy new year.
Speaker C: Happy New Year. Glad to be here.
Speaker A: I'm really excited about this conversation because this is something we've been circling around for a long time. Right. Um, I actually think that this is one of the most important discussions that we'll probably have this year because it could drive the most valuable, most value, uh, a lot of our enterprise listeners. Okay, so what we're going to talk about is domain specific models. Um, let me tee this up just a little bit. So since the rise of ChatGPT and large language models, um, we've seen, we've talked about this many times. We've seen businesses just kind of rush to adopt that and there are really many powerful things about that. Large, um, language models, um, are really good at many things. They're really the first instances of AI models that can do many, many things. They can summarize, they can classify, they, uh, generate content. Um, but with that broad capability comes, um, uh, a lot of, uh, complexity and uncertainty. Right. You know, AI is very jagged in controlling these systems in a way that is reliable for the enterprise can be really hard. On the other end of the spectrum are domain specific models. Right? That's the idea that you could build an AI system that may do as few as one thing. It's only capable of doing one thing, but it can do it potentially at a superhuman level. Right. So when I say, before we jump into Much further before I say anything more about domain specific models. Michael, what, what do you, what do you take it to mean when I say that?
Speaker C: Um, yeah, it's such a broad topic. There's so many different ways to slice it. And it's the reason that we exist, you know, as a company. I mean we can build domain specific models on people's proprietary data. And there's a, there's a lot of upside to that. But I think about it almost in terms of maybe like a toy example. Like we, um, you know, I come from a computer vision background. I know that ZZ does as well. And I think about the arc of history of computer vision classifiers. Like back in 2012, there's that Alexnet moment. It was a major leap forward in imagenet performance. Um, and for domain specific use cases, model architectures along those lines can still outperform, you know, these impressive VLMs that you have that are like hundreds of billions of parameters.
Speaker A: Right.
Speaker C: But when you start talking about cost, even if like, let's say we have two models, it's either this traditional computer vision model or a VLM like Claude Opus 4.5 or something. Even if they're at parity with performance, you're talking about at least four orders of magnitude size difference between the two. And if you don't need, like, if you literally just need to classify cats versus dogs or something like that classic example.
Speaker B: Yeah.
Speaker C: Um, you don't need a model that can also invent new recipes and write poetry and you know, translate Japanese to Vietnamese or something.
Speaker A: Right.
Speaker C: You're, every time you make an inference, you're wasting that, that capability. Right, right.
Speaker A: Um, so yeah, and you hit upon something I think that's really important is the proprietary data. I think this is going to be a big part of the conversation. So to kind of address trust, the elephant in the room, domain specific models, um, we believe very, very strongly at Kung Fu AI that they are really the primary form of AI that most businesses should be leveraging. And that means by all means go embrace LLMs. But if you want to spend money and build AI that is going to give you competitive differentiation and advantages, it's almost invariably going to be domain specific. A, um, um. So let's, let's, let's get concrete. Let's talk about a specific example. One of my favorite projects we've done in the recent years is the work with, uh, the Triumph Financial Corporation. Um, Zizi, you were instrumental on that engagement and that is a, um, what I would consider to be sort of A superhuman domain specific model. Can you describe that work and what that model did?
Speaker B: Oh yeah, definitely. So, so the company, um, so the company, the business context is that um, um, looking m at the submitted invoices and trying to decide whether it uh, is a good asset to make a purchase. Um, but the main risk is that some of the invoices may end up being delinquent. They end up being not paid by the debtor. So that causes uh, lost money and they have a big internal team spending a ton of time to review like hundreds of invoices per day.
Speaker A: Hundreds of people doing this manually.
Speaker B: Yeah. And still cannot cover all the invoices because of the volume that's submitted by the truck drivers. So uh, we came in and um, employed uh, several techniques. Some of them are document AI because a lot of the information that we are using to determine the risk of an invoice is coming from reading and understanding documentation, um, uh, accurately. And then second of all, the decisioning model that we uh, built for them, uh, needs to consider the historical risk from different factors and then try to make a decision that's fast, that's accurate, that's using the domain specific data that's only uh, available to uh, our client. So um, the model that we ended up uh, deploying was uh, the product of their proprietary data. And that's the competitive uh, advantage.
Speaker A: I think you nailed it. And to just kind of reiterate, it's this idea that you can take advantage of data that you have as a company that you may have built up over a decade or more and leverage that proprietary asset to build AI models that give you real competitive differentiation. That is the number one way we see m our clients succeed with AI. We just, we've just seen it over and over and over again. And you know, I may actually even sound a little frustrated here. I think it's because the rise of large language models in some way I feel like has sucked all the oxygen out of the room and. Too often. Yeah, too often. Uh, companies, you know, I mean companies are spending billions of dollars on AI initiatives right now and there are reports that something like, you know, 90% of them failed to make it in production. And uh, I know that we all agree that a lot of the reason is because they're building, they're building these systems that are really, really overly complex. Right. You start with something like a generative model and you try to have it be completely predictable and you end up um, essentially playing whack a mole with bugs forever. Right. Because of the sort of the jagged capability. There's another aspect of this that I think that's really interesting, which is cost. Right. Reliability, interpretability, uh, lower variance. All of those things you get kind of for free when you scope, when you downscope the initiative and you really, really focus on solving one problem well. Okay, so Michael, what's an example of a domain specific model that you've worked on in the last eight years at Kung Fu AI that you think is, um, you really sort of an exemplar of what we're talking about here?
Speaker C: Yeah. So I want to preface what I'm about to say as I, I think it's really hard not to sound like a negative Nancy about language models. I was. I just want to be abundantly clear. I use the heck out of it, like coding assistance. Um, like, I think it's table stakes and it's completely revolutionized how work is done. So I really don't want to discount that. However, there really are certain domains that you just, you still can't even work with unless you have at least some sort of form of text or image documents or, you know, whatever your data looks like. Um, there's this client we had a while ago that has, it's a really interesting use case. They have these electrical submersible pumps that you bury deep in the ground for, uh, mostly industrial applications. And what would happen is they would have these failures that would happen occasionally. And they're very, very expensive pumps, like, you know, seven figure kind of high quality machinery type pumps. And we found that if we just looked at time series signals that were captured from the, the pump. So that's things like temperature, pressure, um, vibration, all sorts of sensors you'd imagine would be on a device like that. You can look at these historical failures and look at all the time series signals leading up to them and predict more than 24 hours in advance, often if there was going to be an issue because it turns out that there's this like cumulative degradation that happens where maybe there's like a little pocket of silt that goes in and there's a dynamic event that you can pick up. It took us a while to figure this out. Um, but we built this model that I think it ended up being more than 80% of the time could predict when a pump would fail even more than a day in advance. And in that case all you do is just pull the pump and service it and put it back in instead of losing.
Speaker A: And remind me when the, when the pumps go down, what was like the daily, um, impact.
Speaker C: Oh, it's, it's definitely, um, well, if it, if it breaks completely, then they have to completely restore. And I think it was, it could be weeks because some of these places are literally out in the middle of nowhere and like Odessa and weren't they,
Speaker A: I mean, you know, wasn't it something like a million dollars a week in lost revenue or something like that? I mean it was enormously impactful.
Speaker C: Yeah, yeah, I think you're, you're right on the money with the order of magnitude there.
Speaker A: Okay. Yeah, yeah, yeah. Um, so, all right, so let's say somebody's listening to this and we've convinced them somewhat that okay, maybe domain specific models are an interesting approach. Um, how do you know, how do you know what to do?
Speaker B: What's the decision?
Speaker A: Yeah, how do you make a decision? How do you identify where to go?
Speaker B: Yeah, I think we all have been thinking about this every day when we work, um, for our clients. Uh, um, but yeah, I would love to hear what, your thoughts too, but I'm going to throw, throw out a few first. So, um, first factor would be uh, latency, um, and then the compliance, whether your data is highly sensitive, whether you're comfortable using a third party API, and also, um, accuracy on your particular domain. That's the three things that's on top of mine.
Speaker A: But okay, so unpack that a little bit. So accuracy, obviously the point there is you're in a situation where sort of the um, the uncertainty of whether you're dealing with a hallucination is just not acceptable.
Speaker C: Right.
Speaker A: You just have really, really high accuracy requirements.
Speaker B: Yeah, I mean, let's just talk about some concrete, uh, examples. Let's say we wanted to build a model to detect whether my garage door is open. How would you uh, do that today? Well, I guess my answer today will be very different from uh, three years ago even because, you know, three years ago there's no, you know. Well, I guess Google Vision API is there, but so let's go back even more, maybe five years ago, then maybe your only choice is to take a model that's pre trained on uh, imagenet and then you fine tune it on your own data.
Speaker A: Right.
Speaker B: Today you can use ChatGPT API. Sorry, you can use the GPT API or you can use cloud API, you can upload the image and I think the accuracy is fairly high already.
Speaker A: Agree.
Speaker B: Maybe 90%, maybe 95%.
Speaker C: Right.
Speaker B: And if your accuracy requirement is not like 989.9999, maybe you're okay.
Speaker A: Right.
Speaker B: However, if you're like you know, kind of self driving car kind of accuracy requirement, then you can see a clear argument.
Speaker C: That's a great example.
Speaker A: It is. And, but I think you also touched on something really, really important which is, you know, identifying whether a garage door is opened. See that's a problem that is so common. Or it's in the sense it's in the real world that you've probably, you could make the argument that you know, just going with some type of API off the shelf is the way to go because it almost certainly could handle that scenario really reliably.
Speaker B: Is it in distribution or not exactly.
Speaker A: Exactly. Um, where I think for businesses that are thinking about domain specific models, almost by definition the problem you're trying to solve is not something that's out in the public domain or that is common. You're trying to do things like um, if you're in healthcare, you're trying to detect um, uh, clinical misdiagnoses or conditions or if you're in banking, we've already talked about loan decisioning, maybe you're trying to detect fraud. Um, these are all cases where you will probably have proprietary data and it just wouldn't be feasible to use something like an API.
Speaker B: Right, yeah, absolutely agree. Like you know, what if you have a drone image, you know, or a satellite image or a fisheye camera image that's you know, not common in those training data set that trend GPT4 Vision or you know, Claude.
Speaker A: Yeah, that's exactly right. Mike, what are some of the things that you look for when you're, when you're talking with clients, you're trying to identify opportunities, um, and you want to make sure that you know there's going to be return on investment and that they have the sufficient data.
Speaker C: Yeah, there are definitely a few. I think ZZ touched on some important ones for sure. Um, yeah, just to take his example further, I think about scale a lot like Doc AI comes up a lot of times. But if they're trying to make hundreds of thousands of inferences every hour or something like that, or let's say it's like a loading dock where there are a bunch of different bays with their own garage doors and you need to assess whether a garage door is open once every 30 seconds or something, it's way too inefficient to just constantly rely on some frontier model that's got hundreds of billions of parameters or whatever.
Speaker A: Right.
Speaker C: Um, so the cost at scale is a big one. There are also just some things that language models or VLMs simply can't do. Like they can't do. Or they're really, really bad at uncertainty estimation because, I mean they're famously bad at that because a lot of the benchmarks reward confident guessing. Um, so, you know, you ask a model, hey, how confident are you? There are plenty of studies that look at this, but it's, you know, they'll often say, oh, I'm M70, 80% confident. Yeah, but for the most part there's very little correlation between those confidences and the actual performance. Um, so if, if confidence estimation is, is a big part from the client's perspective, I'll, I'll ask them about that a little bit more. Um, what about the data?
Speaker A: Like how do you, how do you know, how do you know if, if a client, um, you know, we get asked all the time to, to solve problems and it's not even clear that they necessarily have the data.
Speaker C: Yeah, data volume is really, really important. I mean especially if they're trying to do something that is differentiated. Yeah. Um, I can't tell you how many times I've talked to a client, they're like, hey, I want to do this amazing thing. I have this incredible vision for this AI system. Um, but the type of data they would need is pretty out of distribution from what you'd expect with a Frontier model. And M, in the worst cases that I've seen, they have a retention policy that's really short and they had the data but then they no longer have it.
Speaker A: Yeah, we've seen that a couple times.
Speaker C: You know, um, but there are certain data modalities that just don't even. Like they're non starters. Like I mentioned uh, time series signals earlier. There are a couple people that have tried to do uh, I think Nixta, uh, was one of the companies that tried to do a time series foundation model. I haven't really seen that take off all that much. But if you're dealing with sensors and you're doing industrial automation, um, you can't just copy and paste a time series signal into a prompt and expect anything meaningful.
Speaker B: Yeah, yeah, I totally agree.
Speaker A: I feel like on the data side this is um, the number one issue. Well, sit back. Data is always the issue with AI, right. We're always dealing with data issues, either data volume or data quality or um, the thing you're trying to predict is um, really m rare, for example fraud or cancer. Um, and so you don't have many instances of that. But that's actually, you know, if you think about it, that's actually a real opportunity for businesses if they have a Collection of data and they have a collection of rare events, they absolutely can leverage that to build.
Speaker C: That's the place to start.
Speaker A: That's the place to start. It really, really is. Um, the problem is the opportunity there. Right. So on the flip side now I want to just kind of think about this in an opposite way. How do you know when um, uh, a domain specific model is not the right approach?
Speaker B: Do you count coding agents as domain specific or not? I would think it's, they're general but.
Speaker A: Oh, that's a good question. So if you're working with CLAUDE code or something like that, is that a domain specific model? I would probably lean towards yes.
Speaker B: But you would think okay, that's fair.
Speaker A: No, you don't think so, uh, because,
Speaker B: well I'm kind of on the fence. I mean coding is more specific than just general language model.
Speaker C: Right.
Speaker B: However, coding is so ubiquitous. You know, software is the world and if you can generate code.
Speaker C: I think that's the trick though. It's, it's, it's really more of a spectrum on like how narrow is the task that perform like you like in the most extreme case you have almost think about them like those um, like asics, those application specific integrated circuits like cameras have very specific tailored hardware that's meant to do some like extremely narrow thing like convert light rays to pixels and. Right. Or whatever. Um, on one end of the spectrum you have these hyper specific things that maybe it's like a garage door detector and that's it.
Speaker B: Yeah.
Speaker C: And then as you get more and more generic, I would put coding agents, fine tuned language models and general purpose VLMs like way over here. And I think often what we see is that the differentiated use case that's the most going to produce the most value for a client. It's like way down here on the spectrum. Right. Um, but to your point, I don't think that there's a meaning other than scale. Like there's, there's slight difference.
Speaker A: It's definitely a spectrum. It's blurry. And I do believe that domain specific models don't have to be single use case. I do think you can have models
Speaker B: T shaped kind of model.
Speaker A: Exactly. That's exactly right. So you can, you can have uh, you know, you can be capable of several capabilities.
Speaker B: Right.
Speaker A: Um, it doesn't have to be just narrow means one. But I think I do um, I don't know. I think I do view AI coding assistants as narrow in the sense that they're really trying to do one thing. Here's maybe the litmus test, would you go grab one of those models and use it for um, some other language task as a starting point? Probably not. You'd probably use something that was more general in nature.
Speaker B: Yeah, right, yeah. But uh, for coding agents, would you fine tune that model on your own data? That's a question I often think, you know, okay, do I want to fine tune? Because the cost of fine tuning is getting lower every like thinking machines, their Tinker API, you can fine tune like you know, using $5, $10, you can fine tune a Lora, you download it. But then, you know, is it like, is the quality improvement using your own code make uh, up for, you know, whatever you invest? Or is it better to wait for Cloud and OpenAI or whatever other companies come up with their new version?
Speaker A: Yeah, I think it's probably the latter and I think for a couple reasons. One is things are moving so quickly on the AI coding front that um, it's almost certain that within six months, um, there'll be another quantum leap on that front and you can just wait and leverage that, not have to spend any money. The other thing is that I feel now with the ability to specify and have things like skills and other directives, you can describe the topic of coding that you want to be performed and you can kind of guide in a way that doesn't necessarily require fine tuning. That said, I do think if you were going to like convert legacy applications like you know, move from Cobol to Python or something like that, that might be another Fortran or something.
Speaker B: Yeah, that's a good point. That's a good point.
Speaker A: All right, zz, what do you think, sticking with that theme, what do you think about the idea of not encoding agents but just fine tuning a model? If I took a large language model and I fine tuned it, would you consider that a domain specific model?
Speaker B: Yeah, we were actually um, actively considering that in the, that we are working on tax processing. Uh, right now we build, um, it's sort of an agentic system that uses the cloud API in combination with the OCR model to feed the extracted content and then to parse it and.
Speaker A: Extracting information from documents.
Speaker B: Yeah, extracting information from complex text documents, uh, with very dynamic schema. And it could be not just one tax form, but could be many forms all together in one package. Right. Um, so, um, it's working fairly well um, with the uh, API. I think it's a good starting point. That's another benefit of uh, such frontier general models is that they give you a strong baseline. However, um, we found that there's hallucination for sure. Because if the model has to reason across multiple pages it oftentimes it confuses itself. Hey, which page should I look at to extract a certain um, type of information? So uh, we are running experiments and planning to run more uh, to fine tune both the OCR as well as the reasoning model, uh, to just to improve uh, more accuracy and to reduce the uh, hallucination. And uh, I think possibly we can also fine tune a verifier model that takes the document and takes the extraction output and then just do the verifications and trying to output a confidence score because that's a um, um, missing piece for a lot of the off the shelf models. And I think we can use uh, superv fine tune and we're working on reinforcement learning to fine tune such models. And I'm super excited that um, and looking forward to uh, better performance.
Speaker A: All right, so there's one more area that I want to hit upon that I think is really, really interesting. And actually zz, you touched upon this a little bit earlier when you're talking about latency and other sort of. Yeah, I think most people, most businesses when they're building something, they're building an AI solution, they're really just thinking about how good can it be, how accurate. But there are a lot of other things. There's cost of ownership, um, that can be really, really expensive to build. Models, uh, there are things like governance, there can be uh, proprietary, um, uh issues around the data or maybe even privacy constraints. I mean all these things really factor in and across the. Again domain specific models have so much to offer there because they're um, like you said Michael, they can be orders, many orders of magnitude, uh, smaller which makes them much uh, less expensive. It means you can put them on the edge, right. If you have really low compute capabilities or compute constraints. To your point zz, if you have latency issues, really really effective there. Um, so for me the idea of companies that are taking their first step into AI and they're trying to figure out what to. It is this simple. Be focused, look at what data that you have that is proprietary that you can use to build domain specific models that allow your business to um, introduce new capabilities, reduce the cost of existing functionality within the company, whether it's client facing or internal, uh, and give you real competitive advantage. Not to mention all the other things we've talked about that these things are easier to train, evaluate, ah, govern, um, and less expensive to maintain. So for me, m. Again just to reiterate, I'M completely with you guys. Large language models are astounding. They're amazing. There are boundless instances where they are the right choice. But for most enterprises, um, I think we all agree, um, think more specific and if you can constrain your requirements and leverage proprietary data, you're going to have a much, much more likely chance of having AI investment that everybody's spending right now be successful and go into production.
Speaker B: Such a great point.
Speaker C: Hear, hear.
Speaker A: All right you guys, um, I mean any other, any other thoughts here?
Speaker B: Um, speaking of latency, I, you know, thought of some examples and looking at, and actually more broadly I'm just, you know, looking back at the projects that we worked on and trying to think, okay, with today's technology, what, what do
Speaker C: you redo it different?
Speaker B: Yeah.
Speaker A: Would you do it differently? That's a great question.
Speaker B: And surprisingly when I think about the projects that I worked on, I would say most of the project I will still not use the large language model or diffusion model, the frontier models of today to solve it. One example is um, real time ads bidding.
Speaker C: It's 10 milliseconds.
Speaker B: 10 milliseconds that most of the model is just out of the way and we just have to use a very small either tree based model or linear model and we need to cache the hell out of the system to uh.
Speaker A: That is a really great perspective which is retrospectively. Would you change how you architected something? This is over the last eight years. And the answer is for the most part no.
Speaker B: Right. Surprisingly I would think that for example, clarity, where we build an image, ah, image based model to predict the cancer risk for the next five years. Um, I'm curious Ron, what, what's your opinion with the, you know, latest blams?
Speaker A: Um, I totally know. I completely agree. I think, I think the approach, just taking a complete um, image based approach is 100% the way to go. And, and I love this. It's like this model can only do one thing. All it can do is predict the risk of breast cancer.
Speaker C: But that one thing, it's good at it.
Speaker A: It's really good at it and it's going to save millions of lives. And that's a perfect example of the power of focusing your solution and not trying to boil the ocean.
Speaker C: M. You know what I would wager though is that the process that we would have taken to get to that result would have looked wildly different in almost every project still. I mean with like the advent of coding assistants, coding agents, if we had like um, even just debugging tools.
Speaker B: Oh yeah. Um, deploying tools.
Speaker C: Yeah, deployment tools, like all that stuff. I think it'd be fun if we could just magically rewind the clock and go back and relive some of those fake moments.
Speaker A: Well, there are certainly some projects I remember from seven, eight years ago that were bleeding edge and it took every brain cell we had to accomplish. And now I think we could do those in, uh, a tenth of the time. Right?
Speaker B: Yeah. Remember the first document understanding project where we still train these very specialized object detector to detect the small fields.
Speaker A: Yeah. To even find the fields.
Speaker C: Yeah.
Speaker B: And then ocr.
Speaker C: Yeah. We did some wacky stuff back then.
Speaker A: Well, AI AI just keeps on changing, keeps on moving. Okay, well, I think that's a great point, uh, to stop right there. Guys, thanks so much. Happy New Year, and thanks for being here.
Speaker B: Thank you.
Speaker A: Thank you for listening to Hidden Layers. This series is hosted by Kung Fu AI, a management consulting and engineering firm focused exclusively on artificial intelligence. If you have any questions or thoughts about today's episode, or if you know someone we should feature, please visit us at Kung Fu AI.
Other episodes covering the same guests and topics, from across The B2B Podcast Index.
AI Is Designing the Next Cancer Fighter | EP.5383 / 100
Anthropic Code Leak: A Rare Look Inside Frontier AI | EP.5282 / 100
The "AI Bubble" Bubble | EP.5174 / 100
Did AI Kill Programming? | EP. 5072 / 100
AI Year in Review - Key Moments, Hot Takes, and 2026 Predictions | EP. 4877 / 100
