
The AI in Business Podcast · 2026-11-10 · 23 min
Key moments - from our scoring
Substance score
59 / 100
Five dimensions, 20 points each
Choosing the right AI model starts with defining the exact work it must perform, not with budget or brand prestige. Melissa Ramey challenges the assumption that expensive frontier models deliver better ROI, drawing on research showing that 95% of frontier model quality can be retained while routing 80-85% of tasks to cheaper models. She uses practical analogies - a Ferrari can't haul lumber, and a Stradivarius sounds terrible in untrained hands - to illustrate how model selection must match task complexity. The episode addresses a widespread problem: accountability offloading, where teams deploy polished AI outputs without scrutiny because they assume more expensive models guarantee quality. Ramey advocates for lightweight models for simple tasks like email editing or call summaries, reserving expensive models only for multi-step reasoning work. For C-level executives, she prescribes a disciplined process: involve the people doing the work, test models side-by-side on representative tasks, and treat AI agents like ongoing employees with regular check-ins rather than set-it-and-forget-it tools. The guidance applies across Salesforce's ecosystem and broader enterprise deployments.
Hosted by Daniel Faggella. The show scores 79.0/100 on substance across its analysed episodes. See the full show profile.
Expensive frontier models are designed for complex, multi-step reasoning and tend to over-elaborate on simple tasks, producing verbose outputs that are slower and often less accurate for lightweight work than purpose-built lighter models.
Define the exact job the model must perform, involve the people actually doing the work in that definition, test candidate models on representative tasks, and choose based on output quality, speed, and cost - not brand or feature count.
No; you can retain 95% of frontier model quality while routing 80-85% of typical organizational tasks to cheaper models, and upgrading later is cheaper than overpaying for unused capabilities today.
Treat AI agents like employees with regular check-ins - at minimum annually, but often weekly - to verify output quality, update grounding knowledge as business needs change, and assess whether the tool still solves the original problem.
Accountability offloading is deploying polished AI output without proper verification, assuming it's ready because it looks good; it's caused consulting firms to release fraudulent work and governments to pay millions, making rigorous output review essential before deployment.
Our reviewer’s read on each dimension, with quotes from the episode.
The episode contains several genuinely useful frameworks (task-first model selection, accountability offloading, the Stradivarius analogy, comparing model performance empirically) but these are interspersed with significant filler and repetition. Melissa's core insight - that model choice should follow job definition, not precede it - is valuable but retold multiple times across the conversation without densely packed new claims. The accountability offloading concept is underdeveloped and lacks concrete examples beyond vague references to government and consulting firms.
it's not the model that changes the value, it's the way you use the model that changes the value
the job doesn't follow from the model. So find the job you need and then hire the tool for the job
While the task-first framing is sensible, the underlying ideas (right tool for the job, avoid over-engineering, test before committing) are well-established principles applied to a new domain. The accountability offloading concept shows some originality but is barely explored. The episode largely recycles familiar advice about matching requirements to tools without surprising counterintuitive claims or first-principles thinking. The blender and box truck analogies, while clear, are not novel.
You wouldn't just hire a smart person and hope that you could find them a job, but that's exactly how we're hiring AI
expensive models doing easy tasks can actually be more inaccurate, according to research, than a lighter model doing that light task
Melissa Ramey holds a credible title (Director of AI Transformation, Data Insights and UX at Salesforce) and demonstrates hands-on practitioner knowledge through personal anecdotes of struggling with model selection and prompting. However, the transcript does not establish depth of operational scale or concrete impact (e.g., size of initiatives, revenue impact, years managing these decisions). She speaks with authority but the evidence of having built and scaled AI systems is implicit rather than explicit.
I use AI in my work all day, every day, right?
Director of AI Transformation, Data Insights and user experience at Salesforce
The episode is notably sparse on concrete data and named examples. The accountability offloading claim references a government paying 'millions' and a consulting firm with '95% fraudulent citations' but provides no names, timelines, or verifiable details. No specific companies, customer examples, or quantified outcomes from Salesforce AI initiatives are mentioned. The 95% frontier model quality claim is cited as 'research' without source. Most advice remains at the principle level without grounding in specific use cases.
One entire government had to pay millions of dollars. A major consulting firm had like 95% of their citations shown to be fraudulent
you can keep, according to research, 95% of frontier model quality while rooting 80 to 85% of those questions to cheaper models
The host asks reasonable opening questions but largely allows Melissa to deliver prepared talking points without pressing for depth or disagreement. Follow-ups are soft (e.g., 'When do you get to that point?') and rarely challenge claims. When the host offers a counterargument (the 'blender' thought experiment), Melissa responds but isn't genuinely pushed. The host does ask about executive psychology and implementation friction, which shows some strategic thinking, but overall this reads more like a friendly interview than a critical conversation.
So, Melissa, welcome to the AI in Business podcast
I guess that's why you get to be so practical, is because these are real life things that you've had to learn and work through
Computed from the transcript - who did the talking, and the words that came up most.
Choosing an AI model without defining the task can leave organizations paying for capabilities they do not need or relying on tools that cannot handle the work. In this episode, Melissa Ramey, Director, AI Transformation - Data Insights and User Experience at Salesforce, joins Emerj's Yolandi de Weerdt to examine how leaders can evaluate models against the work their teams need done. She discusses testing models on everyday tasks, involving employees in workflow assessment, and revisiting performance as requirements change. Learn the exact strategies we use to help leading AI brands and startups
Transcribed and scored by The B2B Podcast Index.
Speaker A: Welcome everyone to the Emerge ah. AI in Business podcast. Today's guest is Melissa Ramey, Director of AI Transformation, Data Insights and user experience at Salesforce. Melissa joins us to discuss how leaders can select AI models based on the work their teams need to do. She explains why teams should compare models on specific tasks and account for the people and processes involved before making investment. Just a quick note for our audience that the views expressed timeless on today's program do not reflect those of Salesforce or its leadership. If you offer AI products or services into the enterprise, you need to find enterprise leaders with relevance. That means the right title at the right type of organization, and of course, readiness. Emerge attracts VP and higher ranking enterprise audiences who are already convinced that they need to move beyond traditional IT to learn the exact strategies we use to help leading AI brands and startups connect with their ideal enterprise AI buyers, visit emerge.comad1 that's E M E R- dot com ad1 now the conversation with Melissa. Melissa, welcome to the AI in Business podcast.
Speaker B: Thank you so much. I am so excited to be here.
Speaker A: So our, uh, listeners, they don't know this, but you and I, we've had conversations in the past and just a heads up for the listeners, Melissa is super passionate about this, so you are in for a treat in today's conversation. And she has this superpower of making everything very understandable and very practical. So Melissa, no pressure today.
Speaker B: I feel no pressure to be practical and passionate about something. I love this much. Thanks so much for intro.
Speaker A: Great. So let's get into it. Everyone talks about better models, like it's obvious what that means. But when we just say, oh, this needs to be better and that needs to be better. But when you dig into it, the line between worth it and not worth it gets blurry very fast because we really don't speak the same language on that just yet. So once you dig into the actual difference in performance, what separates a model that's worth the extra costs from one that isn't?
Speaker B: So I would actually just change that a little bit and say it's not the model that changes the value, it's the way you use the model that changes the value. All the models have value. It really comes down to what is the work that you're trying to do and do you have the skill to use the model to the best of its ability? I think of it a little bit like a Stradivarius. A violin. Anybody can pick up a Stradivarius, but if you don't really know how to play it, you're not going to get the sense that it's this incredible, you know, musical instrument. You really have to know how to use it, and you need to be able to use it for the right thing. So I would just change it a little bit and say every single model that's out there, from the least expensive to the most expensive, is a valuable model. And the real question is, what is the human making that choice trying to do? And do they have the understanding. Understanding of that model and the prompting skill in order to get the most out of it?
Speaker A: Interesting. So we're not thinking about cost, really. We're thinking about practicality and where it's going to be used, how it's going to be used, and things like that. So have you seen teams pay for power that they don't actually need, power that they've never used, and then because of that, they feel that the AI is not worth it
Speaker B: 100%. And, you know, there is research out there that will tell you that the more expensive models are slower, they cost more, they tend to overthink simple tasks, and they are very, very verbose. So for all the people that are like, why is my AI model talking so much? It's because you're using an expensive model that's doing a lot of reasoning. And so when you needed a simple task completed, you feel like it's failing because you're like, I just need a sentence. Why have you just brought me six paragraphs? So, absolutely, I have seen teams that think, oh, I need the most exp. Or I need this kind of tooling. Right. There are teams that are choosing agentic models like cursor or CLAUDE code, which are more expensive than, say, a, uh, typical LLM like a chatgpt or a cursor. Without really understanding that the work that you're trying to do must determine both the kind of tool that you choose and the model that you choose within it. And if you choose a lower model for a complex task, you will not get what you need. But the inverse is also true. If you choose a very expensive model for a very simple task, you will walk away just as frustrated. Because expensive models doing easy tasks can actually be more inaccurate, according to research, than a lighter model doing that light task. As,
Speaker A: uh, someone that loves to have the top tier of everything. I know that what you're saying is true. I just don't like hearing it. Because, let me, let me tell you my thinking, and we can maybe see if that's also relevant to the enterprise world, is if I want to Go and buy a blender. I don't go to my local supermarket that sell very entry level blenders and get that because you know, it's just for me, I think, okay, but what if I want to become a professional smoothie maker? So I'm going to go and order the industrial grade blender, uh, and now I have this super powerful blender that I most probably won't ever use for what it's built for. So my thinking is always, okay, but what if I become this professional smoothie maker? So isn't there a risk with AI that today is good enough and this is good enough for what we need? Is that not going to become tomorrow's bottleneck as the needs grow?
Speaker B: Well, it's a great question and there's two things I'm going to say about it. Number one, good luck in your professional smoothie making career. I think that sounds amazing, but there's a couple of things to keep in mind. Number one, you can keep, according to research, 95% of frontier model quality while rooting 80 to 85% of those questions to cheaper models. Let's talk about what this means practically. So let's imagine that you need to run to like a home improvement store to pick up some lumber. And you have the choice. You can either take the box truck or you can take the Ferrari. The Ferrari is exciting, it's expensive, it looks like the very best thing. But is it actually the right tool for the job that you are trying to do? You're going to burn up what, $100 in gas just trying to get there. Once you get there, you're going to realize that the car you've brought cannot hold what you're trying to buy. It's a great, it's a great car. A Ferrari is a great car, but it cannot do the job of a box truck. And I think the question for us always comes down to if we're using a better thing but getting worse results. That is an output, but it's not an outcome. And all we would have to do seriously in order to test this is to go into your models and put together whatever your prompt is. Okay? And I would say keep it for this task relatively simple. We're not talking about the most complex tasks because every single model will tell you for your hardest jobs, you need to use those most advanced models. And when they say your hardest jobs, they mean multi step long track reasoning. Right? That's what they mean by that. So if you've got that you need those most expensive models, I pay for the most expensive version of Claude, and I pay for the most expensive version of ChatGPT. And I know the value of those, so I save them, right? So if you're doing those jobs, great, get in your Ferrari and, and let it do amazing things for you. But think about your everyday work, right? You are trying to write an email and you need just a little bit of editing or you've got a call that you were in and you need to summarize it. Just do it. You don't have to trust me, right? Go in and use your Fable 5.1 and then also use one of your older models and see what comes out. And the result is often that you're looking at these two results and thinking, okay, Fable took me three times as long. It came back with three times as much information, and I feel like it's overcomplicated it. Whereas when I went in and I chose something much simpler, a much older model, what came back to me was a lot faster and tends to be a lot closer to the work that I was doing. Again, you don't have to take anybody's word for this. You can just do a simple test. I think, though so much of prompting is a skill that is assumed because these models use natural language, it can be hard without really thinking through it. That a prompt for advanced reasoning is very, very different than a prompt for a simple task. And if you're going to do a prompt for advanced reasoning, it needs to be long and extremely detailed because the more you have to come in and make adjustments, the more you're weakening the power of that initial prompt. But lighter reasoning models were built for that. So don't take my word for it. Do a small investigation in your own models, see what comes out. And then I think that is the thing that will convince you more than me or anybody that all these models are not built the same.
Speaker A: M. I think I agree with you with this. I think everyone thinks bigger is better. And I'm thinking about the psychology analogy behind that as well. What is driving this instinct to go big? Is it fear of being wrong or missing out on something, or is it something else? What have you picked up?
Speaker B: I do think it's a couple things, but one of the things I've been looking into a lot recently is something called accountability offloading. And accountability offloading is affecting industries everywhere. Big consulting firms, entire governments. It's affecting legal, medical. Every profession essentially is in a place right now where we are taking things that AI has given us, and then we are just putting our names on them and releasing them out to customers and the broader public. So accountability offloading comes down to AI, produces a lot of volume very quickly with a lot of polish, right. I kind of skim through that. It looks pretty good. And then I just put it out there, right. I assume that because it came to me fast, looking really polished, it's probably ready. But as we're seeing all across, as I said, all across these industries, one entire government had to pay millions of dollars. A major consulting firm had like 95% of their citations shown to be fraudulent. Accountability offloading is happening to all of us. So if you take the idea of accountability offloading, where we essentially put an idea into whatever model we're using, we get an output, we read the first few sentences or paragraphs, it looks great, and we think, oh, this is great, this is ready, then the correlation to that is a better model is going to do that even better, right? So if the goal I have of AI is that I put my idea in, I get an output, I never have to read the output, and then I just send that work on to other people, then the bigger model that takes longer and puts more words on a page must be the thing that's going to be even better, right? I think that's a little bit of the psychology that's going on that if the old model gave me some good words, the better, more expensive models are going to give me more, better words, and it's just not the way the models necessarily work.
Speaker A: Yeah, I think that you're so right there. And I'm wondering also, what eventually triggers you to think this way? Because I'm sure that, like you just said the example with go and test your older models and the newer ones and you for basic tasks, you'll probably still want to use the older ones. When do you get to that point? Or what triggers you to get to that realization where you're not trying to convince yourself that bigger is better?
Speaker B: I use AI in my work all day, every day, right? So I've had that moment where I'm sitting there with a very, very good model getting absolutely terrible results. And I'm thinking, what's going wrong? And like so many people, um, right. All of us are in this boat learning AI together. Like so many people. I get to a point where I've put a really good prompt in at the beginning, right? And I've gotten like, pretty good outputs, but now I'm trying to edit it, and with each new edit, my prompt is getting shorter and oftentimes more Frustrated until finally I've been working with say a very expensive model for an hour. And the very first results I got were like close but needed editing to the point that I'm at now where I'm frustrated, I'm blaming the AI. And this is not hypothetical, this is real. Like I still go through this as well and I'm thinking, what's wrong with the AI? And then I go over to another model, just like a Gemini or something, and I'm like, I just need you to do this simple thing. And it turns around a very simple edit or polish for me. And that's what started leading me down this road of um, again, it's never the fault of the AI. The AI is doing for the most part what it is told to do. So we have to bring the accountability back to both the input and the output to say, okay, if the AI output is no longer what I expect it to be, what have I done? Have I not written a good prompt? Possibly. Has my frustration caused me to start prompting increasingly poorly? Like why can't you just do. This is not a good prompt for AI, right? So I think that's a big part of it is that because I use AI so much, I started seeing a pattern emerge of I write a really good prompt, I get a good but needs editing output, I start prompting that in little chunks, in little pieces and my output is going down. Then I go over with that simple task to a lighter model. I get a fast good answer. So I started with the expensive model with a great detailed, multi stepped piece of work that was right for the model. But by the end of that, I'm prompting little things. That's not how the most expensive models work. Which is why the lighter model could do what I was needing. So again it comes back to me, do I understand the nature of the work I'm trying to do and have I chosen that tool and Yolandi? That is all from feeling the frustration of my own usage of AI and trying to learn how to use it better.
Speaker A: I guess that's why you get to be so practical, is because these are real life things that you've had to learn and work through. So thinking like a C level executive, when they hear this, they say, okay, this is very possible when we're speaking about LLMs and just data use. But if we're thinking about the AI investment we made to just automate this or to do that, thinking about where to invest the money and they're still stuck in that bigger is better. Because how do you Taste this. Without it turning into a huge product on its own, what guidance can you give our leaders there?
Speaker B: For every single person using AI that I talk to today, the question I ask is, what is the exact thing that you're using this for? So AI transformation of an org is not an answer, right? Adding more productiv to an org is m not an answer. What people have to understand is what is the exact thing that I am hiring AI to do? You wouldn't just hire a smart person and hope that you could find them a job, but that's exactly how we're hiring AI, right? We're going out and spending and finding the best and spending a lot of money and then hoping that we'll figure out the job that it's going to be able to do. So I would say start. Start with understanding the job that you need the AI to do. Find out if the architecture of the tool you're looking at can handle that job. Understand what it's going to take in order for that job to really work, and then start thinking about, what model am I going to use. But the model follows from the job, the job doesn't follow from the model. So find the job you need and then hire the tool for the job.
Speaker A: Oh, I love that. I love that. And I think it fits in perfectly with our last question for this episode. So, and I think this is really the question underneath everything that you've been talking about. If you're a leader who has to make this call for your team, is that exactly where you would start? Would you start with sitting down and dissecting the job or jobs that this agent would have to do? Or how should a leader go about deciding what to deploy?
Speaker B: 100%. I would sit down and I would say, talk me through this. And I think the skill of interviewing to understand this is not as simple as, like, oh, here's my job. Like, it is challenging to really understand the job. So I don't want to make that part sound easy because I do think it is challenging. It takes time. You need people in the room who are doing the work who can say, oh, but you're not considering this yet, or you haven't thought about this downstream impact. So I think this is challenging work. The question is, are you going to do this work after you've made the investment and already spent your budget on the AI, or are you going to do this before you've made the investment to ensure that every single output from that AI leads to an actual measurable outcome? Because you're going to have to do it. The question is, do you want to do it before you spend the money or after you've spent the money in the wrong direction?
Speaker A: That's only one way that this needs to go. Unless you want to answer some very uncomfortable questions, I'm going to work in one more just because I have you here and I. I'm really enjoying this conversation. So how often should this, should the team revisit the choice that they made in the beginning? Because jobs do change and requirements and needs do change. So how often should we go back and revisit the choice and see if this is still working for us?
Speaker B: Yeah, I would say we have to think about AI Just like in many ways an employee, nobody hires an employee and then just doesn't ever talk to them again. Right. There are annual check ins, there are often weekly check ins, there are skip levels. There are so many ways that we make sure that our human employees understand exactly what we need them to do and that we're continuing to assess their output. Agentic AI needs to be treated similarly. This isn't like you buy it, you set it up and now it just runs like Microsoft Word or whatever software you were running before. This is now a dynamic, non deterministic tool that is making decisions based on the output that you give it. You should be just like with any other part of your organization where decisions are being made, you must go back in and say, are these the decisions I need to be made? Does this tool understand or does this human employee understand the shifts in our business? Treat it the exact same way. Check in with it regularly. Make sure that you're continuing to update grounding knowledge to reflect changes in your business, changes in the micro, uh, or macro climates around your business. Continue assessing the quality of the work. I never would think of these things as really being finished. That doesn't mean that you're not getting outputs that are leading to outcomes in the same way that it doesn't mean that if you have to have a one on one that your employee failed. It means that you're continuing to check in to make sure that that employee, whether digital or human, continues to understand is the work I'm doing. Right. Okay. What needs to change and where do I need to be looking next?
Speaker A: I love that. And I think that's exactly what I want to leave our audience with today is to think on that. That's our final thought. Melissa, thank you so much for being on the show, for sharing the information with us. I think this is extremely valuable for our listeners and there is going to be a lot to take away from this.
Speaker B: Oh, uh, wonderful. I had a wonderful time. Always love talking with you, Yolandi.
Speaker A: Wrapping up today's episode, let's look at the three key takeaways from our conversation with Melissa. Um, first, define the specific job the model needs to perform and and involve the people doing the work before committing to a tool. Second, compare models on representative tasks, looking at whether the results meet the need as well as the time and cost involved. And finally, review agent performance regularly and update its grounding information as business requirements change. If you offer AI products or services into the enterprise, you need to find enterprise leaders with relevance. That means the right title at the right type of organization. And of course, readiness. Emerge attracts VP and higher ranking enterprise audiences who are already convinced that they need to move beyond traditional IT to learn the exact strategies we use to help leading AI brands and startups connect with their ideal enterprise AI buyers. Visit emerge.comad1 that's E M M E-R dash.comad1 for further executive level analysis and to join our network of leaders delivering workflow impact with AI, visit emerge.com on behalf of the team at Emerge. We'll see you on the next episode. Um.
Other episodes covering the same guests and topics, from across The B2B Podcast Index.