Runtime Confidence Meets Real-World Adoption: How AI Is Transforming Traditional Industries | The Pair Program Ep82

Speaker A: Welcome to the PEAR program from Hatchpad, the podcast that gives you a front row seat to candid conversations with tech leaders from the startup world. I'm your host, Tim Winkler, the creator of Hatchpad.

Speaker B: And I'm your other host, Mike Gruen.

Speaker A: Join us each episode as we bring together two guests to dissect topics at the intersection of technology, startups and career growth. Welcome back to the Pair program. I'm your host Tim Winkler, joined by my co host, Mike Gruen. Uh, Mike, I was, uh, reading on, uh, my kind of like, daily newsletter. Uh, anyways, Stephen, uh, King movie that just came out the Long Walk. This theater in LA was hosting a screening where the audience has to walk on treadmills for the entire runtime, which is 108 minutes. And they locked him at 3 miles per hour. And if you stop walking, you get kicked out. It's like a brutal way to just kind of get it all thrown into a movie screening. But, um, yeah, I've seen this before. The kind of like turning these movies into experiences and so quick, quick opener for you. If you were to pair any movie with a live, kind of immersive experience, you know, what, what would you go with?

Speaker B: Uh, so it's tough because all the movies I really enjoy are not fun, but, uh, like, not a place I'd want to necessarily live in. But I don't know, like, the first things that come to mind are like, fun adventure, like movies like maybe Princess Bride or Conan the Barbarian, something like that, where you're running around and there's a lot of just stuff going on. I don't know, like a mythical kind of. Mythical kind of environment. Yeah, yeah. As, um, long as I'm not going to get, you know, hacked to death.

Speaker A: Yeah. I was thinking through it and I was thinking, uh, like an escape room with Shawshank Redemption. So you kind of like, you start off locked in like Andy Dufresne and you're selling clues just to escape. So, um, anyways, I.

Speaker B: As long as you don't have to climb out through the sewer, I think we're good.

Speaker C: Yeah,

Speaker A: that final scene. Yeah. Cool. All right. Uh, yeah, that was random banter, but let's, let's give our listeners a little preview for today's ah, episode. So this one is all about making the leap from prototype to production and what it takes to build AI systems that you can trust when, you know, really going into a pressure environment where, you know, you're in production. And so we've seen a lot more teams building with generative AI These days, um, you know, doing a lot of prototyping, and that's cool to, to demo, but, uh, things get a little messy when it's time to go live into production. And so, you know, we wanted to tackle this and understand what some of those common pitfalls might be. Um, you know, how decisions were made, uh, you know, how confident the outputs are. Um, and so, you know, what, what needs to happen when things kind of go off script. Right. And, uh, to help us kind of tackle this, we've got a startup, mabel, uh, who has, uh, been helping another startup growth company called Spec Book, uh, construction tech platform turn, uh, a messy document processing problem into a fast, accurate and explainable system at scale. So, uh, to tackle this, we've got two guests from those companies. Uh, first is Kevin McGrath, co founder and CEO of Mabel Startup, building runtime infrastructure to help teams monitor and manage how their AI behaves under pressure. And alongside Kevin is Rob Murray, founder and CEO of Spec Books, uh, widely used material sourcing platform in the construction space. Uh, Rob kind of leads a product that bridges the gap between distributors, showrooms, and trade professionals, making it faster and smarter to quote and purchase materials at scale. So, Kevin and Rob, excited to kind of dive into this case study. Thanks for joining us on the podcast.

Speaker D: Glad to be here.

Speaker C: Thanks for having me.

Speaker A: Awesome. All right, now before, uh, we, we jump into the main discussion, we start with a segment that we call Pair me up, go around the room, spitball. Two things that go together. Um, Mike, you lead us off.

Speaker B: Ah, so. Well, you already got the spoiler on this when we chatted yesterday, but fallen fishing. Uh, I went fishing over the weekend. Uh, it was great. It was nice to not wake up at the crack of dawn in order to beat the heat, um, and could go out there a little bit later in the day, um, and actually catch some stuff. So that was nice.

Speaker A: Shore fishing.

Speaker B: Yeah, I did some shots. Yeah. Yeah, I mean, that was what I did. Yeah. Uh, most of what I've been doing is shore fishing. Uh, you know, lakes and reservoirs around the Maryland area.

Speaker A: Still, still waiting on that picture of the, uh, the proof of the. The pound. The. The.

Speaker B: I sent it to you. I sent it to you in Slack. Yeah, you just check the thread.

Speaker A: Check the thread. Left you unread on there with your picture.

Speaker B: Yeah.

Speaker A: Um, yeah. Good stuff. I dig that. Um, cool. I'm gonna go with crabs, uh, and crushes, which is. Uh, just got back from a beach weekend in Ocean City, Maryland. Uh, and there were two things that we pretty much had on Repeat. And that was crab. Crabs and. And orange, uh, crushes. So, you know, anything from crab cakes, crab legs, soft shells, you name it. I mean, Maryland's fantastic for that, but pairing it with a, uh, go to cocktail, the Orange Crush. So we, We. We use the, uh, little. Little, uh, switch up and. And went with grapefruit, uh, but it's, uh, orange juice, uh, vodka triple sec, and a splash of Sprite on crushed ice. It's delicious. Uh, if you're ever in Ocean City, do yourself a favor and you grab some crabs and order a Crush. I don't think you're gonna regret that. So, uh, cool. Let's kick it over to our guest. Um, Kevin, quick intro and your pairing.

Speaker C: Uh, so that's great. Yeah. Uh, also fan, uh, of the Orange Crushes. We.

Speaker B: We don't. You don't have to go all the

Speaker C: way to Ocean City. I live on the bay. And you can get crabs and orange Crushes, right, right, Right there off the dock, uh, dockside. Over. Over. Near. Near Deal is close to my house, so I've done nice, too. But, uh, also playing off the fall theme, uh, I think fall, uh, goes great with fire pits. Uh, over the Labor Day weekend, we had the fire pit out because the weather is just absolutely beautiful. Uh, you get to sit outside, family, kids, uh, get to do some s'. Mores. You get to have a bourbon. Uh, and, uh, it's just a beautiful night. So all nights and a fire pit, to me, are two of the best things that go together and then the atmosphere that goes around.

Speaker A: Yeah, it sounds great. And if you had to throw one more in there, maybe a little, little glass of bourbon just to, uh, to enjoy by the fire. But yeah, fallen fire pits. That's a strong combo. Um, yeah, appreciate you joining us as well. And, uh, Rob, quick intro in your pairing.

Speaker D: Yeah. Hey, um, you guys are coming in strong. Um, I gotta say, you know, as much as I love, you know, morning coffee and a good workout, I, uh, think I'm gonna go with, uh, a very productive week and a very cold martini with the doll. So I think that would be my go to pairing at this time.

Speaker C: There you go.

Speaker A: You're going dirty. Dirty martini.

Speaker C: Absolutely.

Speaker D: Yeah. Ah, dirty the better.

Speaker A: Awesome. Good stuff, man. All right, well, let's, uh, let's kind of wrap, uh, on that segment and transition into the heart of today's discussion. So, um, you know, this episode tees up, uh, you know, perfectly as this kind of case study between, you know, a tech startup, Mabel, and one of their earlier customer Spec books. And, uh, so, yeah, the, the main themes we want to talk about are going to be the, the gap in AI adoption, uh, and why runtime matters. Uh, we'll go deeper into this use case, uh, with these two companies. And then, uh, the power of empowering humans, not replacing them with AI was, was a theme that we wanted to make sure we covered. So let's start with the adoption gap and runtime AI. So, Kevin, I want to lead off with you on this. You've talked a lot about how a lot of generative AI projects maybe stall at that prototype stage, and very few make it into real production environments. From your experiences and conversations with CTOs, engineering leaders, what is the biggest barrier that stands in the way of some of those true deployments?

Speaker C: Yeah, I think it's a good question because AI is so accessible to everyone now that, uh, everyone can get their hands on it and everyone can start trying it. And when you try it, you realize, hey, I'm getting these really compelling answers that are, I don't know, 70% of the way there. Uh, and you feel that it won't be that big of a leap to get from 70% to 100%. But unfortunately, what AI hasn't replaced is the old 8020 rule, right? Uh, where 20% of the work gets you 80% of the way down the Runway, but then 80% of the work gets you the last 20%. And unfortunately that still exists even in the world of AI. Ah, AI is not just about taking a prompt, throwing it at ChatGPT or Claude or one of these other large LLMs are out there and getting a response back. A lot of times you're running a company at scale that has maybe tens of thousands, hundreds of thousands of documents, maybe terabytes of information. Um, you have customers sending you things that are unstructured and structured, and you want to process them. With AI, you quickly realize where the limits on AI are on you run out of context window. And a context window is basically, how much data can I show that an LLM per question? Uh, you need to start solving for things like, okay, well, what data am I going to present to the LLM M and how am I going to tell the LLM to do that? And then on the other side, if you don't want a human sitting next to every single response that comes to your system, how are you scoring it? And so all these questions start adding up. So you started with this really, really compelling demo. You asked the LLM to do some very basic things for you, and it Excited you and it rightfully so because it's very powerful. But now you actually want to bet the revenue of your company on it. And so a litany of questions come in that you didn't previously had during demo. So from our perspective at Maybal, what we do is we help organizations, specifically their product and engineering teams, scale these AI experiences. Because there's a lot you have to do to do it right. Um, not just from an infrastructure standup perspective where you need to start running vector databases and graph databases and system checks and different types of metrics and everything else. Um, but also the scoring. And I think the scoring is one of the, one of the bigger things that uh, you have to do is understand the fidelity of the outputs that you're getting. And the Mavel platform ties all of that together for a product and NGD team so they can have a high fidelity response for what they're building.

Speaker A: Yeah, well said. I wanted to make sure that we covered the concept of runtime confidence and expand on what that actually means.

Speaker C: So runtime confidence means as you are giving data to the LLM, you're getting responses back and are you actually tracking how your inputs and how your context is going in and what the fidelity of the output on the other side is? So what we do at Maybell is we have 14 different pillars that we're judging. These are everything from context grounding, uh, to realm, uh, of answers and different things that we associate your context with your output. We give that a final confidence score. You can use that single score or you can use the broken up values from the 14 different uh, uh, attributes that we're measuring on to then decide how you move forward with that response. Are you going to move forward automatically or are you going to put a human in the loop to validate something? Or you can also make this judgment in aggregate. Uh, and you can say, okay, over the last 10,000 runs, how many uh, scores came back in an area that I don't like? How do I validate those and how do I make my system better move forward?

Speaker A: Yeah, I want to hear it from uh, your perspective, Rob. So someone that's running a real platform with thousands of users, what was your team missing before you kind of brought runtime AI into the picture?

Speaker D: Yeah, I think your initial question was you asked what are the gaps with adoption? And uh, the industry kind of lends itself to a uh, challenge when adopting new technology. But I will say the industry is changing a lot to where um, it's waking up and understand that, you know, specifically AI in this case is, is here to stay. Um, so whenever we're working with Mabel, uh, we started off with a small focus group, understood their needs and we um, were really solving a tremendous problem. Whenever it comes to this very time consuming uh, kind of archaic process around processing large amounts of data through PDFs. Whenever you're specifying products for large commercial projects which can be very costly and um, um, from a time perspective, from an error perspective, um, it's also very difficult to onboard new employees and new users. So working with MABEL really allowed us to upskill um and close these inefficiency gaps and error gaps which has been tremendous for our user base not only from a productivity level but from a sales level. Um, they can win and capture more leads and jobs by using this platform, um, integrated with mabel's AI platform are using spec books integrated with Mable.

Speaker A: Yeah, it's, it kind of also leads into a um, you know for me thinking about like AI adoption in different, call it like change resistant industries. Um, and you know construction for example, maybe isn't always known for being like an early to adopt new tech, especially AI. You know, I'm curious if you experienced any sort of resistance or what kind of resistance maybe did you see in the industry when it comes to kind of introducing new tools or different workflows.

Speaker D: So as we get the product to a place where we're ready to uh, incrementally scale it, you know, we're very conservative with the way we rolled it out. We did come across several users, several entities that are uh, long standing clients of ours that were, I don't want to say offended but we're definitely reluctant uh, to adopt it. Like hey, this is, we even had some people in, some calls saying this is going to take my job and it's absolutely not. Uh, it is closing the gaps and eliminating a lot of the uh, frustrating things about your job, the time consuming things about your job. So again uh, once these uh, users started to adopt the technology they very quickly learned that hey, this has given me the time, uh, and energy to go to more productive areas of my job which ultimately led to more free time frankly and also um, led to more sales. So uh, again there was this obtuse position. Right. But um, as they learn and accepted they really realized the value added adding to um, themselves personally, um, as an employee, but also the overall business.

Speaker B: It's interesting because we were in the process of rolling out something very similar AI and there's a lot of resistance. Am I going to lose my job? Type stuff. And it's like what we're trying to say is we're just trying to replace your snow shovels with snowplows. Like that's the sort of, like we're just trying to make it so that you can be more efficient, you can spend your time, get more done in a shorter period of time, not replace you with these things. Um, and I think that's resonated fairly well. I'm curious, like, is that something that would resonate with, with people in your experience?

Speaker D: Uh, you know, also the uh, accountability. Right? I mean, AI does a great job, like I said, getting the job done. Uh, uh, but in the day a person, you are still accountable for that, for that result. Um, um, so this just makes you better at producing that result. Um, um, it's not here to eliminate you from the, from the result. So, um, uh, again, it's nothing but positives. Uh, and I can't, you know, stress enough how the adverse ratio of increased sales, um, and lack of wasted time, uh, has been tremendous for our user base.

Speaker A: Kevin, before we dive deeper into the, you know, the actual use case between, uh, you guys in spec books, what, you know, what is it that uh, maybe you've seen or similar patterns, uh, in other industries that are kind of maybe traditionally resistant to change, Is there like a common threat that you know, maybe finally gets people over the hump and into adoption?

Speaker C: Yeah, I think there's two main pillars out there with AI adoption. I think there's one pillar that allows humans to be more efficient at their job. Like uh, a co pilot, for instance. Uh, whether it's a developer co pilot or whether it's a sales co pilot or an SDR or things like that. And then those are focused on making individuals better. I think those have a little bit more resistance in the industry because people initially look at them and there's even some advertisements from companies, companies out there saying, hey, replace your whole sales team with uh, you know, with, with our automated SDRs and things like that. I, I think some things will get automated, but in a large part I think AI will create more jobs in aggregate than it replaces. And I think this is like technologies that you know from, from previous times too, right? Like Excel didn't put accountants out of business. Um, and there's a lot of examples about how maybe technology comes in and replaces what some jobs do, but doesn't replace a whole industry. And actually a whole industry can grow up around that technology. So do some jobs get replaced? Some jobs. Some jobs are going to get replaced. By new technology. It's just the way it works in aggregate though, I think there can be job creation until maybe we hit AGI one day. I do not think we're close to AGI. Uh, but when we get close to AGI, then you need to start wondering, you know, okay, is there, is there a bigger thing on the, on the hook? Uh, the second pillar are people using AI to drive a workflow or automation in their business that they haven't been able to do before? Anyway, uh, and there are a lot of examples, uh, for instance like spec books with Rob and other examples of industries that use a lot of unstructured data that pass hands between all kinds of uh, people in the industry and that unstructured data sometimes just sits there unprocessed even by human eyes, doesn't get to see. And that's just one of many examples of where hey, in an automation or a workflow process you can use AI to do things you've never been able to. And that can be job creation of some sort or at least a revenue enabler of some sort that you never had before.

Speaker A: I want to you know, dive deeper into the use case with spec books and Rob, if you could maybe paint the picture for us on, you know, what was the challenge that you were facing handling, you know, spec docs and quotes before? Maybe I'll enter the picture.

Speaker D: Um, yes. I mean again, um, this is slow to the tech space and there is still a lot of processes um, that are very manual touch that shouldn't be. Um, it's um, it's efficiency killers which a lot of these companies that we work with, they work with small teams, they're limited on resources and you know, the larger companies are kind of running away, um, and these smaller companies, um, it sometimes are kind of being choked off um, just from keeping up because of the lack of efficiency. So whenever mail came in, not only with the ability for our users to produce commercial quotes and submittals and things like that, but we've also worked on order email automation. And this is where um, when a lot of our users are receiving tens, hundreds of orders a month, instead of someone manually plugging into their system, we have a webhook with mabel where it just goes out, reaches grabs, email grabs the appropriate content from the email and it automatically inputs an order into their system. So that's freeing up resources to do other jobs that um, uh, necessarily um, would not need to be um, had on the order. Email or emails that come in through an order. I know fumble that answer, but um, um, so that, that's another way, in

Speaker A: addition to the quotation, Kevin, from your side, you know, what was it, you know, when you kind of looked under the hood at what Spec Books was trying to solve? You know, what, what were some things that jumped out to you that you felt were immediate value adds that you could bring to the table?

Speaker C: Yeah, I think Spec Books is, they have a fresh look on, uh, a traditional industry. Right. Uh, uh, quick adopters of technology and trying to figure out exactly this, where is the process getting stuck? Not only because it's manual, but there just aren't enough people to do the job. Right. You can't just keep scaling another person, another person, another person. There's not enough people who want to do that job out there. Um, and you know, Robin, Spec Books were looking to scale a business, compete, uh, against the, the, the big boys, so to speak, but do it in a way that, you know, they can run, uh, at the size that they are, uh, but still participate in the big pool with everybody else. And AI, uh, working with their engineering team, we found that, hey, this is a lot of unstructured information that has structure in it. Uh, if we can work with you and you can use the Maybelle platform to identify the structure like in a po, what's the structure of this po, what's the structure of this rfp, what's the structure of this specification? And they can use that AI to identify that structure and identify how to extract those entities. And then now they can relate it to the business they've been running for 10 years, which is, you know, finding parts and building specification plans and bidding orders and different things. And now they can turn around that to a customer, you know, on minutes instead of days. Right. That's, that's a big win. So working with Rob's technical team, who was also extremely willing to figure out, hey, how do we make this business faster and make this better? Um, there was a really compelling use case for AI, uh, and there was a really compelling use case to make the business better, which is the best way to do it. Right. You don't want to jump into AI looking for a problem. Uh, they had a real world problem they needed to solve, and AI was a, was a great way to solve.

Speaker A: Yeah, and I think that's a good point too. That leads right into like, you know, peeling back the onion on the actual implementation. I think a lot of, you know, founders might interpret this as, it's going to create a lot more work. We're going to probably need to scale up just to, to handle that kind of influx. Um, what did that look like on your side Rob, when it, when, when you were you know, making sure that this partnership was going to be successful, did, did you have to hire up? Did you kind of keep the, the team status quo, use your existing engineers? You know, what did that process look like for you?

Speaker D: Well, the flexibility that Mabel offered working uh, hand in glove with our team, we didn't have to scale up the team. I mean it kind of worked right to our normal sprints workflow, what have you. So um, not only the, the um, short Runway they helped us create to get to market is also once you get to market the adjustments you have to make. Right. Because you get a lot of edge cases. Especially in our space, you're getting a very vast range of data and content. Okay, how are we going to handle this? Especially um, uh when, when we are dealing with those edge cases and work with Mabel and the tools they provided us that we collaborate on, uh, has made it very easy to handle that and fill it because at the end of the day, especially in our space, again because we do have folks that are reluctant, um, if it doesn't work they're going to put it down very quickly and look, they understand technology. Um, you're going to have some bumps but it's the way in which you react to it and us to be able to react, working hand in glove with Mable to fill those bumps or those edge cases has definitely spoken to our success and our ability to continue to grow uh the products which include um, mabel.

Speaker A: So what do those kind of actual results look like? Um, talking through what impact this has had on your team and your customers. Um, I was reading the use case on your uh, uh on Mabel's website. But if you want to spitball some of those numbers specifically on what kind of value add you saw from the partnership.

Speaker D: Oh yeah. So um, uh, as far as your bid win ratio, uh, we'll jump by you know, 250, 300%. Um, which yeah based off the volume doing that's, that's obviously significant dollars. Um and then your time spent on each project, you know, drops by 75%. I mean those are great numbers to, for any business. You know, when you're, when your sales uh, is going up that drastically and your cost to produce those sales is going down, uh that's a massive win and it's, it's proven set for all of our clients.

Speaker A: And I'm always curious with ah, an implementation like this. Um, thinking about this Long term, Right. Uh, from the initial integration, kind of maintaining a feedback loop with how things are going. How do you all, maybe a question for both of you all. How do you all approach kind of like iterating over time? Um, Kevin, maybe for you is this kind of like, all right, it's good to go, set it and forget it type of solution are you continuously refining, you know, how the system behaves and responds to these different cases?

Speaker C: Um, I think AI is something that always is going to need tweaking. Right? So one of the big differences from AI than maybe your traditional automations is that AI is non deterministic and AI is going to change, right? You're going to change the data that you're feeding to AI. You're eventually going to upgrade the different models that you use behind the scenes. Uh, you might use multiple models for multiple things in a workflow, uh, and they might change over time. When AI sees something, it's going to come up, uh, with an answer every time and that's not necessarily going to be the same answer. And that's the difference between your deterministic outputs and your non deterministic outputs that we have with AI. So in the old days what we would do is we would write a unit test. We would say, okay, whenever I input this one thing, I expect this other thing and I expect that 100% of the time. And if it doesn't happen, you're going to alert me right away. What you have to do with AI is you're going to input something and you're going to get a slightly different answer almost every time it comes out. Unless you have a system to either help you normalize the answer or understand out of the scope of answers, what is a correct answer. And that's one of the things that we measure on in our confidence score that I mentioned before, is that we measure out of, hey, the realm of possible answers. What is a good answer, does this align with that? And then furthermore, should it be further refined, uh, to be a single answer that shows up in maybe a JSON structure or some sort of uh, uh, format that you can use to automate with later. So AI is very different. You're going to have to watch it, uh, eventually you want to watch it in aggregate and not every single answer that's coming out of the system. But you do need someone who owns making sure that the AI system is running correct some way, shape or form.

Speaker B: So Kevin, when you mean looking at it in aggregate, I assume you mean, you know, give it a whole range of questions, look at a range of answers that come back, and then you're sort of saying like, okay, 80% of the answers were within acceptability or 90, whatever your threshold is, and saying like, oh look, it's actually improved. While it might have gotten some wrong that it used to get right and now it's getting some. It's getting more right now than it did before. I assume that's what you mean by looking at it in aggregate.

Speaker C: And I think you have to understand that AI is not always going to give you the 100% answer every time. So what do you do? So what do you do when the confidence score is low? Do you stop it? Do you run it again? Do you put it in front of a human? Uh, do you go use a different model? Right. There has to be a lot of questions about that. And then our customers have different use cases. Right. So some of them want to see the outputs in aggregate. So they'll want to run 10 to 100,000 of these things. And then, and then I, you know, maybe not eyeball, but you want to take a certain percentage of those, like 3% error rate or how much percentage fell under a certain threshold. And then how do we go back and do that? Uh, some of our customers want to do that to test and build the platform, but they never want an answer going to a customer that's below a certain threshold. So they're not willing to see what it's in aggregate live. They want to see the aggregate in test and then go out. When they go out to customers, they want to block answer, block anything under a certain threshold getting out. So you want to handle those in certain ways. Different industries, different use cases, different things. But yeah, how you plan to use AI and how you plan to measure AI are two very, very important.

Speaker A: Yeah, maybe leading uh, more into like customer success on, on, on uh, side of things. Because, you know, we, we also, we signed a, um, like a sales AI partner uh, earlier this year. And you know, we had a designated customer success person that had a, you know, at least a once, uh, twice a week, maybe a lot more, maybe in the early stages of, of just onboarding and ramp up. That was just constantly, you know, tuned into our account, knowing our, our strengths and weaknesses, how much, you know, our team was adapting to the tool versus not, you know, maybe getting the max use out of it and max value out of it. Is that part of every engagement at Maybelle? Is there a, uh, you know, a customer success person that's dedicated to every account with what kind of cadence that check in is with the customer.

Speaker C: Yeah. So we work, uh, we work hand in hand. We usually work on big projects, um, and we usually, we, we always have someone dedicated to the account. Right. Um, and sometimes it takes that dedication. Uh, sometimes the team you're working with doesn't necessarily know how to give feedback to AI in the correct way or they don't know what a certain parameter coming back in the system is. And we're there to help the customer. Uh, we're not only a console or only an API with only docs. Uh, there are people here. Uh, we use AI ourselves and we'll use AI to help our customer ourselves, but we also have real people to help you use AI. Uh, we're looking for people to use AI at scale and sometimes at the bottom of that are still humans doing things at the end of the day, uh, to make sure that the scale part works correctly. Right.

Speaker A: Yeah, I want to make sure we kind of, you know, highlight a little bit more of this, you know, almost like this human in the loop, you know, with, with any sort of like, implementation, uh, or any sort of, you know, engagement with, with AI, um, empowering the user, if you will. I think that was one of the, the ways that we put it on our, our discovery call, uh, and that's something we talk about a lot on, on this podcast is, you know, the, the human side of technology. Uh, and it's obviously a top of mind topic with a lot of folks running that question of whether AI will replace their job or not. So just with this use case specifically. And Rob, you kind of alluded to the fact that you weren't needing to hire up additional engineers as a part of this partnership. Um, but in our discovery call, you also kind of alluded to that it upskilled them. Can you maybe expand on that on how this is kind of like beefed up or propped up your team with, you know, uh, being savvy with, you know, with the implementation of an AI piece of your business versus the engineering

Speaker D: team or the, our users?

Speaker A: Um, oh, I was referring to the engineering team.

Speaker D: Yeah. So I would love to answer that question, but that's outside of my scope. That would be, uh, Adam Pippen, I'm on the business side. I could talk to you all day long about how to.

Speaker A: Yeah.

Speaker D: Um, so I do know that in reference to integration and ongoing work that we do with, with mabel, that, um, out of all the objectives, uh, on our roadmap, that uh, those objectives move the quickly, uh, the quickest cross sprint now to take a technical deep dive on that again, that's outside of my lane that I, uh, don't really want to even try to, you know, um, massage that question.

Speaker C: I.

Speaker A: Well, Kevin May, for you, you know, was, was it intentional kind of, you know, from a, a product design perspective when building Maybelle to augment human judgment rather than automated away?

Speaker C: Yeah, well, I think there has to be a human in the loop somewhere. Someone has to tell the AI whether it's being correct or not. Now what type of person that is might change. So, um, you have technical implementers of AI and they're going to implement how the AI works, how it's structured, what's going to be monitored, how things work at scale, who's giving the feedback to your AI. That could be a technical implementer, someone from your engineering team that could be a subject matter expert that you have in your organization. Um, but the short answer is yes, there has to be a way for someone to give feedback on what's correct and what's not. Um, even just to get started. And then again, alluding back to before in aggregate, you're going to need a way to say, yes, the scoring is working correctly, uh, because the scoring has to become yours. Uh, there is no general score out there in the world where 90% means 90% across every industry and every AI use case. When you come in and the system starts scoring and you start getting these metrics and data back, you need to make that number yours. Your threshold might be 70 or 85 for a certain experience while someone else builds something and it might be a completely different number. The most important thing is that you're getting, uh, a consistent metrics back and a consistent way of scoring so that that human on the other side can come back and go, okay, I'm going to focus on this area. All the, the 5% of answers that this system produced that we were not comfortable with. I'm going to give feedback on why those answers were not good enough. And then the idea behind that is that the AI will continue to get better, right, get better and better over time. Uh, but you do have to have someone giving that feedback. That's just the way it works.

Speaker A: Yeah, I mean, we have a lot of guests, uh, on the show that come from, you know, defense or national security, uh, you know, critical infrastructure kind of verticals like that, where, you know, keeping a human in the loop is a non negotiable. Like you, you need to have somebody in there to make sure, you know, with some of those decisions that are being Made that, um, you know, you've got a. You've got a QA kind of process there that's making sure it's not just

Speaker C: something all the way up to the last step. And the last step is human review. And that still might save you thousands of thousands and thousands of hours worth of work to get to that point. Or you might be doing a process that you never did before. And I have to say, more than looking at situations that say, like, oh, okay, well, I'm going to automate what this person did. It's like, what work am I not doing that I wish I could do as an organization and how AI can help me there. Right. There are probably so many things as a business that you're not operating on today because you don't have the capacity to go out and hire 300 more people to go do it. Well, AI opens up those opportunities that, hey, I have a team. I can dedicate a certain percent of my time to helping me do something I never thought possible for my business. Why not do that instead of thinking about how do I squeeze out the extra 3% of efficiency over a person, you know, over and over and over again, like build new paths? And I think that's one of the most exciting things about AI is the ability to go out and do things that you haven't been able.

Speaker A: Yeah, it's a great point that opportunity calls that's lost from having your team, especially if it's leadership.

Speaker C: Right.

Speaker A: Really consumed in the weeds when, you know, AI can quickly open up some of that bandwidth for them. Um, Mike, did you have anything else you wanted to touch on prior to we, you know, closing out at the final segment?

Speaker B: I did, um, a little bit with the. You were talking about QA and you were saying the human in the loop. I'm curious, do you also look at the negatives, like, things where it's scoring really, really low, and look at the opposite and say, like, hey, is this actually really a bad. Is this really a bad answer or what? Or, you know, because I imagine there's. It's not just looking at the good answers to make sure they're good.

Speaker A: Right, Right.

Speaker C: Yeah. You got to look at. You always want to look at the deltas, right? Like, what's my 95th, 97th percentile, 99th percentile. Things like that you want to look at in both directions. Um, it's kind of fun. Like, if it's saying. When AI says it knows something correct, but someone obviously knows it's wrong, usually get feedback on that right away. Like when if something outputs that AI thought was correct and it's wrong. Unfortunately that's one of the ones that you usually hear about very fast. The scarier. Some of the scarier ones are when AI is not fully correct and neither side kind of knows what's going on. And so highlighting those and having again a robust scoring mechanism. So you can see, okay, why, why was this answer the way it was? Was it because of context grounding? Uh, was it because of my context windows? Was it because of uh, the sparse and destin vectors that uh, were mashed in the context and the retrieval. Right. All of these different things, you kind of have to, you know, pull all that together to understand. So yes. Um, I try to oversimplify some of the examples sometimes, but you're exactly right. Like there's a, there's a scope of things that you want to look for, both good and bad on the other side.

Speaker B: Cool.

Speaker A: Yeah, and I just wanted to you know, ask a couple of quick hitters on you know, maybel from a size of the company, you know, how many employees, uh, funding stuff like that just before we wrap into the final segment.

Speaker C: Sure. So uh, we just announced our $7 million uh, uh, price round that happened a few months ago. That's allowed us to scale to 20 employees, mostly out of the northern uh, Virginia area, uh around D.C. uh we're doing both government and commercial. Uh, commercial is the lion's share of uh, customers that we have right now, uh but also working on with government partners and opportunities as well. Uh naturally being out of the DC area it definitely one of our pipelines uh as well and we're looking to grow uh, with great customers like spec books over the next uh, following years.

Speaker A: Awesome. Yeah. Excited to continue to track your all's growth. I know it's uh, right in our backyard. So was always uh, perked up when I see uh, a local startup that's catching some attention and creating opportunities locally and solving some interesting problems. So kudos uh, on that round and this kind of continued growth. And then yeah, Rob, quick uh, hits on spec books. Just where are you guys headquartered? A little bit on the size and uh, then we'll wrap up.

Speaker D: Yeah. So we're uh, based out of Florida. We have roughly um, 35 employees. Um, uh, we've been around for about 12 years now. Completely bootstrapped, no private equity. Gives us some flexibility. But we've also been profitable. So pretty uh, proud to get this far without um, going that route. But that's not a saving. We won't probably will when it's the timing's right. Um, so, uh, we service about 10,000, uh, clients in North America. That's throughout Canada and the greater 50 states. Um, and looking forward to what's next. Also looking forward to growing with Mabel. Um, there's a lot of, you know, we touch in on just a couple tools that we're working with naval on. Um, but there's so much more. Because of our vast audience, uh, and all the different needs, um, I'm really excited to focus on, uh, the consumer experience and more of a, uh, in store shopping experience. A lot of people think in store is dead. Uh, it's not. Uh, especially in our space because of the look, feel and touch of all the products, fixtures that are being sold. And I think that, uh, with Mabel's assistance and the direction we're going, we're definitely going to be able to enhance that experience, um, um, from the sales perspective and the customer side. So, uh, I'm excited about things to come.

Speaker A: Cool. Yeah, I got a lead for you too. A buddy of mine runs a construction company out here and I was like, before, before the episode, I was like, hey, you've heard of Spec Books? He's like, no. What do they do? And so I just passed it over. So, yeah. Awesome. All right, well, uh, let's wrap up the main discussion and, and close with a fun little, uh, rapid fire Q A segment we call the five second scramble. Really just, uh, quick hitters, uh, some business, some, some, uh, personal. Um, Mike, uh, why don't you start it off with Rob, and then I'll close with Kevin.

Speaker B: Sounds good. All right, uh, here we go. Uh, Rob, if Spec books were an animal, what animal would it be?

Speaker D: Uh, in Eagle. Uh, I think because it is, it can see all. Uh, it touches a lot of things and it has, uh, a very large territory.

Speaker A: Cool.

Speaker B: Uh, what roles are you hiring for in the next six to 12 months?

Speaker D: Um, all things engineering. Right now in the process of, um, I must double in our engineering team. Uh, and with that, uh, obviously your sales team is to follow. So, um, specifically, um, we're right now looking for sre, um, uh, a couple of back end positions, front end. I mean really the whole thing. And then, um, senior level sales associates, um, between now and end of the year.

Speaker C: So we're, we're in growth mode.

Speaker B: Um, is there an emerging market or sector you think is underrated?

Speaker D: Uh, emerging market or sector that I think is underrated in reference to our space or just in general?

Speaker B: Either way, dealerships.

Speaker D: Um, that's a tough one. Merging mortgage sector, I think is underrated. Um, I guess I do have to say the construction space, um, people, uh, don't realize that the construction space is really one of the pillars of our economy.

Speaker A: Um, and a lot of people, I

Speaker D: think, take it for granted. Um, there's a lot of hard working folks in our space, uh, that do a lot for this country. And I, uh, think it should be appreciated and valued and protected. And, uh, I love to be part of the technology arm of that that's contributing to the growth of the construction space. Um, and I guess maybe also helping put a little extra, uh, sheen on the space. Right. Um, because I feel like it is kind of forgotten at times. And I think technology can definitely help, um, enhance that image.

Speaker A: I can tell you data center construction's hot. Real hot out here.

Speaker D: Yeah.

Speaker A: Loudoun County.

Speaker B: Yeah. Yep. What was your first car?

Speaker D: Um, my parents bought it, obviously. It was a BMW 318. Huh.

Speaker B: Nice.

Speaker A: Okay, that was fold.

Speaker D: I should not have received that card. Things for my ego do, uh, not buy your kids a BMW.

Speaker B: Nice. Um, is there a charity or cause that's near and dear to you?

Speaker D: Um, yeah. So, um, we normally donate to, uh, there's a foundation called Dustin Charity Wine Auction that focuses on, um, the, uh, nicu, uh, which is, um, you know, basically a baby hospital. And, uh, I have two kids, so children's health are so innocent. Um, I feel like that should be

Speaker C: a focus for everyone.

Speaker B: And last one, uh, what was your dream job as a kid growing up?

Speaker C: A lawyer.

Speaker D: I wanted to be a lawyer and then, uh, took the lsat. I aced that. But then I just realized exactly what. Yeah, how much reading down to be a lawyer. And I was like, I'm out. Actually, you know, also, when I was in school, I was going through graduating, uh, um, um, undergrad. Um, in 2007, I actually had to, um, depart from my education and go straight to help, um, my family. So that kind of derailed that as well. But, um, yeah, come, uh, along, I guess. You know, every kid likes to argue, so I thought that was in my wheelhouse, but reality sucked in.

Speaker B: There you go. All right.

Speaker A: Good stuff. All right, uh, Kevin, you ready?

Speaker D: All right.

Speaker C: I guess so. Let's see. All right.

Speaker A: If, uh, yeah, if Maybelle had a mascot, what would it be?

Speaker C: Maybelle had a mascot, what would it be? Um, that's interesting. Ah, uh, mascot. I don't know. I guess a. We use a robot as kind of our mascot right now. We have, like, these little robots that we hand out at all these conferences. So I'll have to go with that because that's the thing that we identify ourselves with when we hand stuff out. And it's AI based and we do help people kind of automate and push things through. So I'll say our little Mabel robot that we hand out at all our conferences, that's kind of our, our de facto mascot.

Speaker A: There you go. That's fitting. Uh, what's one word that you would use to describe the culture at mabel?

Speaker C: I think selfless. I think everybody at this stage of a startup is here because they're really excited about what you're building and the goal. And um, you know, you don't, you, you have everyone bought into like, kind of building this dream. And this stage of a startup is just so exciting, uh, to see people come in and just want to build the thing for the customers and kind of like the selfless attitude everybody has. Like, you get in, there's no politics, there's no mess, there's no anything. It's just people coming in and everybody working towards the same goal. Like, you know, I just love that atmosphere.

Speaker A: Yeah, it's a great answer. Uh, what are, uh, some of the roles that you guys are hiring for over the next three to six months?

Speaker C: Yeah, we'll be looking for engineering, sales, marketing, you name it. We're growing company. Uh, we are, you know, hitting our, hitting our growth curve. So six months, uh, and out literally almost every position of the company, but, uh, more near term. We're always looking for, uh, you know, great engineers. And uh, I think it's like Rob said earlier, with, you know, with, with more product and more engineers, always has to come more salespeople on the other side to kind of fill the bucket. But also everything in between we're going to be looking for over the next year. So, yeah, we're excited. We're, we're in that growth phase where we got to start filling, uh, most parts of the business.

Speaker A: Nice. What's the first job that you've ever had?

Speaker C: I first job was a paper boy, um, way, way back when. So people used to get these pieces of paper delivered to their doorstep back in the day, you remember, I used to be the person who would chuck it from my bike onto the, onto the doorstep. Uh, and then my first job, I needed a work permit for. I, uh, I, uh, I was a trash. I worked trash for the Renaissance Festival. So I used to just pick up and move trash all over the place. So those are my first two jobs.

Speaker A: Nice. I've Got the, the Nintendo game visual, you know, just kind of like, uh, dodging and weaving the dogs that are running across the yard.

Speaker C: I played that game, and I would actually go out and try and mimic some of those things in the game. Like, during, I had a bike with a big basket on the front of it, and I would take those papers and I would chuck it. And every now and then, I, I, I didn't break too many flower pots.

Speaker A: Just right through the window. Um, what's a charity or corporate philanthropy that's near and dear to you?

Speaker C: Uh, my, my, my wife donates to Susan G. Komen, uh, every year. Um, so that's a big one. Uh, we do try and rotate through some local charities. Uh, there's a halfway house here in Southern Maryland, uh, that we've donated to and made food for multiple times. Um, I would say I, I tend to like going towards more local charities. If there's a charity local, uh, it's really hard for them to get the funds. Like, nothing against the Susan G. Komans, but, you know, they got NFL players advertising who they are and stuff. So if you can find a local charity that means something to you or you, uh, want to help out with, I think those are great to go find.

Speaker A: Nice. Uh, best concert you've ever been to?

Speaker C: Best concert I've ever been to. All right. Um, I'm trying to think. I was at, you know, this isn't the best, but I just went to one, so I'm just going to say this.

Speaker A: No doubt.

Speaker C: I went to a Weird Al concert just last month, all right? And I had a great time. I don't know that it's the best concert that I've ever been to. It was silly. I liked Weird Al growing up. I never saw him live growing up. It's not a concert I even thought about seeing live growing up. Went with a few friends who also grew up, you know, we all knew at least, you know, a, uh, bunch of Weird Al songs. He put on a great show. It was a great show. It was all people from our generation. I had a really good time. I don't know what my best concert is, but I have to say, like, I don't know. Take, Take a flyer, if we'll. Weird Al's coming from the neighborhood. Take a flyer, go to the concert. It's a really fun time.

Speaker A: Just, did they play, uh, Amish, uh, Paradise? I think that was one of.

Speaker C: Absolutely.

Speaker D: Yeah.

Speaker C: Yeah.

Speaker D: Know.

Speaker A: Uh, all right, last question. It's going to get a little abstract with this one. If you had one day left to live, Would you spend it with Morgan Freeman or Denzel Washington?

Speaker C: Oh, all right.

Speaker A: It's pretty deep.

Speaker C: One day left live. Um, I. I guess I'll go Denzel.

Speaker D: I don't know.

Speaker C: I'll go Denzel. Um, he. I mean, Morgan Freeman's lights. Like, his voice might, like. It was one day left to live. Like, Morgan's voice might, like, kind of, like, put you at ease, like, going into that moment. But I think, like, Denzel will get just hyped up to, like, go for it. Like, Denzel would be like, yeah, we're doing this.

Speaker B: That's right. That's right.

Speaker A: Going out guns blazing.

Speaker C: I mean, it's going out with it all, so. So I'll go Denzel on that train.

Speaker A: American gangster says a lot about somebody, so I like. I like that, uh, side of you, Kevin. I appreciate that. Answ.

Speaker B: Definitely.

Speaker A: Uh, that's a wrap, guys. I wanted to thank you for. For joining us. Uh, always great to get, you know, a good perspective from, you know, a customer and a provider and kind of especially, like, playing that into the. The AI and the real world. Right. Um, I know that it's a. It's a space that's moving very quickly, so I love hearing these. These kind of case studies and excited, uh, to watch both of your all's businesses kind of keep pushing forward, uh, with. With technology. So thanks for joining us on the Pair program.

Speaker C: Thank you.

Speaker D: Yeah, thank you for having us.