How AI Agents Really Work - Daniel Vassilev

I feel like 2025 is the year that really explode as a topic and potentially as a reality for many companies too. But the challenge that I see in this is that the topic is really not well defined. There's kind of this vague notion that it has something to do with putting AI into workflows or giving it tools. And something, something, something, it takes over everyone's jobs all well and good, but how do you actually make AI agents that save you time and labor, because anyone who has tried to do this knows that it's not just as simple as, you know, spinning up chat GPT and giving it a mission and letting it run wild.

It still requires some kind of architecture building, debugging, thinking. And it also requires some kind of platform or environment to do that building in. So today's guest has seen this need and he's co founded a company called Relevance AI. They're a platform for AI agents, a really cool product.

I actually recently became a customer so I could explore this topic further and I've enjoyed digging into it. Daniel Vasilev, welcome to the show. Hey, Justin, thanks for having me. I have personally been excited by this topic, in a way that think I've really felt since I first got into like, you know, marketing automation and got the ability to just create simple, if this, then that workflows it kind of, to me, it feels like the next generation of that.

And I'm just curious if, you could just start us off by giving your definition, at least of what an. AI agent or what a gentic AI is just so we have this common frame of reference for everyone that's listening. Yeah, absolutely. I mean, the simple way like to think about agentic AI for us is whether it can now make decisions that are dynamic.

Can it handle non deterministic workflows? Right. So if we think about software and software systems, they're largely defined by algorithms. Where you described a if this then that that tends to be fairly fixed and rigid rules you can put in place to make decisions.

Agentsic systems are a lot more like human systems in the sense that given instructions and given context at a decision point, it could make a variety of decisions and those decisions don't necessarily need to be predefined and those decisions can be based on some sort of qualitative judgment in addition to a quantitative judgment. for us the real difference and appeal of a genetic AI is when it fully enables them How can it accelerate our ability to make decisions and take actions?

without being constrained purely by the amount of people you have on your team Or constrained purely by the amount of hours you have in the day What does that work look like and we are looking at relevance in particular to help accelerate that journey And make it so that Teams are absolutely unleashed, they have the ability to execute on ideas and hopefully remove a little bit of that constraint, which we have today, which is, oh, if only we had an extra person to help us do this.

That's kind of where we see Agents. AI take place. And only if it's able to handle those dynamic decisions. If you're still kind of stuck in rule based decisions, then that's much more akin to software and software systems of the past.

Um, I know there's a lot of marketing noise out there, but I think once, you this year wraps up and we start being clearer as an industry of what a Agentic AI is, that'll be the main differentiator. Let's drill down on that dynamic quality that you isolated as kind of the essential quality of an agent. and it's also, I think, key to how a lot of people are talking about agents that, they're autonomous in this way that they're decision making. And there's a little bit of.

Mistification around that because when I've gone into some agentic workflows or ostensibly agentic workflows that people have shared on LinkedIn or wherever, it's still very much rules based with like an AI step, like instead of just a deterministic calculation, you've got an AI model doing something within that workflow, which is awesome, but that's not aligning with the definition that you've given. So what is the key from your point of view that enables, a workflow to be agentic in that way?

Like what, what is required maybe from a technical perspective for that to happen? I mean, so kind of what you're describing there is a little bit of an in between stage of kind of like software systems and then trying to incorporate large language models. It's that process you're right, a lot of the kind of traditional workflow automation has achieved that by adding a new step that lets you also plug in an LLM. Now, that to me doesn't quite qualify as agentic.

The reason for that is quite simple. It's you're basically introducing a new tool, step into existing workflow automation. Just in the same way you have a PS steps. You might have a triggered some other system.

You might have some code step, and then you might add an LLM step. You're still broadly within that workflow automation space, and now you just have this ability to generate output from an lm. So I think that's kind of what you're describing. That's a lot of what's in the market at the moment, and we actually have that ourselves within our tool builder.

That's something very different to our agent builder, and for us, that's really powerful because tools and workflow automation tools in general, let us create kind of repeatable steps for repeatable workflows. and now you can add LLMs as part of that, like you can a code step, like you can a Python step, but that's different to agentic. So that's one way of using an LLM. when we're talking about agentic capabilities and agents in general, the way we think about it is, okay, so what happens if you have 50 of these tools?

And you could use any 50 of them at any one time, depending on some context. And how do you decide which tool to use when not based on some linear workflow but based on real decision making? So when we're doing our jobs, like let's say my job to be done today is you know I've got some recruiting work to do after this. We're currently rapidly recruiting.

We're screening lots of candidates We're doing washouts and I know I need to submit my feedback for a lot of these different. people we've interviewed and also like accept meetings for new ones. When I do that, I'm working across maybe five to ten different systems, right? I'm jumping in Slack, I'm jumping in email, I'm jumping in my calendar, I'm jumping in my ATS, and so forth.

And the steps that I'm doing for those different jobs to be done can really vary. And they can vary based on the candidate. They can really vary based on some instruction I've been given by someone on my team. And the magical thing about me at the moment, like really why I'm valuable in that process is because I can decide, Hey, I need to do this like this and then do this over there.

So if we think about agents from that lens, right, that's when they're truly powerful, when you can give them a set of these systems, a set of tasks that they can achieve, instructions on how to achieve them. And then they can go out and actually make decisions on how to execute them and actually decide, you know what, now I need to go to the ATS, I need to do something there. Then we need to go to Google Calendar. And this really goes beyond kind of that traditional linear workflow automation style experience that you're describing because we're no longer just defining a flow.

We're now really letting it decide the flow. We're letting it decide how to plan its activity and how to execute this. the way I like to describe for a lot of people to kind of demystify this is just think about hiring someone new on your team, like a junior employee. I say this to prospects all the time, like, could you hire me tomorrow and put me into a meeting room and on the whiteboard sketch out for me this, the way I'm going to be doing my job, the decisions I have to make.

and how to go really well, right? And maybe even teaching how to use the software if we're using some sort of software, can you, can you do that? And if the answer to that is yes, then we can train an agent to do that, typically, right? And that's the process that we're going to follow with an agent.

it's less as if I'm coming in and you're going to write to me, here's the 10 things I do. click those same things every time. It's much more along the lines of here's your job, here's how to do it. Here's the decisions you need to make, here are the systems you have.

and I think once we start thinking of agents. in that perspective and less about the technology and like what framework you're using or what LLM you're using. It becomes a lot easier to suddenly understand the difference between genetic systems and software systems, uh, where workflow automation, even with LLM capabilities is very much still a software system. Energetic systems are starting to become closer to human systems.

In your platform, one of the things I really like, like the way it's fleshed out, you kind of define an agent. It has a core set of instructions and then it has access to tools kind of mirroring what you just described, all the different things that it can do. The tools themselves, like you said, are almost like mini workflows where it can make API callouts to other systems. It can do various things can scrape the web.

What I want to understand is when that agent is triggered. is it really just like, you know, some of the new thinking models like, patchy PTO three or Something like that. deep research or some of these models where you can really see like the chain of reasoning that it's doing, is that kind of what, like the agent component of it does, where it gets a request and then it sort of makes a plan. And then as part of that, it starts pinging those tools and doing those different things, or if that's not it, what is happening under the hood when one of these agents gets an input?

Yeah, I mean, so we use large language models for that decision making. if we think of large language models, less in terms of chap GPT and more as a fundamental technology that provides reasoning capabilities. then I can kind of, you know, you start understanding how large language models can be leveraged. So, large language model basically can be given some context and then it can generate some output.

And if you use that correctly, you can actually have it make really good decisions for you. and so under the hood, we would use a variety of models. Some of them would be, you know, thinking models. And, Each one of these models can provide different kind of benefits kind of advantages over different ones.

So some might be better performing, i. e. they can handle higher levels of reasoning, but they might cost more. Others might be faster and cheaper, but maybe handle simpler use cases.

So what ends up happening is we basically leverage these models to make a decision. Okay, so this is what's happened so far. These are the instructions we have. What should we do next?

And then based on that, Decision making and reasoning ability, we can then take another action and then, you know, our system can orchestrate that we like to think of ourselves essentially as an agent operating system, right? In the same way that your team has Windows, your company is now going to have an agent operating system and our interface is both the IDE for creating those agents on the agent OS and it's also the analytics and monitoring and governance. So you can see what's happening in the operating system.

And with those two things, it means that you can give tasks to agents and then under the hood, they'll start making decisions. You can see how it's made those decisions, which actions taken. it gives an update using the models again on why it's made those decisions and what it's actually finished doing when it completes a task. but ostensibly it all comes down to just leveraging the models for decision making points.

And if you just start thinking of everything you do as like, you know, even me. When I started doing that, you know, the CEO review every single time, I'm probably stopping for a second and making a decision. And that's what we're leveraging the large language model to help us achieve. Obviously today, that level is different to where it's going to be in a year or two from now.

Today, we recommend that predominantly for tasks that are easier and simpler, i. e. something that you might hire a more junior employee for. but as the capabilities of the models expand, then the capabilities as well, the agents will expand.

I want to touch on use cases, but just quickly before we go there, in terms of how an agent process or workflow, whatever you want to call it, gets kicked off. it seems to me there's a variety of ways that could happen. There could be like a user chatting. Um, With the agent that starts something and making a request, could be an API request from another system.

Is there such a thing yet within your platform or that you're aware of just in general of agents that are kind of like always on, like always scanning, looking, evaluating data and then performing according to a set of instructions they've been given or scheduled by a batch like a data quality agent, you know, once an hour. Come and look at all the new leads that have been created and clean up their data and merge any duplicates or, something like that. definitely. I mean, we internally this kind of such a use case, but I think it's the best way to highlight this.

We have an agent that every single day will go ahead and at the end of the day, go through all of our core transcripts in the sales look and then for each transcript, it extracts a bunch of information and pushes it to a notion database that we have. That's very much formatted in a way that works for us. that's what we use as part of enablement. So Reps can go in, they can, every single day, if they have a question about like, how do I handle, pricing or onboard of our implementation, they can filter by that, they can see how other people have asked or answered similar questions, improvements they could have done based on what the agent suggested, our RevOps team can have a look at like statistics, you know, percentage of questions coming in this week have increased about, you know, implementation, so maybe we need to improve our collateral there.

So in that situation, we've got an agent that basically is constantly ready to receive new transcripts, process that data and Submit it and that happens on a daily cadence you can have all sorts of triggers, right? Like those triggers could be time based They could be you know, and time could be every 30 seconds, right? Like you could just be doing this job every 30 seconds. It could be as you mentioned from other software integrations i.

e slack messages come through User messages come through it could be like an email to be received a calendar has just been triggered or started the event and so forth. So absolutely like we already have engines that are constantly working for us Um, and those I guess always on it just depends maybe on how frequent those activations are But it makes sense. diving into use cases, share a quick anecdote, just something that was an unlock for me. And then I would love for you to comment on it and also just expand about the wide variety of use cases.

I'm sure you're seeing within your customer base. But I had a member of my team leave late last year, uh, had been there for a while. And so as part of her off boarding, we did. of an inventory of all the work.

We obviously knew about like the big strategic projects, the OKRs, uh, but we really wanted to see like, what is it that's taking up 40 hours a week? And uh, really itemizing that out just as part of evaluating, you know, what should this role look like? What does it look like today? What should it look like in the future?

And it was really eyeopening for me because it highlighted to me how much of work. It's not necessarily these big strategic things, there's a lot of granular work, things that are not yet predictable enough to fully automate, but are not necessarily like, very complicated or requiring very senior skills. They just require a certain level of judgment. Could be, you know, evaluating a record a CRM and making a decision about it.

what channel do we attribute it to, et cetera, providing that sort of input. And it hit me that that is a great place for AI to play. You don't want it to like come in and create your strategic vision necessarily. Maybe you disagree with that, but level of work, that's just, it's not quite totally deterministic, but it's also still relatively straightforward.

please react to that and tell me if you agree or disagree and what are the cool things that you're seeing out there. I think this answer will change over time. I think currently I do agree. I think right now the state of kind of technology, Makes it better for handling more of those tasks, adding tasks that are not necessarily the strategy, not necessarily the vision, but it's all about execution.

And especially when things, you includes time or volume, it can just outcompete every single day of the week because. It doesn't scale with, traditional resources. It scales with compute and when, when people think about this, I don't think enough people really just stop for a moment and just reflect on how powerful that is right now, right? We have all these websites in the world and if you get more traffic, they can just add more service and they can have that traffic.

Imagine that same concept being applied to the work that your organization is doing. it's so difficult to fathom the results and outcomes of that. And what I tend to really to people is like, don't think about the technology as just doing a little bit more of the same. Think about it.

What will your business look like if you could do 100x more? Because I think what we're about to encounter is Significant increase in the amount of value, uh, that we can generate, globally from a business perspective, right? When you think about the goods and services being produced, I think we're going to produce better services, better goods at a better cost. Um, and I think as a result of that, that's absolutely going to increase the value we generate, whether you measure that through GDP or through something else.

I just think we're going to live through an absolute explosion in opportunity. That being said. today agents are really capable for those tasks that have a well known, well defined process. I.

e. you could teach 50 people how to do this and they could all do a good job. If it's something that is still not working very well and needs to be figured out, It might be better off doing it like in a more traditional sense today, not leveraging automation. The reason for that is even if you have a process that doesn't work, you wouldn't hire 50 people to that process.

You would probably just have one person, two people maybe figure out that process. And so for any work like that, that's very exploratory trying to figure something out. I think you should not use agents. But for work, that is something that you could build a large team for something that you could, scale up, then agents are a great place to start thinking to deploy them.

Now, will that change someday? Probably, right? the more inputs you have, the better decisions you can make. There's a world in which I can see agents being able to look at every single data point in your business to help you make better decisions.

And I think we're not too far from that. But I think today, the best way to approach this is less strategy, more execution. going into that future vision that you just sketched out. it seems to me that or AI in general or large language models in general have both advantages and disadvantages versus human cognition.

one of the advantages that you cited obviously is just the ability to. Take in such a Vaster context window than a human can easily take in or retain. Like you said, looking at every data point in the business, maybe seeing patterns, calling things out that from our limited vantage point, we just don't have enough space in the brain. perhaps arguably, limitation versus human cognition is, uh, is kind of inherently derivative.

It's all, based on corpus of, of information that it's sort of processed and then continually recreating. So can it be truly, uh, original? as the technology improves, would we ever completely outsource certain elements of strategy to AI or will it always be a sidekick, a co pilot in that process? I think we will.

I think we definitely will. and we published this in 2023. We, when we look, there's a series and we published an article, beyond co pilot and kind of stated how we think that realistically co pilot is a very short term trend. And a lot of us, I don't know, five years from now, right?

The reality is every single place that autopilot can do the job better. People are going to prefer it, right? Like, think about it this way. If you had the choice in your company, or in your team to have five people sitting around you and all they could do is sit around your desk and wait for you to turn to them, ask them something, and then they'd reply back to you.

Or you could have five desks around you with those same five people and you're all working and collaborating together and you need something done. You can delegate it to someone else. They can go off and do it themselves, come back when it's complete. Which of the two would you prefer?

Obviously, it's the first one, which is why companies today are built with teams that are autonomous. They can delegate work. They can achieve things. We value autonomy.

We reward it. And we don't just have, you know, many assistants to one person. I think that's exactly the same, uh, when we think about agents and co pilot versus autopilot. To clarify, when I say autopilot, I don't mean something that doesn't involve humans, right?

I still think human in the loop plays a really important part. And in fact, when you delegate work between people, it goes between people, right? So if you delegate some work to an agent, even if they could complete That task on autopilot, there is still a touch point that then goes back to the human, whether that's an approval process, whether that's an escalation for help, whether that's just completing the task and handing it over to the next touchpoint. So when we think about autopilot and copilot, I think the future that I really see and we believe that we've been building towards honestly for a few years now is that world where you can delegate tasks that can be done autonomously.

You still have the first class experience for approvals, right? Because businesses are. Built on approval methods. One of the most questions, common questions I get asked is how do you make sure it does the right thing?

And I honestly just ask, how do you make sure your team does the right thing, right? engineers have pull requests that get code reviews. the sales team have deal reviews and people watching the call see the feedback and improvement. you have all these processes built in an organization today that are all about approvals.

And from our perspective, as part of the definition of an AI workforce, it actually finishes with a human workforce. So we think you need to have kind of the best in class experience when it comes to using, and working with your agents. And so I stress that because I want to be really clear autopilot does not mean without humans. Autopilot simply just means you can delegate work to it and it's more useful and functional to you than just an assistant you can, you know, have an in out, in out sort of experience with.

which is what we believe Copilot is. It just enables you to be a little bit more efficient. What does the world look like if you could be 100x more productive? And that 100x could be a thousand x at the click of a button.

That's what autopilot means to us, rather than kind of these incremental gains that you can use as a tool, because for us, realistically, Copilot is still part of like this trend in the past. So like software, yes, it's really useful. Yes. It's given us so many benefits, but at the end of the day, all of those benefits are just productivity boosts.

And at some stage when you want to do more, whether that's high quality, whether that's more volume, you're still limited by headcount. Autopilot changes that. Autopilot should ideally enable someone with an idea to be able to execute something phenomenal and magnificent beyond, the capabilities of an individual person. And that's the future I'm really excited about.

Because imagine if we've all got that capability, like, what could we create then? What better services, products can we be creating? Um, and it's not just about, you know, like, um, kind of SaaS. You can see this being applied to medicine.

You can see this part of education. You can see about the cost of these things going down and globally what that means for people. So i'm extremely optimistic about kind of that direction the thing we're focused on it's not co pilot, right? that's something that I think is this We'll see in the next few years quickly become less and less relevant in more areas.

Since we're talking about the future, let's, just look down a little bit further down that path. AI, you know, is increasingly going to over that lower end, that more junior end of work that we talked about, then as models get better, presumably it's going to come. so to speak and take on more senior level tasks is there an endpoint again? I want to think about like the role of AI and strategy like are there tasks you think we should just never delegate because they're Too important and a human has to do them Or what is the role of the human in this future of work besides, you know?

Those like AI approving the work that AI does in other words, As long as AI is demonstrably good at the task. There's no reason we should leverage it again of the caveat of We delegate to it But just because we've delegated some work to it doesn't mean that we don't have responsibility To be part of that process part of any decision making and actions that happen beyond that. So I think for me like when I think about an NC, I don't I don't know what that end state is but One thing that I fundamentally, I guess, believe is that every single time, you know, we've had the opportunity to do more as a society, as I guess as humans, we tend to take it, right?

Like we, we rarely think to ourselves, you know what? We've done enough now. We were able to manufacture more of this. Let's just stop here.

Inevitably. More factories come up more, it becomes more efficient. Now we've got like, you know, people working on that. And I have a really strong inclination that Magentic AI and AI more broadly will just be, part of that journey.

The only difference here from my perspective is that the opportunity. And scale of, these new capabilities will be just extraordinary. I think that's the difference here. It's a scale, but at the end of the day, if you just equip people with stronger tooling and better capabilities, my instinct is we're going to just try to achieve more rather than say, you know what, now we can do everything we did 10 years ago.

I just think that goes against human nature. And I think, the society and the way we built our system tends to incentivize trying to operate and play and create more, services, goods and things like that. Right or wrong, right? I think that is the system we have created.

And so, I'm just particularly bullish and from that perspective and optimistic so if we zoom back to today what are some of the cool things that people are doing? I've seen some of the templates and examples that are available in your platform, but like, what are either internally or in your customer base? What are some really interesting things people are doing today that maybe can spark inspiration for people that are listening to this? Yeah, so I mean, we're obviously very lucky that we have quite a large number of customers in the sales and marketing space.

RevOps tends to be a team really well positioned to benefit from an AI workforce. it's interesting how, because of the kind of work RevOps are traditionally done, because it's kind of sat near the subject matter experts, but also has been the more technical kind of expertise on hand, it's a really great place for both fostering and adopting, kind of AI agents. The thing about relevance, right? When we built relevance, you know, before this, I had a decade of experience in automation, built another company before, this with millions of users across our products, I was very lucky to get to build and work a lot of machine learning models, you know, albeit quite different ones than today's, but for very practical reasons.

And the thing in that whole experience that was very clear to me was automation projects rarely fail just because of safe technology. And one of the biggest reasons automation projects fail are because you don't fully understand the unique workflows and organizational wisdom that goes into that process. And so instead of building an engineering framework, we said, well, let's build an agent operating system for the subject matter expert, right? Let's build the ability for the people who are the experts in this to train their agents.

Um, and if you kind of just, do a simple, kind of exercise here and ask yourself, like, who at the moment hires salespeople? Who trains salespeople? Who hires Redbox people? Who trains Redbox people?

Is it engineers? Is it data scientists? Or is it salespeople, Redbox people, you know, and you know, if we live in a world where right now that the subject matter experts are training and hiring subject matter experts, then to us, it just feels very natural that they are going to be the same people that are going to be training and hiring for these agents and also probably managing them, right? Because who knows how to manage once again, those agents, then those subject matter experts.

That's a really cool tenant of our platform and a really cool tenant of how we've built our product and we're definitely not where we want to be yet, you know, we're still more technical than we want to be, but we're rapidly working towards making that as simple as possible for as many people as possible But that's why we're not an engineering framework. And the reason I say this is because when we think about, Use cases in a lot of teams, Rev Ops kind of nicely straddles at the moment that in between status subject matter expertise plus technical acumen.

And so I think for a lot of your audience listeners, like, this is the perfect time to get started with agents. You've now got a good tooling, whether it's relevance or kind of the broader ecosystem is more available to you today it's a matter of when, not if, and I think early adoption is always the right strategy. And then RebOps for me is particularly a place that can become the internal experts. We've already seen this people becoming an AI workhorse manager in their organization, because they're basically helping all the different business organizations build and deploy these agents, giving them that internal.

And, and some of these cases we've seen, right? Like let's take RebOps. Like we've seen some really trivial use cases. Like I don't even want to start with like, you know, just to flashy ones, because.

There's just so much stuff in the organization that is so valuable, even if it's not necessarily that the Flash is. Like, for example, we had one customer, they had, I think, over 100, 000 accounts in this year round, and it was an absolute mess in there. It was just duplicate, old versus new, wrong statuses. And this was causing a great deal of frustration for the sales team because it really made their job harder.

And the only option they had was one, pay an extremely large sum of money, to a, basically like a BPO. So then go ahead and go through every single account one by one and manually check and review it was going to take I think months to complete. Or the second option was because they're already a customer of ours for another use case, build an agent, deploy that agent and get it done in less than a week. And that, when you think about like what that means to the businesses, okay.

So first we just save a whole bunch of money. We saved a whole bunch of time. and so when we think about our sales team being able to be productive, we've just saved them six months. And, that was phenomenal because the agent could go at each account, look at it like a human, determine what's a duplicate, then we go search in Salesforce for other stuff and clean it up and put it together.

So when we think about agents and use cases, don't feel the need to go for some pie in the sky thing. You can start small. and when I say start small, In terms of like this might not sound sexiest idea, but boy, it's impactful. And I think that's something that I'd really encourage people to keep in mind.

But then, you know, one of the ways we also use relevance ourselves is we have a small sales team at the moment and we get, we're very lucky that we get a lot of inbound requests and we get a very large volume, both of signups on our product and book demos, and it's really hard to handle that volume. and so at the moment we have a fleet of agents dedicated to basically treating every single inbound signup that comes in. qualifying it, maybe asking us some questions and then determining where to route it, whether it goes to our sales team, whether it goes to a partner, whether it goes to signups, something like that, right?

We might need maybe at least 10 people on staff to have that volume and they just, they couldn't be in one geo in order to hit our SLAs for how fast we want to reply. There'll need to be multiple geos and so managing that and you know, you can just think about how difficult that is to build out that process, but because we've got these agents. They're doing that job for us extremely effectively. and in fact, so effectively that I'm often on calls where people have come in through that channel.

They ask us, do these agents work? And then I have to remind them that they've come in through an agent and they, you know, they didn't, they weren't aware of that. So, that's another example. Internally as well, our life cycle marketing.

Agent is saying that's really popular every single time someone signs up and I recently just shared on LinkedIn a post that someone made Analyzing the email they got from the agent every single time someone signs up We asked ourselves like what would it look like if We could do the things we did at the beginning of like our company where we could message every single person individually look at who they are and help them get started like Could we achieve this? And so that's when we, created the life cycle marketing agent, not because we want to study better life cycle marketing, but because we explicitly wanted to start asking ourselves, can we do one on one customer success for every single sign?

can we live in that world? Is this what agents can enable us to do? And that was kind of like that first iteration of that, you know, a hundred X future that we believe in. we've obviously got people doing outbound messaging, creating sequences for their team, you know, doing research, creating sequences, putting it into their outreach that people can, send out so they can have more personalized messaging.

And again, not thinking about that as a spring prey tool, but thinking about like, what does the top rep do in this company? Where are they researching? If they have an extra hour per lead, where will they go? How can we create an agent that mimics that?

So now the agent can create really good research for the team and really good content for them to send out And that's like very much a theme and what we talk about to the prospects is Don't just spray and pray on these sorts of things find out the top human quality work you can do and execute that And that's why relevance is really good actually because kind of intuitively in the past if you think of horizontal sass horizontal sass tends to have like You know, the lots of use cases and that's all I'm shallowing when it comes to our workforce.

Build a platform. The thing that's counterintuitive is you can now train an agent on your very niche and specific workflow to execute things the exact way you do versus a vertical agent being a little bit, you know, more rigid in terms of what it can do and how it does it or how it integrates the Salesforce or how it does X and Y. And you can't really train it to do things yourself. And I think the future, you know, world we're going to live in, and this could be, sounds in general, right?

AI needs to wrap itself around your process and not your, team and organization around its process, around the software's process, which has been the way we've done things in the past. I think we're going to be a lot more flexible. And the AI Workforce Builder platform kind of unlocks that today, for a lot of these use cases. Yeah, I mean, two things in response to the first, I could not agree more about the value of automating the, not even the little things, but just the more mundane things.

You know, everyone likes to share these, flashy use cases and big complicated flow charts on LinkedIn, but quite often the things that consume an inordinate amount of our time, especially in rev ops, are the duplicate accounts. And I really want to see the architecture of what that client built, because I've been thinking about that exact use case and like how to solve it, because it's such a pain point. I mean, we have ringly, we have tools, but it's very, very difficult to safely your entire database, just rules based.

There's so many exceptions where a human can look at something. and be like, yeah, clearly this is a duplicate. Clearly this is not, but it's really hard to wrap rules around that. So I mean, the flashy stuff is cool, but I agree that so much of the benefit right now, at least from where I sit is in those little things.

And number two, I just want to say, I think you've done a good job at positioning your platform for your target audience. Cause I found you guys, I guess as many people do, I was looking into agent platforms and I looked at a variety and, you know. like crew AI to take an example of a competitor of yours, but it, very clearly seemed engineer oriented. And then when I looked at your platform, like, Oh, this is built for, like, I'm not a developer.

I am a technical ops person. I'm comfortable with APIs and comfortable with Jason, et cetera, but I don't really code at least not very well, a little bit with the help of, chat GBT. I'm like, this was built for me. it makes sense.

And so. I will say I think that you've done a good job creating an interface and a mental model that works for my profile, which seems to be your target audience. I want to just drill on something you said around, training and memory, because this is something that, um, I want to understand better as it comes to agents, because people talk about, oh, you can train your agents and they learn. But how does that actually happen?

Cause quite often a lot of the agentic AI I've interacted with, some of it doesn't even have context from message to message. Like I was interacting with part of Salesforce Einstein the other day. with all of its resources, and it literally did not have context in between messages. It was like each message, one shot, you get this one chance, and certainly in other parts, it does seem to retain context between messages, but not between sessions, so how do you think about, memory and knowledge and training and making it better aside from just somebody going in and manually updating the instructions?

Yeah, I mean, on the Salesforce manager, not to take a cheap shot, but I don't think I've spoken to a single Salesforce admin who hasn't been disappointed by the over promises of Einstein and where it's ended up. We'll see if agent force lands in a similar similar bucket. But Look, that's a really difficult problem. It's the first thing I'd say, like making agents, have the cognitive abilities that, we have beyond just reasoning is a big challenge.

And there's many ways you can approach this. two things that we do at Relevance, right? one is we were actually about to launch this. We're currently in early access with a bunch of our enterprise customers is whenever agents complete tasks.

And relevance, right? We have a lot of heuristics as to whether that task was successful, i. e. either there's some feedback loop, maybe, you know, someone successfully booked in that meeting when they, came in inbound.

So that's a successful outcome. Or maybe there's something else that we can look at from the flow to determine whether that task was successful. We've got all these heuristics. And so every time a task gets completed, we have the opportunity to, one, improve the instructions of the agent.

or to actually improve the underlying model. So now we're training the model every single time a task has been completed successfully or not to make better decisions for that specific use case. And so those are the two, channels through which we're, approaching this from a product perspective that automates it for all our customers. You know, in the future, these things will just continuously get better.

And, one of them kind of speaks more to improving our brain and the other one speaks a little bit more to improving our onboarding handbook. And so that's kind of like if I was to think about it from a very human workforce perspective, how we're approaching this, for example, when escalations happen in relevance, when the agent says, Hey, I don't know how to do this, Justin, can you help me? I've just had someone asked this question. You know, I don't know how to answer it.

We also have the ability that when people provide that intervention, that I either updates the instructions or the kind of the memory and knowledge base that the agent has. So we've got those different channels to help improve it, but I agree with you, it's something that is still not as good as it can get and, you know, every month right now, but I'm seeing some of the stuff we're shipping for that piece is extraordinary and, it's really easy to forget that we're just in the early innings of what the technology can do, right?

Like we're fortunate enough that our product You know, we've been developing this now for, maybe just under two years of like real customers, we had one of the first agentic use cases ever live for the customer on autopilot. so we've got a bit of a head start, but the reality as an industry, we're scratching the surface. And so I think we're going to see a lot of improvements, with those two ones I mentioned, I think being really big ones that, you know, I expect to see huge performance boosts for our customers.

Talking a little bit more about, like, design patterns of agents and how they work together. There's this notion of agent teams, there's just something inherently fun and interesting about this notion of, like, supervisors and, workers and people with different subject matter expertise. aside from that whimsy of it, I guess, Why not just have a monolithic agent that does everything? That's a really good question.

It is fun. Part of me, you know, it is nice seeing your team of agents executing work and talking to each other and completing tasks, but the more serious answer is, and it kind of ties back again to the human workforce, right? Is there a single person in your company that can do everything? Does that exist?

The answer is probably not. And if there is, man, that's an impressive person. But the reality is we can't like, we all have to specialize somewhere. that same principle applies to agents.

What is your agent going to specialize on? Now, the difference of humans and agents is that agents at the moment specialize a little bit, have some sites, more scope and tells what they can specialize in. But that analogy. plays true.

We have agents that can specialize on tasks. They have spikes of capabilities in order to keep their performance really high because you have an agent, a monolith agent, as you described, trying to do too much, your performance will inevitably suffer. The second benefit is actually really interesting to me. And that's, again, tying it back to the human workforce.

when you have a team of people completing a piece of work, you've got multiple checkpoints to reduce mistakes and errors. Because if you were to delegate some work to me, I was to complete it and give it back to you. That review process would inherently potentially bring out. the, Hey, I've made a mistake here and that applies to agents as well as they're working with each other, delegating work.

If you've had that same hallucination, which maybe is the, how we define a mistake from an agent's perspective, and it's a big concern for a lot of people when you give that to someone else. So another agent with the context, like with the citations and stuff like that, that other agent, because it has a completely different set of instructions. And it's got a completely different set of context. It's very unlikely to make that exact same hallucination.

And so that produces a second benefit of reducing errors and hallucinations. And then the third benefit, which I actually think is the most important one and why, I highly recommend if you're thinking about, you know, to your audience about an AI strategy. Think about an AI Workforce Builder versus a vertical solution because your agents quickly compound that one agent that you built that specialize in prospect research could now be applied to help you dedupe your database, could be applied to lifecycle marketing, could be applied to an account based marketing campaign, can help you qualify inbound leads and so on and so on.

And so you quickly get this compounding effect where these agents that you're creating can be deployed from many different use cases. And the only difference is. You just construct them slightly differently, but you've already created that agent. You already know it works really well for doing research for your kind of business, and you can deploy it in many spaces.

And so that compounding effect for organizations that get this right will be extremely significant, and will generate huge amounts of value, for the business and ROI. So, That's kind of the way we think about irrelevance. It's like why teams are even so critical. and that's obviously going to evolve slightly, right?

Like as agent capabilities get better, maybe you can specialize some agents, to be a little bit more generalized. And then you can specialize some more to be even deeper on that topic and can go even like at the higher level to it. And so it just gives you that really great ability to mimic what happens to their organizations, to maximize performance, reduce errors, and also set yourself up to benefit from compounding effects of having many, many agents. The modularity that you described is one that I hadn't thought about, but it's true.

It makes the work that you're doing more reusable. And if you have an agent that's very well trained at a particular task, then being able to just plug it in, in different contexts, really valuable. in the few minutes we have left, I want to deep dive a little bit about the platform and the company and just your experience as a founder has probably has been clear to anyone listening so far. I'm, I'm a fan.

I really just like what you're doing. And when I watched the videos that your co founder did, just like explaining it, I don't know. I just really did vibe with this platform for some reason. So I'm curious, what did you set out to do two years ago and maybe how has it evolved?

Yeah, I mean, look, I think the thing that Our customers and users tend to resonate a lot with that relevance is we are really building towards something, right? I think, we are not just trying to chase a trend every month and kind of pivot the whole direction of the product. So just satisfying one requirement, you we always had a lot of success in sales throughout 2024. And it's very tempting as a product telling a lot to sales teams to say to yourself, Hey, I wanna build, you know, a dedicated experience with sales teams.

I wanna verticalize. Um, but fundamentally we have very strongly held beliefs when it comes to our vision about that subject matter expertise about moving towards autopilot. That we know that if we wanna deliver the best product to our customers and give them the best ROI from agents, we have to build in this direction. And I think that's something that's enabled us to make some really good decisions.

That have led to really great results. and you'll see that consistently throughout the messaging, right? Like so much of what we do is inspired by the human workforce, right? it's such a simple concept, but I, you see this kind of light bulb moment happen in a lot of people when I really, communicate and respond to their questions with analogies towards what they currently do.

And I find that extremely helpful for people to then be like, Oh, okay, that actually makes a lot more sense now. That's really practical how I can deploy this for myself and really get the benefit. All the technology like this. So I think that's one thing.

And I'm glad it's resonating with you, but I think that's one reason why people resonate with us is, if you read up on Copilot, 2023, you'll see how much of that is still true today. Even when, back at the time, people were like, what are you talking about? This is, uh, you know, not necessarily something that a lot of people believed, but I think, has paid dividends for us today. But in terms of, I guess, us as a, company, we So, as I said, we spent a lot of time in automation.

My co founder, Jackie, he, previously worked with me on the previous company that we built together, had millions of users. He then went to lead machine learning for a large corporate. We actually, first started looking at vector embeddings because we saw that there was a significant shift, in. Capabilities when machines are starting to understand data and we knew, okay, that plus some of the model work we've been doing and the way the models were improving felt like there will be a moment in time soon where the capabilities.

Of machines will start mimicking humans it'll enable automation to succeed in a way it hasn't before. And we've been really passionate about automation for context. Like we saw the benefits in our own business, the things we could achieve, but also like in my daily life, you know, I just think about all the quality of life things that we have because of automation. So I feel really strongly about this.

And then when we saw those two opportunities come together, and, uh, this was around the time, I guess, as well, that, GP 3. 5 was launched. Uh, we were like, okay, the AI Workforce Vision really came together for us and we started building towards that. it's been a really exciting journey so far.

We're very lucky to have some like amazing customers from small startups to public companies. Um, we're very lucky to be able to deliver a lot of value, but more importantly, we've got a lot of work ahead of us to keep delivering on that promise. And as I said, in the next few months, like six months, the barrier of entry to creating agents and relevance. It's going to keep dropping and we've got some really exciting releases to make that possible because I really want to see everybody be able to create agents to help them.

because I think it's just going to be one of those technologies that once we have it, we'll think to ourselves like, how the hell did we do things before this? as far as I can tell you and Jack, you're both technical, co founders that come from a, computer engineering background. it been organic in the sense of, you know, finding fit develop? It seems like you have a pretty engaged community.

I'm in your discord, a lot of people in there. have you thought about this sort of deliberately in terms of how you're positioning yourself? What's the thinking there? we tried to be quite intentional about position.

I think we can always do better. this year in particular, one of the main things have kind of assets on the On the mission is to make sure that everyone who's looking at agents knows about relevance. I think not only can we give them the best product, but I also think it's important that we're in the right conversation. So that's personally one of my major goals for this year.

because you know, we're getting some of the most organic mentions on LinkedIn, on YouTube. We're getting some of the most branded search queries we're getting, Whether it was ranking extremely highly for a lot of key SEO keywords, we've got a lot of that going for us, but I think this year is the opportunity for us to share relevance and kind of the AI workforce mission and vision that we have. And so I'm personally excited about that and has been organic. I mean, last year for us was when we really started commercializing our product.

And, that was an interesting transition because, as you mentioned, we're both very kind of, technical co founders and, the other Dan as well, he's very technical as well. So we don't have a great marketing background per se, but what Jack and I fortunately had is the experience of marketing products that had millions of users. Not just in that once, but we did that twice. and then a couple of other products that will start hundreds of thousands of users.

So we've always interest and understood the importance of marketing. So I think we've always tried to take that through the new thing that we've introduced, I guess, for us as a team last year was more of that enterprise emotion as well, and really leveling up the business to be able to handle those enterprise engagements. Not only building better software and tooling for it, that enterprise is required, but also, you know, making sure we build a team that is enterprise ready.

Uh, we opened up an office in San Francisco, so we're based in San Francisco now to help engage our customers in North America better. We're hiring some of those talented and brilliant people who've, either built, the RPA slash BPA versions, in large enterprises in the past. Uh, and help deploy them onto our team so we can give that same kind of level of expertise and guidance to our customers. And so we've really also done, um, uh, a lot of work around building the right team to help us engage those customers.

But it's still early days where we're actively hiring at the moment. And if anyone in your audience is interested in looking at those opportunities, please check out the website where we're hiring basically across every single team. in order to, help capture this moment in time and help deliver kind of a genetic AI to as many companies and people as possible this year. I think that's all we have time for today, but this was just super, super interesting.

So thank you. wish you folks the very best and we'll check in with you again sometime in the future. I hope. Thank you so much.

Thanks, Justin. And thanks for the opportunity to share more to the RevOps community.