219: Inside Databricks' stack with 3 AI agents, 1 lakehouse, and 6 years of data work with Elizabeth Dobbs

Liz: So Marge is our marketing genie. and she's a way you can natively interact with natural language, with your marketing data and our team spent a ton of time training this genie, How can we augment her with better data, better context? and so really that led to, agent tag atha, which is, kind of like our tagging detective agent. So she goes through all of our content and tags for.

all the boring things that our product marketers would have to manually do, every time we had new content. And then we realized, hey, something that was really hard is we had a lot of complex segmentation in marketing, what if we built an agent that just kind of We're gonna come to work on Monday, and we're gonna have three new audiences we didn't think we had on Friday. by the way, we don't have two quarters figured out. So, uh, thus Agent Atlas was born as just a way to help us navigate, all of our audience decisions for both a rules-based perspective as well as an intent and content engagement perspective.

In This Episode - Phil: What's up everyone? Today we have the pleasure of sitting down with Elizabeth Dobbs, a VP of MarTech Data and Growth at Databricks. In this conversation, Liz takes us inside the marketing ops agents that our team has built starting with their AI analyst agent, their content tagging agent. Their segmentation agent will also cover how Databricks embeds data engineers and marketing and what our team is testing for in marketing hires these days.

All that and a bunch more stuff. I have a quick word from two of our awesome partners. Sponsor: Knak Sponsor: MoEngage - ​ Phil: Liz, thank you so much for your time today. Really excited to chat.

Liz: Thanks, Phil. This is, uh, something I've been very much looking forward to. I love talking about this stuff and happy to be here with, uh, fellow adults versus my two young kids at home. Phil: Uh, I love it.

Yeah. Two young kids, uh, on my end also. Um, yeah, put a lot of time into prep for this episode. Really excited to chat.

you spent the last like six, 12 months building agents and Why Velocity Beats Permanence in Marketing Data Architecture - Phil: your marketing team's been building agents for marketing use cases. You're calling them parallel workers alongside the humans on your team. But before we get into that, um, there's a couple things I wanted to chat about. Your earlier journey at Databricks.

I read that when you were interviewing at Databricks like six years ago now. Um, someone on the sales ops team during the interview process said straight up, like, our data is a complete mess. And you thought they were being humble, you know, like a data company. How bad could it actually be?

But you came in and you found out that, uh, data was pretty messy. They, they meant what they said. So maybe take us back like 60 years ago, first couple of years of joining the team, fixing a lot of this stuff, how to. How, how bad were things?

Actually, six years ago. Liz: Yeah. You know, it's, it's funny and as I said, like the company's called Databricks, like data is in the name. It cannot be that bad.

Um, but it's, it's funny. It just like it, a trick goes to show, uh, you can't judge a book by its cover and it turns out all startups, the data is just bad. Uh, and Databricks was probably no worse than others at the same size and scale that they were growing. Um, but you know, I think something that you learn pretty quickly when you come here is like, kind of.

Building for permanence in this world is just a, it's kind of an effort in futility because even if you get something completely perfect, you know, the game has changed in six months. And so, you know, we're, we're uh, kind of optimizing here for velocity and not durability. And I think, you know, very much lately, when I joined Databricks, we were about a thousand people. Now we're over, you know, 10,000.

And so there's a new, uh, you know, triangulation point now, which is scale. So you have to build for velocity scale and some, some semblance of durability because there's far more people than just, you know, 40 marketers for, you know, relying on you to get these things out. So, um. You know, when we kind of came in and I, you know, was super lucky to come work for an amazing CMO who is still here.

Uh, an amazing marketing team who's, you know, since scaled with this company. But, you know, when we first started we realized we were just kind of a little bit all over the place. We had, uh, agencies for a little bit, kind of agency sake 'cause we just didn't have the people to do the work. We had a lot of kinda systems that didn't talk to each other very well and a lot of human middleware attacks.

Not to say we don't still have that. but really we just kind of focused on kind of the, the core principles, which was like, let's centralize the data in a very imperfect way just to kind of set ourselves up on where we thought the industry was gonna go in the future. Um, build some shared definitions and logic to help us kind of build a structure of standardization, even if we don't always follow it. Um, and really just kind of bringing in partners to augment where we had gaps in the product or we had gaps in our team.

Agencies and really surgically kind of make sure that we were building a team that was set up for success. Phil: what advice would you have for folks listening that you know, when you think of fixing things and rebuilding this stack? Most marketing office people are thinking of, how do I do this? With permanence, we want to build a foundation.

We don't wanna like have to rip this apart every six to 12 months if we're doing this whole project. I wanna do it once, do it well, and then we can bolt on things on top. But like, I want the foundation to be rock solid and not change too much. Um, you mentioned like in, in the article that I read, like bubblegum and duct tape and you did this like intentionally knowing the health.

Fast. The company was changing like in six years. It's crazy. The amount of things that have changed, um, like what advice do you have for folks to get comfortable with that idea of impermanence when it comes to like the data stack and the MarTech stack?

Liz: Yeah, I mean, I think it's like anything in life, you really have to pick your battles. I don't think every six months you can like decompose everything and come back to it. So you kind of need to figure out like where do I think there is opportunities in six, 12 months? And I think agents are of really great heuristic to be thinking about because.

I actually do think a lot of agents are not yet ready to be fully human outta the loop, fully, like prime time on their own, but let's say in 12, 18 months, like they will absolutely be there. And so kind of what our job is, I think as a, a marketing technology team is kind of just to place really strategic bets like you're in Vegas, like. All right. It, we might be hedging here, but we really feel like the industry, the velocity, what we know about the space is gonna catch up.

And we don't wanna be the team that's like, been waiting for it to be perfect and then be ready to start. Um, so I think, you know, you, you have to think about, I think the idea of like the one-way door, right? Like what are the one-way door decisions where we're gonna go through and it's really hard and painful to step back. And what are the things where we think we can make, uh, architectural decisions, which, you know, could be painful up front, but will pay us dividends in the long term.

And so something, you know, really building our marketing Lakehouse is a great example of that because the idea is like, it's nothing new. It's just, Hey, bring all of your marketing data into one place. Have it be governed, have it be cataloged. Have it, you know, everything kind of working in the data models we need.

And like, that's kind of like a no regret decision. Like there's nothing like a SAS that we would buy or something we do differently that like wouldn't benefit from that foundation. So I think those are the things where, you know, I think you make that decision and they're, they're messy and hard decisions to get. Right.

But those are the kind of things where it's like, hey, we're gonna keep pouring resources and time and energy into this because. The more we get this right, the better everything we do on top of it will be. And then there's these like kind of like, Hey, let's rip and replace, let's try this. Like agents could be offsetting this entire group of work or this entire piece of software and like we could build an app for it, but like, you know, it's not there yet.

And so those are kind of the way that we think about it is like kind of more of the short. Uh, short term wins, but really I think where we pour most of our energy is these long term foundational things that we feel like if these are the rising tide, that will lift all the boats we do in marketing. Phil: Very cool. So when you think back, like six years when you joined the list of tools that the team was using versus today, six years later, um, like how crazy different are are things today compared to six years ago?

Liz: I am gonna blow your mind, Phil, like we had more tech coming into Databricks in this job I've ever had in my career, and it was like not the largest company. And I will say we still have tools like LeanData today had back then. Uh, you know, we still use, like a lot of the prospecting tools are very similar. The webinar platforms shoot like the same, like I, I mean, I would say it's funny.

If you look at the landscape of our SaaS vendors from six years ago towards today, I mean there are different, like we've swapped some things in and out. We've made little changes, but like a lot of like the core components to solving business problems are actually the same vendors, and I think where. We've been really fortunate as we, we came in using these vendors as you know, I'd say a small fish, and we're much bigger fish now with more complexity. But a lot of our vendors have like kind of grown and partnered with us on that journey where, um, you know, have been really flexible, helped be like great partners when it came to how could we change this?

What's the right solution we're trying to scale, but like things are crazy. How can we think about this differently? Um, and so really you'll find the vendors in our stack that have stuck around since. You know, I was fortunate enough to join Databricks are the ones who just have really been really great partners over the years.

Phil: Very cool. Shout out to NAC too. Uh, I know Databricks is a long-term customer of of nac. NAC is based locally in auto.

They're a longtime sponsor of the podcast. Liz: They're, they're a great example. I think NAC is probably like, who knows what's going on on Databricks, but those contracts are crazy. Phil: I love it.

Um, Why Databricks Embedded Data Engineers Inside Marketing - Phil: let's, let's chat about the team structure a little bit. 'cause, um, like the team name that you lead is MarTech Data and Growth. And you mentioned something in one of the interviews that I read that I, I don't hear a lot of, uh, MarTech owners say, is that like. The whole marketing stack is under your team independently.

So you have data engineers that sit under your team. You have data scientists that sit under your team on top of a bunch of marketers, I'm sure. How did you get that structure? Like did you fight for it?

Did it something that, is it something that kinda like naturally or organically evolved during your time there and talk to us about that. Liz: Yeah, you know, I think I, again, joined Databricks at a very special time back in the day and I think, you know, was working to, in that under two really great leaders. One, um, our CCIO at the time, Mike Hamilton, and again, Rick, our CMO, uh, were just really great partners as our team kind of was storming and norming. Um, and I think, you know, it was really helpful because having really great partners in it and central teams saying like, Hey.

We've got a lot of fires ourselves. I feel like you guys, you know, enough to be dangerous, you know, kind of the boundaries of what you kind of can't do. Like just files of Jira tickets. You guys can go run the way you need to run.

Um, and Rick is just a really great advocate for saying, Hey, I want this. Okay, here's what we need to be successful. And then Rick going to bat to kind of bringing that into reality. Um, so I, I really think it comes down to having leaders who were really supportive, knowing.

The value of marketing, kind of knowing before maybe a lot of other leaders that the big unlock for us would be allowing us to bring in specialists who are very familiar with the marketing data domain, which six years ago was. Much less like common than it is today. Uh, but I think what we really were able to show is like, hey, if we can bring in, you know, a, a bench of, you know, and by bench I mean one data engineer or two who, who had experience in marketing, we'd be able to move so much faster.

Than we could if we were kind of constantly trying to update or train or enable a team who didn't have the contextual knowledge. And you're really seeing that play out now in this era of agents. It's like, it's kind of the same thing. It's like these agents are a blank slate.

You spend a lot of time training, enabling, providing context, explaining the nuance. And the cool part is like, yeah, with agents, like they learn it and they'll just kind of continue it. But when you think about internal resourcing, you're training a human. A lot of times it's not the same person.

And so that, that time you spend, um, was time just back then, we really didn't have to order to move fast to meet the needs of the business. And so it really was an organic development. And I think it's, you know, kudos to, um, you know, the incredible team I've had the opportunity to work with over the time, but it's. Everyone who comes into our team really does raise the bar and just shows kind of the value that we have.

And, you know, fundamentally the idea of one plus one equals three, we're better together. We can move faster and do more experimentation and fun things when we're able to kind of move quickly. Um, and within marketing. Inside Databricks' 3 Marketing Ops Agents - Phil: So let, let's talk about three new people that have joined the team over the last six, 12 months.

You're calling them parallel workers and, um, maybe we can start with like the operational story for, for some of these agents. So there's, uh. We're gonna talk about like a conversational assistant, a content tagging assistant, and then a segmentation engine. Um, what's the actual order that they kinda came in for, for your team and what did maybe building each one kinda teach you that made the next one possible?

Liz: Yeah, it's, it's a great question. Um, and similar to Databricks and our story about hedging maybe before the technology and things are ready, we started with the hardest one, which was Marge. So Marge is our marketing genie. Um, and she's really, uh, just a kind of a, a way you can natively interact with natural language, with your marketing data.

And it was a, a. Tool that Databricks had launched. Um, and we just had some really smart people in our, in our marketing data team who were like, Hey, this is the future. We should totally do this.

Um, and I was like, yeah, awesome. Go for it. And then, so, you know, Thomas and our team spent a ton of time training this genie, building this genie. We rolled this genie out and it wasn't perfect.

There were gaps in her connectional knowledge. The product was still evolving. So it was kind of this thing where we're like. Okay, let's pause.

We know this is the future. We know we have to get this right, but also like, how do we set her up for success in the future of where we need to go? So I think this is the idea of, um, again, having just a great team and a great company, there's a little vision of like, where the industry and stuff is going is we can te, you know, use these experiences as a speed bump versus stop sign. And so what we talked about Marge is like, okay.

How can we augment her with better data, better context? How do we connect things better with her in the lakehouse so she has more information when she's trying to give an answer. Um, and so really that led to, uh, agent tag atha, which is, um, originally tag your bot. So we've gone kind of on a journey, hero's journey on naming.

Uh, but Agent Taha is kind of like our tagging detective agent. So she goes through all of our content and tags for. Um, all the boring things that our product marketers would have to manually do, uh, every time we had new content. And then, let's be honest, that no one went to do when we updated our products.

So, um, it solved a really big pain point in marketing, which is like, Hey, how do we give someone a first draft of all of the information we need for all of us to be successful in our reporting and in our, you know, agents? And then also like, make sure what we have in the past is kind of constantly being updated, revised. Um, and so each Taha kind of really helped us build that kind of content con tagging layer. And then we realized, hey, something that was really hard is we had a lot of complex segmentation in marketing, but we were just kind of constantly, again, back to the bubble, Goa duct tape.

Rules-based things are very hard to maintain in a complex environment, and so. Um, uh, you know, Anup and our team had a great idea, which is like, what if we built an agent that just kind of took all of that away and really was a way we can bring together both rules-based and intent-based logic and, and kind of build a, some, a segmentation structure that could move what the pace that Databricks does, which means, hey. We're gonna come to work on today, Monday, and we're gonna have three new audiences we didn't think we had on Friday.

Uh, and by the way, we don't have two quarters figured out. So, uh, thus Agent Atlas was born as just a way to help us navigate, um, Phil: Hmm. Liz: all of our audience decisions for both a rules-based perspective as well as an intent and content engagement perspective. And by the way, have the benefit of being able to be nimble and agile the same way our marketing team has to be here at Databricks.

And yeah. And then now I think Marge is all the better for it because she has all of this foundation to find, uh, to, to build on. How Databricks Built an AI Analyst That Marketing Teams Actually Trust - Phil: So Marge was the Guinea pig initially, and now she has a couple of friends that, that are helping her with with the day-to-day. Maybe we can chat about like some of the nuts and bolts about Marge.

Uh, let, let, let's chat about March 1st. So you mentioned the, the Genie product that Databricks has. Um, it's a text to SQL product and your team has. Like, it's not just Genie, right?

Like your team, you said, built like a custom agent layer that's specifically trained on your marketing data at Databricks. How did your team kind of ensure that Marge's answers were trustworthy and and accurate? Maybe chat about like that part of the build process. Like how did you get folks to feel confident?

When I ask a question to Marge, she surface data, she understands the tables, the contacts and the joins and, and all that, that, that kind of story. Liz: Yeah. Um, slowly and painfully is the right answer. Um, no, I mean, and I think it's, it is just a really great partnership.

I mean, our data team, we work very closely with our product team at Databricks and they're just really incredible partners. So willing to take feedback and build it into the product or help us make sure we can service the things that really we know will instill trust with our, our end users, which are our marketers. Um. And so I think the first, you know, I think three, three things we really did is, one is just making sure we had that strong semantic layer.

We all the tags in there. What does this mean? Um, we actually sat and met with all the marketers to be like, okay, ADD team. Do you say ROI?

Do you say roas? Do you say like this? And then we actually train Marge to understand. So if anybody on, you know, our marketing team who all talks about ROIA little bit differently.

That she understands when every team uses a term, how she should be calculating that. So that's kind of something that, you know, you don't realize it, but if you actually go into marketing, there are so many disciplines in marketing and there's a lot of nuance in those disciplines. And I think everyone kind of, you can kind of have a common understanding when you speak externally, but if you actually sit with someone and like shadow them, who is in our partner team talking about MDF, like that is a different universe than someone who's, you know, out here running a third party event.

And so we really needed Marge to kind of, again, with the vision of you have a 24 7 analyst who understands your business. She need to understand the disciplines of marketing. So that's kinda the first thing we did. The second thing is, is Thomas, uh, did an incredible job, like really built out a lot of question answer pairs.

So when you think about. Sequel and Tech Seql and think about Ja, GBT and, uh, cloud. I mean, they're, they're all super powerful and great tools, but a lot of times, like you're kind of obfuscated from like, where is this coming from? Like, how do I know this is right?

And really something they did, um, a really great job of is like we actually have what's it is trusted answers within Merge. So if a marketer asks a question. You'll actually, it'll reference all of these like approved, trusted answers. So you can actually see, hey, this is a green check.

I know it's a hundred percent right, like let's we, we can run and activate on it. And the interesting piece is like where I think Thomas done a really great job is the question answer pairs. They're also like contextual. So if you say, Hey, what is the ROAS for AMEA in Q1?

And the question answer pair is actually the ROAS for AM mayor. So we know AMEA is certified, but we just didn't certify it for amea. But, but like Marge can actually pull that through and say, Hey, this is a green check because here is a, like, question we know is a hundred percent right. Um, and then I, it also shows that the sequel actually ran through.

So if anybody, um, does or is curious or concerned that can go in and say like, Hey, what are the things that Marge actually referenced? It's a little technical, but I think the nice piece about it is like. You know, if, if you've been in, you kind of in tech for long enough, you can kind of make sense SQL, if you look at it and squint a little bit. Like, I think that that really does help.

Um, and then I think the third thing is like we, she, we don't like release her to the wild and we're like, good luck everyone. Like she, we do look at, she has a feedback loop. She had thumbs up, thumbs down. We do benchmarking, we kind of look at all of the data on a very regular basis.

I think we spend about like a no, an hour or two a week going through all the feedback and making sure if we're getting bad answers, how do we improve her? So it's not just like a, an agent that's out there by itself. It is constantly kind of governed, reviewed, and we, um, well, you know, if we have issues or we will reach out to the marketer and say, Hey, we can see your whole query. Why did you go like this?

What, what, what felt wrong? And just really make sure we're constantly improving her. So I think those three things are kind of, uh, you know, what we've been doing to make sure that Marge is tru, trustworthy, actionable, and really kind of able to keep pace with the marketers here. Phil: Yeah.

Very cool. I love the, the trusted answer there. I feel like it's a, a, a good way to have someone not have to second guess everything. And like when they're presenting this data, they can say like, you know, this is legit because Barge said it was legit.

That was the green check Liz: Yeah. Don't blame me. Blame Marge. Phil: So like behind the scenes, it, Marge isn't just like one, like specialized agent, right?

Like is there like a hierarchy of specialized genies like the, the Databricks product with like an agent routing layer that knows which genie to send what question to like based on roles or like definitions? Can you chat about that? Liz: Yeah, it's a, it is a great question. It's exactly where we're going as a, as a team.

So I think where we. Started is we started with Marge and we're like, Marge is the future. We love Marge. She's wonderful.

Um, and she, she's trained on like so much, uh, all of the data you can, can feasibly ask marketing, like she'll have some kind of answer for it. But you realize like when you try and go really deep with Marge, that's where she is much information that having so much contacts is actually a negative in some of these things. Um, and so what we started doing is building out these child genie spaces, um, which are really deeply trained on like our marketing, our digital advertising data, because we wanted her to be able to have very good answers, but go very, very deep on these things, um, which she couldn't do if she had so much context.

And so really what we're, we're building up is this kind of, uh, idea of the one chat. So you have like a chat interface very similar to like the one text bar, Chad, GB like thing. But on the backend we just kind of have all of our genie who are trained with, hey, what their specializations are. So the idea is the marketer doesn't really know that there are all of these genies out there.

There's all these genie rooms that they have to navigate between them because that's just. Hard for, I think someone to keep in their head all the time is, Hey, I asked a question. Did I go to the right gen room? Is this the right answer?

Do I be like, so we are trying to obfuscate all the complexity in it, but the idea is yes, if a marketer has like a 1 0 1 question, Marge is the place that will will take them. But if they're going to go deep, they're trying to really get, um, you know, a deep research answer, which has been really helpful for our team, which is like kind of the, the way you can actually have Marge do kind of deep analysis on insights. That's where we really make sure you have to end up in the right kind of genie room for all additional context, much more narrow scope so she can be much more effective in in her answers.

Phil: Super cool. I appreciate the, the depth of the exploration behind, uh. Marge there. How Agent Tagatha Cut Months of Manual Content Tagging to Hours - Phil: I wanna be fair to, to Taha and, and, and, and chat about some of the stuff that you did on, on tagging content.

So, you know, anyone that works in content operations, even marketing operations, that's a deal with the CMS, knows the pain of strategy changes. ICP changes, product changes, new products, like all taxonomy changes requires retagging of everything. I'm in the content world, so I deal with that myself all the time. Um, so Taha helps automate a lot of the content, tagging that a lot of.

Product marketers on the team probably had to like, spend weeks doing, um, what, how, like, how would you explain like what Taha does that your average CMS with tagging features can't do as well? Can you maybe chat about that? Liz: Yeah, I mean, I think the, the cool thing you know about everything, all the agents we're building is they all kind of sit on the same foundation of our marketing lakehouse. So the idea of all of the, the data we have, the content, the context, you know, how we think about things, it, it go, it kind of gets built into the agents that we're then delivering.

And so I think what's cool about, you know, agent Taha. And again, we call the tag your rip bot because it had all of these steps where each bot would kind of tag the next top of the bot of like, okay, first you read and pull out the, the content, and then you look for these things, and then you then, uh, you know, map it against the approved values. Then you translate it. So I think this is, you know, obviously our next version as we've gotten more mature as an organization, but I think the idea and the differentiation is, is really like when you think about like the.

Tag atha as a, just the idea of it is that we kind of decided that tagging is just a business problem at Databricks in general. And that content was actually the simplest part of it and really like, it really ended up being the foundation. We think about we wanna run a campaign, what are campaigns built on tags, customer or prospect, you know, which region do we want? Like it really ended up being this foundational element of.

Customer stories of every part of marketing. And so, um, you know, when we talk about what the CMS agents can do and they're getting, you know, better and better, we just think it's in-house because you have all the, in the business context that we've built in, it's gonna be more effective and more efficient, um, to get all of our tags on our content. So really what we've done is we started really easy. We fed it things like blogs and webpage.

So we just Phil: Mm-hmm. Liz: tag atha. You know, read all of the content on this. Here are the tags we need, pull it out.

Here's how we like, you know, you know, if there's a tie, you know, pull these values in. And then all of that goes into kind of obviously, you know, a warehouse that we kind of maintain and look at. And then that that's which is our marketing lakehouse. And so I think the nice piece about it is when we need to update a tag, we can just go in and, you know, update agent tag at that and say, this was this.

Now Genie is a product. You know, pull it out, make it like this and it can do that moving forward, but also retroactively. Uh, and I, you know, I think the, the idea is I think where Tag atha, Atlas, Marge, these are, this, the concept is like where are marketing spending a lot of their time? That is like low value.

And it's not that like having good context and tagging is low value because that's actually the lifeblood. Every good marketing organization. But you think about the low value of like somebody going through and being like, oh, what was that white paper about? What did I tag on it?

Like, and that's not someone good, good use of time. And so really where we tried to like build these initial agents that we built over the last 12 months is like, how do we find these high pain point, very friction filled problems that kind of are tailor made for an agent to step in and take off that manual work from a, from a marketer or a D of person. So. I think that's where these aren't like foundational, changing how agents are doing marketing anymore, but it's really like what is the business problem?

What is the friction people in our team were facing in the last year, and how do we build agents to really solve those individual pain points? I. Sponsor: AttributionApp - ​ Sponsor: GrowthLoop - ​ Phil: How would you describe the. Like path that Taha took from like V one to V two V three, like behind the scenes.

Would you describe it as like, it's essentially an LLM classification pipeline that you're feeding different types of content to, and it's just running different scheduled Databricks jobs and the output is like a structured schema and it just outputs like a bunch of different tags. Like what, what's changed from V one to like whatever V four you're in now, like context drift feedback loops. Can you chat about that? Liz: yeah.

You know, I think. The, honestly, I'm not as close to this one, so I probably don't have a really great answer for it. Uh, the web team maintains this, um, and with our data team, because that's where it originally started, was really getting our web pipeline data. Great.

Um, but I think, you know, when we, the, we called it TAGit Bot originally because we had to have so many different. Like check bots in the middle. Like, does this look right? Does this look right?

Okay. No it doesn't. And so we call it tag your RIP bot because you really couldn't get from content to tag without like several check-in points in the middle. And now I think the way, I think the biggest change now is like we've actually identified where the drift was happening.

We've really controlled for it. We've looked at it. The agents have just gotten better foundationally from V one to where we are today. So I do think it's like, I think while we've improved the architecture of, of tag atha.

A lot of it is also these foundational models are just getting so much better than they used to be. So the work that, you know, if I would say we started a year ago and we wanted to build tag atha and we had the agents we have right now, I don't think we would've had to go on this like kind of complex hero's journey of where we started, where we ending. I think these foundational models are getting so much better. Um, and the idea of like, I think it should been great is.

Working with these models where you can have a platform that you can say, Hey, sometimes Claude is better than open ai. Sometimes open AI is better than Claude. And so it's kind of nice where you know, you don't have to get locked into one model because there's really the efficacy of a model and then there's the cost because some are very expensive, some are cheaper, and so it's kind of like this, you know, balance where you always wanna strike, like what is the most cost efficient?

But highest outcome model, and how do we find the right model for the right problem? And that's kind of like where I think our team is starting to think about more is like, okay, we know all these things. We want all these agents to work really efficiently and really effectively, but it turns out it's like a little bit different for each of 'em. And so it's our team's job to just also keep our finger on the pulse of like, how do we constantly look at where we're spending the value we're getting, and making sure we're using the right agent and the right model for the right job?

Phil: Very cool. How Agent Atlas Replaced the Rules-Based Segmentation Wheel - Phil: So last but not least, we have Atlas, um, Atlas is maybe a bit closer in terms of use case to when I was in-house and dealt with a lot of segmentation In the world of like a lifecycle marketer, what is the best message to send to the right person at the right time with the right contacts on the right channel? Um, so you said that. Liz: silver bullet marketing.

Phil: Yeah. You said before it was a lot of static rules, right? Like before Atlas, um, you were making decisions based on like who gets what. There were static rules that, uh, a lot of the marketers wrote down.

Atlas is now making judgments about which humans get which message based on a bunch of data, especially behavioral signals. Um, can you chat about like behind the scenes atlas, the builder? I dunno how close you're to that one, but like, is there like. Propensity modeling behind that.

It's like obviously predicting like the likelihood that someone might do certain things. So like, we're gonna send this message to that person. Uh, can you chat about that a bit? Liz: So Atlas is actually more on the segmentation itself, not on the next best action.

Just, um, so just the, the way think. I think Atlas is what, what we used to be before is we used to kind of be a little bit of like a house divided where we have all of the segments based on like title or job function, very traditional way that you look at this and then you're like, okay, in AMEA they call it head of is that a director? Having these conversations, which I think everyone has everywhere. Um, and then you're thinking this other side, oh great, we have agent Aha.

Now we have all this like world of intent and we have all these things. Okay. People are interested in. Genie, they're interested in data warehousing, they're interested in like, you know, agents.

And so we're getting all this great contextual data around like topics, and something we realized is like. It was just so bulky and very hard to maintain when you're trying to bridge a world of two things. And then you're thinking about a program and you're like, Hey, I wanna run a program that's going to be, uh, talking about, you know, machine learning. And it's like, great.

So we're doing the data, data scientists, but also these people who are interested in this, but they're also data engineers. What does that mean? And so you realize you're kind of having these like, not very valuable conversations, but you're having them quite frequently. Uh, 'cause you're trying to marry these two worlds of intent and, and title based.

Um, and so something that or Atlas was really helpful is like the third, you know, vector on that is like, Hey, there's a new new audience tomorrow. What do we do? Uh, and so really the idea was, hey, can we bring all of these things together? Can we bring together the idea of these, you know, rules based, based on title?

I think seniority is always gonna be an important vector for us in marketing because we need to know who. You know, the decision makers are versus the practitioners. So we can kind of best offer content. We're getting into the topics, they're interested, what's the product?

Is it a solution? Is it an industry? Like how do we bring that together? And so each and Atlas was really, um, even before this idea of next best action, because I think that's actually a separate grouping of things.

This is just foundationally how do we organize our audiences, ourselves, our programs, in order to enable the idea of next best action. So this is much more, this is much more like kind of. Base level, how do we get ourselves out of this kind of, uh, wheel of kind of constant iteration on these rules-based, intent-based conversation, the hierarchy, and kind of bring those together in a way that it made it easier for our marketers and clearer for our systems when we say we wanna do something, that we have a way to organize ourselves so it makes it much more effective.

Phil: Gotcha. So next, next, yeah, next best action would be like V two, V three, or like another, uh, Liz: think it'd be a different agent. Phil: Yeah. Liz: I'm outta, I'm outta names at the moment, so we gotta come up with something, uh, very pithy before we get to it.

But yeah, I think that's, that's. What we're, I think where we're going now and what we've been able to do is now we have Atlas, we have better segments, is now we're able to test the efficacy of those segments across multiple channels like web and email, across, you know, web, email, and digital. So I think that's kind of this next era of like, okay, we have the audiences pretty well defined. We think they're kind of future-proofed.

Now it's kind of getting the channels all aligned, getting the message all aligned, and then obviously the timing. And I think that's where agents are gonna be so helpful. I think we haven't quite yet gotten to the point where, you know, we're human outta the loop when it comes to agentic content execution. I think we still have a ways to go, at least ourselves before it's something that we're really happy with.

But I think we're very bullish, obviously, on this, and we're. You know, we're con continuing to iterate on our ability to use more and more agents to offset these manual work, or at least come with these first drafts that our marketers can, uh, get to final product much faster. Phil: Super cool. Why Marketers Don't Care Whether You Call It an Agent - Phil: So you mentioned human in the loop there and like the, the definition I feel like of like the semantics of, of agents and marketing specifically.

Everyone has a different like, opinion on like, uh, the variety of definitions, the consensus. Um, I've read, you know, a bunch of different sources, like it needs to have a goal direction, needs to be fully autonomous, like it runs without approval. Any decision points need to be adapted. And like it can rewrite its own tasks.

Um, you said like just now that you're kinda nibbling at the edges of, of the work and that the agents aren't quite ready, a whole thing, end to end yet to do it autonomously when you're like inside what you've built, um, like would you call Marge or Tag Ather or Atlas agents by the, the standard definition? Like does it even matter what you call them? What are your thoughts there? Liz: Um, yes.

I mean, I, I think the, to me, it doesn't really matter what you call them because I think at the end of the day when you're talking to a marketer, and that's what our team, our end, end of the day, our job is to build tools that serve marketing and reduce. The work from a human. And does it matter if it's an agent? Does it matter if it's like a really well defined workflow and the automation on the backend, the marketer's not gonna know?

Like I think this idea and this obsession of like, Hey, we have to build like an agent. It has to be this perfect agent. It has to do like, yeah, all those things are well and good, but I think at the end of the day, like sometimes a marketer would prefer something that. Just works then something that's almost trying to do too much, uh, and not do it a hundred percent or 80% or 90%.

Like I think if you're trying to do everything you do at 50%, sometimes you're actually putting more work back on the person than I could do. Like I could just do it myself and like half the time versus you giving me a crappy first draft that I've to spend so much time editing it, like I can just write it better myself. It's not that hard. So I think that's like the.

the agent semantics. At the end of the day, like I think there, there is value in like having conical de canonical definitions of things, but ultimately when I, you know, we talked to our team, like we think about our team's north stars, and we really wanna make sure that, you know, especially this year, we're really revisiting what does work look like? And we're thinking about work from like, what does a work look like that marketers doing today, but how do we actually break that down into digestible chunks?

And then allowing ourselves the space and the time to think what could this work look like in the future? Not just focusing on building a bunch of bolt-on point solutions that don't necessarily kind of fundamentally disrupt the way we've been thinking about doing marketing in the last, you know, 20 years. Um, and the other thing is like, our job is to help make marketers make better decisions, whether it's an agent, partner, whether it's like a really great dashboard, uh, tailor made to them, whether it's an insights in their inbox.

Like, I don't think they really care how it's built. They just wanna make sure that the, the tools are in serving them to do their job better. Uh, and they're getting the insights and the opportunities to make the best campaigns they can make. Phil: I love it.

I appreciate the, the, the reality grounding there of like, you know, market. I just don't really care if we're calling it agents or not. Um, like when I was taking my first dive into a. Cloud code a couple months ago, like building pipeline automations for like the content production on, on the podcast.

Initially cloud was just like calling everything agents, agents for this and agents for that. And I was just like fighting excellent. Like we're just like doing very simple workflow automations here. Like why do we need to call everything in agents?

Um, but I think it was just going with the hype cycle of what everyone else is calling an agent. Liz: Which, which is fine. I mean, you gotta kind of roll with, you're like, you know, you're in MarTech, you gotta roll with the punches, you know, with marketing automation, everything was marketing automation, like. You know, it is, it is definitely, as you said, it's the hype cycle.

Um, but you know, I, I think sometimes people get so focused on agents they forget what like really great workflow automation can do and it's like that's the kind of stuff sometimes that like, you can, we can package it up in this agent bow, but like we go back to it, it's like it's actually just a really good workflow. Phil: Yeah, it's such a good point. Um, Liz, I, How to Get Data Warehouse Access When Your Team Doesn't Own It - Phil: I do wanna like have a bit of time to ask you about being customer zero for, for Databricks.

Uh, not everyone has this experience, uh, but like most marketing ops and MarTech pros, like don't spend the majority of their day in the data warehouse or the lake house, your Databricks customer. Zero in the sense that like you're. Marketing team is simultaneously running real campaigns, but also proving the data platform works, um, for like the product team. So in a way, you're kind of forced to be closer to the product.

Um, so you can be that proof point for them. Um, maybe you're like the, the, the alpha in, in a lot of cases for new features. What, what advice do you have for marketing s folks who. Don't have the data warehouse keys in like the areas of ownership with the data team because they don't have data engineers on, on, on their team specifically, like how does a marketing ops person or a MarTech person start to become more familiar with the data warehouse?

What do you, what are your thoughts there? Liz: Yeah, it's a, it's a great question. Um. And I think funny enough, like I I, something that we look at for our team when we're hiring, um, is we like to hire people who are curious and want to try something new and have an open mind, a growth with mindset, you know, a little overused term.

But I think the idea is like, if you have a question, there's usually someone at your company who knows the answer or knows how you can get that answer. And so I think it really comes down to something that's served, you know, myself and my career and my team really well. Here is like. Just having relationships with everyone in different functions.

So you know who to you can go to if you have a question. So for instance, if we, you know, we work very closely with our data platform team. They own all the ingestion for Databricks. And if I have a question about, you know, the way data's coming in, like I kind of need to know who made the decision, who to call.

And sure you can like look through Jira and try and figure it out. Ideally, like, you know, I have people that I've known for years at this point, and I can just say, Hey, would you mind telling me like where I can go to find X, Y, Z? And usually like, because at the end of the day, we are a company of data people. Sure.

But like again, to your podcast, we're humans. And if you have good relationships with people in different departments, like you can learn a lot from them. And a lot of times they're curious about what you're doing too. So I think it's always helpful like never to come with a, Hey, I'm trying to do this, I need to do this, do this, who do I get to do this?

For me, it's like coming with a question and like actually being curious about like, Hey, can you help me understand where, what's going on here? Like, that gets you pretty far. 'cause I think people fundamentally like to be helpful and they like to like explain what they do for a living. And it's fun because, you know, a Databricks like.

You wouldn't be here if you weren't passionate about like what you were doing for a living. Like it's, it's a hard company. Like the, the work life balance is crazy. Like we're running every day and like you have to find the joy in it.

And so it's like coming with questions, I think is the first thing too. Building relationships, like I said, is so critical. Like the more you can build strong relationship with cross-functional counterparts within your marketing team with different disciplines of marketing. Friends across the aisle in sales strategy, you know it.

All of these things are going to make you know you more successful in your career and also better at your job because you're gonna find people you can lean on if you run into blockers, and especially at a company size like we are today. It's things that you just go through the standard ticketing process, like no offense to ticketing, but like, you know, that's kind of the slowest path to where you need to go. Sometimes you just need to be like, Hey, this is a priority. I need to call on some favors.

And you have that relationship to fall back on if you didn't. It makes things very hard, um, at large companies, in small companies too. Um, and then, you know, I think where, you know, you. Uh, like the last thing is like, sometimes you just need to ask, like my dad always says like, the worst they can say is no.

Um, and so sometimes you just say, Hey, can I get this or can I, how do I get this? Can I get access to this? Like, they might say no, but like they might also say yes, or, Hey, as long as you do this, then you can have it. So I think those are the like kind of the three things, uh, that have been helpful for us on our customer Zero Journey.

Like we were actually, Marge was one of the first production genie spaces. We were one of the first customers internally on Unity Catalog. So I think, you know, we've been very lucky to benefit from a really strong product, great direction. Um, but we started on this journey, like Databricks did not work for marketing to be totally transparent.

Uh, you know, we've had to grow up and kind of adapt and modify the product to fit our use cases. Now it's much easier than it used to be. But I think the idea is like you kind of just have to have faith and conviction that like this is just the direction the industry is going. And kind of as I said earlier, like treating these things like speed bumps, not stop science.

If you let yourself just get like totally uh, disillusioned when you kind of, something doesn't work or you run into your first problem, like you won't be successful in the next era of marketing. I think this next era of marketing, like the people who are gonna be the best, are the people who are gonna be willing to take risks. Fail and then try again. Phil: Uh, What Databricks Is Actually Testing for in Marketing Hires Now - Phil: what are your thoughts on like this next era of marketing teams?

Like you mentioned curiosity and like growth mindset or like two of the things you look for when you're hiring new people on your team. What's actually changed about who you want on your marketing team when you're thinking of this next era? Like, not the job title, but like. You know, what's the thing you're testing for interviews now that maybe you weren't asking five years ago?

Liz: Yeah, I mean I think the, the basic one I think everyone asked for is like, what are you using AI for today? Um, I don't even ask that anymore 'cause I'm like, appar, everyone's prepped for this question. So I kind of believe nothing of what they say. I wanna Chad GBT and come up with a way better answer than what I actually do.

So, um, you know, I think the, the difference is gonna be the business context. Um, and when I say that, I think with both people who are in marketing, running marketing programs and tech teams, 'cause I think. Where on the tech side you have tools like Cloud Code, cloud cowork, you know, we have a Genie Code product now that helps us with our internal, you know, dashboarding and stuff. But like, I think this, um, the idea of like these skills like data engineering are getting more and more commoditized with these agents.

And so like this idea of like building a good prompt and actually having the business context to build a prototype or a product that's gonna serve the needs of the business is actually, is not. Uh, is actually hard. Um, and so I think the idea of, of hiring people who have context, the business context and can really explain what they did, but why, what is the problem they were solving and really going deep on not just like the technical solution or the product that did we built, but really like what was the business problem we were solving?

What was the goal? We started out, what did we find along the way? How did we change what we wanted to build out? Like all of those questions I think are becoming more and more important.

And on the, the, you know, flip side for marketers too, when you're building a, a campaign or a program, it's not just like, Hey, we need more pipeline. Like, that's kind of like the given now that's what everyone does. It's like, how did we, you know. Like desire that, what kind of pipeline do we want?

Who do we wanna target? How did we wanna change what we wanted to do in order to better fit these audiences? So I think that's like kind of that business context element is, is gonna become more and more important and that'll translate into skills like prompt engineering, all these things. But like I think that that idea of being able to just take a a problem.

And really like break it down into its fundamental parts, re knowing deeply and reflecting like really clearly on the, your business individually, not just like a generic chat GPT thing will be more and more critical as we move forward. Phil: When, when I ask you like, uh, a recent candidate that you've interviewed when you were chatting with AI about them, um, what's one example of someone that like came up and really surprised you? Like you interview people all the time. You, you said like you even got tired of asking people, like, how do you use ai?

What's one candidate's answer when you're chatting about the AI that made you go like. Wow. Like you still remember that person's answer. Um, and, and maybe we can use that as advice for, for people, uh, on the job hunt right now.

Liz: Yeah. Well, I mean, I think the, the first thing actually I, I will say is like it's been very hard to find people via applications anymore. I think almost everyone has cracked the nut on Take job description plus my resume, ask chat GP to customize it, and then you load that version of your resume and cover letter into an application platform. So I think if you're on the hunt right now, like having a perfectly tailored resume or JD is actually, it's very hard to stand out even if it's perfect, because even if yours is real, you're going like up against people who maybe have taken some creative liberties.

But if you're a hiring manager, like some like myself or someone on our team, you look in greenhouse, they all look the same. It's so hard to separate the week from the shaft. And so I actually have like the negative answer. Is actually like, we'll interview people we're like, wow, this, I mean, they could not be better on paper.

Like look at their resume, look at their experience, look at the companies they work for. Like so great. And then you talk to them and you're like asking these questions about business context or growth mindset and they're just like stumble. You're just like, okay, so clearly use chat GBT to do this.

Um, so I think the way I think about like, you know, the advice for people who are looking for their job is, I think it's the warm intro. And building your network, leaning on people, you know, asking for introductions, asking for referrals is really the best way to get a job these days. 'cause I think it's actually very hard to stand out. And a lot of people are applying to the same jobs.

And so I think this, I think where AI can even actually more effectively help you is help you with warm intros, reaching out to people saying, Hey, I think this is the hiring manager. Would you mind introducing me? Here's why I think I'd be a good fit. Like using AI to craft these like kind of hand, you know, almost more hand raised outbound ways to kinda get your foot in the door is the best use of ai.

And then also I am, I'm a sucker. I'm like. You know, old too, but I always doing a thank you note, even if you do it through ai, like being thoughtful, like little things like that. I think where AI can really help in hiring.

Um, I think unfortunately like the tailoring your resume for, you know, the job is table stakes at this point Phil: Very cool. I really appreciate the, the advice there, What Gives Liz Energy Outside the Office - Phil: Liz. Um, I got one last question for you. You're obviously an A VP, you're the team leader.

You're a speaker, but you're also a mom of two. You're also a dog mom and a self-admitted Facebook marketplace deal hunter. One question we ask everyone on the show is how do you decide what deserves your energy at any given moment, and what's your personal system for staying aligned with what actually makes you happy? Liz: Yeah, it's a great question.

Um, and honestly, I, I, I come back to, you know, obviously I have, I have two kids who keep me on my toes, um, who are very young. So two and a half and five. Um, and our dog, who is our first child and is getting on in ears. Uh, but Phil: Our first child.

Yeah. Liz: yeah, so she, yeah, that makes me a little sad, but, uh, she's wonderful. But I think, you know, I think about what brings me joy and energy is, is I, I realize that it's at my computer all day. It's doing things in my hands.

It's either being with my kids, it's being outside, and it's gardening, it's doing things. But as I, as I mentioned to you, it's like I think the intersection of. The my job, which is technology and, and the intersection of my, my passion, which is getting great deals and trying not to buy anything full price, um, are where I think the future's going and for myself. And so I think like what brings me joy is really the intersection of those two things.

So, uh, you know, my husband will make fun of me. I'm like, vibe Cody, 10 30 at night. But I'm like, I bet I can build a really great agent to somehow figure out my Craigslist postings offer up. Facebook marketplace for these things that I need or for my kids need a new coat or like clothes and looking at Poshmark and Merc and like pulling all these things together so I can get a curated list.

Because if anyone buys things secondhand, they know timing is everything when it comes to how quickly you can respond. Very similar to an SDR follow up from a marketing campaign. So I think that's the kind of thing is like, you know, the, that the intersection of the things you love and bringing them together. Uh, I think that's where I kind of get re-energized and I, I get a lot of joy and then I slap my kids in the car with me and drive 45 minutes to go pick up a bike.

And my husband's like, can we not? Phil: But we Liz: But I got a good deal. Phil: bucks. Liz: I got a good deal.

And the only, the only secret to the success is your time has to be literally not valuable to you. Phil: I love it. That's great advice. Liz, uh, thank you so much for your time today.

Um, really appreciate you going really deep on, on some of those agents there. Um, we'll, we'll share out the link to, uh, the report that you did with, with Scott. Um, folks have, you know, probably read it already. Um, lots of Brinker fans, you're uh, in the audience, but really appreciate your time Liz.

This was super fun. Thank you so much for joining us. Liz: Awesome. Thank you, Phil.

Have a good day.