The Future of Content Intelligence With Adology | James Donner

James Donner: What happened about a year ago was that LLMs became better than humans, or as good as people at watching videos, watching imagery, and seeing what's in them, describing the attributes, but also the context. So really rich descriptions, not just labels. Because, uh, computer vision in advertising prior had been heavily based around labels. And so seeing what that could do and kind of just realizing in advertising, all of the data is locked up in imagery. I mean, there's tons of media data. Myself, my teams would analyze, you know that till the sun came up, but there's so much more data in creative.

Paul Kovalski: James, welcome to the show.

James Donner: Thanks for having me, Paul.

Paul Kovalski: I'm excited to, uh, chat all things creative and AI with you today. I was hoping we could kick this conversation off just by giving some context into the Adology founding story. I think it's pretty fascinating how quickly so many startups have popped up in this space. And I know that this is a recent thing for you, so can you kind of take me back to the point of, you know, you identifying the need in the market to build something like this, your kind of motivation behind doing so and how it all got started?

James Donner: Yeah, yeah, for sure. So, yeah, I am not a tech person by trade, but somehow got the idea in my head to start a tech company, which probably isn't so uncommon. So prior to this, I was helping to run an agency called Decoded Advertising. I was one of the founding partners in charge of media optimization, media buying. So the teams that would spend, you know, hundreds of millions of dollars a year on marketing for clients like Estee Lauder, Visa, uh, QuickBooks, brands like those. So, yeah, the genesis of the business was kind of like twofold. One was what we were doing at Decoded around creative analytics and seeing how powerful it was. And then what I was seeing in LLMs and what they were able to accomplish. And so while everyone's been very focused on using multimodal LLMs for creating content, automating content, first blogs and emails and then copywriting, and then imagery and now videos. I've always been very fascinated with the computer vision side of LLMs and the fact they can see. And so we were always kind of looking at, you know, what, when can we automate some creative analytics? And, you know, Google had their Google Vision API where it could be like, there's a red car and there's a stop sign, there's a bicycle. It's all very literal. What happened about a year ago was that LLMs became better than humans or as good as people at watching videos and watching imagery and Seeing what's in them, describing the attributes, but also the context. So really rich descriptions, not just labels. Uh, because computer vision in advertising prior had been heavily based around labels. And so seeing what that could do and kind of just realizing in advertising all of the data is locked up in imagery. I mean there's tons of media data. Myself, my teams would analyze that until the sun came up. But there's so much more data in creative. And so we used to create more data, we used to add labels to content manually. So we'd have teams of people labeling every single ad that a client ran. And we'd produce dashboards that had these deeper creative analytics. But even then, even when it was only a few minutes of people's time amongst a creative process that still took tens of hours per asset, if not more people didn't want to do it. It was inconsistent, all these issues. And so when LLMs about a year ago were able to surpass what people could do faster at a higher scale, that was the moment. Like, oh, there's millions of pieces of content out there that can now be analyzed and described and you can build these huge data sets of what everyone is marketing, what everyone is saying, what consumers are doing. So as an analyst by trade, that was like the moment of, okay, I'm going to build a business that automate some of our best processes that we used to do at our agency and one that unlocks this whole new world of data that prior wasn't really possible.

Paul Kovalski: It's really cool that you're taking a little bit of a different approach to what a lot of other folks are concentrating on, which is the creative production side. And you want to more operate as a creative analyst, a, uh, copilot, as you state. Why is that? Do you think that there's more kind of green space there?

James Donner: Yeah, I mean there's more green space. It's also, it's unattended and it's kind of necessary, I think, you know, making things automatically is cool, but it's also not perfect yet. People still aren't really running that many AI ads. Like the AI avatar space is pretty, is pretty hot if you want to run influencer ads. Um, but it still requires a lot of human input on top of it. But yeah, it's mostly, I think I just saw that people aren't really helping on the strategy side yet. There's a, a lot of competitive intelligence companies for media which will report on your competitors ad spend data, their web traffic. The ad spend data stuff is so incredibly inaccurate, by the way. I mean I used to audit that in my last agency. We would pull what they thought all our clients ad spend was and it would be off by 10%, 500% up, 10,000% down. So uh, I didn't put a lot of stake in that. But competitive intelligence was always really robust for media and didn't really exist for creative. You maybe your agency would put together a little bit of a swipe file and you'd look at it. So it was just seeing the need, you know, the gap in the marketplace for that information, seeing knowing that creative is the most important thing going forward. You know, media, you kind of have to get your settings right, but once your settings are right, you're kind of in a good place in terms of like tactical media buying. But creative is just an endless game of optimizing, optimizing. And so it's pretty clear the intelligence there is, I think, the key lever in the future knowing, you know, where the white space is, what the consumer pain points are, what the ad trends are. To borrow, when we were at the agency, we weren't good at creative performance for a while until we did this project where we started aggregating what was working in market. And so we started aggregating everyone started sharing into a Facebook group all the ads that they saw with all the likes, comments and shares. We started breaking down the tactics in them. And this was back in 2017, 2018. And when we started breaking down those trending high performing tactics and using them in our ads, it wasn't guesswork anymore. We were adopting high performing trends and our performance, marketing performance creative just started getting really, really good. And that was another like formative moment like eight years ago, uh, for this business was, wow, the power of learning from other people is actually greater than even your ability to test and learn yourself. There's so little creative data in your own ad account. There's so much when you start looking at the entire marketplace, sure.

Paul Kovalski: I mean I went through the exercise several times of auditing competitor ad libraries, competitor creative. And a lot of times it's more of affirming process for me of, okay, we're doing a lot of the same things and this is working. One of the unique things about creative, and specifically direct response and ad creative is there are some fundamental components of what makes it work. A strong hook that captures attention, a good explainer, a uh, good kind of call to action. There's elements that I think you can find in good performing. Common, common elements. Right. But then what really differentiates creative, there's a Lot of kind of constant change in what's hot and trending that you need to apply on top of those foundational elements, meaning that you know, if something's happening in the news, for example, the Coldplay concert, CEO, founder, whatever, being able to quickly like uh, adopt your framework for what works for your brand and the elements of a strong creative on top of that viral moment is like a recipe for a really good ad. And so that's kind of what I find interesting here is like there's the foundational element, sure. And you can put that data and feed that into assist to produce good ads, but you also need the market data to kind of overlay on top of that.

James Donner: Yeah, yeah, there's like the fast moving data and then the uh, not so fast and then the slow moving data. All of which are important. You know, the data in your account is almost a little bit more slow moving because you're slowly building those learnings over time. But yeah, when it gets to organic content on TikTok, it's going to have to be based on super recent activity. And if you want to automate content and you want to actually automate the strategy behind content, you need data sets, data sources, knowledge sets that actually contain what's going viral, what's trending right now for news that's a bit easier to get for content advertising trends is a lot harder. That's where we like to help help folks. But to your earlier question about, you know, why focusing on the, on the analytics or insight side is because I think yeah, if you want to automate the whole process of creative, uh, which isn't the end goal but is a sort of intermediary goal of automate big parts of it, save time, be able to scale it, leave the people to do the strategy and the decision making. Uh, you need automation for the creation but you also need automation for helping drive the creation. And especially if your aspirations are to do some seriously scaled AI driven work, you need a brain that's going to instruct the production company. Otherwise it's kind of like you're hiring just the production company, the AI production company, but not the AI, you know, strategists.

Paul Kovalski: Let's, let's talk about the adology brain. So I'm going to kind of walk through my notes on my understanding of your technical process and then I would love your kind of like commentary on maybe some of the interesting things you're doing, what you might want to share. So essentially, and please correct me if I'm um, wrong but you are ingesting ads at scale by, you know, scraping websites, looking at public ads, brand ad libraries, running multimodal AI, tagging on visual, visual, audio, text, building a data set and then training trend detection models and then feeding that into a proactable copilot. Is that a generally accurate description?

James Donner: Yes. When we build our data sets, it's a lot more than just train the trend detection. It's really building up knowledge bases for AI to then be able to either mine for insights such as trends or leverage for automations. So if you want to automate a new post every day that's based on recent news, plus using the top performing hooks in your category, plus using the best performing ad concepts from M, your historical ad performance, uh, you can do that with our knowledge bases because it gives AI intelligence across all three of those sources.

Paul Kovalski: Do you see more value in the kind of like automated organic content side or the maybe slower paid creative production?

James Donner: I think they all need support. I think what all of this needs. Yeah, you know, we're, we're essentially a knowledge based company. We believe we're building knowledge bases for kind of the future of, of marketing or AI driven marketing. And so whether you're doing rapid social content, you need that knowledge base, otherwise you're having to prompt every single thing yourself that you want made. Or if you're doing ad content, you may use it a little bit less frequently, but you still need it if you want to quickly know where there's white space or quickly know what's trending in your category. In uh, terms of content approaches, what

Paul Kovalski: do you think are some of the more non obvious areas of white space that a common marketer might not detect or maybe not identify that adology can help solve?

James Donner: Um, I think there's, I mean there's so many. So when you, when you look at ad libraries for trends and you're doing it yourself, you know, you look at one brand and it's like, okay, you can kind of see what they're doing. You look at another brand, you kind of see what they're doing, but now you're only taking in 70% of it. Then you're on brand three and you're down to like 40% of what's going on and you've forgotten 50% of the first brand. So that the, I think for people to really understand the scope of what everyone's saying and doing, it's just too much. It's okay when you're trying to pick up a few trends, but that's where AI really shines. There are so many weaknesses to AI, by the way, as a, as an AI company. To me it's number one strength is its ability to summarize and synthesize and to see patterns. But it really is lacking in knowledge in a lot of ways, which might sound crazy to people. So. Yeah, I think, I can't remember. I actually just lost my original train of thought. But. Yeah, what was it? Sorry, I forgot to get the original where I started with the question.

Paul Kovalski: Yeah, I think what, what I'm kind of interested about is identifying white space. Yeah, but, but also like, we can, we can leverage AI to track competitors at a much more efficient and automated rate. And then we can see, okay, this competitor is tending to use this type of concept or this type of value prop, and we have the information at our fingertips. And then the question is, how do you respond to that too? Right. Like in my case, working for a credit building company, it's helpful for us to know that competitor A is kind of going at a 45 point credit score increase and that's where they're pushing. So we know we have to kind of respond to that. I'm kind of wondering your thoughts there too. Like, how are we using a company like Adology to assess not only what our competitors are doing, but how we can respond to it?

James Donner: Yeah, I mean, some people want to copy more, other people want to, you know, go towards the white space, I think. And uh, we don't necessarily have an opinion on which one is better or worse. I can say when we, things started getting more powerful, when we started incorporating a lot more user sentiment and social listening data. So product reviews, questions on Quora, even Twitter comments, Reddit comments, and all of that. And I think the more sources you start to ingest, ingest and put your knowledge base, the more likely you are to catch the things or have AI catch the things that you may have missed, which is consumers are complaining about this, and it's what they're complaining about the most. It's what they have the highest passion and sentiment about, because we're extracting all the sentiment and the depth of how much, uh, they care about these things and then they're complaining about this. Here's where everyone is addressing these pain points or not, across their retail sites, across their ads. Here's the gaps, here's what's left, here's the unattended things for you to go after. Ah. And so we're a relatively new business. You know, we're watching our clients start to take advantage of these insights and build campaigns around them. And we're just starting now to even collect some of that data of how much, you know, a campaign built around these insights helps. But I think, you know, there's a lot of little things that will fall through the cracks as far as things you realize you didn't, you never realized your business wasn't addressing these. And either consumers are talking about them or your competitors are or areas where you realize your competitors are leaving yet kind of empty space. But ton of applications almost, you know, can have almost nothing to do with the competition and can really just be about listening to consumers and then looking to make your content more effective by using better hooks or using better, you know, techniques.

Paul Kovalski: Is there anything in the, in the building of your product that prioritizes certain data sets over others or values pieces of data over others? And I want, the thing that I'm kind of thinking about is like we have, you can, you can kind of ingest the visuals, text, audio of a competitor ad and ad sets. There's a social listing, which you mentioned. There's reviews, there's, you know, answers on Reddit, there's all these different data sets like how do we figure out how to orchestrate all of that and then what is more valuable and what's not?

James Donner: Yep. So there's kind of two ways to have AI produce higher quality predictions or recommendations. So right now, if you just go to ChatGPT and you ask it, you know, what ads should I make? It'll give you a bunch of random ideas. Those are not performance informed. They're probably not grounded in your category, all of that. If you feed in our knowledge base without any of the enrichments that we do, it will now know what everyone's doing and it'll be able to give you recommendations that are grounded in the category and basically at least, you know, relevant to reality and what people are actually doing in their ads today, then the level three is okay. Now we want to actually have it produce high quality recommendations. And so there's two ways that we do that. One is feeding in the right input data. So, okay, we're not going to give it every ad that people are running. We're only going to give them people's most effective ads or our most effective ads historically. We're not going to give it every consumer comment. We're going to give it the ones that are the most frequently mentioned and have the highest sentiment and, you know, depth of feeling around them. And so giving better inputs to get better outputs. And then the second thing is on the outputs Having those outputs then re ranked based on a more probabilistic model that will actually look at all the recommendations, analyze them against all the data that it has, and then actually rank them, uh, on which ones we think are most likely to be effective for your business. So that's actually where we're at right now in the company in terms of building that. So right now our product will actually have all these signals on what's working and we'll feed in the higher performing creatives and insights and let it then ideate and produce concepts based on that. So you just talk to our AI agent and you say, I want new ideas based on my competitors top performers or based on the top trends. Then the question is, how do you actually identify what's top performers? In some places it's really easy. What has a lot of likes, what has a lot of shares, on the organic content, on the paid content. We have our own methodologies. We're essentially reverse engineering what marketers are doing. Uh, a marketer who's running their Facebook campaigns, they're going to do a few things. One, they're going to pause the ads that don't work very well and they're going to keep running the ads that do work well. How long an ad is running is a signal of success. The second thing they're going to do is they're going to make more iterations of ads that work well and they're not going to make iterations ads that don't work. And so we watch that as well and that's a huge signal for us. So we're watching all of these signals and then we're enriching essentially the knowledge base so that AI that's accessing our knowledge base is able to go and select high performing ads or organic posts, etc.

Paul Kovalski: That's really great. I think. You know, I struggle with this too. My philosophy on kind of like creative testing has shifted and changed over the years because sometimes I think to myself, I don't just want to continue to create a variation of the same winning ad, or I don't just want to say exactly what my competitors are saying in a different way because it's too vanilla, uh, and it's not different enough and I want to be new and unique and sexy and whatever. But then it's like, well, it's working, so who cares, right? If you have a winning ad that's been spending a lot for the last year, who cares? You don't have to pause it, keep it going, Create a simple variation where do you kind of stand on that?

James Donner: I mean, I agree. I remember we'd have clients that were running ads for like two or three years and just couldn't beat it. I think a lot of this relates to performance marketing, the flaws of performance marketing. So, yeah, I love performance marketing. I'm, um, a performance marketer. I now also love brand marketing and believe in the. They're just, they're totally different things. Uh, I subscribe to most of Byron Sharp's theories. A lot of performance marketing I now view as a form of distribution, or what Byron Sharp calls physical availability, which is it's making the product easy to buy. So Byron Sharp's framework is mental availability. The product is easy to think of in purchasing moments. Physical availability, it's easy to buy. There's a concept now of digital physical availability, which means it's just easy to buy via, uh, online. So I Google shopping ads, Facebook newsfeed ads, I see those no different as walking down the aisle in Walmart. If you search for something in Google, you're like in whatever, you know, you know, the pharmacy aisles in Walmart or the outdoor aisle in Walmart. In Meta, if you start engaging on something, it basically put you in that aisle. If you start looking at mattresses, you're starting to see all these mattress ads. If you start looking at jet skis, you see all these jet ski ads. So to me, it's the equivalent of being on shelf, and you're paying to be on shelf. And when you're optimizing performance marketing ads, you're largely optimizing for attention. And this might be a little bit controversial, but only 20 to 30% of people, only 20, 30% of ads in the newsfeed are ever actually looked at. So immediately, 70, 70% of people are not looking at the ads. When you do something, you have a shocking hook, a pattern interruption hook, et cetera. It's capturing more attention. And so I think so many of the things that we think are, oh, this is working because it's a more effective message for my business. Maybe not. It may just be working because it's taking it from 20% of people looking at your ad to 40, and now all of a sudden your ad's performing twice as well. And so through that framework, I distinguish between creative that drives physical, that is effective at driving that physical availability, making it stand out in Google shopping, making it stand out in Facebook newsfeed, making it stand out in brand search. These are all just forms of retail distribution versus ads that are good at driving mental availability. Which are very different and basically impossible to measure digitally. And so I think is it okay to you know, run the same ad for years? If the ad is just a sign saying hey, you know, we're cast or we exist, come buy this mattress, then I think it's fine. If it's truly, you know, intended to drive mental associations and shift the how people associate your brand, um, then maybe it's a, it's a bigger issue or

Paul Kovalski: problem as that relates to kind of what you're building. You obviously are being able to analyze a number of different ads within competitor ad libraries. Do you think about kind of sharing or reporting on the kind of brand versus performance mix or developing different levels of analysis or reporting for kind of direct response performance ads versus more branded creative?

James Donner: Yeah.

Paul Kovalski: So

James Donner: what we're aiming for is as a knowledge base building and management company. So for us that means knowledge of your competitors brand ads and performance ads both. And it means that you can flexibly query about how they're doing different strategies. There may not be a clean line between the two, but you can ask your knowledge base via, uh, ChatGPT, via Claude, whatever, what are their bottom funnel sales driven strategies? What are their associative strategies? How are their bottom funnel ads driving mental associations and brand associations? How are their upper funnels seeming ads mentioning sales if at all? So for us we're focusing on being a more multipurpose platform, almost essentially a data platform or what we're calling like the Bloomberg terminal of marketing knowledge. So that people can either data mine it for these insights or they can build automations on these data sets. Uh, and so automations if your competitor drops their pricing automations if there is a news event you want to react to or just automating insights that go to your boss, uh, you can automate email reports and things like that. So that's a little bit of a dodge to say that we don't, you know, we're not pushing very defined workflows through our platform. We're building the platform that people can then customize their workflows on top of and so they can do that. And uh, this is a big bet of ours is that people are going to want to orchestrate their own AI workflows. Many people want to buy an all in one AI tool like Icon Me or you know, maybe if Motion heads there. But I think even more people, agencies especially are going to want to build their own workflows and pipelines because it's their own special sauce. I think anyone who's worked with These AI workflow based solutions in marketing or out knows that they can be limited and then you want to do other things. It's like why do we all still use Excel, you know instead of QuickBooks for certain things or people will always want to customize and diy. So our strategy is going to build the data platform that they can build on using Claude to build automations on it, using N8N having their developers build automations on et cetera as well. So we're a little bit out of that like workflow game on our end.

Paul Kovalski: Understood. I want to go back to something that you mentioned before which I thought was really interesting on the performance side. Ads are measurable in that you can measure the response, the click through rate, you can measure view through conversions. You can see deterministically what ads are resulting in conversions, whether they are causal or not. Um, and so it's very easy to say and even looking at like media mix modeling to say these ads drove this immediate result. So that's more on the direct response end. But then there's incrementality question for more of the changing mindset, changing perception of a brand that is much harder to measure but is still super important. An ad that, seeing an ad for Coke that makes me think nostalgia, that changes my perception and uh, emotion in that moment. But I don't visit the website right away so it doesn't show up in last click and it doesn't show up in mmm because it didn't result in revenue. How do you measure that? What do you think about that? It's a big problem.

James Donner: One of the benefits I think of AI automating everything is it's going to automate the performance world. Like performance marketing will probably be solved which will be really nice for everyone who wants to discuss other things like strategy and brand because once that's solved for we'll all stop trying to chase it so hard and we will spend more time on strategy and brand. Those things I don't think are measurable, not in an easy way, not in a big data way, not in a way that AI is there to solve. We want to help solve so that you are as intelligent as possible for that strategic planning. So you know what nostalgia, emotions everyone around you is tapping into and where there may be gaps or what consumers are talking about so you can address them. So our product, it can be used for performance marketing. But I do expect performance marketing to be solved and not necessarily by us just showing the trends. It'll be by Google and Meta just running their Black box performance Max, ASC plus. And they won't just change out the colors and the products, they'll start changing everything about the ad, which is great. It'll leave everything that, that those don't solve for which is so much. So my two favorite ways of measuring those things, there's brand left surveys that you can, you know, show people ads and collect 300 survey responses. And she would see which creative shifted the needle on brand perceptions. We used to do those early agency, you can do them for like 30 to $40,000 per creative now which for small brands is like okay, that's insane. But for larger brands is possible. So that's a good way. And then I think, you know, neuro testing is going to be a big thing in the future because when, when we're no longer able to optimize performance creative, people will still want to optimize stuff and they will want to optimize brands. I think they'll become more focused on how do we measure those mental associations and tweak and optimize things for that.

Paul Kovalski: Elaborate on the neuro testing piece.

James Donner: Uh, literally like measuring people's brainwaves and their emotional responses to content. I mean there's the facial recognition companies that will watch as you watch an ad and will try to infer your emotional response to it based on your face. There's a lot of questioning of how effective those are. But then there's the neuroscience stuff which will actually measure the brain waves. And that's not great either because those people are not in real settings. Uh, they're not at home watching the ad in that sense. So there's flaws in, in everything. And then you can do run real world experiments. You know the other issue is that people are unwilling to run experiments for as long as they need to to measure them. So if you really want to measure, okay, is nostalgia going to drive more coke sales? Uh, what I would do is set up a match market and in a small region of the U.S. pump up our nostalgia advertising and run sales lift campaigns. And so those are kind of the methods that we be at people's fingertips. But I think there'll still be a lot that you just can't big data your way out of and good old fashioned human strategy. Hopefully you know, informed by AI research will be the, will be the solution.

Paul Kovalski: Yeah, and I think there's an element of this where marketers have conviction about a strategy or a philosophy and they need to deploy that and then have trust and faith that it will work out. And maybe some of it is not measurable. I mean even for example, I'm in the process of structuring a retargeting experiment right now where we are looking at email submits, one day email submits and looking to drive incrementality there through paid retargeting and the like. Common best practice knowledge would be to optimize those campaigns to a purchase or an action. But because of the small audience size and the small amount of event volume, I know that running those purchase optimized campaigns in retargeting are less incremental and it's actually not about the tracked conversions, but the incremental conversions are the ones that aren't necessarily like going to be last click. So I might run a retargeting campaign optimized towards reach or impressions just to flood that one day email submit audience and that might be more incremental.

James Donner: Yeah, I mean I'll, I'll nerd out on this with you for, for a few minutes. So I, I mean there's, I don't. Well, I can say two contradictory things. One, there would be no reason to run purchase retargeting or uh, purchase optimization if the audience is already so qualified because the whole point of the optimization algorithm is to uh, qualify the audience. So you've already done that. Those people are already in your retargeting pool because they've been qualified and probably reached on a retargeting or on a purchase optimized campaign anyway. So they're qualified. So bid reach, the reasons and not bid reach is, you know, from all my testing understanding, it's all an economic marketplace and peak, different people command different prices. So if you're buying a lot of stuff through Instagram and Meta is going to know that and you're going to Command very high CPMs or revenues on you. So you buy a lot, you're only going to get hit by purchase targeting ads and people are going to pay a lot to reach you. And if you bid for reach, you're telling Meta, I want to maximize my reach for every dollar. So Meta's trying to minimize your cpm. So at that point meta isn't going to bid on someone who's super expensive. They're only going to bid on the cheap people. And when we run reach campaigns and purchase campaigns against the same audience, we see like 5% overlap if that. And my understanding has always been that it's because the purchasers command higher prices and you'll never reach the purchasers when you're running reach campaigns. And so that's actually, I used to think of reach campaigns as offline purchaser campaigns. They're targeting people who either don't have a history of buying online or aren't buying a lot because they're not, you know, necessarily affluent or, you know, in that category. So that's how we would think of like the, the purchase targeting, purchase optimization as for online buyers, and then reach optimization as for offline buyers. And we often would see that a reach optimization worked better in the, you know, offline list studies that we used to run with like Oracle and Datalogix back in the day.

Paul Kovalski: Yeah, yeah. You get what you bid for, essentially.

James Donner: Yeah, exactly. Yeah, right, exactly. Yeah, 100%. If you bid for clicks, you get those.

Paul Kovalski: I want to talk a little bit about where AI breaks down because, you know, I, and many marketers are thinking about a future where everything will be automated. But I'm going through a process now where I'm listing out all the tasks that our broader marketing team does on a recurring basis. Everything from financial forecasting to, uh, OKR development to concepting, ad launching, organic posting, you know, email audience creation, et cetera, et cetera. And then going through this whole prioritization framework of like, where can we automate and, and where can we be efficient? One of the scoring criteria is like, the importance of getting that data right. I can't think of the exact term, but it's the idea that you wouldn't want to automate something that if gotten wrong, could really screw something up. And so, you know, we're having, uh, it's fun to think about. I won't have to do anything. I will automate all of these processes. It will just know the campaign to create, create the campaign, launch the ads, optimize the ads, whatever. But there's, there's components where it breaks down. So from your perspective, where, where does that actually happen?

James Donner: Yeah, AI has so many weaknesses. So I'm excited to discuss this. The best analogy I heard for AI is that they're like compression engines that they compress the entire world down into a small model that when you're putting information in, uh, it's essentially uncompressing it to give you the answer. So the way I like to think about it is essentially it has the whole world boiled down into a series of patterns. And, and I ask it, you know, what is, you know, what are some ideas for dinner tonight? And so it has all these patterns about food and recipes, et cetera. And so then it can take my query and understand it and then expand out from that and produce the final recommendations. But what kind of happens a lot is that it mixes and matches without you even kind of knowing, and it'll freely just mix and match different patterns. And that's why, like, they've done tests and Moby Dick. The most an AI can actually reproduce of a piece of text is 41% of Moby Dick. So it doesn't. While we think AI knows everything, it actually could never play back for you what's in a book or what's in a specific thing perfectly, because it knows the patterns that predict Moby Dick, and it knows the patterns that predict this book or this video, but it doesn't actually know the entire thing in detail. So those have been kind of like informative analogies and frameworks for me to kind of be grounded in the limitations of AI. Uh, I think the LLMs themselves, they're these thinking machines. I think of them as thinking machines even more than, like, knowledge. And I'm super biased because we're building a knowledge company, but it's this idea that they, they know how to think, but if you want them to actually react to facts, it can't be stuff coming out of their own training data. It has to be facts that are in their context windows. And that's why context engineering is the new prompt engineering. And any good AI company largely is all about the way it's managing the context window. Even ChatGPT is like the. When ChatGPT has a conversation with you, that's a lot of human ingenuity and design that makes it feel like a conversation. Every call to OpenAI's LLM is completely independent of all the others. It's stateless, so it has no idea between one query and another query. The reason that ChatGPT actually retains what's going on is because after you send the first message and you send your second message, OpenAI, the company, has decided to take all of the information from your first message and paste it in ahead of your second. And then it responds, and on your third message, it's taking all the information from the first three and pasting it ahead of your query. So every time it's getting a new query, it doesn't know what the hell went on before. But OpenAI has decided to feed it everything from before and now. It can have what feels like an intelligent conversation. And then when it starts to get really wacky, that's because your conversation has gone on so long that they're now summarizing the middle of the conversation. They're taking the first message they're summarizing the middle, they're taking the last one, they're sending it through. And so when you're using chats and you've probably the experience you have to restart the chat. It's because it's that memory system when you're using an AI that does automated email outreach. Again, it's how people are informing that context window. So AI really is a tool. Uh, its limitations are that it doesn't know Moby Dick word for word, and all the implications of that. Like it, it struggles with actual facts, it has the patterns of facts, which means it's likely to mix and match and recombine. It can make it very creative. But it also is what has, you know, led to hallucinations in the past when it's just like, oh, these are kind of patterns. I feel like this is. Right, I'm just going to answer that. So I think for AD to be reliable, it's, uh, okay, what is it good for? It can do certain things. Like, as a tool, it can summarize and synthesize incredibly well, which means you put information into the context window, you ask it to bring that information together, to cross check it, it'll do that incredibly well. It can watch and describe things incredibly well. It can record the attributes and the labels, all of that. It can create images. Everything else is kind of like built around it. I view as the product of human intelligence and engineering, where people have decided to string together your LLM calls. I'm going to string together memory, which sends all the old messages in. I'm going to string it together with another AI that's going to summarize the messages in the middle, another LLM call. And then we're going to string all that together with another one that produces an image based on what you're asking for. So all that to say those kind of. I talked about a bunch of different components of AI, but those all lead to my worldview of what is it good at, what is it not? You have to look at the thing you're trying to solve and say, okay, is AI's ability to summarize information going to be useful in automating this? Are we going to be able to feed it the right information so that it has the right facts that it needs to automate it? And then if you can do all that and you can engineer the system around it and you feel like that's going to work, then it's like, okay, this is a good use case for AI. We're comfortable using AI to watch all these videos because it's very good at that. We're comfortable using it to synthesize what's in our knowledge base because it's very good at that. When you start to, I mean, start, if you ask it to do math, it'll confidently do math for you and then it'll completely blow it. It'll make up statistics like with no issue. So that's kind of. Yeah, I mean, so I guess if that answers the question.

Paul Kovalski: No, that's helpful. I'm realizing that I have a, uh, meeting to hop to that's on my calendar. So I think I'm gonna, I'll ask one more kind of rapid fire hot take question and then we can, then we can hop. I guess my, my question would be, what is the best resource for marketers looking to operationalize AI tools?

James Donner: Operationalize them.

Paul Kovalski: Um.

James Donner: Oh, man. I. LinkedIn. Following smart people on LinkedIn is like everything.

Paul Kovalski: Who do you recommend following AI?

James Donner: I'll give a shout out to Jake Abrams at Odyssey Agency. There are the influencers. Motion seems to have like half the influencers on their payroll. The AI marketing influencers. Sarah Levenger does great creative strategy work. Jimmy Slagel, I think is one of the guys who's putting out AI work. There's a few others. And then for non AI, Byron Sharp, Mark Ritson. But you know, people who aren't just promoting themselves, but are truly sharing what they're doing, I would say LinkedIn is probably the number one resource. Cool.

Paul Kovalski: James, thank you so much for joining the Call today.

James Donner: Yeah, thanks for having me, Paul. See you on the other calls.