Web Scraping, AI Agents, and the Future of Open Source - Kevin Lewis (Apify)

Speaker A: Hello and welcome to another coffee with developers. Today I'm here with Kevin Lewis from apify and we just realized we met each other before as well, we're both in Berlin. So I thought like, oh, get an American dude from apify in. But no, it's not at all. How are you doing today, Kevin?

Speaker B: I'm good. Also apify is a Czech company so you know, we definitely have Americans working for us, but you more than likely wouldn't have had an American. And I sound very not American. American. But yeah, hello, thanks for having me.

Speaker A: There's a slight tint of not American. Yeah, it's hardly, hardly recognizable. So, so what do you do for apify? Like you're a developer advocate. Like what are your main focus areas?

Speaker B: Yeah, I mean really at Appify, our developer advocates are generalists. So our goal is to help more developers discover and use and be as successful as possible with the apify platform. And one thing that's I would say quite interesting is uh, just a, if I can, for 30 seconds, just to contextualize, uh, some of our later conversation. Apify, uh, has a marketplace where on one side we have creators who create what we call actors. Actors do some kind of action like data extraction or scraping from the web, acting on a platform on your behalf. Um, and on the other side we have users and then we provide the runtime, the bit in the middle. So if you're looking to scrape Instagram profiles or LinkedIn comments or e commerce product data, you would use something like apify to make that happen.

Speaker A: And you do the whole caching in between the proxying, the like. I mean, because it's getting harder and harder to get to this information because if uh, you basically go through like cloudflare turnstile every two seconds and have to click on like what is, where's the traffic light kind of thing. So uh, you've been working on that for a long time. So basically you know all the bits and bobs how to get around that.

Speaker B: Yeah, I mean you absolutely hit the nail on the head. It's not just the fact that we're this kind of serverless runtime, right, where you can run these pre built scripts from our creators. Uh, but also that there is persistent storage, um, and also that there is uh, yes, proxying available as well. Uh, these are the things that developers often come up against when they're like, oh, it's fine, I'll just scrape data myself as you start to hit these problems. Um, and either we or one of our creators have already solved that using one of the 20,000 plus actors that exist in the Apify store.

Speaker A: And it's a marketplace yourself. So if you come up with an actor, uh, then you can put it in the marketplace and apify, uh, will find users for you or find customers for you.

Speaker B: Yeah, or customers can find you and run you through the runtime. One other thing that we think is really interesting is open source creators who build really competent projects. And most times it stops at publishing it on GitHub with a run guide and installation instructions and then they run it themselves and that's it. Sometimes though, uh, you want to be able to provide an opinionated runtime and so you can hit a uh run with apify button. And if your uh, repository is set up such that you have created one of these actors, you can also get a kickback when people pay for the usage. So it's also a nice kind of sustainable open source piece as well.

Speaker A: And how do you provide that? Uh, do you provide an API endpoint? Uh, what is an actor in terms of when you get started, when you want to create something to get into the platform?

Speaker B: How would that. Yeah, that's a really good question. So I think there were two questions there. One is like, what is an actor? And the other is what are the interfaces with which you can run an actor? Firstly, what is an actor? Given that this is a uh, podcast for developers, at its core, an actor is, is a small program that runs in a Docker container. You provide a Docker file. They're typically built with JavaScript or Python. Um, they, they take a fixed input like a specified, ah, input. You can create a schema. When you create an actor to specify the input, something happens in the middle and then there is a specified output schema. And if you want to store data, a dataset schema as well, then there's the, how do you run them? So firstly there's the apify console which is a web app. You log into it, you can fill in this nice little UI form that we generate from your input schema and just run it all within the platform. Uh, and from there you can export a CSV or you can integrate with Google Drive and have data piped or fire it off and um, trigger a webhook. You can also interact with them programmatically via API, a REST API via one of our client libraries or via our MCP server. So you can enable any number of actors. Uh, and then your apify MCP server gets added into your MCP client like Claude for uh, Copilot in Vs Code for example. And now your agent has access to real time data on the web via the actors that run.

Speaker A: Which brings us to the main topic that you wanted to talk about which is agents and especially agent skills which again gives us now the next confusion. We have an actor, we have an agent, we have a promoter, we have a broker, we have like whatever we have right now. So uh, agentic, uh development and agents are the biggest thing right now. What do you see is an agent and how do you enable them in your platform?

Speaker B: Yeah, so uh, just firstly so we can do some terminology debugging so to speak. So we have the LLM, we have the large language model, the models provided by a lab. It could be anthropic, it could be, be deep seek, it could be OpenAI. Um, that is just an LLM though, you know you need to be able to interact with that LLM and this is where more opinionated agents get built. So GitHub Copilot is an agent. The Claude desktop app uh is an agent and really an agent at uh, its very thinnest is going to be an LLM call with a wrapper around it or M. Maybe a wrapper is the wrong term with a set of custom instructions and parameters for how it should act given the individual context. At its core that's an agent. We can start layering more on top of that. Typically agents will also allow for multi step, multi turn interactions, your chat where you can follow up with further information. Then there's this whole world of other things. You have the harnesses, the applications that run the agents, you have skills, you have tools, uh in apify terms you have actors. But really the word actors I don't want people to get hung up on here is you could call it a function, you could call it a uh ah script, you know we call them actors. Uh but this is not a, this is not a term that is intrinsically linked to the world of AI development.

Speaker A: And the main thing is also that agents can use tools in the background. They can uh, they can spawn up browsers to scrape things, they can spawn up services to convert things, FFMPEG to analyze videos, whisper to do audio, uh extraction, these kind of things. So the idea of like going from a, from a script that we used to have like a JavaScript somewhere on the server to an agent is like you have the LLM information power but you also have the opportunity to talk to other services and use uh tools as they need to. Now the danger there is of course that basically it will just use all Kind of malicious packages without you knowing it. So where do you, where do you see the uh, where do you see the guardrails? There. Like you, when we talked earlier before the recording, you said like you're using a lot of a, uh, lot of Chinese models yourself, but you actually want to use them locally because you're always worried about somewhere, uh, things going on the web and your data going somewhere else. As a developer that wants to get started in this world, where are the guardrails? Where are the interesting things to play with?

Speaker B: Wow, that was so much there. Um, firstly, reasons for wanting to run models locally or where do they run. This is something just worth considering. Generally you send a request to some external system, um, and it returns data to you. This isn't just the world of AI, right? This happens when we're just using REST APIs. You need to think about where you are sending data to because there's different laws that govern different uh, countries and different companies and so on. One of my major concerns though is also we are recording this right at what I believe is the start, uh, where people are going, oh, I need to start thinking about token efficiency because our free lunch is ending and things are going to start costing a lot of money and an unknown amount of money and it could be fine, I could shift over to something else. But then patterns repeat themselves. So I'm also really interested in running local models just to have a known cost to running these models and sure they might be a bit less powerful, uh, but they're mine and they sit here on my desk and I know nothing will escape unless uh, I've given it permission to. The other thing you spoke about was what I'm going to refer to as trust. Guardrails. Right. Agents can go and do a whole ton of things. How do you know that they're doing um, what you expect? And to this I would say, I would say a couple of things. Firstly, one thing that's starting to come into the conversation is mixing deterministic and non deterministic actions. So only uh, doing some actions using scripts that have defined outcomes and then instructing an LLM that is equipped with, or an agent that is equipped with only the tools that it should be equipped with for the given task to go in head and act on those fuzzy outcomes. So that's one thing. Um, there are also a whole bunch of different, uh, trust, let's call them heuristics. Right? Um, what is something going to cost me? Uh, am I, am I allowed to make that payment? Um, am I Getting the data I receive back? Or is there something interesting in the payload that looks dodgy? And do I, do I now stop interacting with this platform? What data can I send off? Do I need to redact or log or block values before they make it to the external platform? And there are tools, all of these layers that enable it. I don't think there is a turnkey solution for this. Um, one thing that is also becoming um, supported by many MCP clients at the moment is dynamic tool registration where the agent can find new tools, enable them and without restarting a new session, which historically has been what we need to do, can then go and leverage those tools. And that is both incredibly exciting and incredibly scary at the same time. Right, because your agent can now equip itself with new tools and especially tools

Speaker A: that you might not know about and uh, that have vulnerabilities that basically you look at the things that you want to have. And especially as a developer, you like your tools, basically you like your editor, you like your environments. And nowadays we just have things on the fly and uh, being discarded and being introduced all the time. So uh, one of the things that you talked about in the preamble to this recording as well is that uh, and you already hinted on it right now with the payment problem like uh, sometimes you just go out there and like agents go out and scrape everything. And I, as a content provider or as a blog provider, I'm confused just how much of my data goes somewhere and I don't know where it goes, I don't know if they pay for it. And a lot of like newspapers especially have a really big issue with like agents and agent the web coming and just getting their data without going through or trying to work around their paywalls. Now there's a HTTP uh protocol 402 which was hilarious because it was just like a stop for so long. It basically said like payment required. And then uh, Brian Whipple from Algorand gave a talk at our Congress last year about it, about how they want to take that one and actually make it a thing that agents and uh, systems can pay with. So how can we think about that right now? Uh, how is uh, apify implementing that and enabling agents to pay safely and securely without actually breaking the bank.

Speaker B: Yeah, sure. So there's two things there. One is, I'll come to apify later but let's talk about uh, this status quo code in the HTTP spec, HTTP 402 payment required. And as you said it was a stub. It was just, it was Reserved, I think, is the formal word in the spec without an implementation until the last year or so really. So there are a couple of different implementations of it. Uh, there was one before, I forgot the name of it, but it was rooted in using, ah, Bitcoin. Um, it didn't really have wide adoption. The implementation of the spec that you can use today is called X402. Um, it was created by Coinbase. It was an open spec built by Coinbase. The idea is that when you implement HTTP 402 in your application and use it as, let's say, middleware in your route handlers, the browser can respond to requests, the server can respond to requests with this status code and a set of metadata, including, how much will this cost, who are you paying, what's the timeout and so on. Um, then your agent or you as a developer. Actually, let's forget AI for a moment. This is just HTTP, you can, you can do it yourself in, you know, Postman or similar, you sign, um, a declaration, uh, cryptographically signed declaration to say yes, you can take this money that gets sent back in a retrade response. Then something interesting happens because then there is this third party. Normally an interaction happens between you and a server, but in this world there's a third party called a believe it's called a facilitator. And Coinbase, it's a service that um, Coinbase host as well. And what they will do is validate, you know, validate that permission, uh, slip, let's call it that authorization. It will move stablecoin, which we'll talk about in a moment, from one wallet to another, verify that that has happened and then alert the server that it has happened. So neither the server nor the client touched this blockchain, but it is the irrefutable proof that money has moved from A to B. Coinbase handle that and then they respond with the paid asset. And I think this is fascinating because right now there's a whole bunch happening, right? Agents have issues accessing stuff. Uh, agents also access stuff which it shouldn't, you know, it shouldn't rightfully take and use and respond because of copyright, for example. Um, and I see this as being like agents being able to perform microtransactions. Um, we could talk more about Stable coins if you're interested. Uh, because, you know, I am not a Web3 fan at all and this immediately go, you know, put the heckles on my neck up, it wasn't really my thing. Um, but a stable coin is just a dollar represented in a digital form for every stable coin there is a dollar held in the real world. They're always one to one and it just allows us uh, to move money around using the Internet without using the archaic pipes of the 1970s. Um, but yeah, that's basically the mechanism and how it works. I could talk a bit about how we implement it in a moment, but did that make sense? Did I?

Speaker A: Yeah, that's a good point coming uh, from it. And I love that it's an HTTP uh, uh, um, implementation rather than just like having to use a service and having to use a broker in between. So you can just set the header and just get, and get the payment in the background done that way as you said, like you don't have to go through a blockchain web3 kind of interface, but you can use it in the same interface that you're scraping the data and you're getting the data and

Speaker B: you're sending the data from and you're making sure creators are paid right or let's call them uh, services are being paid for their service. Because this doesn't have to be around data. It can be about taking an action that costs computational resource and making sure that that's being built. So the way it works in practice is you equip your agent with a wallet. This shouldn't be your life savings wallet, this should be a low balance wallet. Um, and then it can do things like understand the uh, price to value when it gets that, when it gets that http402 response it could say, you know, a weather result might be worth one cent to me, but it isn't worth five bucks to me, you know, mhm. This is where you start to put those guardrails in. So with Apify we have these actors, right? We have 20,000 plus actors in our store. They, many of them cost money to run and you can pay for them using HTTP 402. So you don't need an account, you don't need to sign up for an app FI account, don't need to put a credit card in, don't need to identify yourself as an individual, your agent or you as an individual. Of course using just HTTP requests can go and pay for the run of an actor. And remember it's a few, it's another round trip. You make a request, you get a 402, sign the authorization, you send that in a retried request and then once it's all settled by the facilitator, you get data returned. And um, you get that for free. You don't have to do any additional work there. Apify natively supports X402. So if you're a creator and you publish an actor, you also support HTTP402 which means all of these AI agents now have access to 20,000 more skills effectively that they can run via the Apify MCP server, which I think is really cool. There are other specs as well. Just as an aside, you know, forget ah, appify for a moment. There's one called the Merchant Payment Protocol mpp. This one's being run by um, Stripe and I want to say Tempo Labs. Um, the idea here is that it's going to support more payment schemes. Uh, this is still, the spec is still being written. The integration surface will likely be bigger because it's more complex, um, but it ultimately should be more capable. It uses the same overall mechanism though. You make a request, you get a 402 sign return, um, with the authorization and get the data. Um, I think they're going to implement it slightly differently and that like the header will be different but it's the same basic flow of using um, of using it for authorization. And then there's in the ecosystem AP to the agent payment protocol by Google, which interestingly is not about settling funds moving. It's all about that authorization piece up front where there is an irrefutable chain of proof between a human to an agent to a server. But you have said yes, you can spend that $2.50 and so that you know there are different layers of the same problem. Authorization and then settlement.

Speaker A: I mean that was the thing when, when people started saying like oh, and now agents can actually use your credit card and go out on the web and buy things. And I'm like no, that's not a good idea. So uh, having an, having an upload so to say uh, a pay as you go as you, as you said makes much more sense. How is the life cycle of that one? I authenticate once and I say like okay, you can now uh, spend it. Can I send a uh, and now stop spending or is it just every single time a transaction goes through I have to re. Authenticate.

Speaker B: Um, a lot of this is handled automatically by clients. We do have to do that whole thing flow each time. Well, apify also create MCPC which is a CLI for interacting with MCP. And I believe we're the first CLI to support X402 as a standard. The workflow there is interesting because it really, it shows you how slick the user experience can be. So you create a session where you say, here's an MCP server, uh, I want to call it apify. It's MCP apify. Um, you link a wallet to that session and then you make requests. You make multiple requests under the hood. It's still doing that whole dance every time, but it's kind of invisible. You just tell it to make a request and it does it. Um, there's other interesting things there. Like right now the workflow is that you put up a bit more money than you need. I think it's a dollar. And there's a 60 minute window, um, in which off of the chain, you know, the platform figures out how much you've used and then it refunds the rest. 6. Anything that's unused 60 minutes later coming up will be a way of being able to um, specify like a maximum amount up front. Uh, so yeah, it's interesting, it's very, very fast evolving. Um, but I do really like this idea of being able to attach a wallet which I prepaid effectively. You can think of, you know, a stable, a standalone stablecoin wallet for your agent as being like a prepaid debit card for all intents and purposes. Um, and it can then go and make requests on your behalf autonomously, potentially, or not. The other thing that's really nice about http402 is it removes that need to register for some platforms. And that also bodes well for anonymity. Right. And privacy.

Speaker A: Yeah, I mean I'm using uh, vpn, uh, I'm using malware and they allow you to pay in any which format as well. And that's, that's the interesting thing. It always cracks me up when people go like, I want to have privacy, I want to use a vpn and then I use my credit card to pay for it. And you're like, wait, there is, you know, there is an error in there.

Speaker B: But if I remember correctly, they explicitly say in the faq. I also have used Mulvaned in the past. Like you can put your five bucks in an envelope and mail it here with your user id, like your randomly generated string that isn't attached to a person. And we'll just give you that month of usage, which I love. I wonder how many people do that makes me.

Speaker A: Yeah, it makes you wonder. I mean it might be happening more than we think. So, um, in terms of skill sets, uh, uh, that agents have nowadays, like we talked about guardrails before, but what are things that you're seeing that are still a big hassle, uh, for agents to do that, we should be Finding easier ways to actually uh, make them navigate the web as we do.

Speaker B: Yeah. And I'm going to just take a moment to say, you know, the word skills is also loaded in the world, in the world of AI because skills means a thing. Right. You, you create reusable skills which are essentially just custom instruction files with perhaps uh, you know, a set of uh, tools that it has available to it. But talking more about skill sets a bit, let's call them abilities. Right, Abilities. For agents, one thing that is still tough is accessing real time data consistently and reliably. That's where apify comes in. Right. And I know I just sound like I'm going to just keep talking about applied, but it's true. As I use my agent, it just cannot get access to data. See all the time. Uh, let me go check that for you. Oh, I've been blocked from accessing that. It's super, super frustrating when you're trying to run analysis or like do more bulk work. I can't feasibly go into all of those sites and copy and paste values. So you know, that's where having an intermediary like an actor which has anti blocking mechanisms is really powerful. Um, so I'd say that's one thing. One other thing I'm seeing that's kind of hot at the moment. I'm seeing a few talks come up about it at the moment. Different developer conferences is where clis and NCP servers interact with each, you know, um, converge inside of an agent and where you should use one over the other. Bearing in mind that um, MCP tools can eat up your context window, which we're increasingly aware of.

Speaker A: Right.

Speaker B: Um, when it comes to things like GitHub for example, I don't have the GitHub MCP server installed in my clients. I just have the GH CLI. It has a help command which the agent can run. Fantastic. It's a good cli. It does everything you need and the agent uses that directly. That's another interesting thing is when do we need to equip agents with new skills versus when can they run commands on our behalf in a way that is more token savvy?

Speaker A: I think that's the biggest issue to a degree. The tokenization of things and the payment window. As you said, the uh, uh, holidays are over. Like basically everything was free for a while and now all the computing companies realize just oh my God, it's so expensive for people to use these things. Especially rogue agents that just install deinstall things and try things out and discuss things all the time. So the uh, uh, it's interesting when I just, I just realized myself like remember when Alexa had skills and that was the thing as well. And like we keep reusing these terms and it becomes increasingly confusing for people to say what they are. And in the AI agent world, a skill set is a markdown file that basically explains what it can do and what it should do. And um, it's fascinating how fast MCP became like a, oh, it's too expensive now because it was like the solution for everything. And now we're going back to the CLI. Are we always going back to a 1970s interface to computing because it's the fastest and simplest?

Speaker B: Well, they don't need to look nice. That's the other thing. Right? Like they don't. Agents don't care about how things look. They're not opinionated. They just need to achieve the task at hand. And I still use MCP servers. You know, the way I think about it often is if your agent needs to go connect to a complex third uh, party service and that doesn't have a cli, use an MCP server. But where there's a CLI available that's trusted, um, and well documented, I default to that entirely. Sometimes it means a little more prompt engineering, a little more nudging. Hey, you have the ghcli. Can you use that? Um, but that's where skills come in because I don't have to remember to tell my agent that I have a skill that I can invoke in, you know, a couple of keystrokes that already has that instruction baked in. Um, I'm really enjoying using agent skills though at the moment. I actually developed my own set of them a couple of weeks ago for our ah, fellow developer advocates to generate talk submissions, um, to take talk submissions and turn them into talk outlines and be really opinionated at every step about what makes a good output, um, that's available dev advokate K I t at the end. Um, but yeah, I'm really enjoying creating customers skills. Um, also forgetting things like Personas. I have a skill that invokes that um, it is instructed to basically act like a financial advisor that I enjoy listening to, like a media personality. When I have financial questions I invoke that skill and then ask the question. So it's pretty nice.

Speaker A: Interesting. Uh, it's also the, the uh, the automatic generation of things with it is of course the, the, the bane of my existence with like with submissions for conferences and things like that. But it's Great that we can actually, uh, make these things and that it enables people who are not developers to actually do all these amazing bits that basically are there, which are the most used agent, uh, actors in the apify platform. What are people using it most for? Is there any statistics on that?

Speaker B: Yes, though I don't know them off the top of my head. I will say that the ones that we readily build and maintain are pretty well adopted, um, and some of our key creators. So if you think of a platform that doesn't have a public available API, so you know, you have a web page, let's say YouTube, right? A YouTube listing, a channel, a video. There is a bunch of data on that page that you can physically see, you can click it, you can highlight it, but you cannot programmatically get it out reliably. You might be able to do it, you know, 100 times. And then you get, you know, you start getting, uh, a band rate limited, your IP address gets banned, whatever it might be. So there's no, you know, there's no reliable way of getting it. And when there's a platform like that, there's a pretty good chance there's an actor. So we maintain actors for things like Instagram, TikTok, YouTube. We, uh, have an E commerce scraper which can scrape Amazon and, and others to get product data, to get video data, to get engagement data. And then you might be thinking, okay, this is all a bit abstract, like what do people actually do with that? So a couple of examples. Um, uh, competitive intelligence when it comes to products from your competitors. I don't just want to have to manually go look every day or every week at their Amazon listing. I want to look, I want to know when things have changed across 30 different E commerce sites. So I can keep an eye on that. It could be, I want to, I'm ready to launch and I want to work with some influencers. So I hit hashtag, we are devs, uh, as a conceited example, hashtag fitness. And I look the creators whose posts have the most traction using those keywords. Um, and now I have a hit list. You know, it's done some of that work for me. Um, it could be scraping Google Maps, uh, places to understand, you know, I'm looking to open a new location or a new franchise. Where is a good spot where there aren't things from the same category, you know, where, where am I going to serve a niche with this store? And all of that is rooted in data that you can absolutely get, but you cannot get reliably at scale. Um, and that's really that gap that apify fills. But it's not just about data extraction. They are ultimately just script. They could do any, they could, they could do post. They could post things on platforms. They could write emails for you. Um, so yeah, you know, create images.

Speaker A: Create, create images, create videos, all kinds of things. Yeah.

Speaker B: And the nice thing about it is because we enable creators to like state how much things should cost for the end user. Uh, they, you can build in the kind of um, upstream costs into your pricing model and then still monetize like built into the platform. You don't need to work that out. You just set some values in the metadata.

Speaker A: Mhm. It feels to me like uh, web 2.0 back then like when the, when the web of rss feeds and APIs and web APIs came out and I was super excited about. I wrote a book about it that nobody bought because by the time uh, the book came out most of the free APIs were dead and actually not available anymore. Like the Maps API, like the YouTube API, like the Bing API, like all these things that were out there. So it seems to me like as a company instead of providing an API which is like expensive and hard to do, you could build an actor on your platform to actually give that information out there as well to make sure that not people don't come randomly and actually work around your security uh, or limitations but you just offer it as a feat so to say.

Speaker B: Absolutely. I don't really have much more to add there. That is also an incredibly viable way to build an actor. They're also fun. I built an actor, uh, or I've almost finished an actor which is a web ring manager, if you remember web rings set of links at the bottom of your website and you'd hit next and if it go to the next website in the web ring and eventually it can come back around to you. But you need a system to manage that with some perspective. Persistent storage with an endpoint that can be hit. Um, so yeah, I'm in the middle of building an actor that will do that and that's super fun. You know they don't need to be big serious things either.

Speaker A: M. I used to write them in Perl. The web bring scripts and that was just one server with the database behind it. And yeah it didn't scale but like

Speaker B: for some reason doesn't sound like it would mate.

Speaker A: It's interesting to see though that all these, all these things I uh, mean uh, when you talked about like that most of it is about Instagram, about YouTube and getting those. How do you prevent abuse though? Like uh, um, how do you stop people from using apify to actually just get the data and store it somewhere else and make copies of things like uh, is there any limitation in there, have there been cases in the past where people abuse the system?

Speaker B: I don't know so much about. There have been cases in the past people have abused the system and I'm not sure what abuse would look like in this case because frankly, you know the, the idea is to equip your applications, uh, to equip developers and to equip agents with real time data. Data in 2026 that you scrape is stale in not very long at all. If you just want to scrape once, store it, copy it, distribute it. Okay. Like our platform was to give you access to that data programmatically in a structured form where before you could see it, click it, right click it. But you couldn't do that programmatically reliably at scale, um, but it, but it becomes stale. So you know another example, um, I kind of gave this with the E commerce example is you know, running a web crawler uh, across your competitors sites and understanding in uh, front of storing pages and then when there's a diff, when there's something that's changed on one of those pages or new pages or pages have been deleted, send me an email, let me know, send me a slack notification to let me know stuff I could already do but that is very, very reliant on up to date information in order to be anywhere near useful.

Speaker A: When you talked about um, uh, this session you basically said like we want to have a more equitable relationship between agents and creators they interact with, with. What do you mean by that? Like is it just like that agents feel predatory now and coming to your system without you knowing about it. But is it a more a give and take in the near future like our agents are replacing us or is it something that we should just see as another way of interacting with technology?

Speaker B: Yeah, you have a brilliant skill of asking like three questions in one question and I need to decide where to start. Right. Uh, what I meant by that truly is that I think we just have to own um, the fact, and I don't think it's much debate that many of these AI tools that we rely on day to day as developers, as consumers has been stolen, it's been scraped from the public web, it's been added to a corpus, it's been training new models and there's lawsuits that ah, are currently ongoing. Where big publishing houses are like, no, you cannot do that. You cannot just scrape all of our data, uh, and then make it available elsewhere because it's copyrighted and what this allows. So there's two sides. This for content, um, creators, the ones who actually create this data in the first place, this provides a way to effectively monetize that right, and provide um, access to that information programmatically, but on your terms. The second I would say is for open source creators, I mentioned it before for um, we have this term that we use called actorization, where it's taking, where it's encouraging open source, um, authors and maintainers to allow a deployment on apify as part of their instructions, as part of their workflow. And we have an agent skill to help open source creators with this and um, you know, allow apify as one of the runtimes and when that happens they get a kickback too like they are the authors of that, of that actor. And I think these last few years have been really interesting when it comes to sustainable open source. Um, having previously worked at GitHub and large open source projects that have gone through license changes, um, I don't think anyone's truly cracked sustainable open source. And I think this provides an interesting option to add to that. Table Ah, for the projects where it makes sense.

Speaker A: I mean people like Brave and others tried to do that before but they always went for the bitcoin route and the Web three route and that's how they lost people rather quickly again. I like the idea of, of uh, uh, of changing the web as it is right now into an agent platform, but not replacing it with something that is more confusing that Web3 was. It's like explaining blockchain to normal people is really, really tricky. Explaining the web, they already got used to that and it took them 15 years to realize it. So now we can actually build on top of that.

Speaker B: And I will say that a lot of, a lot of how agents actually surface abstracts uh, away a lot of that complexity. Even this X402 topic that we previously discussed, a lot of that complexity is handled away from the client. You know, you don't think about TCP packets when you're thinking about loading a webpage, right? You think about the HTTP spell like that's the level of abstraction that we as developers think about. And those around us who aren't developers don't even have to think about it at that level. They just put in a way, I put in a URL, I hit enter and there it is. Um, you know, the MCPC CLI is a remarkably slick experience in that once I attached a wallet to the session I could just make requests and it handles all of that. You know, checking the price, signing the authorization, sending it off to the facilitator, um, automatically and then returning once the server return data. It handled all of that away from me. I didn't get any logs or anything. It was great.

Speaker A: Cool. I mean we're at the 40 minute mark and people are not watching long videos. We do a good job of cutting them up into little pieces on our platform, we. But uh, this was very interesting to see. Like um, if you want people to get started on these things that excite you, what are the places that they should go and actually start playing?

Speaker B: Yeah, head to apify.com, sign up. There's a free tier, you can try out actors, get data, see how the integrations work. And once you've done that, you can configure the MCP server and uh, give your agents access to real time, real time data.

Speaker A: I especially like the idea of the monetization of open source, uh, systems that basically. Like right now. Yeah, okay, we got a GitHub, uh, that's uh, uh, if you already have something and you're worried about that, that people use your stuff without knowing it, that might be a good way to do it.

Speaker B: Hey, if you write little utility scripts that are genuinely useful, you can wrap them up as an actor and publish them on Appify and give people not only the ability to like effectively run them in very few clicks, if not one click, uh, but to chain it in multi actor workflows so you know, and use that common platform and interface in order to interact with, with your actor. So give it a try. It's really good. We have an actor creation skill as well, an actor development skill that I used in a, uh, in a demo a couple of days ago and it literally one shot, a reasonably complex actor and it was, I was so impressed. Like I use AI all the time but I understand its limitations and I was incredibly impressed.

Speaker A: Excellent. Well, thanks very much Kevin. Uh, we meet each other again soon in Berlin I guess. Uh, and uh, are you gonna go, are you gonna go come to the Congress?

Speaker B: Absolutely. Oh yes, of course.

Speaker A: Sorry.

Speaker B: Yeah. The other thing you can do is come find us. We're, we're there, we're there. We are devs, uh, World Congress in Berlin in July. So like come say hi, I'll be there. Yeah, come tell me I was wrong in this or tell me that I'M great either works.

Speaker A: I tell you, you're great. And I think it's a wonderful time to have you here. Thanks very much for being with us. Um, that was Coffee with Developers with Kevin Lewis from apify. Um, look at their stuff, what they're doing. And there's going to be an article in this as well soon on the magazine. I guess if you want to write one, we can publish that for you as well. That's it from us today. So thanks very much for listening and thank you very much. Bye Bye. Kevin.