Ep#250 AI Accountability and Open Source with JD from Command Prompt

Jon Meyer: Today we're diving into AI accountability, what it means, who it serves, and whether open source is a lifeline that keeps this technology honest. We'll explore the very human condition that drives us to create tools we don't fully understand and what we must never forget as we hand over more and more of our world to machines. Because at the end of the day, AI doesn't have a conscious. We do. Hi everybody, and welcome to the John Meyer Podcast where we bring you thought leadership and conversations from around the world. Today's conversation is one that I believe every person needs to hear. Joining us today is jd, the founder of Command Prompt. Jd, thank you so much for joining me.

JD: Oh, thanks for having me, M. John, I really appreciate it.

Jon Meyer: So jd, our topic today is AI accountability and Open source survival. What is AI accountability?

JD: Well, it basically boils down to ensuring that when you use the tool, you use it responsibly. Uh, one of the things I like to tell people when I'm speaking is that while AI isn't going anywhere, we do have to accept the responsibility that if we're using AI, we're using stolen intellectual property. Now that's kind of the way it goes, right? All massive industries as they bloom unfortunately do things that aren't all that ethical to get there. The idea being the solving the bigger picture, right? But if we're going to do it right, especially in an age where we can move so quickly now, we have to be accountable for the code, for the intent and for the end result, which is something that society doesn't like or is not really used to doing.

Jon Meyer: J.D. when you take accountability for the code and let's step back here a little bit, and I can't even say, uh, all that long ago because AI has been writing code for a little bit and people have been start using the term vibe coding of AI writing the code. When you look at a, ah, developer who used to write the code fully from start to finish, he knew the code, he understand the code. When you talk about understanding and being ethical around the coding now AI writes the code. Do you see the code? Should you review every aspect of the code and is the code really valid and going to do what it needs to do?

JD: Well, I mean AI does hallucinate and it does make mistakes in that way. It's very human, right? It's not perfect. Um, so yes, when you write code with AI, you should review it. However, it has gotten to the point where it is accelerating the developer experience. And I'll give you a perfect example, uh, just this week. I have written more code with AI in about 16 hours. And this includes full documentation, unit tests, security review, bug testing, the whole bit that normally would have taken me a month to two months. Right. So the accountability portion of this is while, yes, I can trust the tool to a point, you still need to have your unit test. You still need to do at least a cursory review of your code that you're creating. You own that code regardless of the tool using to create it. It's no different than the old school. Borland, Pascal, Delphi, uh, ide, right? Where you'd be typing, it would start to autocomplete for you. That's essentially what AI is doing just at a very accelerated rate.

Jon Meyer: Is there a concern that this code that AI is writing, that humans are not being interactive or acknowledging some of the code or some of the lines that are being input into the solution?

JD: Oh, absolutely. By nature, people are lazy, right? We seek convenience. Uh, and so you're going to see a situation where, well, and we have already seen it, right? We're, you know, malevolent. M. Malevolent bad actors, right? Bad actors have created with AI, uh, attacks on infrastructure at a velocity that they never could before. So they're not in that way, they're actually taking accountability. Cause that achieves their intent. But on the flip side, you're starting to see, uh, you know, I've got buddies in the market that used to do bug bounties, right? Find a bug will pay you, and they've had to shut it down because people would just use AI and just hammer the servers to a point where it's no longer effective.

Jon Meyer: It's not providing them the outcome that they were originally looking for in order to find those bugs. In fact, it's just consuming it and making people even more lazy to use AI to financially gain, find those bugs and then really the outcome is no longer relevant, right?

JD: That. That's where you get the idea of slop, right? AI slop is that. And let's, let's be clear. This is not a new problem. There's always been people that are going to generate crappy work product, right? Usually we fire them, right? But the difference now, the difference now is the velocity in which it can be done. One person who is not me, right, I know what I'm doing. But you get someone who is just remotely talented or has an idea talking to Claude, and all of a sudden they're generating thousands of lines of code and just dumping it out there for anybody. And as we've seen in the past, even before this hit, you'd have uh, I forget what the name of the library was, but there was this library. It was a JavaScript library that something like 70% of the Internet used. Uh, and it was hosted on S3. All of a sudden, uh, instead of downloading the library and installing the library, people would just call it from Amazon. And all of a sudden it was no longer available and large portions of the Internet just disappeared. They were down because this one little JavaScript library, that's the laziness we're talking about, right? And so now we have to be more diligent. It's going to cause problems within the workforce, right? It's going to increase the disparity.

Jon Meyer: One of the concerns that I see here is that not only people are creating all this, uh, code using AI, is that nobody has time to authenticate the code, review the code, go line by line to make sure all the security is there. They might use AI to validate the AI code. And really AI is going to say, hey, this is good because the AI wrote it, so we know it's good. But then you have to challenge the aspect. How does the human condition come into this?

JD: Well, so using Claude as an example, because that's what I have the m most experience with, although I have used Gemini as well, um, if you take the default LLM, which is Sonnet, it has a certain rating level for quality of code and it actually does pretty good if you prompt it correctly and say, make sure you write the unit test. I've actually watched it catch its own problems. Okay. But that doesn't make it any different than the human condition in that we can't catch all the problems no matter how many tests we write, how many times we code review. Right? So you can take that code and one of the things that I've done is I run it through Sonnet and then I'll upgrade to say Opus. And I'll have Opus. I'll say, look, take a fresh look at this code, make sure that there's no craziness in the unit tests, review this for problems and it will find them. Will it find them as well as a, you, uh, know, experienced engineer? No, it will find the low hanging fruit. The problem is, is that business doesn't care. Business doesn't care about quality of code, they care about quality of product. I mean, look at Microsoft. Okay, no offense to Microsoft, but they have released a lot of really questionable stuff over the years, but it works enough for businesses to continue to buy, which makes their Profit margins. Good. So why would they fix it now? Uh, it's just a velocity.

Jon Meyer: That's a valid point. What is the point of actually fixing it if people are buying it and we know there's bugs and we're just going to release the fixes later on using our AI?

JD: Well, it's not all that different than automotive recalls. Right? Automotive recalls. What they do is they calculate the risk of basically the value of the lawsuits that will be incurred by the bug in the car or the mechanical defect in the car. And if it's cheaper to leave it on the road, they just leave it on the road. I have a perfect example. I have a 2015 F150. It has a known problem, documented, publicized, that it might at 80 miles an hour just throw itself into second gear, which will completely destroy the truck. Right. If not the humans inside. But they have no fix. It's the only reason they told people is because now they can say, hey, we told you, if you've got a concern, don't drive the truck.

Jon Meyer: That's actually a good point there. Well, and speaking of active vehicles, I have a 2025 F350 and now periodically I'm getting the automatic updates. It says, hey, at 1:00am Your vehicle might not be available because we're going to be updating it. What if I needed at 1:00am um, what the crap? Can I still use it?

JD: Is it going to be, let's be honest, we're too old to care about 1:00am um,

Jon Meyer: uh, that's very true. What am I definitely be in bed. But maybe there's this one time that I might be using it, like decline.

JD: It's it. But it really does boil down to that. I mean I hate to keep using the word velocity, but that is really what we're talking about here. We're talking about Instead of doing 60 as an experienced developer, I can now do 120 or 240. Right. I am now on a high speed train and I can deliver MVP level product in hours instead of weeks. And AI is what allows that. So when we talk about accountability, you have to ask yourself, you know, how are contracts going to change? For example, you're going to start seeing contracts where it says that will explicitly state not only you work for hire, if you use an AI tool, you are liable for the results of that AI. Right. And no developer in their right mind is going to sign that, but you're going to have to anyway.

Jon Meyer: Yep, that's uh, hugely valid and I think that makes them accountable in that aspects. JD what role does Open source play in keeping this AI transparency and accountable?

JD: Well the nice thing, first of all the world runs on open source whether you know it or not, right? 85% of the world's running Android. That's just a Linux distribution. Uh, if you're browsing the web, you're using open source. If you're shopping at Amazon, you're using open source. So the role of open source will be this. The engineers and project maintainers of open source projects, they're going to become even more hardcore gatekeepers, right? Linux, the actual Linux kernel for example, has done a good job of this so far and they adopted AI. They said, look, you can use AI, it's just another tool. Absolutely true. But you are uh, back to what we were previously talking about, accountable for your code. You must own the submission or the patch and your reputation will be derived from the quality of code, AI or not, that you submit. So that's a good thing. But what's great about open source is since everything is visible, you've got the many eyes problem. You don't have just one guy using AI, submitting a patch. You've got one guy submitting a patch with AI, another guy running that patch through his LLM, Deepseek, Gemini Open, you know, whatever, ChatGPT and saying find the bugs in that patch before we commit it. You've got a human reviewing the patch before they're willing to commit it. And then you have automated test suites which are already in place through continual integration and continual development tools that are going to check that. So from that perspective I think we're going to see a great amount of open source improvement. Right? Because open source is a general rule. It's just like any other software project, it's got bugs. I think AI is going to help that the open. But from an open source survival perspective, everything that we just talked about, any major open uh, source project is going to have to implement those things. You know, chain of evidence basically of all the code, how it was tested, documented, how it was tested, cicd, the whole bit.

Jon Meyer: Didi, I like how you said in a lot of the aspects, companies are holding people and developers accountable for the code that they write using the AI, the bug fixes that you actually talked about. But, but really who holds the power with this? The developers, the corporation, the people who that are using the technology?

JD: Well, the power comes from whoever owns the ip, right? So in the Linux situation the power comes from who is, you know, it's a huge feather in the cap to Say I have a patch in the Linux kernel for example, right? My, I contributed code to make Linux better. It's better for your career, it's better for your ego, all those types of things. So there will be an incentive to produce better product or better software regardless of AI or not. It's just uh, that production is going to happen quicker. Right now it may be IBM Red Hat that owns the code. It might be Linux itself is actually owned by whoever submits the code itself. So there's thousands and thousands of owners of Linux. However, the name is owned by Linn is. So if he doesn't like what's going on with the kernel, they got to change the name because he's got that trademarked. Right. So it really just boils down to whoever owns the IP in the end result will be accountable, but it's going to all flow downhill. That's not going to change. That hasn't changed ever. Right. It's always company boss, employee, pleb, Right?

Jon Meyer: Exactly. Jd, My question that I have, and this is going to be a very challenging question is where do we actually draw the line between AI assistance and AI dependence?

JD: So we won't have AI dependence for a little while. We're not there yet. Um, but it's coming.

Jon Meyer: Wait, I gotta, I gotta jump in there. A lot of people are dependent upon AI to write their thesis, to write their everyday lines of code. Is that dependence or assistance or. I'm dependent upon it to do my work.

JD: If you are dependent on AI, you don't deserve the job if you know how to use the tool to make you better. That's where it, it really comes down. I actually, I just wrote about this on um, LinkedIn. Where AI is really going to help right now and for the foreseeable future is people who already have knowledge. Because to use knowledge you need knowledge. So if you're 20 and you're coming out, don't get me wrong, they're going to use AI and they're going to, they're going to learn it and move it and go and go and go. But the funnel of information is going to be so much that they're not going to know what's good, what's bad, what's average. Right? It's people that have been doing this work for 15, 20 years that are going to be able to say, whoa, whoa, whoa, whoa, that's cracked. That's a total hallucination. Let's re prompt, let's retest, let's re engineer, whatever. And it's going to make Them move faster. So that's the assistance part. You brought up something that was interesting. Writing your thesis. Okay. To get like a doctorate. I am of a very brutal opinion of that. If you are a student and you are using AI to write your stuff, you should be failed. Because the whole purpose of being a student is to learn how to think, not how to regurgitate. Right. So if you're a PhD candidate and it's obvious you used AI, you had them write Section 3 site with four citations. You fail, period because you are dependent. That means you don't have the critical thought or the ethics of effort to succeed.

Jon Meyer: I uh, like that you mentioned the PhD because I don't uh, believe it was my wife who said it is that. Do you know your current doctors right now that are in school are using AI to pass their test?

JD: Yeah. And that should be, I mean literally it should be an expulsion level offense. I don't get it. Wrong. Use it for research. Right, Use it. I mean because AI is a search engine quote unquote. All it is is a faster way to go to the library. Okay, that's fine, use it for research. But you need to have your own independent thought, your own independent hypothesis and your own independent conclusion.

Jon Meyer: I think one of the biggest things right now and a concern, and I like that you also suggested that those who have been in the industry for 20 years doing the work will be able to say that something is crap, something is not. And those who are coming up now and actually just using strictly AI ChatGPT, they're going to agree with the results that is coming up. They're going to say this is true. The concern that I have is that us, uh, our generation, and I'm going to put myself in the older category is that we're not going to be able to. What's that said?

JD: We're Gen X.

Jon Meyer: Yes. We're not going to be able to retire. We're not going to be able to do anything because we're going to have to stick around and tell the younger generation that all the stuff that AI is generating is potentially not all this stuff, but some of the stuff is hallucinating, incorrect. Right. And the data is not there and valid. And I feel like we're going to be going back to a cycle where people should go back to the library or do their research themselves and come up with their own thought or hypothesis.

JD: Well, you're right, but I don't think we're not going to be able to retire. I mean there is a reason I moved off grid. Right.

Jon Meyer: And here you are on the podcast

JD: and here you're not retired via space. I'm on the podcast via space, yeah. Um, no. So there's two parts to this one. We're already seeing a, the social contagion contagion change and we can thank social media for that. You're starting to see laws across the country enforcing zero screens in schools. Okay. As it should be, as it always should have been. And uh, that's going to require students to critically think. We're starting to see limitations on social media because what we're talking the problem that you just brought up uh, of they're just going to believe whatever chat GPT says that was exasperated by social media. You have a whole segment of the population and it's about half of the population that will watch a video on YouTube or TikTok and say that's now real. That's where the anti Vax movement came from. Okay. That's going to shift because the powers that be, whether I like them or not, all of them across the board are like whoa, whoa, whoa, whoa, whoa. We're killing ourselves literally. We are at an extinction level event. Not tomorrow but if we allow this to continue forward the way it is, we're going to have a bunch of robots that don't think in progress will stop.

Jon Meyer: I actually just saw that that about not having your screens in school. There is going to be. They want to implement something bell to bell. No screens it we're called. I agree with that. I mean granted, uh, it's nice to talk to my kids periodically or if I need to get some message or information out to them but we survived without it. We were here without technology or got to here without technology of always on our screens, uh, to a certain point. And I do believe kids are growing up with too much of the screens and us as parents are enabling that in most cases or some cases to do it. So it's a double edged sword. But going back to the AI aspect I think it's very critical for, for these students to think for themselves, for those coming up to think and analyze for themselves. Because the part where you mention that we're at an ah, extinction level of not thinking and not innovating and progressing is because they're going to be heavily relied on old information from AI.

JD: That's right. And number to address your school thing real quick just for a second remember you can always call the front desk and say please pass a note to my child or please go get my child. I need to talk to them, that's always been the case. So this idea that. Oh, my God, I need to talk to my kid right now. Get a grip. No, you don't.

Jon Meyer: What do you want for dinner?

JD: Yeah, exactly. Right. You know, I'm gonna pick up your sister. I might be 15 minutes late. That can be a note that's passed via an aid. Um, now, in terms of the thinking part, the re. Every generation, every young generation comes up with something brilliant. Right. In our lifetime, you and I have been through Nixon, Carter, Reagan, Bush, Clinton twice, Bush twice, Obama twice, and two more. Right. One of them that achieved something no one in history has ever done, which was to lose and then win again as a president. Right. We have also seen eight tracks, tapes, CDs, DVDs, Blu Rays, 4Ks, the cell phone.

Jon Meyer: Right.

JD: Everything what people are walking around with today, which is a tablet that pretends to be a phone. Right. Do you remember, remember the bags? Yes, the bags.

Jon Meyer: They were shoulder host. You could put over your shoulder and it would pick it up and actually I wanted one of those as a kid. But you want. Sure.

JD: But that was every generation coming up, something brilliant.

Jon Meyer: Yep.

JD: What we need is the young generation to be able to think so that they are inspired. AI will never be inspired. It will never be impassioned. It will never create. It can only respond to your ideas. So we need the next generations to actually truly believe in themselves and be inspired to learn to critically think, to come up with the next great thing. Maybe it's going to Mars.

Jon Meyer: So I want to touch on that a little bit. And I'm not going to challenge it, but I want the audience to understand this. You said AI will never be able to create, but it will only be able to respond to my commands. If I tell AI to create an application for me that does X, Y and Z, is it essentially creating the application or is it just responding?

JD: It's regurgitating code that it has already learned about. Right. It is not.

Jon Meyer: So it's just compiling different various pieces of code to build. And I don't want to use the word create, but to build the thing that I have asked it to do. But with old information.

JD: Correct. With the whatever. Well, it's not to say old because it learns from people. I shouldn't say learn. It stores the information, the information that people give it. Right. So it's not necessarily old in. In the sense of. It's from the 1970s. It is old from the sense that it was yesterday. Now, the point of why it can't, uh. You made A good point there. It's create versus build. If I tell Claude, I want you to build an application written in Python, uh, and C that allows me to authenticate to PostgreSQL via SSH keys, it will go and it will read all the existing source code around that idea. Whether it's postgres, whether it's Python, whether it's open SSH code, it will analyze that code and then build code around that. When I say it can't create, it won't have that idea. It won't say to itself, how can I make infrastructure better for people running postgres? It doesn't have that ability. It has to be prompted. Will it have that ability in the future? I surely hope not. Because no matter what, AI is a binary decision. It is a true, false yes, no. And in the world of humanity we lose if there's no maybe, kind of, or sort of, or hesitation or hesitation. That's absolutely right.

Jon Meyer: That's definitely what I think. Jd, my last question for you is what is important to remember about the use of AI along with open source?

JD: Okay. Number one, it's just a tool. It's not taking over the world. Only humans can do that. We can either continue to build or continue to destroy, or both. It's usually both, right? Historically speaking, but it is just a tool. In terms of open source, the most important thing is to create the human integrity chain of evidence. Right? Code needs to be signed, you need to know it came from a human. Regardless if it was built with AI, you need to know it came from a human. The human has to enter a social contract that yes, I am responsible for this code, I'm just not lobbying it over the fence and hoping it works. Right? That's the key to open source success with this. Closed source is a little bit different when it comes to this because you can just be fired, right? If you, if you destroy the company, you're fired or whatever. Uh, but there's a social contract with open source and most projects are going to have to implement it. They won't have a choice.

Jon Meyer: Jd, I gotta thank you for joining us on the show. Thank you everybody. AI is only accountable as a humans behind it and responsibility falls on all of us. Thanks for tuning in to the John Meyer podcast. If today's conversation made you think, share it with someone who needs to hear it. Until next time, stay curious, stay informed and stay human.

JD: Thanks for having me, John.

Jon Meyer: Thanks J.D. that was awesome.