Dark Factories, Cargo Cult AI, and Drunk Agents with Geoffrey Huntley

Ankit Jain (00:01.31) All right. Hello and welcome to Hangar DX. This is a podcast about developer experience and how engineering teams solve productivity challenges at scale.

I'm your host, Ankit co-founder of Aviator. at Aviator we are building a verification platform for AI generated code that creates guardrails against your AI slop Today my guest is Geoffrey. Joffrey or Jeffrey? Geoffrey Huntley (00:27.

911) Jeff. Ankit Jain (00:28.866) Jeff, there you go. And everyone knows Jeff from Ralph Loop, but he is also an longtime software engineer and a GOAT farmer, as well as a great DJ.

I can see amazing setup in behind you. So Geoffrey Huntley (00:45.872) You know, you know. Ankit Jain (00:49.

996) Jeff, I'm really excited for this session. please introduce yourself for our audience. Geoffrey Huntley (00:56.797) Hey Ankit it's good seeing you again.

I've just spent the last ninety-five days traveling around the world giving conference keynotes and talks, cha talking about how software development has forever changed and is changing. The unit economics of business have changed. some folks know may know me from the the the autonomous Ralph loop thing, which is now like if you're using Codex goal or any of any of the equivalents in Copilot, Claude, et cetera. These are these are all implementations of the idea.

Now, with this idea, it it's actually it takes a bit of systems engineering to reach the destination where it's so simple that it's essentially a meme of while true, cat prompt into agent done. Like that that's Okay, sure, that's the low IQ meme version, but it's actually about memory management and context engineering and a few little discoveries along the way. And I don't know, maybe we get into it, maybe we don't. I've been thinking about this is essentially a pattern for an orchestrator.

And the crudest teachable form of orchestrator is just cat the prompt. Ankit Jain (02:17.591) Mm. Geoffrey Huntley (02:23.

867) And have that prompt pretty deterministic and then set the set the direction on how an agent could advance. In the case of Ralph, it was you're only allowed to do one task, don't reuse the context window for other tasks, so thus recycle the recycle, but you still want to progress the state. So that was a git commit and a to-do list. And all the different tools you use today have an implementation of this.

Yeah, software engineer. I've had many titles in life. but not that interesting. Let's riff about stuff.

It is about nine twenty AM. Well you wanna go about talk about, mate, 'cause we've been meaning to do this for a while. I ran I ran into you over at Heavy Bit. So let let's go.

Ankit Jain (03:04.821) Perfect. Yeah, absolutely. Yes.

Yeah, I mean, so I think a lot of my questions are gonna revolve around, you know, what's coming up next? How is our industry evolving? Right. And a big part of this is you already mentioned orchestrators, we're hearing about software factories.

So we're gonna like want to dive into that, right? So maybe one question to just trigger the conversation would be. I'm hearing at least some people have started talking about one person model first companies. I don't know what your opinions on that is.

Maybe this is where we can start. Like, do you believe there could be enterprise software which is driven by one person in near future? Geoffrey Huntley (03:59.911) Well, let's get get to terminology and definition.

The word enterprise automatically means you cannot have a one person company. Ankit Jain (04:09.677) Okay. Geoffrey Huntley (04:11.

22) Enterprise is fundamentally incompatible with with the notion of a single person. They won't buy from you if you are a single person. Like to pass to pass audit and compliance topics, you you need to actually have multiple people type things. And then it's very easy if anyone has done work of enterprise, you see that it it consumes your entire cycle.

Like like w like one one enterprise Ankit Jain (04:21.496) Okay. Geoffrey Huntley (04:40.144) Get it up and running POC Yeah, that that's two weeks of work for questionable ROI.

Enterprises move slow. Enterprises move slow. So I think it's fundamentally incompatible to say that you could have a one person enterprise play, because enterprises are fundamentally incompatible with that notion. So l let's flip this around.

Let's flip this around. Ankit Jain (04:48.716) Yeah. Ankit Jain (05:05.

58) Interesting. yeah, go ahead. Geoffrey Huntley (05:07.388) W where are where are we today?

and where have we come from is a pretty good way of looking at this. So I remember when I first posted back almost two years ago, two and a half years ago, saying, fuck, everything's changing. The IDE's dead and people won't use the IDE anymore. And I remember my coworkers at Canva call calling me bloody mad.

Like and I catch caught up with them and like no one's Really using the IDE anymore. They're just it's a code review diff tool. And it's like, wow, that's that's how far we've come. Now, in my travels, I I like Europe, I I I saw different cultures evolve at different paces.

And like at least in like in Estonia, very far forward for for of forward thinking and a lot of AI first, like not using IDEs, but I'm not gonna name any particular company, but well known. And I remember having some late night drinks with a tech lead and like, no, no, no, no, no, jet brains forever. I love my IDE. AI is just this tool that allows me to auto-complete my own like this was last month.

I'm like, what the hell? So that's enterprise. If you want to sell to that They still haven't like you want to make it some sort of enterprise sandboxy thing. It's three years from now before they'll actually be a buyer of your product.

Ankit Jain (06:48.451) Mm. Ankit Jain (06:56.065) Interesting.

Geoffrey Huntley (06:56.732) Pace, at pace and time. There are a lot of enterprises who have banned AI outright, not on a token cost basis. But because th th they they're still worried if the company will train a model on their data.

But if anything we've seen recently, the real value is in the traces. The real value is in the agentic traces because then you can work to fine tune a model. But yeah, so more targeted answer. I do think the the fundamentals of business have forever changed.

And I think that the reason that you raise capital is your post revenue, not pre revenue. So meaning Ankit Jain (07:28.022) Mm. Geoffrey Huntley (07:50.

013) Previously, you need to raise revenue to hire a software engineer and convince a bunch of a bunch of people to join you on your crazy venture into the seas. And they needed salaries, payroll, et cetera. And you would have your runway. And it might take six months or a year to build your concept.

So you needed a raise to be able to fund the initial expedition into the seas. Now, that has changed. You literally can just rip a fart into Claude Co. and go, I want this and you get this.

Now, the quality of that can be variable based on the s the skill o of the operator, i.e., if like if you're an accountant, you're gonna get better gu skills and outcomes with AI as an accountant, depending on your expertise and accountant, than me or you. But if you're a principal software engineer, you know how to really juice these things.

So Ankit Jain (08:24.451) Yeah. Geoffrey Huntley (08:48.166) The way to get outcomes here, like the the funding I think funding has changed, like pre seed is gone.

Now, absolutely it's not a one person company. I think it's like a twelve person company and you're living a very very nice life with twelve people, and maybe the way that we structure equity is very different. Ankit Jain (09:04.59) Okay.

Geoffrey Huntley (09:16.092) It's not a point one or a point two. No, like you hire the the twelve best people in their domains and it's all leveraged by AI. And that's fundamentally different because you're looking if you're thinking about changing cap table topics, what you're actually talking about here is hiring the damn best people you can.

Not not like we hire the best people like enterprises. No, no. Like you have you seen what Anthropic's doing? then the new minimum hire is the CTO of a company that they actually want to replace.

Like like really creating this ensemble of merry men. And like the leverage here, using AI's leverage, we could be like you have to be looking at ROI, like to like ten people have to have the impact of a hundred people. type topics and it's less people, less communication overhead. So that really works if you're in your own little orbit building recursively related space.

But as soon as you that intersects with enterprise, good luck. That's why you raise. You raise to build a go-to-market team because enterprise needs their hand held and their their pace is much slower. Ankit Jain (10:33.

141) Mm. Ankit Jain (10:39.222) Interesting. Do you okay.

Instead of going in that direction, given that you just mentioned about people hiring the, I don't know, maybe 10x engineer for the lack of better word. let's just think a little bit about it. Like, I mean, maybe not 10x engineer, right? Like whatever term you want to use.

the experts. Let's just call them the experts, because even the definition of developer is fuzzy these days. yeah, go ahead. Geoffrey Huntley (11:06.

04) the l let's you do definition. A developer is a developer is a product manager. it's a designer. It's anyone using these tools.

if you consider yourself a developer, you're typing in code and you work in an employer where you type it code, you need to quit that employer now. Like honestly, like you need to put your family first. Ankit Jain (11:10.19) Yes.

Geoffrey Huntley (11:33.358) And you need to re identify yourself as an engineer because the the the coding or develop has been commoditized by prompting. Ankit Jain (11:37.334) Mm.

Geoffrey Huntley (11:45.222) So like everyone is now a developer and existing people who saw th themselves as a developer, they need to reinvent themselves. So keep going. The definition is a developer is anyone who uses code.

Ankit Jain (11:45.432) So Ankit Jain (11:55.161) So yes, so it kind of like brings in so it brings kind of like two different questions probably with the same answer. So what should these let's just say experts sh how should they reskill themselves?

Like what skills they need to know to that will essentially compound over time. Like that will essentially help them scale over time. And I think I'll just put the second question as well because it's related, which is what skills should a organization be looking to bring in those experts? Geoffrey Huntley (12:34.

644) Okay, so skills that a the developer or sorry, like developer that wants to be an engineer is the same skills that an engineer has always had. It's its first principle's thinking. There was a time that software engineers or software developers before AI, we used to own their tie damn stack. And we weren't gatekeepers.

We would do research. We would understand the full domain of what needs to be built. And this is before agile. And this is, I guess, what we're talking about is product engineering.

And what we're seeing is return of that type of thinking. So as an enterprise, it's not necessarily about the skills that you bring in. It's not like a skill that you hire for. It's actually thinking more a fundamental basis, like How do you structure your organisation in such a way that doesn't hide the best business context from your best in engineers?

Like if they have to pull from a Jira ticket queue and they've got some sort of like Scrum, agile, scaled, safe delivery system, and it's just PMs, all the PMs have all the context in their head, you're not going to get a great outcome. So It's not something that's a skill you need to bring in. You actually need to actually rethink about things here and like how delivery is done. Now, for developers who want to become engineers, it the skills is not apart from that that soft skill angle, the most fundamentals is like start data engineering is a fun a fundamental.

Like if you figure out how you're gonna be Ankit Jain (14:15.352) Okay. Geoffrey Huntley (14:27.044) Working with your data and how it's going to be stored, everything can work backwards from there is something that's always been true.

How to actually verify software. Topics like test generators, property-based testing, deterministic system testing. These are topics that a lot of people are only just hearing now from the first time because they're kind of being fringe and the outliers. But like the principal engineers and like distinguished engineers, we've we know what.

Cock is, we know what Lean is, we know what TLA is. But it's a very small community. What we're seeing right now is the these topics of fear and perimeter's cross the chasm. Ankit Jain (15:05.

046) Hmm. Interesting. Geoffrey Huntley (15:06.716) 'Cause verification of software is very very very important.

Ankit Jain (15:11.68) It very much is. So going to the other side of the question, which is you already touched on this a little bit, right? If you were to guess, how would engineering organizations look like maybe a year from now or two years from now?

What how would they look different? Geoffrey Huntley (15:32.506) I don't know. And if anyone says that they do know and have an answer to this, they're selling absolute horseshit.

What I can put forward is for people to think about this. Remember the Spotify Agile video? There was two videos and it was done a brilliant drawn animation. You can find on YouTube.

And it took Spotify did software delivery. And had squads, tribes, leads, guilds, and all these things. And there's committee on committee on committee and committee. Sound beautiful on paper.

Now, the listeners here probably or the juniors that they might be working out a company and they they've got these squads, tribes, guilds, and they're where the hell did this come from? They came from a YouTube video. It came from a YouTube video, and every bloody company. Cargo culted that into their company without consideration whether it was a good thing to do or it's the right thing to do.

Spotify was on a journey of trying to figure out what works. Now, directly to your question, but with enterprises and other companies, they're just like, that's what we're gonna do. And we had this, and they just did it without the actual experimentation science. Let's figure out what works.

That's the key. Figure out what works. Now, what it looks to the future is no one knows, but there are some companies that are with a front seat. We have, of course, Shopify, and we also have Block on the front seat.

A lot of people giving them crap for laying off and restructuring companies and maybe doing some rehires back. They're completely missing the meta point here, Ankit The meta point is. Ankit Jain (17:12.259) Mm-hmm.

Geoffrey Huntley (17:31.47) When one of those mad lads figures out the right thing and they produce a case study or shop a a video like Spotify, then everyone's gonna copy it. Rightly or wrongly, it takes one business study. This is how and then someone can go, This is what block does, and everyone's gonna do the block.

Development workflow, like Ramp is trying this as well. Like they they are in this mass experimentation phase to find the right thing. So one thing is kind of crystallized in my head is we need less people. because less Ankit Jain (18:18.

689) Okay. Geoffrey Huntley (18:22.694) People that you have fundamentally means the organization structure is easier. Like there's less communication overhead, there's less lost in translation.

In Australia we call this like Chinese whispers, like ding, ding, ding, ding, ding, ding. Now, what I'm seeing is companies coming to market right now in SF, they're they're building essentially tools that allow us to compress the middle management layer and information layer. And build what's called an AI operating system or knowledge operating system for your company. No one knows what this looks like, but everyone's kind of resolved that we need less.

But we need more less busy work and that can be automated away, but more people building, working directly with customers, leveraging AI. We all kind of agree that's the right thing, but like what is the right Ankit Jain (19:15.683) Hm. Geoffrey Huntley (19:21.

484) No one knows. What we got right now is a bunch of companies taking huge risks, which could make the company wildly successful or could kill the company. They're taking their organization chart. Imagine a pack of cards, and you're going, you're like, Ankit would you like to play some 52 pickups?

Geoffrey Huntley (19:45.86) You you're probably gonna go, Jeff, what is fifty-two pickup? And I'll say, Here you go. Throw the cards up in the air.

There you go, you can pick up for me, Ankit So this is what we're going right now. We're going for a mass experimentation phase. Ankit Jain (19:59.565) Interesting.

Interesting. So maybe a related question to that, if let's say Block succeeds or like whoever succeeds in making it work for them, how likely is it that it's gonna work for others as well? Geoffrey Huntley (20:16.55) Well, if you just cargo cult things, it'll never just work.

And I think it's important to to talk about what's called a J curve. A J curve is a people it's a way of measuring people transformation and new initiatives, what a stuff you. So if you if if the magic recipe on how to build an end org is published by Block, people are gonna run to just adopt it. Ankit Jain (20:27.

192) Mm-hmm. Geoffrey Huntley (20:46.182) without actually doing the the transformation needed to adopt it in crude terms. The people who are on the journey towards Kubernetes in the early days, not to say Kubernetes was a good or bad tech.

I have thoughts there. But it required people to think about containerization. And as soon as they start thinking about containerization, they started on their path of basically having reproducible developer environments and they started thinking about testing and all these other things. And it turns out all this all that prep work ten 15 years ago.

Well, Jesus, almost fift yeah, it's twelve years ago. Twelve years ago. Yeah. Well, that's all the precursor work to be able to use autonomous agents and make it work for a coding agent.

Now there are many people who still haven't even started the the path. So what normally happens is they're like, hell yeah, let's do Kubernetes, even now today. Like their first Kubernetes project. Ankit Jain (21:28.

046) Five, yeah. Geoffrey Huntley (21:51.77) And they haven't invested in testing. They haven't they they haven't like they're they're still learning what a container is and how to build it.

This is enterprise land. And then you're gonna skip through all the precursor steps and go, hell yeah, let's just slap an agent on it. Ankit Jain (22:07.33) huh.

Geoffrey Huntley (22:10.052) It turns out if you run an agent without some form of verification or back t the way to add back pressure to the inferencing loop, then you you just get slop in slop out. But the companies that went in first. So that's what it would look like.

It it always happens. It's a journey. It's a journey to of maturity and expect like and like experimentation. So a company that like their block publishes its thing, whatever that is, that's a couple year journey.

And they've gone through the maturity of figuring out how to get there. You can't just like bake a cake instantly. By the time the the company that just wants to do the block thing, blocks are two years ahead and they've they've got the learnings, what does work, doesn't work, what they don't talk about publicly, privately, like all that tactic knowledge. So Yeah.

verification, agents, ankit, and factories. Ankit Jain (23:00.866) Yeah. Ankit Jain (23:13.

826) Okay. Ankit Jain (23:19.19) Yes. So on to factories.

I think this is a great segue. you had some opinions on software factories. I know we have also during the Heavybit session there was also a talk on dark factories. what are your opinions on it so far?

Geoffrey Huntley (23:39.59) So I for context, back with Sonnet three five is or Sonnet four is when like I started doing some of my long horizon tasks and these were low with lower quality models compared to what we have today, if you can compare. The longest r long horizon run I did was like almost three months. It's like I streamed it live on YouTube.

This was like a year and a half ago. I've seen the value domains of where it works, not what doesn't work. And I've tried different ver variations, permutations of this continually over the years. Like I've rebuilt my own gas town now, probably four or five times in in like i i i i since my fuck moment to really understand things.

And I caught on early that the industry is just now re kind of rebranding itself as a verification. But like verification, caught on early. And like check out the crew of people that hang around Anthithesis not necessarily the company, but the community. This is if you find the nerds nerds.

And I I when I was after Heavy Bit, I went to Washington to hang out with them. And it was very interesting to see Jane Street was also in the room. And they were saying, We're gonna hire every single one of you. Ankit Jain (24:51.

459) Okay. Geoffrey Huntley (25:08.718) If you want to work at Jane Street, you can come join us. So let's get a couple of things out there.

A lot of people want to have dark factories. And verif to solve dark factories we're gonna need to solve autonomous verification and of software. Do you have the budget of Jane Street per employee? For the record, J Jane Street has about twenty two hundred employees with nine point eight million profit per employee head.

So d th this are the the the dynamics here. It is known and is recognized that verification is a hard problem. you're competing with labs for talent, you're competing against your high frequency trading firms for this talent right now. I don't think it's going to be accessible.

Like maybe someone can like package up a factory for an enterprise to use, but enterprises, like a lot of them don't write tests. Like if you haven't even got the frontier thing, you're not gonna get there. So I guess that's that gets into the go to market concerns. Ankit Jain (26:18.

593) Mm-hmm. Geoffrey Huntley (26:18.624) is you want like the some of the best engineers to actually if you're building a start up in this space for verification, you're not gonna be able to hire You you just won't have the budget to hire them. The next step is let's say you build a dark factory.

What will happen is a enterprise will be like, we're an airline. This is how we build our software. So they're gonna want to customize the dark factory on how they specifically do software. That's neither wrong nor right at sense.

But what they'll try to do is they will force the factory To be like them. And if like them is literally, they don't do tests, they've never heard about test generators and all like formal proofs, et cetera. Like, you're just gonna get garbage in, garbage out. So with this, we call this like a slop factory.

We call this a slop factory. And I have generated a lot of slop. And like anyone who has built these. these software factories is a lot of slots.

So there's a gonna be a lot of vendors bringing out software factories to market and they'll roll it into enterprise. And I guess we're gonna see hear a lot of noise. They're like, AI doesn't work. And they're gonna go, look, this big brand company that with this huge big brand credibility, maybe like I don't know, ATT.

Hi ATT. I have known no one at your company, but let's say it. Ankit Jain (27:38.06) Mm-hmm.

Ankit Jain (27:53.711) Ha ha ha. Geoffrey Huntley (27:56.176) Big brand pedigree.

They're like, Agenic coding doesn't work. But like anyone close to it, like, yeah, no no crap. It didn't work before AI and then afterwards. I don't know anything about ATT but like that's an example, right?

So I have my concerns that a software factory is essentially a C and C machine. And without training, a lot of people are gonna cut off some hands. It's like this huge power saw type thing. It's a t it's a so let's play around quickly.

Ankit Jain (28:25.73) Hmm. That's a nice analogy. Geoffrey Huntley (28:40.

09) Let's play around quickly. Like something I've been unescapable from my head ever s for a while now. Like I think we talked talked about it at HeavyBit. I if we pull up the family tree of operating systems Ankit Jain (28:50.

68) Mm-hmm. Geoffrey Huntley (28:57.12) And what I'll say will probably piss off a lot of engineers or software developers because it i it they'll see it as attack. I want you to turn that off.

If we look at and just think about from first principles, if we look at the operating tree of family system operating tree like you can see that eventually there was a convergence event. We used to have Solaris, AIX, HBUX. Ankit Jain (29:21.58) Mm-hmm.

yeah. Geoffrey Huntley (29:27.484) True sixty four, I used to do all this stuff. And eventually it converged down to Mac Linux Windows.

Ankit Jain (29:33.614) Mm-hmm. Geoffrey Huntley (29:36.528) Well, when are we going to converge programming languages?

Is a very interesting thing to play with in a thought domain. So I'm gonna say the next thing is flamboyant. Ankit Jain (29:41.932) Interesting.

Geoffrey Huntley (29:51.342) I reckon that Ruby is Solaris. Ankit Jain (29:56.013) Huh.

Okay. Geoffrey Huntley (29:59.008) And it it exists and maybe some people are still using Solaris, but it's very rare. It's very rare.

And a Ruby developers, like, Jeff, why are you attacking Ruby? It's like, I'm I'm not. I also think Python is Solaris. You're like, Jeff, why are you attacking Python?

It's like I'm not. I just I I anyone knows that retrofitting types onto a dynamically typed language is near impossible. It's just hard. So if we want to talk about dark factories and getting there, we've got to be thinking along the types of verification.

And types are a form of verification. Doesn't mean the right thing's being created, i.e., solves customer problems, but it does mean that it won't magically convert Ankit Jain (30:48.

969) Mm. Geoffrey Huntley (30:57.764) A bullion to a float. So and it turns out the these LLMs are somewhat drunk.

And they love doing that conversion. They love doing these upcasts. and if your type system prevents those types of things and those category problems, then there's less that the agent needs to reason about, and you have more surety when you read the code that it works. No.

Ankit Jain (31:01.934) Sure. Ankit Jain (31:24.43) Mm.

Geoffrey Huntley (31:26.522) So I I deeply suspect that there will be convergence of programming languages. Geoffrey Huntley (31:35.028) deeply suspect there's gonna be a convergence of programming languages and what comes next is not designed for humans.

It's not designed for humans. Ankit Jain (31:35.297) I can Geoffrey Huntley (31:51.32) Specs, programming languages are specs for machines.

Let's get let's get that clear. Markdown are not specs. Right? it's just this weird thing that humans use to the next.

The I think what comes next is if we look at all the specifications for machines, also known as programming languages, that are all been designed for human ergonomics. Geoffrey Huntley (32:18.876) They're being dumbed down because a human wouldn't be able to adopt. Like, I guess the classic top topic is what what's a monad?

What's a functor? Everyone's like, what the hell are you talking about that for? Like and mo it's like, no, we can't add monads to our program. Like no one will be able to understand it.

Like like this language over here added monads and no one's using it yet. And look at all the burrito jokes. So Ankit Jain (32:43.064) Ha ha ha.

Geoffrey Huntley (32:45.616) Deliberately in language design, we've dumped it down to lowest common denominator. Ankit Jain (32:50.05) Mm-hmm.

Geoffrey Huntley (32:52.028) Well Geoffrey Huntley (32:55.568) We don't need to do this anymore. So I highly suspect we will either see an old language that was considered too hard to adopt by humans, but's very good on the verifiability domain, or a a new language programming language for machines.

And the spirit of that language isn't we're gonna slow down development and we can't make breaking changes and what us have you. Ankit Jain (33:08.792) Hm. Ankit Jain (33:14.

422) Interesting. Geoffrey Huntley (33:25.838) It's like, no, we can make a breaking change. We can do Python 2 to 3 and we get everyone upgraded within a month.

Like the type can you imagine Python 2 to 3 breaking change happening in a month? Because you can literally now ship a fixed pack agent which will autocode migrate. A programming language d author could make such a thing, a skill or what as have you, and you run it with an agent. Ankit Jain (33:33.

357) Hm. Geoffrey Huntley (33:54.054) You wake in the morning, you've been gone from Python two to three. But if you want to do that type of thing in the Python community now, it's like, that would never happen.

Like we might break people. But the thinking is different. What I'm getting at is the pace of development of programming languages is all and every the committees, it's all designed around the human. I think we're going to see some sort of weird alien type thing.

that's gonna feel alien to us. We won't be able to really understand it, but we can prompt for understanding. Ankit Jain (34:18.87) Mm.

Ankit Jain (34:26.346) Interesting. I mean you could argue the instruction sets were those programming languages at some point that the computers understand, but humans don't. Geoffrey Huntley (34:41.

712) Yeah. So th there are people playing around. On my GitHub you'll find a one of the Dropbox engineers, they forked Rust because they didn't want to deal with the Rust community. Ankit Jain (34:52.

631) Okay. Ankit Jain (34:56.458) Ha ha ha. Geoffrey Huntley (34:58.

2) and that's not a stab at the Rust community, but like they don't want to deal with c with committees. Committees suck the the life out of it. And now we have a formally verified Rust tool chain. Like a a a a a as in not to maximize rust or rewrite in Rust, I'm not anything but that.

But I'm getting at there is you can just do things now at a pace without permission, structures, etc. I have my own fork of Rust which added high kind of types. I think if you Google high kind of types in Rust, you'll find it died in committees and different things like three or four years ago. Ankit Jain (35:38.

7) Hmm. Geoffrey Huntley (35:40.516) Like you can just do things now at a pace that i this that is intense. so the the there's people playing around with formal verification.

So like you also find on my GitHub a like a model checker, like like TLA is normally external. What they did was they did ralph loops with all the model checkers. Ankit Jain (36:00.866) Mm-hmm.

Geoffrey Huntley (36:09.092) And they turned them into they internalized them as r as rust cargo crates. Now the reason they did that is because then they can do they can move the proofs to be macros seeing side by side the c application of logic that that's invoked during compilation time. Ankit Jain (36:30.

187) Okay. Geoffrey Huntley (36:32.102) single sources of truth, instead of having it four or five different files split everywhere, the i they're actually inlining within a programming language. And I'm not saying that we should all switch to Rust, we should all use this.

This is all highly prototype, but they're people playing with these domains and trying new and interesting things. Ankit Jain (36:36.461) Wow. Ankit Jain (36:52.

974) Very interesting. Geoffrey Huntley (36:55.438) like have a look at some of the work that Eric Meyer's been publishing on ACM for the last two years. He's been talking about a need for a new language, for LLMs and that's and his work in verification.

And maybe it is a new maybe it is a new instruction set. Ankit Jain (37:07.095) Interesting. Ankit Jain (37:20.

59) Or maybe it's just English. okay. I know we are coming close to end, but I do want to at least cover one more topic, which we are already somewhat somewhat revolving around. we are already talking about you know writing code without really ne going needing to know it in details, in some ways, like you know, being able to transform one framework to another.

Where does that leave the Geoffrey Huntley (37:29.66) Yeah, what do you want? Ankit Jain (37:49.369) the review part of it.

Right? Like I understand you're talking about verification and there is like lot of obvious reasons why verifications are important. But do we need human reviews? Geoffrey Huntley (38:01.

908) it's hard to give a concrete answer to that one. okay, if you're a early stage startup and going to market without revenue, don't do review. Don't even do pull requests. Just literally push master.

Ankit Jain (38:15.566) Mm-hmm. Ankit Jain (38:23.277) Okay.

Geoffrey Huntley (38:24.54) Pushmaster. So like I used to be an engineer. I was with SourceCraft, we made AMP code.

that's post revenue, but that's a that's a team with the context that they're principal software engineers. And we did no zero code review. We're talking like a tens of millions of dollars a year annual revenue, brand new start up type thing. I don't know the specifics now.

Ankit Jain (38:32.12) Mm-hmm. Ankit Jain (38:46.039) Interesting.

Geoffrey Huntley (38:55.77) We did zero code review. What we had was this accountability culture. Geoffrey Huntley (39:04.

142) If you broke it, then you just you just you f you you put you own it and you just instead of spending all the time and figuring out like accountability or like blame culture of like this, that, that, that, that, that, or chasing around people for code reviews. We're just like, Yep, I'm on customer support. I broke it, I fix it, and then you get a fi spocus you focus on time to fix. So we could get a new release out.

Ankit Jain (39:08.558) You have to own it. Geoffrey Huntley (39:33.786) within three minutes.

Like actual deployed update in VS Code within three minutes. so in that case, because you design your engineering culture around, you don't really need code reviews. but there is, of course, compliance frameworks that necessitate it. So I think the most nuanced take here is Ankit Jain (39:39.

053) Mm-hmm. Ankit Jain (39:50.264) Okay. Geoffrey Huntley (40:04.

602) reducing the need for human intervention as a framework. Ankit Jain (40:10.24) Okay. Geoffrey Huntley (40:11.

546) Meaning if you were to change building billing logic, then you should force a conversation about code reviews. You change a database schema, you should force a conversation about it. But if it's a marketing copy update on your website, why are you asking for a human pull request review? Like and then basically doing a risk based can it auto ship and using agents as much as possible?

Ankit Jain (40:25.208) Okay. Geoffrey Huntley (40:38.94) Like it's not a binary type thing, it's a very nuanced type thing that depends on the particular domain.

But things that there are people out there you're using their products. They aren't doing code review. What they have is engineering excellence. Ankit Jain (40:58.

464) Interesting. I think you touched up on the idea which I was also going to bring up. my thought process on this is could reviews have a angle of knowledge sharing, which in again organization Geoffrey Huntley (41:15.1) It's always been bullshit.

It's always been bullshit. Go speak with a junior or and learn how they really feel. Like this the the where it's actually been like I'm 43 now. We've always sold that code review is meant to be for mentoring and knowledge sharing and all these things.

it does Ankit Jain (41:30.296) Mm-hmm. Ankit Jain (41:36.878) Mm-hmm.

Geoffrey Huntley (41:44.412) Serve a interesting thing of code review, so of knowledge sharing. Like it it does provide a synchronization event knowing of a team knowing an incrementers as it is. That is valuable, but doesn't mean that we have to kill that completely.

You can just literally have a living, breathing architecture document that an agent maintains. There's your lookup source. You don't need to have this, I was off on annual leave this week. And everything's changed.

my god, I don't know understand how that all works. I've got to go look at the code, the previous com just the knowledge increment is falsified in my head, because you can literally have an agent that maintains this doc technique documentation for you, including all sequence diagrams, state machines, all the rest. You can automate that stuff. So that that's one half of it falsified in my head.

Ankit Jain (42:24.833) Interesting. Geoffrey Huntley (42:41.466) Now the other path that I say is bullshit is the mentoring aspect.

That's the quiet that's the quiet that's the loud thing that we like to project. But the reality is go ask a junior, go ask a woman in engineering how they feel about that question. And you you might tell that it's like it's mainly used in our industry for bullying, unfortunately. It's very rarely used for uplifting Ankit Jain (42:46.

67) Hm. Ankit Jain (42:58.766) Mm-hmm. Geoffrey Huntley (43:10.

396) coaching conversations. Only in a few select c companies were they really used for mentoring. very rarely. Most of the time it's like you can tell if a company's they like you open a change set early or pull request early in draft.

Not when it's done, in draft and they they they go from a very pairing collaboration journey along the way. That that's different. Ankit Jain (43:22.178) Interesting.

Geoffrey Huntley (43:40.998) I'm talking how ninety-nine point nine nine percent use. Hey, I created the pull request, DM on Slack, can you rub a ticket, please? And and next thing you know, you get all these, I'll remove the white space, tabs, what about this over here, that over there?

And it's it doesn't make sense anymore. Like it it it really doesn't make sense anymore. The I think the mentoring aspect, the the craftsmanship aspect is kinda unsolved. Ankit Jain (43:49.

334) Mm. Geoffrey Huntley (44:09.542) But we like to pat ourselves on the back to say that code review was how we is how the apprenticeship was done. It's rare.

It's very, very rare that it actually probably was done. Ankit Jain (44:20.289) Interesting. Okay.

Okay. I have a lot of thoughts on this, but I think we may have to do another session on this. but let's wrap it up here. So Geoffrey Huntley (44:32.

736) dude, we can just keep on riffing and you can kind of clip up the questions as need be and compress time. Ankit Jain (44:35.31) Are you sure? Like I mean we are you don't have anything right after this?

Okay. Okay, perfect. So I have some opinions and this is probably where we might debate a little bit. when I say knowledge sharing, for me the part which is important and where also typically agents cannot do a good job is understanding how your code evolves over time.

Geoffrey Huntley (44:41.274) Nah, I'm just hanging out. Ankit Jain (45:04.632) There's a lot of decisions that organizations make.

Right. So if you're looking at a state of code at any given time, versus if you look at like how the code evolved from day one to today, you'll have two different understanding of the word. So I think even if you look at architectural diagrams, you still need to know why certain decisions, like maybe we moved from, I don't know, like Kafka to SQS, some random example. Right?

Like some of these things which you have possibly like killed over time or moved away from or evolved from, those kind of things generally get lost in the architectural diagrams. And and and these are the things which are like tribal knowledge, sort of like you know, places where you actually get to learn from others. And maybe this might be solved in the future. But I think there are like these kind of like micro decisions which actually get shared through the process of review.

Now, my thought process on this is it doesn't have to be a code review, but there has to be something that gets reviewed. Now, the something could be like it could be just like a Geoffrey Huntley (46:12.826) It has to b I would agree. Like it i it it doesn't have to be.

Ankit Jain (46:16.15) It could be just like a behavior. It could just be like understanding of the architectural changes, right? Like maybe it's like a 10 bullet point summary of like what changes you're making and why it impacts our business.

And I think just being able to share those parts and being able to see the evolution makes things more interesting versus like I completely agree on like, you know, the bullying part of like actually like the code standards and all of those things. I think those things are like very mechanical, you can, you know, in some ways codify it. But like the judgment and the decisions part are some things which are still worth discussing. Geoffrey Huntley (46:57.

916) Absolutely. I don't think that's possible to actually automate and it's something that shouldn't be. I'm tick just kicking off the r the rhombic just kicked off. so yeah, no, I'm actually in agreement with you, Ankit The the too often we try to codify like Ankit Jain (47:07.

48) Mm-hmm. Yeah. Geoffrey Huntley (47:25.052) Codify like this happens in code review, code review is the thing, but you're correct in saying it's the better thing.

It's the it's the tactic knowledge. There needs to be something in a way where these are discussed and these are distinctly human. This is, I guess, what SF would call taste topics. but if we split this around just a little bit, historical kind of rarely doesn't matter.

for on architecture. What matters is the current state of the architecture and where you need to go and whether it's working for you. Ankit Jain (47:54.37) Hmm.

Ankit Jain (48:03.022) Hm. Geoffrey Huntley (48:03.58) So let's look at the let's look at the architecture topics.

Okay, if you have a you have a vanilla web app running in a Docker container and it sends a a the users registers an account. Well if that sends the email in proc. Ankit Jain (48:17.752) Mm-hmm.

Ankit Jain (48:30.862) Mm-hmm. Geoffrey Huntley (48:32.392) Then that that's fine.

It works until it doesn't. Because you do a deployment and that terminates mid-improc, or maybe there's some sort of outage type thing. Or like when there's an outage, your automatic retries it causes like you to exhaust your thread pool type topics. Like this is an architecture question, right?

Well You can literally find an application like this, and most applications are like this. Like out this, if we're real. You can literally just create a skill pack for these archite these architecture questions and go, Hey, how could I split this up? Should I do some sort of like event bus?

Should I use some sort of durable execution saga type pattern? Maybe there's a SAS bender I can buy that would allow me to move Ankit Jain (49:08.014) Mm-hmm. Ankit Jain (49:16.

204) Mm-hmm. Geoffrey Huntley (49:32.056) sending email from instead of being in proc to be out of proc and have dead letter cues and what you could that's now s that's now l that's literally a prompt. That's a prompt pack to an architecture.

Well that same prompt pact on architecture can be ran on each increment of changes. Ankit Jain (49:52.023) Okay. Geoffrey Huntley (49:53.

774) And part of the automating part of the code review. So what I'm getting at is the architecture side can be automated in that sense. It's not whether you it doesn't the human still has to decide what the architecture should be. And they design that architecture, but the entire like the burden of maintaining that architecture vision in your head can be codified.

And then the maintenance of that. Ankit Jain (50:09.579) Exactly. Geoffrey Huntley (50:23.

686) Th the the the pain of teaching people and ensuring it maintains that's automated now. Ankit Jain (50:31.541) Yes and no. Geoffrey Huntley (50:31.

63) It is all the the the the education aspect is all like I think this violates this principle. Like you can codify that. It doesn't mean a person can just like or an agent can't just ramp past it. But the point being is it's easy to like historically run this through like every like as the current state of the code through this automation pack type things.

So we're we've got these Ankit Jain (50:38.008) Mm-hmm. Ankit Jain (50:54.83) Mm-hmm.

Geoffrey Huntley (50:59.608) Yeah, we've got this like you can taste it. You can taste these these concerns are you basically have automated rumbas to handle these concerns to remove the bo the toil from you. But y it's not a complete done automation as much as I might say, but it it in my head it's done because the toil is gone.

Ankit Jain (51:20.43) Mm-hmm. Geoffrey Huntley (51:21.402) We're able to automate the toil right way.

Now let's look at the other thing. Architecture sometimes Ankit Jain (51:26.498) I mean, can I first go into this before you go? So one caveat I will mention, just to kind of like double-click on what you just said.

I agree with all of the things, except this is not one set and done. Your organization evolves. Imagine fr imagine my team, right? Like we are a team of seven people.

You know, if our emails Occasionally fail to send because our architecture cannot handle it. We can live with it. We don't need to go and redesign everything. But maybe we get to a certain scale where this becomes a serious enough issue.

Right? So the point I'm making is if you are a sales force versus if you are a five-person startup, you're going to make very different architectural decisions. That essentially means is as your organization evolves. Geoffrey Huntley (52:16.

816) Course. Ankit Jain (52:21.122) you have to continue evolving these architectures, which is why understanding kind of like also the the direction you're going into and like evolution is somewhat useful. Geoffrey Huntley (52:34.

332) th again that can be cod the it can be codified. Literally it can use codified as a skill of markdown. Like this is where this is where it is, this is where we're going. But the way that architecture is done in enterprises, even like say Salesforce, it's you know, in it it's slow committees, it's consensus culture, it's political buy-in type topics, all that can be erased in my head.

Ankit Jain (52:35.202) Because that's how you're making the trade offs. Ankit Jain (52:43.734) Yes.

No, a hundred percent. Geoffrey Huntley (53:02.52) because we have autonomous Roombas who can o o audit this stuff, propose what it would look like in the future, and not just generate the code, we could actually connect in data sources such as we can connect in data sources such as like honeycomb or sentry or what else have you to actually provide data insights whether the architecture is actually even working. Ankit Jain (53:17.

527) Okay. Ankit Jain (53:24.802) Yeah. Geoffrey Huntley (53:32.

57) Like l literally it's like the It's it's gonna sound like full on psychosis in a sense, but like l all the burden of being a principal engineer is gone. The responsibility's not. Ankit Jain (53:54.432) Interesting.

How has the burden gone? Geoffrey Huntley (53:56.144) The the the the the hu the a lot of the human toil topics of having to repeat yourself again and again and again, the education, the teaching, the why, not just the how. Ankit Jain (54:02.

231) fair. Ankit Jain (54:08.463) Sure. Yeah.

The effort is gone, but the the knowledge is still important and the the thought process is Geoffrey Huntley (54:14.224) Very, very, very, very so a lot of the human capital toil is gone, but the craft is still very alive, but the craft has never been about typing the code into an IDE. Ankit Jain (54:27.821) It's probably even now more important.

Geoffrey Huntley (54:30.148) It's way more important. because th the these power tools h held the wrong way will chop off your hand. Ankit Jain (54:40.

591) Mm-hmm. Geoffrey Huntley (54:42.8) So yeah, the historical why something was as it is in architecture, I would debate and contest, it doesn't matter. It's a historical artifact, it's something we tap onto our shoulders and maybe it's a story that it is like, y y you you built this and that this was like ten years ago and the person go, Yeah, I built that.

Ankit Jain (55:07.37) Mm. Geoffrey Huntley (55:09.916) Like it's a wartime campfire story.

Like it doesn't matter. What does matter is does it work? Does it meet in work meaning? Does it meet does it make allow us to make money?

Does it it is does it actu is it reliable in allowing it to make money? That's a subcategory of it. and if we if we were to extend it, is it extendable? Like like is it rigid or it can be done.

Now where historical tactile knowledge of how things were and w w were I think Night Capital is the classic example. They reused a feature flag and blew up their firm. we were talking they incinerated they incinerated like look it up, but I'm using bullshit numbers 'cause it's early. But I think they incinerated like Ankit Jain (55:55.

502) Okay. Geoffrey Huntley (56:09.436) 250 million in an hour. It was it was look it up.

I I'll link it to you, Ankit. the because they reused a feature flag. So they had a system called SMAS, and in SMAS they reused a feature flag. And in that feature flag, well it's the flag is really a binary opcode.

And Ankit Jain (56:12.483) Well what? Okay. Geoffrey Huntley (56:38.

854) They reserved a binary opcode and for an old system used for auto trading. And when they did that, they like never used this one again. But they ran out of address space and a software de and they went through the entire organization, made sure this old trading algorithm is gone. And then they reused that opcode because they're out of time.

They're out of address space. Ankit Jain (57:03.759) Mm. Geoffrey Huntley (57:06.

649) And one of their deployments failed and one of the servers still had the old code on it. And next thing you know, they lit up that feature flag and they incinerated like a lot of money really fast. Let me pull up the exact numbers. Ankit Jain (57:18.

295) Wow. Four hundred and forty million it's in front of me right now. In forty five minutes. Geoffrey Huntley (57:22.

052) Yeah. Yeah. So like that is wild. there are many engineering failures there, in the sense of like it's probably not a good idea to reuse feature flags type things.

Now I see a perfect use of an agent there is has this been previously used? Has this feature flag been previously used? Ankit Jain (57:26.519) Wow.

Geoffrey Huntley (57:52.038) that doesn't need necessarily prescribe the agent need to be AI. It could be just some sort of lookup source to launch darkly type topics. Or maybe like, is this feature flag or this op code used anywhere in our entire code base?

Well that's an agent. You're repping an agent for the code base. There there's so many things here. Ankit Jain (58:13.

347) What about like you were mentioning about extensibility? How do you actually can you build an agent for that? Checking whether it's extensible. I feel like it also has a lot of business context, like extensible in what way?

Geoffrey Huntley (58:21.83) Yeah. Well Geoffrey Huntley (58:30.17) No, that's exactly you.

So extensibility is one of those words like quality that we we kind of throw around the world between each other and have these fights. It's not extensible, it's not testable, it's tech debt and all these things. It's the the the it's the the sweet somethings we say to each other to justify doing something unhinged like rewriting the code base and Rust. Let's let's let's let's instead focus on the word maintainability.

Because we go, is this not maintainable? Well, by who? By who? By who?

Why is an human the if all that code is generated by an agent controlled by a human operator, then previously the lens for maintainability was focused on is it maintainable by a team of people? Do they have the tribal knowledge to be able maintain it? We we covered this for the increment. Or maybe the correct lens of maintainability is can an agent understand it?

When the agent is working in that code base, in this particular one, is it able to achieve a happy outcome with minimal human intervention and supervision? Ankit Jain (59:48.943) Mm-hmm. Ankit Jain (59:58.

572) Interesting. Geoffrey Huntley (59:59.96) Why is one company with one their code base running their agents, they're they're having great outlier success and another company is not? Like it they're both using the same model.

They might be using GPT5 or they might be using Opus. Why is one getting success and one the other one's not? I think we are gonna have this new notion of maintainability by the agent. And that's through practices of practices within the code base.

It turns out garbage in coverage results in garbage out. These LLMs love to copy and paste patterns throughout the code base. So if you got junk in your code base, it's going to produce a lot of junk. So hygiene of code base is really important.

And there's so much you can do as as a like as as an engineer to automate these types of things. This is why we've got language analyzers. Ankit Jain ( ) Interesting. Ankit Jain ( ) Mm.

Geoffrey Huntley ( ) and you can express queries saying that we don't want this type of infraction in the code base anymore. in very concrete and abstract terms. And you can cause that to fail tests, you can cause that to fail git gabits, etc. Like you can automate all these things.

People who understand this will understand that designing for agent maintainability over human maintainability is going to be really important going forward in my eyes. Ankit Jain ( ) Very interesting. So where does that leave? Are you making a case for AI slop being okay?

Geoffrey Huntley ( ) I am not making an excuse for slop being okay. I'm saying that everything has changed and going back to typing in a keyboard is no longer a an option. Like leveraging AI is the thing, but not complete slop, which is why I've been playing around with the different programming languages. I've been looking at like what produces slop, what doesn't, what is Solaris?

Ankit Jain ( ) Mm-hmm. Ankit Jain (01:02:03.63) Mm-hmm. Geoffrey Huntley ( ) type topics.

What is end of the line for a language? And how can we have a way where Geoffrey Huntley ( ) I don't need to or a human doesn't need to get involved as many times. Like w really like minimizing the amount of human intervention. I think that's really important.

And there's gonna be other languages that are gonna require a lot of hand holding in that regard. But at no point am I saying slop, like like YOLO slop, like like I've done slop. cursed lang is the perfect example of slop. But that was generated with three point five and that was Ankit Jain ( ) Hm.

Ankit Jain ( ) Interesting. Geoffrey Huntley ( ) It's like a year a little bit over a year old now. And it was a joke. Can I can I rip a f rip a prompt into a coding agent and and run it in a loop of three for three months saying, Can I have a Gen Z programming language?

Like it's it's it's it's slop with no verification. but playing around with like the compiler as verification of itself. I did it three times. I think it's broken right now.

Ankit Jain ( ) Yeah. Geoffrey Huntley ( ) But like I did it in C, then I did it in Rust then I did it in Zig. I should never have done it in Zig. That is not a stab at Zig.

It's just that Zig does not have the back pressure off the type but the Rust type system. And I was I almost had it completely successful. like w like up to stage three with Rust then decided, ha, I'm just gonna rewrite it. Tokens are free, I'm gonna rewrite it in Zig.

And then it it it I could see the difference in Ankit Jain ( ) Ha ha Ankit Jain ( ) I have never looked at that curve. Ankit Jain ( ) Hmm. Geoffrey Huntley ( ) Verification. Like this is probably where the if someone wants to play with these topics, I challenge them.

Like go download Gastown, tell it you want to build something, ignore the UI layer. Just focus on backend layer. And then t try to do it in PHP. Then do it in Python.

Then do it in Haskell. Cool. You may not be able to understand Haskell, but If it compiles, it works. Now try evolving the application and notice that Haskell stays stays on the target line and you human doesn't get involved as much anymore.

But the PHP and the Python one, it eventually degrades into slop. Why is this so? Why is this so? Yeah, it turns out types are really important.

But types aren't Ankit Jain ( ) Interesting. Ankit Jain ( ) The typing gets Geoffrey Huntley ( ) the only form of verification, but they are a damn good form of verification. and also like I'm not even a prescribing Haskell, but Haskell has the properties of being a very terse language. It's very hard for human legibility.

But the terseness means that it uses less tokens to generate. So for input tokens and output tokens. And it it it won't put up with any bullshit. If it if i if if if you don't if it if it's not c correct, it won't compile.

Which means if the agent is drunk and the agents are drunk, you won't allow it to proceed with cheating. So like th these types of higher powered type things that are a little bit alien. And I'm definitely not prescribing everyone use Haskell. I want people to get used to this first principle thinking and like experimenting with these things and eventually you start to notice these patterns and perhaps we'll see a birth of new languages and we'll see a convergence event.

Ankit Jain ( ) Hmm. Interesting. I mean, I understand the desire to have new languages, but where are we falling short? Like, I understand, you know, typing and non-type languages are two different things.

If you're compiling something, there are certain levers and controls which makes things better. But that doesn't mean like, you know, you don't have slop. You don't have I mean, I would say like the other definition of slop for me is also tech debt. Right?

Like something which kind of like evolves, like harder to harder to think about extensibility, like something that you have to rewrite. But like you can also make arguments like why do you even care? You can just rewrite things very quickly. Geoffrey Huntley ( ) That's what I was about to call you up on, Ankit, because tech debt isn't one of those other words we flo throw around to justify rewriting stuff with rust by hand on a typewriter.

It's something that we do. Like it's it's bullshit. There's so many things that are falsified now. Like like Jared rewrote Bun from Zig to Rust in six days autonomously.

I've been doing that now for a while as well. Like the like the early like I I don't genera I I don't consume open source anymore. I literally like I remember an AMP code, I wanted a really good merma mermaid diagram. Ankit Jain ( ) Mm-hmm.

Geoffrey Huntley ( ) library, but it was it was only in Golang. So I went over to the Golang source code and I I I did a route loop. It's like I want this to be in Node because I want to be able to show and visualize Mermo diagrams in a Tui application in Node. And you just you just do it.

You just literally do it. Like like languages are fungible now. So this entire tech debt aspect, there are aspects l legitimate of debt. L Ankit Jain ( ) Okay.

Geoffrey Huntley ( ) but like I I see a place where we're at a place now, like the models are really good. You can transpile from one language to the other language autonomously. Like if you're a CTO and you're running six different tech stacks for no particular reason, you've got Scala in there, but not for data reasons, and then you've got some.NET in your company, and then you've got some like you got some PHP and it's like, holy shit, man.

Just like literally run a couple agents in loop and st start start strang automatically strangling and migrating so you get down to a consistent like tech stack. So your team that doesn't have this complexity the this this complexity. Like we can actually merge languages now. So like let's say that you're a let's say that you're a Golang shop, but you acquired a.

NET company. And you could see the.NET as to tech debt. what are you gonna do about it?

It's like run an agent. Run an agent and actually port it across to Golang. And then you apply standard engineering rigor is you can slowly strangle a service, like piece by piece by piece, auto autonomously. And I'm no way advocating for slop, but the things have fundamentally changed here.

Ankit Jain ( ) Mm. Ankit Jain ( ) Mm. Geoffrey Huntley ( ) Like a we're gonna see the bar for engineering manager or CTO type things, it's like Tell me how you've used AI to remove tech debt. And the stories are all gonna be there's g there's been six tech stacks and I got it to one in a month.

And if if that seems unhinged, because it is, but that's where we're heading, folks. Ankit Jain ( ) Hmm. So if you're making a case against tech debt, why does AI slop get different treatment? Why do you care about AI Slop?

Geoffrey Huntley ( ) good hygiene matters because agents are drunk and they will copy and paste bad patterns everywhere. The exact same way a junior would copy and paste bad patterns everywhere without understanding the why, which is where you we we did the mentoring aspects. So having hygiene it's it's it's about hygiene. Ankit Jain ( ) Okay.

So it's about maintainability. Ankit Jain ( ) But similar to tech debt, arguably you can also just clear out all the AI slop occasionally and be okay. Geoffrey Huntley ( ) Yeah. You can l literally run a a background cloud job with an agent and go find me this give me a report of this type of problem that you think might exist.

Or give me let's say you're let's say that you you got suspicion, you have an incident for this type of failure. And you got suspicion somewhere between your your six different code bases that this is not this is not just Ankit Jain ( ) Maybe Geoffrey Huntley ( ) an isolated thing. It's it's more systemic thing. Well express that as a prompt, run at an agent and get yourself a report.

Cool. Now what? That that would normally take that would normally take someone, maybe a month of human work, to identify this type of systemic thing. No, that time just got compressed into a day or two days.

Ankit Jain ( ) Mm. Ankit Jain ( ) Interesting. Ankit Jain ( ) It's too long. Geoffrey Huntley (01:11:22.

49) Like like it c can get compressed. and so you the idea of gathering knowledge type aspects, that can be automated. So you can make better decisions faster. And that gets into like what do you do with that?

And the th there's a couple y this is where the engineering brain comes in, ankit Do you express that as do you express the resolution with that as just having a prompt? Ankit Jain ( ) Hm. Mm. Geoffrey Huntley ( ) On a code review type aspect or the Roomba that automates things.

Or maybe it's best expressed more deterministically in the sense that maybe it's you're updating your pre-commit checks, and the pre-commit checks you've updated the pre-commit checks run your linting rules, and the linting rules have been updated a little bit more tighter. Or maybe it's like a class of tests were missing. So you generate those tests to prevent that class of failure to happen. Or maybe Ankit Jain ( ) Mm-hmm.

Ankit Jain ( ) Mm-hmm. Ankit Jain ( ) Mm-hmm. Geoffrey Huntley ( ) you you start getting into meta programming of the language itself and expressing that this class of problem should not exist. So like in the.

NET space we've got Roslin as a language as a language analyzer and you see use Roslin to introspect the code base itself and write language analyzer rules. And then all of a sudden you can start at a meta saying that, hey, look Ankit Jain ( ) Mm. Geoffrey Huntley (01:12:49.38) A domain object should never have a direct coupling to the database directly.

Like and express that as a general rule. Like as a domain object, it's it's it's not like an entity object. Well, before AI, it'd be it's really hard to write those type of rules for the normal person. You can literally just prompt the creation of these rules by an agent, and then it's not that the agent runs it every time, but it makes it very easy to Ankit Jain ( ) Yeah, regular scans.

Mm-hmm. Geoffrey Huntley ( ) Create these types of things and then just commit the it commit that and that's always there. Like i it's in it's in it's it's not s it generates slop. I'm not advocating for slop.

I'm just literally there is no going back and we need to engineer the the need for us to babysit these things. And what is needed to stop us from needing to babysit them. Ankit Jain ( ) Ha ha. Ankit Jain ( ) Right.

Right. Ankit Jain ( ) Interesting. So I think one part or like one point which I just thought about as you were discussing this is a difference between AI slop and tech debt is tech debt is known. You know to some degree why and what the debt is.

In slop you might not. Because if you imagine a world, imagine a world like where you've stopped reading the code entirely. Geoffrey Huntley ( ) Look up. Ankit Jain ( ) If you stop reading the code, like how do you even know what is slop?

Geoffrey Huntley (01:14:21.68) Yeah, so why tech debt is not no necessarily known. What call what destroyed white cap white capital was tech debt that was unknown. Ankit Jain ( ) Fair.

Geoffrey Huntley ( ) Go l go go look up the videos on YouTube that go into very deep and like how exactly from engineering to software engineering failure. It's fascinating. so I'm definitely not advocating that we don't read the code. I'd like to get to a point where we're not reading the code.

Ankit Jain ( ) Hm. Ankit Jain ( ) Mm-hmm. Geoffrey Huntley ( ) Like when Curse was built, I was sitting there watching the YouTube stream. I just sit in there watching it.

I'm watching the failure modes and how and not really learning and understanding what's going on. I still like I run an agent like two or three at a time that that had my attention that I need that I really need, and then a couple in the background for researchy tasks. I don't need to watch the agent for the researchy tasks, but like for the critical thing that I really care about shipping today, yeah, I'll watch it. I'll shape it.

but I I've learned that a a bunch of precursors the amount of effort you put in into the research phase and the amount of context engineering to create that plan before you let it go really determines a lot of your outcomes. So if you inv invest in that area and develop that intuition, you do get better outcomes. Now I guess it it depends. Like Ankit Jain ( ) Mm.

Ankit Jain ( ) Mm-hmm. Geoffrey Huntley ( ) I would not if I'm generating some JavaScript, there's no way that I feel like I could be hands off. But if I'm generating Haskell I definitely know I can be hands off. Ankit Jain ( ) Hm.

Ankit Jain ( ) That's because of typing. Geoffrey Huntley ( ) typing yeah. Doesn't mean I'm generating the right thing to solve the business outcome for revenue. That's a different question.

That's got completely nothing to do with the tech itself. Ankit Jain ( ) I mean I I kind of like again, I I've this is maybe going too deep into the same rabbit hole, but I feel like maybe you are overemphasising on just the typing. I definitely see know that typing hundred percent has a lot of value. But I don't think that itself is enough.

Geoffrey Huntley ( ) I don't think it is enough, but it it's what I'm saying is it should become the bare minimum. Ankit Jain ( ) That's fair. Geoffrey Huntley ( ) It's the bare minimum now. Like it the the Ankit Jain ( ) I mean it's like same as like having your you know, everyone needs a CI system or like kind of like some sort of a testing system.

Geoffrey Huntley ( ) Correct. It's i if I were to say that you don't need a CI system, people are like the hell? What do you mean you don't have a CI Like you're saying you don't need a CI system? Well, if you to say dynamic pro dynamic like dynamic pro type programming languages are dead, like even now, you you'd be a heretic.

It's the same level of heretic of saying that you don't need CI, but eventually it's gonna become such tactic understanding that yeah, we we need CI. Yeah, but like it it made sense to retire a class of how we did programming languages. So the Ankit Jain ( ) Right. Right.

But even if you have a CI it doesn't mean you will never have bugs in your system. Geoffrey Huntley ( ) Correct. And the same with types. The same with types.

But what it did was it removed a class of failure domains, it removed a class of failure domains and prevent that class of failure domains from happening again through automation. Geoffrey Huntley ( ) Turns out running your build and running your tests after every change is the is the equivalent of being a window cleaner and having a a a window cleaner with a safety harness and a rope attached to it. Every window you want to make every window that you want to clean, you know at least you're connected to a safety harness that will ensure if something goes wrong, you're there.

Ankit Jain ( ) Right, right. Fair. Geoffrey Huntley ( ) It'll it'll catch it'll catch you in a failure scenario, hopefully. But doesn't mean it always will.

Ankit Jain ( ) Right. Well, yeah. I think what was gonna say now? Geoffrey Huntley (01:18:51.

66) Probably bad analogy, but like I just what I'm getting at here is our job has always been about automation. Geoffrey Huntley ( ) and we're learning as a profession at hyperspeed how we can utilize this new substrate for automation. It doesn't mean that you we should solve everything by claude decord decord or codex to codex to codex. there's a good meme floating around right now that talking about the Holy Trinity.

Anthropic published a blog post and they Basically meant they put claude goes to claude, which goes to claude. So it was like kinda like the Holy Trinity. Like that that is an example of that's more art, that's more research right now. We don't know if doing agent generating code, agent reviewing code, and another diversity of different a different agent in there is the Holy Trinity, and that's what we needed for for automation.

No. It's It's gonna be a mixture of that plus classic engineering practices. Ankit Jain ( ) Mm-hmm. Mm-hmm.

Okay. Geoffrey Huntley ( ) The less that you get your agent to do, the better outcomes you get because the agents are are drunk. Ankit Jain ( ) Right, right. I think essentially you need more guardrails I I I would still I can still imagine a world where we actually stop reading the code in time.

Geoffrey Huntley (01:20:18.95) Yeah. Geoffrey Huntley ( ) Me too. me, absolutely.

Ankit Jain ( ) I don't think we are there yet. I don't think we're there yet, but I definitely think we will get there. Geoffrey Huntley ( ) Well so when I've when I've got like a library for one language I want to bring it over. So right now I'm playing around with Haskell again.

I previously it was O Camel. And there's a lot of stuff that doesn't exist in the Haskell ecosystem because humans don't write it. But agents are writing it for me. And I just take it from Golang or TypeScript and now I've got a Haskell library for it.

It's amazing. I don't necessarily watch it generate that stuff. But once it's done and it's shipped and I I I I I see it like I have a look at it and I have a feel for it and like do something. Well what I do do is I go back post generation when it's working and then I'm like, yeah, I have this I do read it.

But when I read it, like there was something when I was first into software development, we used to print out the code. And we used to start our day with a highlighter and a red pen and used to mark up the code on physical paper and used to plan what you're gonna do. This is the old school prompt engineering. And I find myself doing that as well.

So like if I was to change one thing in GitHub.com, I would improve the ability for you to read. code because right now it's not a c good code reader. And then straight there from the markup, I would be able to launch an agent.

Like it it'd be like a start of a drafting tool. Because I find myself reading the code post-generation, after it's like just before it's in production. Or like i you got your prototype, prototype, prototype, you just ship it, ship it, ship it, ship it, ship it. But before you Before revenue is attached, you should read it.

You should read it to understand it. and then you're like, it'd be good to fix that up. So you like it's very easy to make stuff malleable now. It's it's really weird to explain.

But like I definitely want to minimize the amount of time I'm read reading code, and I definitely want to optimize on how malleable that code is. Ankit Jain ( ) Hm. Geoffrey Huntley ( ) And I I find myself on a workflow basis getting back to the old days where I would start my day by reading code and going, that's a little bit weird, separate that, et cetera. But I don't just prompt that.

I'm thinking about like at a more meta level, how could I prevent that type of class of thing from an agent from doing that class of thing again, which creates even more back pressure, which means I don't have that value domain again. Ankit Jain ( ) Mm. Geoffrey Huntley ( ) Like so unless you study the output, you're never gonna get that meta of how do I stop this? And if you don't study the output then you're in slop town territory.

Ankit Jain ( ) Yes, yes, yes. So I completely agree on that. It's not something that you can just turn off one day. Right?

Like it's something you have to you have to build towards. It's something that you have to Geoffrey Huntley (01:23:49.04) No. Not yet.

It's it's something you build to w it's it's let's look at Gas Town. it's Y cur beautifully articulated what the best thing of Gas Town was talking about essentially a level eight and ran people through where are you in a skill level? Level one, level two, three, four, and like maybe you got y you finally you got YOLO off and Eventually you start juggling lots of agents together and even more agents. And eventually you go, This is fatiguing.

I need to have some sort of automation of the juggling of agents. So I that's how gas town comes to be. Gas town or that type of software factory is an earned privilege because you've been juggling plates and you know exactly what it's going to solve. But unfortunately, people jump straight to the gas town.

before they've learned about the failure domains of juggling many plates together. it's not plates, it's agents. Like yeah. so all these things don't just it's a journey.

there is no going back to how things were. we've got this new s form of automated tool. And I I spend unbelievable amount of time thinking about how do I stop this Ankit Jain ( ) It's a deeper J curve. Geoffrey Huntley ( ) drunk junior from making that same mistake again.

Notice I'm not thinking the next model will fix this. Ankit Jain ( ) Interesting. Geoffrey Huntley ( ) or the models will get better. No, so I'm not saying the models will get better or this model will make it better or this one has this less no, no, no, no.

I'm assuming always that the AI is going to be very unintelligent and we need to engineer our way so to create some more intelligence in the in in the in the rumbo itself. And if you do it in the right way, as the models get better, that even accelerates you even faster. Ankit Jain ( ) Okay, I think my battery's gonna run out soon, so let's Geoffrey Huntley ( ) That's all right. I told you this would be kinda unhinged, catch up.

so I'm just playing around and exploring the the maximal the maximal extent what is possible right now with what we got. Ankit Jain ( ) Right. I would at some point, I'll I'll I'll maybe kind of like, you know, in a week or two, I'll share some demos of some of the stuff that we are thinking about. I would love your opinions on that.

because effectively, I mean we're not doing formal verification, but there are like layers of verification that we are trying to build and there are similar ideas and concepts to what you're describing. Geoffrey Huntley ( ) You're probably doing from yesterday's holy trinity, Claude, Holy Trinity, like Claud to Claude to Claude. Ankit Jain ( ) I should look that up. I I I do not spend that much time on social media that I d miss out some of the, you know.

Geoffrey Huntley ( ) Jesus Christ. Okay, let's let's wrap up. actually I can share from here. Let me show you the Holy Trinity.

It's hilarious. Ankit Jain ( ) All right. Geoffrey Huntley ( ) Ankit Jain ( ) There is a share small button there. Geoffrey Huntley ( ) Yeah, I'm finding it.

Geoffrey Huntley ( ) the most important thing to convey is no one knows w ha have has any answers how this works and where this goes. Ankit Jain ( ) Mm-hmm. Ankit Jain ( ) I mean it is about experimentation, right? Yeah.

Geoffrey Huntley ( ) it's really about experimentation. if someone says that they have an answer, they're selling you shit right now. Straight up. and my biggest concerns is this is the Holy Trinity.

See it? Ankit Jain ( ) Did McWorkloads? Yeah. Okay.

Geoffrey Huntley ( ) God, Jesus and the Holy Spirit on the left there, God's Ankit Jain ( ) Yeah, yeah. Nice. This wait, this is this is n not their diagram, right? This is somebody's meme.

Interesting. Geoffrey Huntley ( ) but the but Geoffrey Huntley ( ) This is their diagram. So the answer isn't just a hundred percent this, but there is some truth here. There is some truth here.

But it's gonna be found at the intersection of both these worlds. Yes. Ankit Jain ( ) Mm. Ankit Jain ( ) Interesting.

Geoffrey Huntley ( ) Yeah. So having Claude generate some code, having Claude review that code, having Claude do what the heck it's doing in the in the final the final leg there, and just solving it or more tokens, isn't it. But as we previously were, isn't it either? There's an intersection.

You can you can do these topics of software factories and these holy trinity type topics. through proper engineering practices that enable it. But if you just adopt the Holy Trinity, you're s you you're just you you you're kidding yourself. So I think a lot of companies will adopt these software factories this year that are brought to market and we're gonna see disastrous reports, but really it's a more a reflection upon the company and their culture than the actual tool themselves.

And it's gonna be very noisy this year. Ankit Jain ( ) Interesting. I I don't think at least from my perspective that every company will be able to even adopt software. Geoffrey Huntley (01:29:46.

15) That's what I think as well. But if you're if you if you got two thousand two thousand people in a low two thousand engineers in a not so in a business with not so great margins and and you need those two thousand engineers to be able to work in that business vertical. And a new competitor comes in and they've got fifty engineers and they're they're gas towning and they think they're cracked it. Ankit Jain (01:30:01.

53) Mm-hmm. Geoffrey Huntley ( ) What does it mean to the thousand engineers? the two thousand engineers at the Delva company. That's the dynamics.

That's that's that's the fear factor. is some of your best engineers. Let's go there. Let's go there.

Why is it that the new minimum why is it that the CTOs of the companies that are meant to be doing the transformation of those companies are quitting those companies and joining labs? They're meant to be doing that transfer that J curve transformation, but they're quitting that those these are publicly listed companies. They're meant to be doing that J curve, they're meant to be doing that two thousand engineer, ten thousand engineer transformation with AI. Ankit Jain ( ) Right.

Yeah. Geoffrey Huntley ( ) They're punching out and they join the lab, and at the lab they're going to be working to destroy that company. You leveraging AI. That's that's the ultimate fear factor, is your best staff with the people with the most context leave your company and they start gas tanning their way to your profit margin.

Because it's no longer kind of hypothetical. Ankit Jain ( ) Right. I mean Geoffrey Huntley ( ) We're seeing these very public departures of people like GitHub, our synclets, GitHub. The CEO of GitHub, the person who has the responsibility to actually do the proper transformation, quits GitHub to build the thing that's gonna that and the pitches to destroy GitHub.

That was the first indicator. And there's there's many more execs that this is happening again and again and again and again now. They're meant to be the ones doing the cultural transformation and all the AI enablement and they realise, stuff it. I'm just gonna go raise sixty million and just destroy them.

Ankit Jain (01:31:55.16) Because it's Ankit Jain ( ) Yeah, transformations are definitely hard. Geoffrey Huntley ( ) Yeah. Alright, Ankitt.

interested to see what you put together because this is just a not a standard podcast. This is not a standard podcast. This is just a riffing. Ankit Jain ( ) Yeah.

Okay, all right. This is my this is my by far the longest podcast. Okay, let me just do at least a formal and exit. so thank you so much, Jeff.

This was again one of the longest and most interesting conversation I've had in a while. And thank you everyone who is listening. If you're interested in developer experience, work in DevX or platform engineering, join us at hangar dx. It's dx.

community, so we can Talk to people like Jeff and learn where the world is going. Thank you, Jeff. Likewise. Geoffrey Huntley (01:32:47.

1) Good seeing again, Ankert. Thank you everyone for tuning in. this is probably the most unstructured thing because it's just literally I I think this is where we need to be. We need to have more riffing and less pitching.

no one knows exactly how this is going. And anyone that says that they know exactly where it's going and how it should be is trying to sell you bullshit right now. But one thing's sure, as we were, is definitely not what it's going to be. Ankit Jain ( ) Mm-hmm.

Ankit Jain ( ) Or they're or they're selling tokens. Geoffrey Huntley ( ) Yes. Ankit Jain ( ) All right, Jeff. Have a great weekend.

Geoffrey Huntley ( ) See ya. Bye.