The B2B Podcast Index
Index
All categories
MarketingSalesSaaSFinanceHROpsLeadershipCustomer SuccessAI & DataProductStartups & FoundersRevOpsEngineering & DevTools
MethodologySubmit
Best of:MarketingSalesSaaSFinanceHROpsLeadershipCustomer SuccessAI & DataProductStartups & FoundersRevOpsEngineering & DevTools
An independent project byFame
SearchBest episodesGuestsInsightsMethodologySubmit a podcast
Index/Engineering & DevTools/ShipTalk
ShipTalk artwork

Beyond Dashboards: How AI Is Redefining Developer Productivity with Adeeb Valiulla

ShipTalk · 2025-10-17 · 37 min

0:00--:--

Key moments - from our scoring

Substance score

65 / 100

Five dimensions, 20 points each

Insight Density14 / 20
Originality12 / 20
Guest Caliber15 / 20
Specificity & Evidence13 / 20
Conversational Craft11 / 20

Adeeb Valiulla leads developer productivity at Harness and brings a unique perspective shaped by his experience building homegrown metrics systems at Sensomatic starting in 2016. Today, Harness helps Fortune 500 companies translate engineering signals - tickets, pull requests, pipelines, incidents - into business intelligence. The conversation tackles how AI is redefining productivity in software delivery. Traditional metrics like velocity and lines of code no longer tell the story; instead, Adeeb advocates measuring problem-solving capacity, throughput balanced with stability, and value alignment to actual business outcomes. Research cited shows 95% of teams use AI tools, yet 30% don't trust AI-generated code and 45% of AI deployments have problems - revealing a quality gap. Adeeb frames practical guardrails: shift-left testing with AI-powered test generation, feature flags for AI code, blue-green and canary deployments, and security reviews at scale. He distinguishes between AI-powered features (like IDE autocomplete) and truly AI-driven products that reshape entire workflows. The episode explores data governance and readiness as critical prerequisites for successful AI adoption, emphasizing that AI amplifies existing team strengths or weaknesses - broken systems just break faster with AI.

Key takeaways

  • →Productivity definition has shifted from output (lines of code, commits) to outcome-based metrics: problem-solving capacity, reliable delivery, and business value alignment.
  • →AI adoption amplifies existing team patterns - strong teams get stronger while weak teams get weaker, so fixing foundational issues before AI deployment is essential.
  • →Quality engineering must evolve from reactive testing to proactive governance using shift-left testing, feature flags, canary deployments, and security reviews 'at AI scale.'
  • →True AI-driven products reshape workflows entirely (managing PR lifecycle, detecting risk, assigning reviewers) rather than just sprinkling features like autocomplete.
  • →Evaluate AI tools on three dimensions: voluntary adoption beyond novelty, measurable impact on throughput and stability (not cost explosion), and downstream efficiency gains.

Guests

Adeeb Valiulla

Topics in this episode

DORA metricsAI code generationFeature flagsCanary DeploymentsBlue-Green DeploymentsDeveloper productivityEngineering intelligenceHarness platformShift-left testingPull request lifecycle automation

Questions this episode answers

How should engineering teams measure if AI is actually improving their productivity?

Evaluate on three dimensions: adoption (voluntary use persisting after novelty), impact (improved throughput and stability without spiking change failure rates), and efficiency (reduced toil without creating unsustainable cloud costs). If any check fails, it's not real productivity gain.

What are the main misconceptions about AI and engineering productivity?

The two biggest myths are that AI automatically equals productivity (it actually amplifies what you already have - strong teams get stronger, weak teams get weaker), and that individual output is the right measure. In reality, if one engineer generates massive code that the team can't securely test or deploy, it's waste, not productivity.

What quality engineering practices are mandatory when AI is generating code at scale?

Shift-left testing with AI-powered test generation, feature flags for AI-generated code (currently less than half of orgs use them), blue-green and canary deployments, chaos testing, and security reviews at scale - since 48% of orgs worry about increased vulnerabilities from AI code.

What's the difference between an AI-powered feature and a truly AI-driven product?

An AI-powered feature makes a task easier (like IDE autocomplete), while an AI-driven product changes how you fundamentally work - for example, managing the entire pull request lifecycle by detecting risk, generating tests, assigning reviewers, and reshaping workflows.

What data governance steps must teams take before deploying AI in their engineering workflows?

Start with data hygiene - ensuring clean, trustworthy engineering data is the foundation. Without good data, AI tools will amplify noise and broken processes rather than drive meaningful insights and improvements.

What our scoring noted

Our reviewer’s read on each dimension, with quotes from the episode.

Insight Density

14 / 20

The episode delivers solid, actionable frameworks around AI-driven productivity measurement (DORA, SPACE, HEART, FLOW integration; platform-first thinking; governance embedding). However, significant portions consist of soft reiteration and recap - e.g., host repeating guest points, lengthy preamble on seasons/frameworks without new tension or data challenges. Guest avoids false simplicity but also avoids pushing into genuinely surprising territory; most claims are well-known industry wisdom (AI amplifies what exists; speed without stability is waste).

AI has blown up that definition. Now, productivity isn't about typing faster, it's about Problem-solving capacity.
If AI helps code faster, but your change failure rate spikes. That's not productivity. That's literally chaos.

Originality

12 / 20

The framing of platform-governance-developer-experience as a triangle is clean but not novel; DORA/SPACE synthesis is standard practice; the distinction between 'AI-powered feature' (autocomplete) vs. 'AI-driven product' (workflow redesign) is intuitive but circulating. The unconventional prediction (AI-assisted decision-making overtaking AI-assisted coding) has merit but lands late and underdeveloped. Most arguments rehash established productivity doctrine without first-principles challenge.

So the difference is… Are you shrinking or sprinkling AI? Or are you re-architecting around AI?
Productivity is about outcomes in context.

Guest Caliber

15 / 20

Adeeb holds relevant leadership position (developer productivity at Harness) with credible background (Sensomatic practitioner, research publications, DORA/SPACE fluency). However, interview does not stress-test his experience or surface novel operational challenges he's solved. Guest speaks as subject-matter authority but without deep war-story specificity or willingness to expose hard tradeoffs. Solid mid-market credibility, not exceptional operator depth.

I lead the developer productivity function, where I help Fortune 500 companies translate these engineering data into business intelligence and outcomes.
I've run engineering metrics program at cybersecurity companies, API security companies. Gaming, hospitality, information technology companies

Specificity & Evidence

13 / 20

Guest cites specific data points (95% of teams use AI; 80% report productivity gain; 30% distrust AI code; 45% of AI deployments have problems; 63% ship faster; 48% worry about vulnerabilities; 70% worry about cloud costs; 94% rely on platform engineering). However, all numbers are sourced to 'Harness reports' or 'DORA research' without granular examples, company names, or timelines. No concrete case study (e.g., 'Company X had 45-day lead time, deployed AI, fell to 18 days but incident rate rose 3x, here's what we did'). Data is illustrative but not deeply evidential.

63% of orgs are shipping faster with AI, but 45% of AI-generated deployments have problems.
the 2025 State of AI-assisted Software Development by DORA shows that AI adoption Amplifies what you already have, which means strong teams get stronger. Weak teams get weaker.

Conversational Craft

11 / 20

Host asks logical follow-ups and avoids pure softballs (e.g., 'what misconceptions have you heard?', 'how do you balance dev experience with performance metrics?'). However, host largely validates guest claims rather than challenging them; no genuine productive disagreement or pressing on vague statements. When guest says 'AI only as good as signals it learns from,' host does not ask 'what does that look like for orgs with messy 10-year-old codebases?' Host recaps and affirms rather than deepens tension. Conversation is pleasant and logically coherent, but lacks sharp interrogation.

Yeah, I can relate. So previously, whenever I needed to, let's say, create a pipeline...
Yeah, and I think that's where I can relate what you said. It's not just sprinkling AI, but…

Conversation analysis

Computed from the transcript - who did the talking, and the words that came up most.

Most-used words

harness209adeeb204valiulla198dewan59ahmed59engineering31developer24productivity20experience17delivery16space16dora15data14code13pipeline13governance13

Episode notes

In this episode of ShipTalk , host Dewan Ahmed sits down with Adeeb Valiulla , a leader in developer productivity and engineering excellence at Harness , to explore how AI is transforming the very definition of software productivity. Adeeb shares his journey from building early engineering metrics systems at Sensormatic to leading developer efficiency initiatives for Fortune 500 companies. Together, they unpack how AI is changing what it means to be “productive” - from measuring outcomes instead of outputs, to ensuring governance, trust, and quality at AI scale. If you’ve ever wondered how frameworks like DORA, SPACE, and Flow adapt in an AI-driven world, or how to separate “AI-powered” hype from truly “AI-driven” value, this conversation is for you. If you'd like to appear on one of our episodes,

Full transcript

37 min

Transcribed and scored by The B2B Podcast Index.

WEBVTT 1 - > Dewan Ahmed: Good morning, good afternoon, good evening, time-appropriate greetings. My name is Dewan Ahmed. I'm your host. This is ShipTalk Podcast.

2 - > Dewan Ahmed: Where we talk about the ins and outs, the ups and downs of software delivery. With me today, I have Adeeb Valliulla, who leads the developer productivity and efficiency space at Harness. Welcome, Adeeb! 3 - > Adeeb Valiulla | Harness: Thank you so much, thank you for having me.

4 - > Dewan Ahmed: I can see fall colors out of my window, in, the east coast of Canada, beautiful New Brunswick. How is there, Adeeb? Do you see fall colors? 5 - > Adeeb Valiulla | Harness: Yeah, I'm dialing in from San Francisco.

Today, we have some rain, some outer cast, no sun today, but, you know, another good day to be here. 6 - > Dewan Ahmed: So you don't enjoy, I think, I guess, shoveling snow. Like, you're deprived of that pleasure. 7 - > Adeeb Valiulla | Harness: That is correct.

I… I had my fair share of shoveling snow, walking in the terrible winters when I was in Chicago, but now that's beyond me and enjoying San Francisco for now. 8 - > Dewan Ahmed: Hey man, if you ever miss shoveling snow, you can just take a short flight, come over here, I'll have a driveway full of snow. But, I don't want to scare our listeners and viewers out with all the snow stories. 9 - > Dewan Ahmed: What I do want to mention is we actually kicked off Season 4 with Nathen Harvey.

We talked about DORA and the framework to measure engineering productivity. Adeeb is another leader in this space of engineering excellence, developer productivity. So, Adeeb. 10 - > Dewan Ahmed: I want to hear about your journey.

Like, how did you end up in this space? Where did your career start? 11 - > Adeeb Valiulla | Harness: Yeah, sure. So my journey really started back in Sensomatic, where I came in as an engineer, passionate about the craft and honesty, just eager to grow in my own career as well.

At one point, I asked a pretty simple question. 12 - > Adeeb Valiulla | Harness: how do I know I am really making a difference? 13 - > Adeeb Valiulla | Harness: And the truth was, I did not get a clear or a measurable answer. 14 - > Adeeb Valiulla | Harness: That moment stuck with me.

15 - > Adeeb Valiulla | Harness: I started sketching a few of my ideas on a whiteboard, how I could actually show the business 16 - > Adeeb Valiulla | Harness: the impact I was making. 17 - > Adeeb Valiulla | Harness: While also surfacing opportunities to improve. 18 - > Adeeb Valiulla | Harness: Now, this was way back in 2016, long before developer productivity 19 - > Adeeb Valiulla | Harness: Engineering intelligence, efficiency, or common language. 20 - > Adeeb Valiulla | Harness: I had built a homegrown project that measured things like lead time, developer punctuality, delivery punctuality, quality, and efficiency.

21 - > Adeeb Valiulla | Harness: Now, it began as a tiny experiment with just my team, my product, and it caught on pretty fast. 22 - > Adeeb Valiulla | Harness: before long. 23 - > Adeeb Valiulla | Harness: all the business units at Sensomedic and every engineering team at Sensomedic were using it to understand not just what we were delivering. 24 - > Adeeb Valiulla | Harness: But how effectively they were delivering it.

25 - > Adeeb Valiulla | Harness: So that experience was eye-opening. 26 - > Adeeb Valiulla | Harness: It taught me measuring engineering isn't about vanity metrics, it's about crafting a shared language of impact that both engineers and business leaders can rally around. 27 - > Adeeb Valiulla | Harness: Now, fast forward to today… That's exactly what I do at Harness. 28 - > Adeeb Valiulla | Harness: I lead the developer productivity function, where I help Fortune 500 companies translate these engineering data into business intelligence and outcomes.

29 - > Adeeb Valiulla | Harness: Now, the mission is the same, just… just a little bit larger on the scale. 30 - > Adeeb Valiulla | Harness: We help organizations measure what matters, improve developer experience, and ultimately align software delivery with real business outcomes. 31 - > Dewan Ahmed: That is fantastic to hear, and we all have… have this goal to show what we did, right? So let's say whether it's an IC, 32 - > Dewan Ahmed: in their quarterly performance discussion, like, what did you do this quarter?

Or even, let's say, a sales rep trying to understand that how sales, change quarter over quarter, and what's the relation with product. So, underneath all this, it's the engineering excellence, right? Because that sort of 33 - > Dewan Ahmed: Creates the flywheel that touches all these different, pieces. 34 - > Adeeb Valiulla | Harness: Absolutely, absolutely, right?

You know, so at Harness, right, my role is almost like a translator. I take raw engineering signals, your tickets, your pull requests, your pipelines, incidents. 35 - > Adeeb Valiulla | Harness: And help leaders see the bigger picture. 36 - > Adeeb Valiulla | Harness: Are we improving time to value?

37 - > Adeeb Valiulla | Harness: is developer experience healthy, right? Are we creating a sustainable delivery system? 38 - > Adeeb Valiulla | Harness: It's not about just dashboards, right? It's about driving change.

You can have… 39 - > Adeeb Valiulla | Harness: 10 or 15 different metrics lined up in a dashboard, but the real value is, are we looking at the right things to drive transformations 40 - > Adeeb Valiulla | Harness: in your organizations, right? For example, I've run engineering metrics program at cybersecurity companies, API security companies. 41 - > Adeeb Valiulla | Harness: Gaming, hospitality, information technology companies, where leaders wanted both velocity and resilience.

42 - > Adeeb Valiulla | Harness: Now, my team helps make sense of DORA metrics. 43 - > Adeeb Valiulla | Harness: Augment them with space-style human factors. 44 - > Adeeb Valiulla | Harness: and increasingly layer AI insights into the mix. 45 - > Adeeb Valiulla | Harness: So, in short, my role is about turning engineering data into business intelligence, so leaders can make decisions 46 - > Adeeb Valiulla | Harness: With confidence, not just gut feel.

47 - > Dewan Ahmed: you touched on something key, everyone's favorite two-letter word these days, AI. Theme of Season 4 is actually AI meets software delivery. Now, we want to hear, how has engineering productivity, like, how you measure it, the definition, and the whole… 48 - > Dewan Ahmed: I guess the area around it has changed now that we see AI everywhere. 49 - > Adeeb Valiulla | Harness: Yeah, that's a great question.

So, the word productivity 50 - > Adeeb Valiulla | Harness: used to mean output, right? Your lines of codes and your commits, your velocity points, but AI has blown up that definition. 51 - > Adeeb Valiulla | Harness: Now, productivity isn't about typing faster, it's about… Problem-solving capacity. 52 - > Adeeb Valiulla | Harness: research backed this up, right?

The recent… 53 - > Adeeb Valiulla | Harness: 2025 DORA AI study suggests that 95% of teams are using AI. 54 - > Adeeb Valiulla | Harness: And yes, 80% say it has improved productivity, but 55 - > Adeeb Valiulla | Harness: 30% don't even trust the AI-generated code. 56 - > Adeeb Valiulla | Harness: Now, similarly, a report that Harness put together in their AI and software engineering report. 57 - > Adeeb Valiulla | Harness: Harness shows that 63% of orgs are shipping faster with AI, but 45% of AI-generated deployments 58 - > Adeeb Valiulla | Harness: have problems.

That's a significant number, right? So the definition has shifted from how fast we code to 59 - > Adeeb Valiulla | Harness: How much value reliable software can we deliver sustainably? 60 - > Adeeb Valiulla | Harness: augmented by AI. So that's a profound change.

61 - > Dewan Ahmed: Totally. And to our viewers and listeners, the reports that Adibh is mentioning in his stat, we'll be sure to link in the description of the podcast, as well as the YouTube video. 62 - > Dewan Ahmed: Now, Adeep, when you're working with engineering teams, what sort of indicators are you looking at to understand AI? Is even AI helping with those engineering productivity?

Because 63 - > Dewan Ahmed: Nowadays, we don't have an issue with less metrics. We have an issue with too many signals, so which indicators actually matter? 64 - > Adeeb Valiulla | Harness: Fair point, you know, I look at three layers, essentially, right? The first layer is your developer metrics, which is essentially what DORA helps you to track, right?

Are your lead times shrinking? 65 - > Adeeb Valiulla | Harness: is deployment frequency up? Are you… are you delivering value to your customers frequently? 66 - > Adeeb Valiulla | Harness: And is stability maintained?

67 - > Adeeb Valiulla | Harness: Right? If AI helps code faster, but your change failure rate spikes. 68 - > Adeeb Valiulla | Harness: That's not productivity. That's literally chaos.

69 - > Adeeb Valiulla | Harness: Right? So that's the first layer. The second layer is, yes, the developer experience matters. So, are developers spending more time in flow?

70 - > Adeeb Valiulla | Harness: Or are they bogged down by context switching between 10 different AI tools? 71 - > Adeeb Valiulla | Harness: The real tool sprawl is killing teams with onboarding, now taking Two months and beyond. 72 - > Adeeb Valiulla | Harness: Right, and then the third layer is around value alignment. 73 - > Adeeb Valiulla | Harness: Sure.

74 - > Adeeb Valiulla | Harness: you're shipping, but are you shipping features that actually move the business KPIs? 75 - > Adeeb Valiulla | Harness: AI can flood the pipeline with code, as I mentioned earlier. 76 - > Adeeb Valiulla | Harness: But if it's not aligned to outcomes, It's noise. 77 - > Adeeb Valiulla | Harness: Right?

So, I ask, Is AI helping developers spend more time on valuable work? 78 - > Adeeb Valiulla | Harness: And less time on toll? 79 - > Adeeb Valiulla | Harness: If not, we are not measuring the right things. 80 - > Dewan Ahmed: Yeah, I'll pick up on the point you mentioned, that AI can flood the pipeline with new code, so it might… it might… 81 - > Dewan Ahmed: makes sense that, yes, we're getting more code, we're increasing productivity, but then it also introduces risks.

How are we making sure that these are good code, these are code ready to be added to production? So how should quality engineering adapt with AI? 82 - > Adeeb Valiulla | Harness: Yeah, so… So think of AI as a speed booster on your highway. 83 - > Adeeb Valiulla | Harness: Right?

If you don't upgrade your brakes, and guardrails. 84 - > Adeeb Valiulla | Harness: Essentially, speed just means more accidents. 85 - > Adeeb Valiulla | Harness: Now, practices I recommend are You want to shift left testing. 86 - > Adeeb Valiulla | Harness: with AI-powered test generation, It also increases human-in-the-loop validation.

87 - > Adeeb Valiulla | Harness: Number 2 is your feature flags and 88 - > Adeeb Valiulla | Harness: safety in deployments, right? So today, less than half the orgs 89 - > Adeeb Valiulla | Harness: and the data comes from Harness's AI and software engineering reports. Use feature flags for AI-generated code. 90 - > Adeeb Valiulla | Harness: That's a big gap.

91 - > Adeeb Valiulla | Harness: The third one is guardrails in Pipeline. 92 - > Adeeb Valiulla | Harness: Now, blue-green canaries, chaos testing, these are not options anymore. 93 - > Adeeb Valiulla | Harness: These are mandatory in your software delivery lifecycle. 94 - > Adeeb Valiulla | Harness: And most importantly, security reviews at AI scale.

95 - > Adeeb Valiulla | Harness: Since 48% of orgs worry about increased vulnerabilities. 96 - > Adeeb Valiulla | Harness: again, this is data from our harness report, right? You need to, in short, evolve your quality engineering from reactive testing to proactive governance. 97 - > Dewan Ahmed: I like the term AI scale.

Like, previously we talked about, doing things at planet scale, or doing things at this scale. 98 - > Dewan Ahmed: But we haven't seen this scale. We haven't seen anything like AI scale, like the pace at which things are being… 99 - > Dewan Ahmed: developed and deployed. 100 - > Dewan Ahmed: That brings me to ask, like, there has to be some misconceptions around as well, right?

There's definitely things that are going right, but people are thinking that, can AI really do that? So, what are some misconceptions you have heard in the engineering productivity space in the AI era? 101 - > Adeeb Valiulla | Harness: Yeah, so, you know, I hear this a lot, and right off the bat, I feel like a lot of… a lot of people in the industry think AI immediately equals productivity by default. 102 - > Adeeb Valiulla | Harness: Right, so that's the first myth.

103 - > Adeeb Valiulla | Harness: The 2025 State of AI-assisted Software Development by DORA shows that AI adoption 104 - > Adeeb Valiulla | Harness: Amplifies what you already have, which means strong teams get stronger. 105 - > Adeeb Valiulla | Harness: Weak teams get weaker. 106 - > Adeeb Valiulla | Harness: So, if your system is broken, AI just breaks it fast. 107 - > Adeeb Valiulla | Harness: Okay?

Second myth is the idea that individual output is the right measure. 108 - > Adeeb Valiulla | Harness: With AI, one engineer can generate massive amount of code. 109 - > Adeeb Valiulla | Harness: But if the team… And not S. 110 - > Adeeb Valiulla | Harness: Secure, or deploy it, It's not productivity.

111 - > Adeeb Valiulla | Harness: It's just waste. 112 - > Adeeb Valiulla | Harness: So the misconception is Productivity is about output. 113 - > Adeeb Valiulla | Harness: The truth is, productivity is about outcomes in context. 114 - > Dewan Ahmed: Yeah, yeah, totally.

And then, I think that's where engineering teams nowadays, they crave the product insights, experts like you provide, because without that, you can't fix what you can't see, what you can't understand. 115 - > Adeeb Valiulla | Harness: And we see a lot of terms like AI-driven or AI-powered. 116 - > Dewan Ahmed: So, what do you think the difference between an AI-powered feature versus truly an AI-driven product? 117 - > Adeeb Valiulla | Harness: AI-powered feature, and it truly… 118 - > Adeeb Valiulla | Harness: AI-driven product.

That's an interesting, way to put it, and I love the question. So, in my opinion, AI… 119 - > Adeeb Valiulla | Harness: Our feature is, like, an autocomplete for your IPE. 120 - > Adeeb Valiulla | Harness: Right? It makes a task easier.

121 - > Adeeb Valiulla | Harness: But a true AI-driven product Changes how you work. 122 - > Adeeb Valiulla | Harness: Right? So, for example, a coding assistant that suggests snippets is a feature 123 - > Adeeb Valiulla | Harness: But an AI-driven product would manage the entire pull request lifecycle. 124 - > Adeeb Valiulla | Harness: So, detecting risk, generating those tests, Assigning reviewers Essentially, reshaping your workflows.

125 - > Adeeb Valiulla | Harness: So the difference is… Are you shrinking or sprinkling AI? 126 - > Adeeb Valiulla | Harness: Or are you re-architecting around AI? 127 - > Adeeb Valiulla | Harness: The winner will be those who do the latter. 128 - > Dewan Ahmed: Yeah, I can relate.

So previously, whenever I needed to, let's say, create a pipeline, like a CI-CD pipeline, I'd read the docs, and then I'd try to, let's say, build the CI stage for build and push. 129 - > Dewan Ahmed: then I'd probably have some sort of approval gate if I need that, then I'd deploy, and then I'd try to understand, okay, what are the configurations I need? Like, it would be a tedious process, like, for a production-ready pipeline, you need to look at all those details. 130 - > Dewan Ahmed: So now with, Harness AI, I can go to, I can see, like, on Harness Platform, you have this button, Create with AI.

I can say. 131 - > Dewan Ahmed: create me a pipeline with a build stage with a push to, let's say, ECR, and then I'm gonna deploy to, to GKE, and then I need to do this. 132 - > Adeeb Valiulla | Harness: You know, and on top of it, adding the security scanners, right, governance, as part of that pipeline. 133 - > Adeeb Valiulla | Harness: not worrying about creating from scratch, right?

So use the Harness AI feature just to create complex pipelines is the beauty, right? So you're not just making simple pipelines, this is complex pipeline with the governance added, with the security scanners added, with the testing added, right? That's the beauty of it. 134 - > Dewan Ahmed: Yeah, and I think that's where I can relate what you said.

It's not just sprinkling AI, but… 135 - > Dewan Ahmed: thinking how everything changes with AI, because not only is it creating pipeline, as previously, when I used to debug pipeline failures, what I would do, I'd look at logs, right? Logs after logs, we'd, like, tail the logs, try to do… Absolutely. 136 - > Dewan Ahmed: now I have this button that says, debug, or let Harness AI, like, debug it for you and find out the issue, and it parses through it, it reads the entire log.

137 - > Dewan Ahmed: and then tells, okay, exactly, like, you might have, have, maybe a YAML validation error, or, or something silly that you… it'll need you probably hours to, to troubleshoot. 138 - > Adeeb Valiulla | Harness: Absolutely. 139 - > Dewan Ahmed: So, now that we think about how AI is making these changes, how do we evaluate? Like, what sort of, I guess, framework do you have to evaluate if AI is even providing real value to the engineering teams?

140 - > Adeeb Valiulla | Harness: Yeah, so, right, with any technology or any tool, you want to make sure it's adopted, right? What impact is it making, and is it efficient, right? So… 141 - > Adeeb Valiulla | Harness: On the adoption side, are developers voluntarily using it? 142 - > Adeeb Valiulla | Harness: Even after the novelty wears off, that's where true adoption kicks in, right?

143 - > Adeeb Valiulla | Harness: I talked about impact, does it… improve Throughput and stability. 144 - > Adeeb Valiulla | Harness: The DORA research shows the best orgs. 145 - > Adeeb Valiulla | Harness: achieve both throughput and stability, so there's a huge impact there. 146 - > Adeeb Valiulla | Harness: And then, efficiency.

Your cost efficiency. So, is it reducing downstream toil Or creating cloud bills. 147 - > Adeeb Valiulla | Harness: which… Are going to shock your teams, right? 148 - > Adeeb Valiulla | Harness: comes from inefficient AI code, right?

So, again, the report from Harness suggests 70% of organizations worry about runway costs. 149 - > Adeeb Valiulla | Harness: coming from AI. 150 - > Adeeb Valiulla | Harness: So, If it fails these 3 checks, It's just a… Lipstick on a pig, right? 151 - > Dewan Ahmed: Yeah, yeah, and on top of that, many enterprise customers, for them, data governance is the key issue, the make-or-break moment, that, what do they do with data, like their data readiness, data governance.

152 - > Dewan Ahmed: So for those engineering leaders, and especially, like, you are one of the very few people in the industry who has worked extensively on this, so what sort of data governance and data readiness steps they need to take into account before they can include AI in their workflows? 153 - > Adeeb Valiulla | Harness: Yeah, I mean, all this is backed by data, right? You have to have good data. 154 - > Adeeb Valiulla | Harness: So, you start with data hygiene.

155 - > Adeeb Valiulla | Harness: AI is only as good as the signals it learns from. 156 - > Adeeb Valiulla | Harness: What does that mean? That means… Clean, labeled engineering data. 157 - > Adeeb Valiulla | Harness: clear AI usage policies.

158 - > Adeeb Valiulla | Harness: Right? Responsible AI. And then governance for model output. 159 - > Adeeb Valiulla | Harness: Don't just trust, you gotta verify.

160 - > Adeeb Valiulla | Harness: And don't skip platform investment. 161 - > Adeeb Valiulla | Harness: DORA found 94% of organizations now rely on platform engineering. 162 - > Adeeb Valiulla | Harness: As the foundation. 163 - > Adeeb Valiulla | Harness: Without a strong internal platform, your AI adoption Will fragment and collapse.

164 - > Dewan Ahmed: You talked about measurement, 165 - > Dewan Ahmed: So, you'd mentioned about DORA, but you also measure the framework itself. So, we talked about DORA with Nathan Harvey, but I was super interested to hear from you that you actually measured the framework itself. 166 - > Dewan Ahmed: So, DORA, Space, DevEx, so all these engineering excellence and performance frameworks. How do you measure the framework itself?

167 - > Adeeb Valiulla | Harness: There are so many different frameworks. Literally, there is a framework every single day, right? 168 - > Adeeb Valiulla | Harness: each framework kind of solves a different piece of the puzzle. You mentioned there's DORA, there's space, there's heart, there is flow.

169 - > Adeeb Valiulla | Harness: DevEx, and I can go on and on, but I'll pick the top few, right? So, DORA is great for delivery performance. 170 - > Adeeb Valiulla | Harness: speed, stability, It's simple, actionable, But it's narrow. 171 - > Adeeb Valiulla | Harness: It only focuses on the lead times, deployment frequency, change, failure rate, mean time to restore.

172 - > Adeeb Valiulla | Harness: Space, on the other hand, adds the human dimensions. 173 - > Adeeb Valiulla | Harness: satisfaction, flow, collaboration, But it's much harder to measure consistently. 174 - > Adeeb Valiulla | Harness: start… Again, created by Google. 175 - > Adeeb Valiulla | Harness: Is user-focused, which means it measures how delivery impacts customer experience.

176 - > Adeeb Valiulla | Harness: It's often overlooked, but it's very vital. 177 - > Adeeb Valiulla | Harness: Flow framework is… is the one which connects your engineering work to business value. 178 - > Adeeb Valiulla | Harness: It bridges the gap between your CIOs and CFOs. 179 - > Adeeb Valiulla | Harness: Right?

So… In my research paper, I map these on a quadrant. 180 - > Adeeb Valiulla | Harness: DORA is easy to implement. 181 - > Adeeb Valiulla | Harness: Space and floor, Space and flow are more holistic, but harder to operationalize. 182 - > Adeeb Valiulla | Harness: The real power is combining them together.

183 - > Adeeb Valiulla | Harness: And we'll be sure to link Adib's research paper in the description as well. 184 - > Dewan Ahmed: So, these frameworks, right, so they were developed way before the explosion of AI tools and AI is becoming mainstream. So how should these frameworks adapt to 185 - > Dewan Ahmed: the AI workflows, where traditional metrics might not tell the whole story. 186 - > Adeeb Valiulla | Harness: Yeah, I feel like… There is an opportunity… 187 - > Adeeb Valiulla | Harness: For us as an industry to extend 188 - > Adeeb Valiulla | Harness: metrics.

For example, in my opinion, DORA could track AI-assisted lead time versus the traditional lead time. 189 - > Adeeb Valiulla | Harness: Right? If you look at space. 190 - > Adeeb Valiulla | Harness: Space could include, cognitive load, from Cool Sprouts.

191 - > Adeeb Valiulla | Harness: Right? Art could measure trust in AI-generated features. 192 - > Adeeb Valiulla | Harness: what I'm alluding to is frameworks must recognize that AI changes not just speed. 193 - > Adeeb Valiulla | Harness: But trust, governance, and the human experience side of things.

194 - > Dewan Ahmed: Yeah, and also accountability, because if you now have a code that is generated by AI, reviewed by AI, 195 - > Dewan Ahmed: where does the accountability line blur? Like, is it you? Is it the admin who allowed certain model? I guess that's something these frameworks would also need to highlight, right?

196 - > Adeeb Valiulla | Harness: Absolutely, absolutely. Responsible AI policies is another, another factor to be included in, in just operationalizing your AI strategy within your organization. 197 - > Dewan Ahmed: Yeah, yeah. And then the other question, let's say if you're a CTO, you might think that, how do I balance developer experience metrics with developer performance metrics?

198 - > Adeeb Valiulla | Harness: Yeah, so… It's… it's about… The cause and the effect. 199 - > Adeeb Valiulla | Harness: The developer experience is the leading indicator. 200 - > Adeeb Valiulla | Harness: developer… Rather, delivery performance is the lagging indicator. 201 - > Adeeb Valiulla | Harness: So, if developers report high friction, or burnout.

202 - > Adeeb Valiulla | Harness: Which is your space. 203 - > Adeeb Valiulla | Harness: You will see it Downstreams, as long… downstreams, as longer lead times. 204 - > Adeeb Valiulla | Harness: Right? Or higher failure rates, which is your DORA.

205 - > Adeeb Valiulla | Harness: Right? So… I tell the CTOs and the leaders that don't treat them as compelling dashboards. 206 - > Adeeb Valiulla | Harness: Read them as a feedback loop. 207 - > Adeeb Valiulla | Harness: Healthy experience fuels healthy delivery.

208 - > Dewan Ahmed: I think that itself could be a blog, like, healthy experience, fuel, delivery, right? And I couldn't agree more, because, 209 - > Dewan Ahmed: This… this mindset where we… we… 210 - > Dewan Ahmed: do one time, it's a one-time thing, has an issue in itself, because it's a feedback loop, because now it's AI, tomorrow it might be something else, 20 years before, the tools change, the frameworks change, but the mindset needs to be that it's never done. 211 - > Dewan Ahmed: We're continuously improving, seeing what works, seeing what doesn't, and then, change, change accordingly.

212 - > Dewan Ahmed: That brings me to the last segment of our podcast, where we ask our guests on future. Now, we're not going to ask you to predict what's gonna be the winning loader number. We will ask you how they work. Also, it might be fun if you could predict that.

213 - > Dewan Ahmed: So, how do you see the defining traits of AI-enabled software delivery organizations in, let's say, in the next 3 to 5 years? 214 - > Adeeb Valiulla | Harness: I think, players who have… 215 - > Adeeb Valiulla | Harness: Platform as a strategy are definitely going to have a competitive advantage. 216 - > Adeeb Valiulla | Harness: stronger platforms that make AI safe. 217 - > Adeeb Valiulla | Harness: and scalable will definitely be the winners, right?

Platform that can help you 218 - > Adeeb Valiulla | Harness: Automate your deployments, can automatically create your complex pipelines. We spoke about this a bit earlier. Adding your governance, adding your security as part of that pipeline. 219 - > Adeeb Valiulla | Harness: will help organizations deliver faster, but not just faster, right?

Safer, in a secure manner, and efficiently. So, platform is… 220 - > Adeeb Valiulla | Harness: where I feel… Players will… will definitely have a good, good advantage over 221 - > Adeeb Valiulla | Harness: or players who, don't have platform as a strategy. That's… that's the key thing. 222 - > Adeeb Valiulla | Harness: Governance, so AI policies and guardrails.

223 - > Adeeb Valiulla | Harness: embedded in your pipelines. That's… that's the key word here. Will significantly help AI enable software delivery. 224 - > Adeeb Valiulla | Harness: And human-centered, which is… They'll review… they'll view your developer experience as the competitive edge.

225 - > Adeeb Valiulla | Harness: not just a nice-to-have, right? So, developer experience is key to all of these, right? So, to summarize, I want to say platform, governance, and developer experience. 226 - > Dewan Ahmed: Yeah, I can almost see that triangle, platform governance and developer experience.

Maybe, maybe in one of the frameworks, it shows as a triangle. For initiatives, the main thing is mindset, right? Engineering leaders, they need to shift mindset, because. 227 - > Dewan Ahmed: Things have been done for the last, let's say, 30, 40, 50 years, and it has worked.

228 - > Adeeb Valiulla | Harness: Now, suddenly, there needs to be a change in mindset. So, for those listeners, those executives, engineering leaders. 229 - > Dewan Ahmed: CTOs who are listening to this podcast, what would be one mindset shift that you'd, recommend to them, now, that would help them in the future? 230 - > Adeeb Valiulla | Harness: Yeah, so, look, AI is here, it's not… it's not really the next big thing, right?

It's here, it's generating a lot of code, 231 - > Adeeb Valiulla | Harness: which is good, but it's bad in terms of the delivery process, right? If you don't have a scalable approach on the right-hand side, you are only inviting chaos, noise, and 232 - > Adeeb Valiulla | Harness: an instability into your delivery pipeline. So, leaders need to shift their mindset from cool thinking? 233 - > Adeeb Valiulla | Harness: Two-way system thinking.

234 - > Adeeb Valiulla | Harness: So, buying the next AI assistant 235 - > Adeeb Valiulla | Harness: Will not fix your systemic bottlenecks. 236 - > Adeeb Valiulla | Harness: But investing in… Feedback loops, culture, governance. 237 - > Adeeb Valiulla | Harness: Which are part of a platform-centered approach. 238 - > Adeeb Valiulla | Harness: will make every AI tool more valuable.

239 - > Dewan Ahmed: Yeah, totally, like… AI doesn't fix broken practice, broken system, it just creates, 240 - > Dewan Ahmed: problems, at a scale. So now you have a broken problem times 100. So, totally, like, you need to have a solid practice, fix the issues, follow the usual SDLC best practices, and of course, then when you add AI at a system level, not just as a one-time, then the benefit will be there. 241 - > Dewan Ahmed: So we heard the regular prediction, but I think our listeners want to hear one bold or unconventional prediction where AI and software delivery intersect.

What would be one unconventional prediction from you? 242 - > Adeeb Valiulla | Harness: Yeah, so… I think in 5 years, we'll stop talking about AI-assisted coding. 243 - > Adeeb Valiulla | Harness: We have already seen a boom today, so in 5 years, you know, in my opinion. 244 - > Adeeb Valiulla | Harness: Coding will fully be automated, for… 245 - > Adeeb Valiulla | Harness: majority of the use cases.

The real differentiator will be AI-assisted decision making. 246 - > Adeeb Valiulla | Harness: Helping leaders choose what to build. 247 - > Adeeb Valiulla | Harness: Not just how to build it. 248 - > Adeeb Valiulla | Harness: I think that's… that's my next frontier.

249 - > Dewan Ahmed: And for my developer friends who are listening to this podcast, don't feel that developer jobs are going away. You know the term, right? We just need a button to make everything automated. Someone still needs to make that button.

So, the work of builders will always be there. 250 - > Adeeb Valiulla | Harness: Absolutely, absolutely. Very well said, and, you know, thank you for this opportunity to share my experience, my expertise in the developer experience, developer productivity space to your audience. I really appreciate the time.

251 - > Dewan Ahmed: Of course, thank you, and if our viewers and listeners want to connect with you, read up more about your work, where can they find you on the digital world? 252 - > Adeeb Valiulla | Harness: Yeah, so I'll definitely link my LinkedIn. I have my publications on SSRN and ResearchGate. I do… I do podcasts, and I do host my own podcast, Metrics That Matter, where I… I speak to other engineering leaders, learn about their journey.

253 - > Adeeb Valiulla | Harness: and, you know, collectively discuss and talk about just the space around developer experience, how AI is impacting our day-to-day lives. So yeah, those are some of the spaces where they can find me. 254 - > Dewan Ahmed: Perfect. This was ShipTalk, Season 4, Episode 4 with Adeeb Valiulla, a leader in developer productivity and developer efficiency space.

Hey, if you are someone as a CTO, or engineering leader, or an executive, would like to talk about engineering excellence, engineering productivity, we'll link Adeeb's podcast in the description. Today, 255 - > Dewan Ahmed: I'm the one signing off, and we'll see you in the next episode.

Related episodes across the Index

Other episodes covering the same guests and topics, from across The B2B Podcast Index.

  • Autonomous Software Development at Enterprise Scale: Inside a 1,000-Developer Pilot (with Blitzy) | CXOTalk #918CXOTalk · on AI code generation97 / 100
  • Charity Majors on AI, Observability, and the Future of SoftwareScaling DevTools · on Feature flags96 / 100
  • Matthias Laug from TIER on charging innovation, power of self-managed teams, and the role of technology | The Dev is in the Details #13The Dev is in the Details · on DORA metrics90 / 100
  • Continuous Quality for AI Coding: ChecksumSourceForge Podcast · on AI code generation81 / 100
  • Ep. 018: Metrics That Matter - Data Points for Technology Leaders | w/ Special Guest Sebastian Kline of Nelnet Community EngagementOpen Source CXO: The Tech Leader's Podcast · on DORA metrics80 / 100
  • CI/CD with Robert ErezThe Pragmatic Engineer · on Canary Deployments76 / 100

More from ShipTalk

All episodes →
  • ShipTalk Season 4 Finale: Engineering Excellence at AWS re:Invent67 / 100
  • Crown Jewels In, Crown Jewels Out - The Hidden Risk of AI with Devan Shah (IBM)82 / 100
  • CTO Predictions for 2026: How AI Will Change Software Development (with Harness Field CTO Nick Durkin)78 / 100
  • Beyond the Magic Box: Solving AI Hallucinations with Precision RAG (with Evgeny Ilinykh)85 / 100
  • Shipping Practical AI: How to Build Real-World ML for 2D Drawings (with Marina Petzel)88 / 100
Explore the best B2B Engineering & DevTools podcasts →
All ShipTalk episodes →