EP 227 - Why Quality Assurance Is Now a CEO-Level Concern in APAC

Speaker A: Software is the business and quality cannot be an afterthought. Right? Because if you don't do the right thing and take ownership at the board level, then no one's going to place the right level of focus and emphasis. Think about it. If you are an airline and a software glitch prevents your planes from flying, or if you're a bank and your software outage prevents your customers from making financial transactions, those are no longer just nice to haves, those are exit existential issue.

Speaker B: Welcome to the Digital AZI Spotlight podcast where we explore the intersection of innovation, industry and technology. I'm your host Peter Oddicen. Today we are joined by Damian Wong, Senior Vice President for APAC at Tricentis, a uh, global leader in software testing and quality assurance. We'll be diving into how AgentIQ AI is transforming enterprise QA especially, especially in highly regulated industries. And why it's a fast becoming a boardroom level concern for CIOs and CTOs across Asia Pacific.

Speaker A: Welcome to the industrial IoT spotlight, your number one spot for insight from industrial IoT thought leaders who are transforming businesses today with your host, Eric Walenza.

Speaker B: Damien, thanks for joining us.

Speaker A: Thanks Peter for having me on your show.

Speaker B: Starting with a uh, short introduction, let's start with your journey. What led you to your current role? What motivates your work in software quality today?

Speaker A: Well, great. Quick introduction to myself, Damian Wong, Senior Vice President for Tricentis across the Asia Pacific geography. I joined Tricentis in 2022. In terms of my career history, I've been in the enterprise technology industry for just over 30 years now. I like to say that I started my career when I was just five years old, but who's kidding, otherwise it would give my age away. Anyway, one of the things that I really like about this industry is that it never ceases to innovate and change. My last 20 years, uh, of my career has been spent in enterprise software and I just love the pace at uh, which this industry moves. There's never a boring moment in enterprise technology. 20 years or so ago I joined a company called Mercury Interactive and we created this category to help organizations move from manual testing to script based test automation, which was uh, what I would consider first generation of uh, automated testing. It was revolutionary back then Peter, and it was very fulfilling knowing that we were um, helping organizations drive quality and performance of their software portfolio. But unfortunately Mercury was acquired and uh, subsequently changed ownership or changed hands a number of times. That was a bit of a shame because you know, with focus it allowed the company to thrive. But once that focus was lost, the level uh, of innovation that was needed to address customers evolving needs started to uh, diminish. And that's why when in 2021, at the end of 2021, I was approached by Tricentis to lead the Asia Pacific business. And Tricentis just reminded me of all the wonderful attributes that Mercury had right in its heyday. This laser focus on helping customers drive better business outcomes using software quality and performance was something that I could really identify with. So the only difference was we were taking enterprises forward with a radically different approach. A fully codeless, AI driven, model based test automation approach. It's a bit of a mouthful, right? But what it did was it addressed the current needs of enterprises given the fact that many of them were starting to struggle to keep up with using script based test automation, given that the whole game had changed with the new technology architectures that they had to deal with the complex application landscapes that were highly integrated. In summary, it was like deja vu, uh, Peter. It was like a new and improved version two of Mercury. And that's really the reason why I got excited and joined the company in 2022.

Speaker B: Thanks Damian. Makes sense. And could you briefly describe Tricentis mission? Uh, you already gave a little bit away. So what's its role in enterprise software? In the enterprise software ecosystem, particularly here in the APAC region, We would definitely

Speaker A: uh, be happy to share that. I think I did allude to the fact that Tricentis mission was around driving quality, uh, for software in organizations. But essentially it's four things that we do. We want to empower every single team in organizations to deliver efficiently high quality software. And the third thing is do it at speed and scale, but not just for the sake of driving quality. But finally the fourth thing which is to drive better business outcomes. So empower every team to efficiently deliver high quality software at speed and scale to result in better business outcomes. I hope that kind of encapsulates everything that uh, we aspire to do. So in short, because it's a long mouthful, we drive quality at the speed of innovation. That's what I always like to say, right, because organizations are uh, driving digital innovation at speed, but then we ensure that there's quality as they drive speed to innovation. So that's really what we do. And if you look at the role that we play in the enterprise software ecosystems, I think the thing that I keep hearing from the CIOs that I speak to the IT leaders and uh, business leaders that I speak to is really around, uh, two key themes. Right. The first theme is application modernization, and the second theme is digital innovation. Right. So first, uh, application modernization. Peter, it's really that many organizations have all these legacy applications and systems, uh, that need to be modernized. Right. You know, they generally are large monolithic applications that are very brittle and they're not very, um, good at uh, supporting innovation. Right. So, so while a number are custom developed, many of them are also packaged systems. Like we spoke earlier that uh, you have a long history with SAP. Well, SAP, uh, is one of those applications that is commonly found in large enterprises as well. So the technical debt that we see accrued on these apps over the years is something that organizations are trying to address. Because if you don't modernize them, then this can impede your innovation. Right. And it can also be very risky for organizations to not have them be able to uh, meet evolving customer needs and demands. Right. You know, uh, and the other issue is that these apps have also been integrated with many other growing portfolio of applications that they have acquired over the years. And this is uh, the basis of all those business processes that power those organizations. Right. You know, so making a change to an app within that portfolio oftentimes results in a, ah, ripple effect and unintended consequences are uh, happening if you're not careful. Right. So this ability to take an end to end approach to testing is so critical today in ensuring that organizations are able to successfully update, upgrade, ah, and refresh some of these applications that they have. Right. And I mentioned earlier, the second point was digital innovation, Peter. Uh, businesses uh, generally compete today on digital innovation. Right? Software is the business for many organizations. So how an enterprise innovates, uh, digitally faster against the competition is a key advantage for many of them. Absolutely. Would you, you know, I'm sure you've heard of this term Gen AI, right? You know, and all of us are uh, now hearing all the time about Gen AI, right, Peter? So Gen AI has actually made this phenomenon even more acute. So recently, uh, I was reading about the uh, CEO of Anthropic, uh, I'm not sure if you're familiar with Anthropic, but they are today fairly synonymous with this whole, uh, Gen AI domain. The CEO of Anthropic, who was also the vice president and head of research for OpenAI prior to anthropic, gentleman called Dario Amodei, he shared, uh, recent times that in three to six months 90% of all code will be AI generated. And he said in 12 months he predicts all code will be AI generated. So that's just mind blowing. But it's also game changing if you think about it. Code can now be generated automatically at speeds unheard of, uh, previously. This creates this phenomenon. There's a bottleneck that now is forming and being created because you can generate all this code at such speeds. You have to now validate and test that code. This is especially important because most of the research that's out there have shown that a lot of the generated software code comes with bugs and defects. Right. You know, so I think that's one of the things that we obviously are there to try and help organizations address in terms of uh, uh, trends. Right. We have a unique role to help the organizations drive quality at speed of innovation, especially in the APAC geography. Right. You know, so we've grown our footprint, Tricentis, uh, footprint across, uh, APAC tremendously. We started in, with offices in Australia, in Singapore and in India. And over the last couple of years we've actually grown our uh, presence to cover Japan, Korea and even have an office in the Philippines. And this is because if you look at uh, the landscape here, APAC is really leading the way in many cases in digital transformation and innovation. So it's a very exciting time to be out here in Asia in apac. And Tricentis is also obviously very excited to play a huge role in helping organizations succeed digitally with what we do.

Speaker B: Excellent. Yeah. M. Actually you're really making a point about in terms of testing because a quick example from my end, we're just now implementing one additional process and as that we're working, we are an SAP implementer. We are implementing one additional process and configuring that process takes just, uh, two, three days. Testing it, making sure that everything is fine, from logistics to finance to output documents, takes one and a half months.

Speaker A: Wow.

Speaker B: So the essence is really into testing, making sure that there are zero mistakes. Especially you're coming from the enterprise software world as well. Mistakes are not allowed. Uh, it's not like you have an app where, oh, it doesn't work well, I don't like it. No, you might be losing money.

Speaker A: You got that right. For the company, they might be losing money. There's reputational risks for the individual, they might lose their jobs. Right. You know, so, uh, that's the uh, that's the risk of it. Actually it leads me to that, uh, very famous article that I like to cite. It was published last year in Fast Company magazine. It's entitled thanks to AI the Coder is no Longer King. All here. The QA engineer Right. You know, so I think that's really, uh, representative of the times that we are in.

Speaker B: Yeah, actually that makes sense. Well, as you said, you have to ensure the quality and you can trust the code or you can trust an AI to write the code. It's the same thing.

Speaker A: Yeah, okay. Uh, I think that's the key to it. Right? It's a tool, it's a great tool in technology. But then there comes, um, implications that we have to deal with.

Speaker B: Yeah, yeah. Right. Okay. So how has quality assurance in software changed over the past five years? Of course now we have the big change with AI, but how did it evolve over the past five years? Would you be able to give an intro?

Speaker A: Yeah, of course. I can share a little bit about how this has changed over the last few years. I did mention that article and I think that article is welcome for all people involved in the quality assurance space. Because in the past quality assurance was seen as an afterthought. Right. Know if I have time, I'll do it, and if I don't, then, okay, let's see. So I think it's putting a whole spotlight on this whole domain of quality assurance. But in, in the time that I've been here, and you know, I've also looking back at Tricentis, uh, history, uh, we've seen the software quality assurance space evolve, right? You know, from the traditional way of manual testing, it's gone to script based test automation, which worked very well for a time when things uh, moved a lot slower and systems, there were less systems that were integrated to power those business processes. And then, you know, Tricentis brought to the table, uh, codeless model based test automation, which was again a paradigm shift because we are no longer talking about coding scripts. You're now looking at modeling business processes and being able to abstract the business process from the underlying technologies, which was fantastic because it took away a lot of the pain associated with traditional script based test automation. Uh, then now we're starting to move into this whole domain of agentic test automation. So if you want to put it in layperson's terms, Peter, you can think of, uh, manual testing almost like running and a script based test automation. Like you have a bicycle now, so you're cycling, you can get there faster, but you're still using your manual effort to get there and then moving on to driving. So now you have codeless model based test automation. You have a car and the car can take you to places further and faster than you could with either running or cycling. Now with agentic test Automation. We're into the world of autonomous cars, autonomous vehicles. So I would use that as a way to help people understand this evolution that, uh, we're seeing in the software quality assurance space.

Speaker B: Very good intro, Damien. Thanks. Then let's move on to the problem. QA bottlenecks and leadership risks. The first question I have here is, I read you've mentioned QA bottlenecks are now a concern at the C level suite. Uh, why is that the case today more than ever before?

Speaker A: It definitely is. That's something that we're constantly trying to ensure we do. We're trying to address concerns at the senior executive level, business leader level, maybe. You know, if you look at the research we are also conducting, we published recently our tricentis Quality Transformation report, which is based on surveys that we have conducted with, uh, organizations. And some of the statistics are pretty surprising. Peter, if you look at the responses, quality gaps are costing organizations on average more than half a million dollars a year. And that's a lot of money. As an average figure, two thirds of those organizations that were surveyed said that they are, uh, at risk of a software outage this year itself. And nearly half also prioritize delivery speed to software quality. So to them it's like, okay, let's roll out the software faster. And whether that's the right level of quality, that's secondary. But first, roll out at speed. And the last point I want to point out is I think the most concerning. Nearly 2/3 of, uh, those organizations admit to regularly shipping untested code. And this is often because of the demands for speed. So think about it. You know, you're shipping code out there that is completely untested. That is crazy. Right. You know, so we've seen as a result of this. Right. Many organizations having these spectacular software outages. Right. You know, there are companies that splash on the front pages of newspapers. Examples that were in recent times are like crowdstrike, the outage that, uh, happened globally. I'm not sure if you have familiar with the crowdstrike outage, uh, Peter, but, uh, that was spectacular. Also we have in Singapore, for example, some local examples. The former CEO of DBS bank actually publicly said that, uh, four out of five of their major outages, uh, that was two years ago.

Speaker B: Uh-huh.

Speaker A: Were caused by software bugs. So think about the impact that these, uh, software quality issues are, ah, facing the fact that they're not testing. So if an organization prioritizes speed over quality and ships untested code, I like to say think of it like playing Russian Roulette. And for, uh, those who are not familiar, Russian roulette, you put a bullet into a gun and then you point at your head and then you spin the barrel. You hope you don't get the barrel with a loaded bullet, but that's exactly what it's like. You're lucky if the code works, but what if it doesn't? Right. You know, and this is why it's become such a, uh, uh, C suite concern. Because C level executives are, uh, now focusing on this quality assurance bottleneck that is forming and they need to ensure they get it gets unblocked. Right. You know those who don't do that, well, let's just say I wouldn't want to be in their shoes if or when stuff hits the fan.

Speaker B: Yeah, absolutely. Someone's got to take the responsibility for those mistakes.

Speaker A: 100% buck stops with them.

Speaker B: Yeah. And then especially sectors like finance. You mentioned finance, health and government. I guess then traditional QA is failing to keep up with, uh, the demand nowadays. Right. I guess that's where you come in to help them.

Speaker A: Yeah, Peter, And I think you've highlighted a few examples of highly regulated industries. Finance, uh, health, government. I think it goes beyond those, uh, industries where organizations, uh, are generally facing this whole challenge because they are using or adhering to traditional quality assurance, uh, techniques and tools. I think all the reasons I mentioned earlier, traditional QA approaches just don't stack up anymore. Manual testing and script based test automation, uh, in this age of gen AI, I think it's like trying to bring a bicycle into a Formula one race. Uh, Peter, imagine you turning up at the Formula one race and you bring this fancy bicycle and expecting to compete. Uh, the speed and frequency of change that, uh, enterprises face today is just growing exponentially. And hence what I think got those organizations to where they are today is definitely not going to get them to where they need to get to tomorrow. So if you're being forced to compete in a high performance vehicle race, you're going to need to show up with a high performance vehicle or you're just not going to compete.

Speaker B: Yeah, that makes sense. So there are misconceptions out there. Right. Would you have some examples for common misconceptions business leaders still have about QA and software testing and their impact on their businesses?

Speaker A: Oh, I hope you have enough hours in the podcast for me to go through them. Right, yeah, but I'll just cite some of the common ones. A lot of the business leaders that I speak to, uh, still think that software quality should be owned by the software QA team that resides in the IT department and they tend to delegate that down whenever we have a conversation. The reality is that quality needs to be owned at the board level. It is a board level topic because it has company wide existential impacts. Given that in today's world, software is the business. Right. You know, and I'm not sure if uh, you recall Mark Andreessen, who is co, uh, founder of Andreessen Horowitz, one of the largest private equity firms and VC firms in the world. He wrote the book Software is Eating the World. Right? But today that's come true. Software is the business and quality cannot be an afterthought. Right. Because if you don't do the right thing and take ownership at the board level, then no one's going to place the right level of focus and emphasis. Think about it. If you are an airline and a software glitch prevents your planes from flying, or if you're a bank and your software outage prevents your customers from making financial transactions, those are no longer just nice to haves. Those are existential issues.

Speaker B: Yeah.

Speaker A: Wouldn't you agree, Peter?

Speaker B: Yeah, that's correct. Well, at least there is a Genti AI now. I guess your solution will be filling, uh, a big gap there. Or you are already filling a very big gap. Yeah, we hope so. At least what I see, it's expanding, the company is expanding. I'm very glad to see it. So, uh, talking about your solution, uh, entering agentic AI, could you explain what agentic AI means in the context of software? How does it differ to the traditional Automation?

Speaker A: Yeah, yeah, 100%. Right. You know, I mean, everyone's putting up all these buzzwords, right? You know, and sometimes it's confusing for people hearing all these new buzzwords. So agentic AI is definitely a new buzzword, but I think it's a very relevant buzzword. So let's, let's look at the definition of what agentic AI means. Right. Agentic AI generally refers to an AI that's able to, autonomously meaning on its own, make decisions and take actions and to achieve specific goals and often without human oversight. And so I think that's an important thing to take note of. It can make decisions and take actions on its own, often without humans intervening. And that's the whole concept of agentic AI. And in the context of software testing, that's where we are looking at agentic test automation. Because that's really the nirvana that all of us have been looking toward for a long time. And it's finally becoming A reality which is fantastic. Right. And then you compare that with the traditional definition or context of test automation. So traditional test automation essentially is taking manual development of test assets based on certain test requirements and test cases. And you've done software projects before, so you know about this, right? You know someone defines what the software needs to do, right? Those are the requirements. And then based on the requirements you define what do you need to test. And then if you are automating it, you develop those test assets that actually execute on the test to validate those software actually does what it's supposed to do in the traditional way. You do it manually. So think of it in the context of driving. It's knowing where you want to go and then driving there yourself. Agentic test automation is then akin to an autonomous car, ah, driving you to your destination. Once it knows where your destination is, it will do it with all the constraints, et cetera, that you define and then it will just do it for you. So I think with agentic test automation, the speed at which testing can be executed will just accelerate exponentially. And all this bottleneck that's uh, based on human effort is actually going to be reduced, uh, significantly. So I like to use analogies. I'll give you an analogy. Think about a situation where you're a taxi company owner. I own a fleet of taxis. Right. In the past I would be dependent on the number of taxi drivers I had and the number of hours they would drive in order to generate the kind of revenue for my company. So now you go into the world of autonomous vehicles. Uh, I am no longer constrained by the number of taxi drivers I have and the number of hours they can drive. Right. You know, the cars can drive on their own, they don't get tired, they can drive 24, 7 if they need to. Right. You know, and uh, and I think that's going to be absolutely game changing. So if you look at that analogy and apply that to software quality assurance, that's really going to be the same kind of paradigm that we're looking at. So I, I hope that kind of explains uh, this whole concept, you know, I guess so.

Speaker B: Yeah, that makes, makes perfectly sense. I think it also brings us to the next, uh, the next topic. What, what the core benefits?

Speaker A: Speed.

Speaker B: And I can confirm speed is very good. Uh, if we can increase speed in software testing, that's what we all need, right. If we can shorten projects and still deliver the same quality, that's perfect.

Speaker A: Well, I've been told in the past that testing is a speed bump, right?

Speaker B: Kind of, yeah. You have to go back a lot, uh, and, uh, rework.

Speaker A: Yeah. If you're still listening, then you must take technology seriously. So I'd like to introduce you to our case study database. Think of it as a roadmap that can help you understand which use cases and technologies are being deployed in the world today. We have cataloged more than 9,000 case studies and are adding 500 per month with detailed tagging by industry and function. Our goal is to help you make better investment decisions that are backed by data. Check it out at the link below and please share your thoughts. We'd love to hear how we can improve it.

Speaker B: Thank you.

Speaker A: And back to the show.

Speaker B: So what other benefits are there besides testing? Sorry? Besides speed?

Speaker A: I think we can touch on it. Yeah. Besides speed compliance, better coverage, etc. So maybe I'll name a few things. Right. Uh, if you look at AI and how it's applied on the domain of quality assurance, Tricentis is, uh, obviously a specialist in this domain, right. So we're applying it to areas such as I mentioned earlier, agentic test automation. So removing the bottleneck for humans and then being able to speed up the creation and execution of test assets, which I think is one of the big, uh, benefits that you get. You get a lot of speed, you get a lot of, um, accuracy as a result of that. Then you also have that whole domain of knowing what to test, when to test. So that's quality intelligence, because good, that you want to have a car that drives you to where you want to get to. But first you have to define where exactly do I want to get to and, uh, how do I really want to get there? There are many ways to get there. So quality intelligence effectively tells you, okay, what's exactly changed in the code, for example, and then what do you really need to test, uh, or do you need to test anything as a result of that? And that reduces the effort required for testing the resource required for testing and increases your speed to, uh, deliver innovation. Another area, a third area, could be around, uh, intelligent test management. We all want to ensure that when we're getting requirements from our, uh, business owners or business stakeholders, oftentimes the requirements come in the form of natural language. This is what I need to get done, right? You know, how do you automatically translate that into, these are the test requirements, these are the test cases and test scenarios, etc. And is this comprehensive? So having something that's able to read, right, you know, from natural language, all of that, and be able to translate that automatically into the relevant test cases, uh, test requirements, test scenarios, and be able to create those tests assets that support those uh, uh, requirements. I think it's going to be fantastic because it ensures that you have better uh, coverage and you have, you're able to do this at speed and scale. Something that uh, you know, in the past a business analyst would have to spend days, weeks, months to try and translate. Uh, I'm sure you've seen that in the past, you know, so imagine if you can have AI now be able to do that for you in the matter of hours or even minutes. Right. You know how wonderful that would be?

Speaker B: Yeah, it sounds wonderful. Yeah. You have to give it a lot of trust as well, Right? You need to trust, uh, the solution 100%. So are there any examples you could mention where organizations have successfully embedded agentic AI in their workflows?

Speaker A: Yeah, I would say that agentic AI as a category is very new and many of the enterprises that we're working with today are doing this in terms of pilots and trials. Right? So they like it. But like what you've just said, it sounds new, it sounds like magic. Right? So uh, a lot of people when you're encountered with this type of disruptive technology, they like to actually get their hands around it and ensure that there's no uh, smoke and mirrors underneath. Right. You know, so, so the tightness and trials are uh, definitely happening. But having said that, AI in quality assurance beta is actually not new. I'm sure, you know, you would have heard that AI has been used in QA for a while now and TriCentis has embedded AI into our portfolio for a very long time. Right. So I'll give you an example of maybe a company in the Asia Pacific region. I uh, don't know if you've heard of a company called Zespri.

Speaker B: Zespri is of course, and the Kiwis.

Speaker A: Good. Yes, Kiwifruit, absolutely. So New Zealand company, the world's largest marketers of Kiwifruit. Right? Yeah. So wholesome company. It's a wholesome company. Yeah. So Zest Free uses intelligent capabilities in the uh, Tricentis software to help them with ensuring quality of their applications, uh, through some capabilities like self healing of test assets. So imagine this, right? You know, if they had an ERP software update that came in, it would then be able to review and upgrade and update automatically their test assets. Because imagine this, if a code, uh, change came in and it broke all my test assets as a result, then I might unwittingly miss out on A software defect. I wouldn't have found it. It might have a positive, uh, or, uh, false positive or false negative result in my test. Uh, and as a result, it might cause an outage. You know, knock on wood. That's not a good thing because for me, that is a personal disaster. My eldest daughter, she adores Kiwifruit and, uh, she has at least one every single day. Uh, so I cannot imagine her not being able to get her daily fix of Kiwifruit. Right. So you're German, and I actually want to use a German example of something that happened like this a number of years ago. Right. There was a company in Germany called Haribo, and Haribo, yeah. I'm not sure if you are familiar with them, is, uh, I believe, the world's largest manufacturers of gummy bears. Yeah. You know, so I've got a sweet tooth. So I do eat gummy bears from time to time. So because of a glitch that they didn't detect in, uh, their ERP software upgrade. Right. You know, it actually caused a global gummy bear outage. Yeah. So I would say that that was, uh, a true disaster, definitely for all gummy bear fans. But, you know, imagine the company not being able to, uh, ship and fulfill on all the demand that was there. Right. You know, so that's, that's an example of AI in action. Right. You know, and, um, you know how it can be used, uh, to help in real life situations.

Speaker B: Very impressive. I had no idea. And, and both from countries. I'm a German, but I'm also a PR of New Zealand.

Speaker A: Yeah, yeah.

Speaker B: Two of the, uh, most prominent brands. Yeah.

Speaker A: I hit close to home.

Speaker B: Yeah, yeah, yeah. Very interesting. Uh-huh. That's already some pretty big examples, actually. Ah, very impressive. I had no idea.

Speaker A: One positive and one not so positive.

Speaker B: Yeah, yeah, yeah. I really had no idea. Okay, thanks. Thanks, Damien.

Speaker A: Sure.

Speaker B: Actually, going into the question, next question, this is probably already partially answered, but how does AI help reduce compliance and trust related risk risks, particularly in regulated industries?

Speaker A: Yeah, that's a great question. And that's a question I often get right. How do I help address some of the compliance risks? Because I'm a bank or I'm a healthcare company or whatever, I think AI can help in many ways, uh, address some of these risks. I'll give you an example. I mentioned earlier, CrowdStrike, uh, had a global outage, and that global outage was caused by a single line of code not being tested before it was rolled out into production. I don't know if you were affected by this, Peter, when the Crowdstrike, uh, situation happened. So maybe you weren't using Windows machines, etc. But many of my colleagues actually suffered blue screens because they were using Windows, uh, laptops. That, I think was a situation where many organizations worry. How do I know if there is a, uh, code change or new piece of code that has actually been completely tested before it is rolled out? In most organizations, there is no way to confirm that it has actually been tested. A, uh, software developer can say, look, I've tested it, but in uh, the past, there's just no way you could actually confirm it and validate it independently. That's why Tricentis acquired a company called Sea Lights, uh, last year. C Lights allows us to detect if a change has been made in the code and then also detect if tests have been run against those, uh, code changes. And if a, uh, test has been run, what kind of test is it? A unit test, an API test, etc. Etc. Right. Uh, and this kind of capability allows organizations to tell their regulators, hey, I can show you how I do it and I can guarantee that I fully tested my code before it is released into production. So I think that is one of the key ways that organizations, uh, can help to meet regulator demands. And remember when I shared earlier the uh, Tricentis quality Transformation report finding that two thirds of organizations admit to regularly pushing untested code into production. So that should be a m, big worry for any regulated company. Wouldn't you agree, Peter?

Speaker B: Yeah, yeah, absolutely. Yeah, it is very scary. I had no idea about that actually before our talk here. So far I've already learned a lot.

Speaker A: I'm glad, I'm glad you're learning.

Speaker B: Let's move on to the implementation. Building the roadmap. What should CIOs and CTOs consider when starting their AI based QA transformation? What are the practical first steps for them to take?

Speaker A: Great questions. There are two things I would advise tech, uh, leaders to, uh, consider when they are starting this journey. I think, number one, attend conferences, listen to podcasts, speak to industry experts and peers who are also embarking on this journey. Definitely listen to the Digital Asia podcast. Right. You know, Peter, I'm sure you would agree with that.

Speaker B: Yeah, thank you, thank you, thank you.

Speaker A: Yeah, 100%. Right? You know, I mean this, this is uh, something that Tricentis believes is important. Right. You know, so that's why we are having our flagship, uh, AI tour event in, uh, in Singapore on September 4th. Right. You know, so if, if you are in, in Singapore around that uh, time, uh, you definitely sign up and, and attend the event. Right. We'll cover uh, this exciting topic in detail. So anyone listening in if, if you're gonna be in Singapore September 4, 2025, do plan to attend the Tricetis, uh, AI tour here. The second thing that I would advise is come and speak to a company that specializes in the domain. Completely unbiased, of course. Peter. Completely unbiased. I'm talking about company that actually is able to do this. It starts with the letter T. Anyone come to mind? Uh, Peter.

Speaker B: Well, I think we have you on the call here. Right. It is okay to put Tricentis forward.

Speaker A: We encourage you to come and speak to Tricentis.

Speaker B: Obviously this is why we have you. Right. We want to feature you because we are actually independently impressed by what Resentis is doing. Yeah, Independent marketing here, very clear.

Speaker A: Great to hear. Yeah, no, I was just kidding. Right? I mean you can obviously speak to different, uh, organizations, but we do believe that speaking to people who do specialize in this, uh, and uh, have a focus around it will allow you to better understand uh, the domain, some of the challenges, some of the implications and considerations as you start this journey. I think it's important to know what you're getting into and what are some of the things that others are doing to also do the same thing. And I think uh, that is going to be critical as CIOs and CTOs start this uh, transformation.

Speaker B: Yeah, okay, makes sense. How can enterprises balance innovation with risk? We touched the topic a bit earlier. Being able to trust new solutions, but how is it especially handled in highly regulated environments?

Speaker A: Fantastic question, right? You know, when we talk about AI in regulated environments, many uh, of our customers tell us, hey, look, my regulator is very concerned about exposing your data to public large language models and so on and so forth. And that's actually one of the reasons why Tricentis introduced uh, MCP support. Model context protocol support. Right. You know, mcp, for those who may not be familiar, is essentially a, uh, you think of it as a standard, uh, that allows you to uh, plug into just about any AI framework. This allows you to bring your own AI and use your own AI. If you've got your own private large language model that you've been developing, uh, you can plug it into. As long as it's MCP supported, you can plug it in and then it will allow you to uh, utilize your own context, your own uh, training data and so on. I think this is, this is going to be an important consideration because While you all want to leverage the innovation that's there for regulated industries, this becomes uh, a big consideration. I hope that helps because this is one of the key things that organizations have been asking for.

Speaker B: Yeah, it's a key thing for not just highly regulated environments. Everybody's concerned about it, um, uploading their data to somewhere and you don't know where it actually ends up. Yes, even for private people. Right. I'm running my personal AI as well, uh, here on my MacBook.

Speaker A: I keep it like that. That's a good idea. Right. Otherwise you might find that whatever you are using, uh, actually ends up in the public domain. Right?

Speaker B: Mhm. Yeah. Yeah, it's something we don't want. Okay. Okay. Thanks Damien. So what skills and organizational changes are needed to truly benefit from Magentic AI?

Speaker A: Yeah, it's interesting, ah question. Especially because people in organizations are also worried because they're thinking m, is this going to replace me, do I have a job in future, etc. Uh, I think number one is adaptability. It's not a skill but a trait. If you're not going to embrace the technology, I think it will uh, at the end of the day make you obsolete. You need to uh, accept that AI is actually here to stay. Embrace that and then don't be afraid. So learn how it can augment you, make you more effective, make you better not be afraid that it's going to replace you and fight it. I think that's the first thing. The second thing is this whole concept of prompt engineering. I think it's a skill that all professionals need to start to learn. I think as an example, one of the concerns uh, some of our customers have when looking at our performance testing, uh, platform, it's called NeoLoad, is whether they would be able to learn how to use it technically. But now we've launched MCP for neoload. So with MCP for neoload, you can use natural language prompts now and just get results like you would a professional. So you no longer have to learn the tech and then be able to do some of the more technical, uh, stuff in order to get what you need right now you can just speak to it, you know, communicate with it like you would a human and say, this is what I want, this is how I want you to do it, etc. And it will give it to you. So imagine how wonderful that is. I truly think that uh, that is going to be important. But that's also based on prompt engineering. You need to be able to understand how to communicate and speak even with humans, Peter, you and me, we're speaking in English now. But uh, English may not be our first language, so we have to be careful how we phrase things and how we position it and the terms we use, etc. Uh, so that we don't misunderstand each other. So prompt engineering helps you do?

Speaker B: Mhm. Okay, thank you Damien. Very good. Then we're coming to our last section, Future Outlook and advice. So where do you see Enterprise QA heading over the next three, five years? Where do you see this whole industry moving to? Will manually testing entirely go away? What do you think?

Speaker A: Yeah, I'm not going to scare people who are uh, doing manual testing as a job today and say that it will go away. I don't honestly know. I think, I don't know if it's going to go away completely. I am sure that the need for intelligent automated testing is going to go up exponentially. So if I'm a betting man, I would start to scale up on how I can get into that field. Enterprise Quality Assurance, I believe, will be inundated with this deluge of code that is being generated by AI and along with this barrage of updates and upgrades to existing applications that we have, like your uh, ERP systems, etc. Uh, I think it's impossible to deal with all this manually. So again, I'm not going to make a prediction and say manual testing is going to go away completely, but I will make the prediction that intelligent automated testing is going to go up exponentially.

Speaker B: Mhm. Yeah, I think, I mean it makes sense. I think my opinion is there, there will still be fields where manual testing will be required, like what I mentioned earlier, testing a process at the end where the actually business has to accept the process as working. They have to validate financial data, logistics information where they then actually have to sign off and say, okay, we accept it as it is here, assuming that all the technical testing tests been performed before.

Speaker A: Completely. Completely. Although I did hear in a recent interview that bank CEO, a large bank CEO just said that the board thinks that uh, AI can replace the CEO in future as well. Right. You know, so who knows who will be signing off on those SMB end of the day?

Speaker B: Could be. Yeah, yeah, yeah, you're right. Well let's, let's see. I mean this is all in the very early stages, the AI, right? Uh, let's see.

Speaker A: Oh yeah, yeah. I would see what our kids will

Speaker B: experience at some point in time. Yeah, yeah, yeah, yeah. We just have to make sure they get the right education to do something which cannot be. So what, what advice would you give to tech leaders who are still a bit hesitant to adopt AI driven qa?

Speaker A: Well, anything new is always going to come with a little bit of hesitation. Right. You know, whenever I learn something new, I always also am excited, but I'm also concerned. Right. You know, and I think that's the kind of, uh, reaction that uh, most people would have. So it depends on the balance of that. I think the important thing for tech leaders to understand is that everybody is still learning and adapting at this stage. No one can say they, uh, are the foremost expert and they know exactly what they're doing. Right. You know, but having said that, virtually all enterprises have said that they're going to be adopting AI for quality assurance in the near future. Right. So not planning to do anything with AI in quality assurance is probably not a good strategy because you risk being left behind. The good news that I think tech leaders can, uh, also celebrate is that anything that now comes with the AI label is going to get C suite attention and C suite investment. So I would leverage that to drive, you know, some, uh, initiatives because there's a good, uh, likelihood that it will get funded. And from what I see, this is going to be, uh, uh, very helpful for tech leaders. Right. You know, for their organization, actually, in my opinion, for themselves too.

Speaker B: Mhm. Excellent. Makes sense. Getting fast approvals. It's always something we tech guys wish for.

Speaker A: Yeah, tell me about it. Right, yeah.

Speaker B: Okay, well, I have one last question. Are there any trends you're personally watching that could shape the future, uh, shape the next wave of enterprise software innovation? Yeah, I mean, not limited QA and testing in general.

Speaker A: Yes. I mean, obviously I'm going to have a QA slant here. Right. You know, we already spoke about genetic test automation. I think that's going to be very big. It's a trend that is very much gaining momentum. The other area I also mentioned earlier is quality intelligence. Peter. So I think it's not just about automating the testing. It's also about knowing what to test, when to test it, and ensuring that it has actually been tested. Right. I mean, these are important things to know. So when you don't know what needs to be tested, uh, you're going to default to either, number one, testing everything, or number two, testing nothing if you, uh, don't have enough time and resources to do so. That's not a great approach. Either approach is not good because if you are going to the formal approach of testing everything, many organizations have thousands or tens of thousands of, uh, regression tests, for example, that they need to run. It's a huge waste of time and money just testing everything because you don't know what needs to be tested. But if you can then change that situation and know exactly what you need to test, you can find that it takes a fraction of the time and effort to do so. And I think that's something that organizations are going to increasingly focus on. I think that's a trend worth looking at. Wouldn't you agree, Peter?

Speaker B: Yeah, absolutely. Uh, that's very well aligned with my thoughts.

Speaker A: I'm glad to hear that.

Speaker B: Yeah. Uh, amazing. Okay. Okay. Very, very good. Very good to learn from you here today. So amazing. Thanks again to Damian Wong and the team at Tricentis for this good interview.

Speaker A: Thank you, Peter, uh, for having me, uh, once again. Right. Yeah. I hope to join you again in the future for future, uh, podcasts.

Speaker B: I would be glad to do so. Thank you.

Speaker A: All right, Thanks for tuning in to another edition of the industrial IoT spotlight. Don't forget to follow us on Twitter IoT1.8 Ah. And to check out our database of case studies on iot1.com. If you have unique insight or a project deployment story to share, we'd love to feature you on a future edition. Write us@eric walenzaiot1.com.