Beyond AI Hype: Building Governance-First Systems (Joseph X Ng)

1 - > Andreas Welsch: Welcome back for another episode of"What's the 2 - > BUZZ?", where leaders share how they have turned hype into 3 - > outcome. 4 - > Today we'll talk about what happens when different 5 - > technologies converge, right? 6 - > We've been talking so much about AI that we are almost neglecting 7 - > things like cybersecurity and quantum and how they fit in.

8 - > So I'm super excited to welcome Joseph Ng on the show to talk 9 - > more about that. 10 - > Hey, Joseph. 11 - > Thank you so much for joining. 12 - > Joseph X Ng: Thank you for the invite.

13 - > I'm happy to be here. 14 - > It's very exciting times indeed. 15 - > Andreas Welsch: Oh yeah, for sure. 16 - > Joseph I was wondering if you can introduce yourself real 17 - > quick to the audience for those who might not know you yet.

18 - > Joseph X Ng: Oh, hello. 19 - > My name is Joseph Ng, and I am the chief strategy officer for 20 - > Gene Genius, and I recently published a book named The 21 - > Hybrid Mind: The Human-AI Convergence. 22 - > Andreas Welsch: So- That's exciting. 23 - > Yeah.

24 - > So- Yeah I know you're right at the center of where all of these 25 - > things come together. 26 - > We've obviously been connected over LinkedIn for a while. 27 - > We got to meet at the AI summit at the end of last year and said 28 - > we, we should definitely spend some more time talking about 29 - > what happens when all of these things converge. 30 - > Now obviously there is so much focus on the AI race in- I'm 31 - > wondering what are you seeing?

32 - > What's your take? 33 - > Is this really where companies should focus all of their 34 - > efforts these days? 35 - > Joseph X Ng: That's a great question. 36 - > When people talk about AI race, they usually frame it as a race 37 - > for bigger models, more GPUs, faster inference, or lower 38 - > costs.

39 - > That matters, but I do not think that should be where all the 40 - > resources should be concentrating. 41 - > These things will continue to evolve. 42 - > The more important question is whether an organization actually 43 - > knows how to operationalize intelligence inside the 44 - > business. 45 - > AI is moving from being a tool that analyzes information to 46 - > becoming a system that participates in those c- 47 - > decisions.

48 - > As that happens, the real challenge is no longer access to 49 - > AI, the challenge becomes architecture. 50 - > How do you govern the system? 51 - > How do you manage risk oversight and human authority when 52 - > intelligence is embedded directly into the workflows? 53 - > Andreas Welsch: Yeah.

54 - > Joseph X Ng: So if I was advising companies, do not spend 55 - > all your time chasing the AI race as if winning means having 56 - > the biggest models or most compute. 57 - > Spend your time in the architecture. 58 - > And then allows AI to operate responsibly and productively 59 - > inside your institution. 60 - > Andreas Welsch: I think that's a very sensible call, es- 61 - > especially these days when, one headline chases next There's a 62 - > new model that comes out almost every other week it seems, or at 63 - > least a new feature.

64 - > So making sure you, you actually get your foundation right. 65 - > And it's great because I think a lot of times we talk so much 66 - > about data as being a foundation, but I think if you 67 - > go one level above that it is indeed the architecture, the 68 - > orchestration, the guardrails, the guidelines the evaluation 69 - > and so on. 70 - > So good to hear that for sure. 71 - > But what does it mean from your perspective when AI participates 72 - > in these discussions like you said, or it participates in 73 - > decisions?

74 - > What does it mean? 75 - > How does it change the dynamic? 76 - > Joseph X Ng: Historically, AI sat outside the decision 77 - > process. 78 - > It generates insights, dashboards, predictions.

79 - > The human was still clearly in control of the final decision. 80 - > What's changing now is that the AI systems are actually starting 81 - > to act with- inside their workflow. 82 - > They are approving transactions, generating responses, even 83 - > triggering downstream actions across systems. 84 - > At that point, AI is no longer just supporting decisions, it is 85 - > part of the decision-making process itself.

86 - > You can no longer rely on traditional assumptions around 87 - > accountability. 88 - > You need to be able to answer very basic but critical 89 - > questions. 90 - > Why did a system make that decision? 91 - > What data did it use?

92 - > What rules did it apply? 93 - > And most importantly, if something goes wrong, who is 94 - > responsible? 95 - > I address this problem in my book The Hybrid Mind. 96 - > The idea is that when humans and AI systems are operating 97 - > together, they ne- need a structured way to manage that 98 - > interaction.

99 - > That is where concepts like OODA loop, observe, orient, decide, 100 - > act, and most importantly, feedback- is very key. 101 - > Andreas Welsch: Yeah. 102 - > Joseph X Ng: Yep. 103 - > Andreas Welsch: I think es- especially this part about, 104 - > observing and feedback, that's, that sounds like a huge 105 - > opportunity.

106 - > I was just talking to a former colleague of mine and we were 107 - > working on machine learning projects like almost 10 years 108 - > ago. 109 - > And one of the big challenges was whenever we had a 110 - > substantial amount of new data, say six months after a model 111 - > training, we would have to go back and retrain the model on, 112 - > on, on that new larger set of data. 113 - > And you would have to test, and you would have to see does it 114 - > still decide and act in, in ways that you had already previously 115 - > approved, or do you need to make changes again?

116 - > So to me this feedback loop, this a- almost constant learning 117 - > that's a real big benefit of looking at things like agents 118 - > that they can pick up information on the go or for the 119 - > next go around, basically. 120 - > So a lot of promise in, in my mind, a lot of opportunity to 121 - > cut down on this batch type learning when it's more 122 - > interactive and more on the go. 123 - > Yeah. 124 - > I see a lot of companies de- deploying these AI tools whether 125 - > it's,"Hey, we've given everybody Copilot and we're done.

126 - > That's our AI strategy," or,"We, we, we have this little feature, 127 - > I don't know in, in our travel and expense app and you can scan 128 - > your receipts and now we're all AI enabled." But I feel a lot of 129 - > times this type of mindset falls short of expectations of 130 - > delivering something bigger, more, more meaningful that 131 - > captures a lot more value than just giving somebody a tool and 132 - > expect them to work on that. 133 - > What are you seeing?

134 - > How do you feel organizations can evolve? 135 - > What are some of your... 136 - > the recommendations that you have there? 137 - > Joseph X Ng: Definitely.

138 - > This is a very key question. 139 - > What most companies are missing at this point is that they're 140 - > still treating AI as a tool deployment problem- When it's 141 - > really a decision systems redesign problem. 142 - > Right now, a lot of pro- organizations are adding AI in 143 - > very narrow ways. 144 - > They introduce a chatbot, a co-pilot, like you're saying- An 145 - > automation fe- feature, and they call it this, their AI strategy.

146 - > In reality, what they have done is a layer of intelligence on a 147 - > legacy operating model- without changing the architecture 148 - > underneath. 149 - > Andreas Welsch: Yeah. 150 - > Joseph X Ng: That works for small productivity gains, but it 151 - > does not hold once AI starts influencing judgment, routing 152 - > actions, or participating in decisions that have business and 153 - > regulatory and human consequences. 154 - > The gap is that most companies have not mapped these decisions 155 - > or action made independent of humans at this point.

156 - > Andreas Welsch: So- 157 - > Joseph X Ng: Which is the logic behind cognitive AI a- and 158 - > native architecture or CANA. 159 - > CANA is not about adding AI to the stack, it is about 160 - > restructuring the stack so intelligence, oversight, and 161 - > human authority are built into how the system operates from the 162 - > start. 163 - > Andreas Welsch: So question there, I know you've previously 164 - > worked in financial services for a large and well-known financial 165 - > services company.

166 - > I'm assuming there's already a muscle built o- over the years 167 - > from predictive analytics to machine learning, to generative 168 - > AI, to agentic AI. 169 - > And I see this a- across many different industries, that the 170 - > larger the organization, the longer they've been at this 171 - > problem and have been working with this kind of technology, 172 - > the more it becomes second nature and the next evolution of 173 - > technology changes a few things, but not fundamentally what the 174 - > organization is.

175 - > But I also see smaller organizations struggling with 176 - > this, right? 177 - > When you talk about architecture it's not that easy for a 178 - > mid-sized bank, for example or mid-sized in- insurance broker 179 - > to do that heavy lifting of thinking about what does our 180 - > architecture look like. 181 - > What do you recommend there? 182 - > What can companies and leaders do that's practical when they 183 - > don't have a large centers of excellence or dozens or hundreds 184 - > of people working on this, but two or three?

185 - > Joseph X Ng: So what I would do or recommend for these 186 - > organizations that are small in scale is to first do a audit of 187 - > their current systems and their s- systems in general, and then 188 - > see where those breaking points would be. 189 - > Yeah. 190 - > And then how we can integrate AI into the mix. 191 - > It's not a full redesign.

192 - > It's more modular in essence. 193 - > Andreas Welsch: So that, that also means reducing your risk a 194 - > little and being able to show something more quickly and make 195 - > progress and continue making progress as you go. 196 - > Joseph X Ng: Exactly. 197 - > And this is what I call phased approach.

198 - > Whenever we're doing large projects, we break it down into 199 - > certain phases. 200 - > Yeah. 201 - > And es- especially creating some objectives and within there- To 202 - > accomplish s- goals, right? 203 - > So yeah.

204 - > I think this is the best approach to a small 205 - > organization. 206 - > Andreas Welsch: Thank you. 207 - > Thanks for sharing. 208 - > That's great.

209 - > Maybe you're seeing this next thing too, right? 210 - > A lot of times I feel that I see this kind of low quality output 211 - > or work slop in my inbox. 212 - > "Hey, can you take a look at this? 213 - > I created a first draft.

214 - > What do you think?" And you start reading and you're like, 215 - > "Yeah, I don't know. 216 - > It's not bad, but it's also not good. 217 - > It's not very authentic.

218 - > It misses details. 219 - > It misses depth. 220 - > It misses teeth." That's one of the big challenges I see and 221 - > that I hear from former colleagues as, as well in, in 222 - > corporate who say,"Hey, I get all of this stuff and it reads 223 - > like somebody just pulled this out of Copilot or ChatGPT."

And 224 - > to me it feels where seven years ago, five years ago, even before 225 - > ChatGPT we're talking so much about business processes about 226 - > business functions and their productivity, changing processes 227 - > with AI. 228 - > Now we're in some ways a step back talking about personal 229 - > productivity, like you said. 230 - > But what are some of the other mistakes maybe or other 231 - > challenges that you see that companies are running into when 232 - > they just roll out an AI tool, a gen AI tool, maybe now agentic 233 - > AI tool?

234 - > What else are they missing? 235 - > Joseph X Ng: One of probably the biggest mistake that 236 - > organizations are making right now is they're scaling 237 - > capability much faster than they are scaling control. 238 - > There's a lot of pressure to adopt AI quickly, deploy models, 239 - > launch features, automate workflows. 240 - > A- and everything on the surface looks like it's progress.

241 - > Systems become more capable, and organizations feel like they're 242 - > moving forward. 243 - > But underneath that, many of the systems are being deployed 244 - > without a clear understanding of how they behave- how they are 245 - > exposed, and how they can be influenced. 246 - > So what you end up with is a huge gap that increases risk, 247 - > and that risk compounds very quickly. 248 - > Andreas Welsch: I think that's an important point, right?

249 - > A lot of times we think about this being additive, but it's 250 - > actually more exponential. 251 - > Thinking about agents or multiple agents in a system. 252 - > It's not risk plus risk, but plus risk, but times, right? 253 - > When each part in the chain be- becomes a failure point or it 254 - > becomes a breaking point.

255 - > Joseph X Ng: Exactly. 256 - > Andreas Welsch: Yeah. 257 - > So there's specifically it's not just again, a- about rolling out 258 - > tools. 259 - > It's about understanding what does that risk mean.

260 - > And it's funny you say we're rolling out capability faster 261 - > than governance. 262 - > I'm thinking in, political terms. 263 - > If I look at the US, if I look at Europe if I look at Asia on a 264 - > larger scale we're seeing how this can play out, right? 265 - > Being more liberal with innovation in trying to be first 266 - > to market or in, in that AI race on a on a global scale competing 267 - > others in the EU taking a much more deliberate a- approach and 268 - > starting with regulation first.

269 - > Say,"How can we protect our citizens? 270 - > How can we protect the data? 271 - > How can we uphold our values?" And I'm seeing this, to your 272 - > point on a smaller scale in businesses too.

273 - > But certainly when AI labs push out innovation weekly some even 274 - > daily it seems and you're still thinking in quarterly roadmaps 275 - > or in, half-year planning cycles. 276 - > I'm sure your executives are asking,"So what are you doing? 277 - > How come we're not shipping as, as fast?" So it's definitely a 278 - > challenge for sure.

279 - > But a- along with that there, there are risks beyond the high 280 - > level what if something fails if you introduce AI, if you 281 - > introduce agentic AI. 282 - > What are some of those that, that you typically see, and what 283 - > do you advise companies and leaders on addressing them? 284 - > Joseph X Ng: Risks in AI is very interesting because there's a 285 - > new technology that's coming around the corner, right? 286 - > Quantum changes the conversation because it is no longer a 287 - > incremental improvement in computing.

288 - > It fundamentally expands what computationally is possible. 289 - > With classical systems, there's entire categories of problems 290 - > that are simply out of reach right now because of the how 291 - > long it would take. 292 - > Quantum computing changes that. 293 - > It opens the doors to solving highly complex optimization 294 - > problems, simulate mo- molecular interactions at a level of 295 - > precision we have never, ever seen before.

296 - > Andreas Welsch: Yeah. 297 - > Joseph X Ng: And advancing probabilistic modeling in ways 298 - > that could reshape industries like healthcare, science- And 299 - > finance. 300 - > The biggest concern for me is the cryptography. 301 - > Today, most of our digital infrastructure relies on 302 - > encryption methods that are secure, but because they are 303 - > computationally difficult to break- and it takes many years 304 - > in order for it to be possible.

305 - > Quantum computing changes that assumption, and the issue is not 306 - > just future risk, it's actually now. 307 - > There's a concept called harvest now, decrypt later- where 308 - > sensitive data is being collected today with the 309 - > expectation that it could be decrypted once computing powers 310 - > like quantum matures. 311 - > The real question is whether we are building the governance, 312 - > security, and infrastructure to handle what technology becomes 313 - > possible when it does arrive.

314 - > Yeah. 315 - > Andreas Welsch: So I'm part of a of a global network of chief 316 - > architects and recently this topic has been moving more to 317 - > the forefront as well to say, "Hey, what are we actually doing 318 - > to a- address this and what can we do proactively?" Because we 319 - > see the writing on the wall as, as soon as quantum is out there 320 - > and it's commercially viable and usable, there is a high risk 321 - > that our current cryptography methods are out the window 322 - > there.

323 - > They're useless or in- ineffective. 324 - > And I think that's one of the key risks I'm not seeing enough 325 - > people talk about yet because we're so focused on,"Hey AI 326 - > productivity," and summarizing meeting minutes and all that 327 - > good stuff. 328 - > But when this quantum leap is really next, what are some of 329 - > the, again, things organizations can do right now? 330 - > How can you harden your security and what can you actually do to 331 - > prevent some of the harm that is likely going to come in the 332 - > future?

333 - > Joseph X Ng: And you're totally right, Andreas. 334 - > Most organizations are still asking which model to use or 335 - > which to deploy, right? 336 - > Those are downstream questions. 337 - > The real risk of, o- of the systems is what we're talking 338 - > about quantum, right?

339 - > The first step to map is to map out the decision systems that we 340 - > have, not just the workflows. 341 - > Leaders need to understand where decisions are made, where AI can 342 - > advise or well advise or act. 343 - > Yeah. 344 - > Okay?

345 - > The second a- is architecture. 346 - > This future is not a single model layered onto a legacy 347 - > system, but it is a actual orchestration environment of 348 - > models, agents, data, and humans. 349 - > If the architecture is not designed for that, governance 350 - > will always lag behind capability. 351 - > And finally, Third is the governance layer, right?

352 - > It has to be embedded into the execution. 353 - > It cannot live in a policy somewhere in your network that 354 - > no one l- reviews for a few years, and then finally someone, 355 - > something or some event happens, and then you have to pull it up 356 - > in the middle of the night, right? 357 - > Andreas Welsch: Yeah. 358 - > Joseph X Ng: It needs to exist inside the system through chase 359 - > ability, access control, and clear human authority over 360 - > critical decisions.

361 - > And the role of the leadership is not just to adopt AI, but is 362 - > to build an institution that can operate both intelligence 363 - > responsibly, visibly, and at scale. 364 - > Andreas Welsch: I think that's a key point. 365 - > Thinking back roughly 25 years I remember in cybersecurity there, 366 - > there were these rainbow tables. 367 - > Basically, what are the passwords or common passwords 368 - > and how can you guess them or through brute force get in.

369 - > We've moved on to more complex passwords, right? 370 - > Should be at least, I know, 16, 32 characters special 371 - > characters, uppercase, lowercase, numbers and whatnot. 372 - > Now we're talking about pass keys, but even that, like with 373 - > this kind of technology doesn't stand a chance. 374 - > So to your point it takes a multilayered approach.

375 - > In, in, in some ways I feel people are still going to be the 376 - > weakest link in your technology strategy, let alone now being 377 - > technology or technology even being weaker than it used to be. 378 - > But the other part that I think is more more hopeful, I would 379 - > say, or more, more promising more, more positive in, in, in 380 - > that sense is you say, hey, quantum computing you can find 381 - > new molecules, you can do a lot of other things. 382 - > So let's talk about that very briefly too.

383 - > Aside from cracking cryptography, what are some of 384 - > the good things that you can actually use the technology for? 385 - > Joseph X Ng: There's many possibilities within quantum 386 - > that can be very positive and very impactful, right? 387 - > I think in terms of quantum, there's a lot of opportunities 388 - > in genomics. 389 - > So within GeneGenius we will be excited to explore that 390 - > capability once it's available.

391 - > And how do we map genomes or the human genomes, right? 392 - > And then provide capability of discovery on drugs, and then the 393 - > n- next enhancement of drug capabilities to solve some of 394 - > humans' or society's most rare diseases. 395 - > Andreas Welsch: That sounds that sounds very promising and like a 396 - > like a great cause for sure, right? 397 - > Now you've also talked a bit about your book The Hybrid Mind.

398 - > In it you, you also talk about what that organization looks 399 - > like that fuses machines or AI and humans to-together, or where 400 - > they collaborate. 401 - > When I say fuse I don't know, it doesn't sound quite right, but 402 - > where they work effectively together. 403 - > Let's maybe go with this one. 404 - > What does that organization look like?

405 - > Joseph X Ng: W- what led me to write The Hybrid Mind was not 406 - > just a single moment, but a pattern I kept seeing within the 407 - > industry, academia, and also the real world, right? 408 - > In enterprise environments, especially in regulated sectors 409 - > like banking, I saw how decisions are actually made. 410 - > As AI entered those environments, it became clear 411 - > that the technology was advancing much faster than the 412 - > systems designed to govern it.

413 - > At the same time, in academia there was a gap. 414 - > Conversations were focused on models and performance, but not 415 - > on what happens when AI is embedded into real 416 - > decision-making systems and who is accountable. 417 - > Through the, a work in AI governance and platforms like 418 - > GeneGenius that we just talked about, I saw the shift 419 - > firsthand, where the challenge is no longer just building 420 - > something that works, it is building something that can be 421 - > trusted, audited, and aligned with human responsibility.

422 - > The book is an attempt to formalize that shift and 423 - > bringing together concepts like governance first design into 424 - > something organizations can actually apply. 425 - > At its core, it is about how institutions need to evolve when 426 - > intelligence is no longer outside the system, but is 427 - > enclosed and operating within. 428 - > Andreas Welsch: Yeah. 429 - > Awesome.

430 - > So folks, for you in, in, in the audience, definitely recommend 431 - > pick up Joseph's book, The Hybrid Mind, to learn more about 432 - > how to facilitate that change and prepare for what's next. 433 - > Now Joseph, we're getting close to the end of the show, and I 434 - > was wondering if you can summarize the key three 435 - > takeaways for our audience today. 436 - > We've covered a lot of ground but what are the three things 437 - > that folks should take away from our conversation today?

438 - > Joseph X Ng: All right. 439 - > AI is moving from a tool that analyzes information to a system 440 - > that participates in decisions. 441 - > Once that happens, the focus is no longer just capability, it 442 - > becomes control, accountability, and design. 443 - > The AI race is not about where differentiation will come from.

444 - > It will come from who can build the systems that govern how 445 - > intelligence operates inside organizations. 446 - > On the security side, the attack surface has changed. 447 - > Systems do not need to be broken into. 448 - > In the traditional sense, they can be influenced through normal 449 - > transactions and interactions, which means governance and 450 - > security must be embedded directly into the architecture.

451 - > Andreas Welsch: Yeah. 452 - > Joseph X Ng: Quantum reinforces the same point. 453 - > Every leap in compute increases both opportunity and risk. 454 - > The more powerful these systems become, the less tolerance there 455 - > is for weak infrastructure and delayed preparation.

456 - > For leaders, the shift is clear. 457 - > This is no longer about adopting AI tools. 458 - > It is about redesigning how decisions are made, how systems 459 - > are structured, and how responsibility is maintained 460 - > when intelligence is shared between humans and machines. 461 - > That is what the hybrid mind is all about.

462 - > It is a blueprint for operating in a world where intelligence is 463 - > distributed, automation is s- consequential, and governance 464 - > must be built into the system from the start. 465 - > Andreas Welsch: Awesome. 466 - > Thank you. 467 - > Yeah.

468 - > Thank you so much for sharing, Joseph. 469 - > It was a pleasure having you on. 470 - > Thanks for sharing your expertise with us. 471 - > Always great talking to you.

472 - > And yeah, thanks for joining. 473 - > Joseph X Ng: Oh, thank you so much. 474 - > It was a fun, fun discussion, and very critical timing too.