AI Trust Architect: Building Responsible AI with Jill Heinze

Speaker A: Welcome to Future Proof, the podcast for founders, marketers and sales leaders who want to scale with clarity. Here we focus on how AI can drive growth, streamline decision making, and support smarter systems you can actually use. Practical, grounded, and built for the real world. Today's guest is Jill Heinze. Jill is the founder and responsible technology strategist at Saddle Stitch Consulting, where she works with organizations to design ethical, user centered and responsible AI strategies create value for both the business and the people behind it. With more than 20 years of experience across product strategy, research and AI governance, she has guided initiatives for academic institutions, digital agencies, and Fortune 500 companies. She also serves as responsible AI program director for the American College's Carrie M M McGuire center for Ethics in Financial Services and as an expert in residence with the newlore AI Governance and Literacy Fellowship. Jill is known for helping teams move beyond checklists and compliance to put responsible AI into practice. She brings a clear, practical perspective on managing risk, leading change, and innovating with integrity. Jill, welcome to the show.

Speaker B: Hey, Solomon, it's really great to be here. Thanks for having me.

Speaker A: Hey, my pleasure. You are the AI expert and I got so many questions. We're going to dive all into that. Tell me more before I get started. A little bit more about Saddle Stitch consulting. Just so we know where the name came from. And why did you call it Saddle Stitch? You know, it doesn't say AI consulting like it doesn'.

Speaker B: Well, that might be a mistake. We'll see. But, uh, no, actually the name of the company started many years ago, um, because I had it as a side business and then in the past few months have leaned into it as my primary consultancy. But I used to be an academic librarian. So a Saddle Stitch is actually a simple sort of binding that's very common in, like, books and pamphlets and things like that. But it also does the work of communicating what I care about, which is like connecting people together, connecting technologies and people. So it does a lot of work for. From like a metaphor perspective, which is why I kept it. But that's. That's the story.

Speaker A: I am familiar with Saddle Stitch binding. Oh, you are starting to make sense. Yeah, I've seen. I've seen books. It, it's. It. You can hold the book and then the. The covers will flap over. Correct? Like, yes.

Speaker B: Yeah. They don't fall out. And then a lot of people think of it like the horse. They're like, it. Stitches, Saddles. That's not what it come out.

Speaker A: All right. I love it. Love it. So your title is also an interesting name. Like that AI Trust Architect.

Speaker B: Yeah.

Speaker A: What does that, um, mean in AI question?

Speaker B: Well, I would argue there's a huge trust deficit in AI at the moment. So much hype, so much, you know, opportunity for things to kind of go sideways. So I see my job as getting really rooted in user behavior, understanding the ground truth, getting governance lined up so that from the get go we are building AI with integrity that maps to our uh, values. And that I think does foster trust. And I say architect because you do have to be pretty intentional about that. It doesn't have happen by accident. So yeah, we're playing with those two concepts, but I think it's very true to who I am and what we're trying to accomplish.

Speaker A: I love that. I mean first of all, having unique titles is the conversation starter.

Speaker B: Yeah.

Speaker A: Because you could have just put president and uh, and nobody would ask you a question.

Speaker B: That's right.

Speaker A: That's yeah, having unique titles. I like to put things like Chief Growth Architect, like, you know, I just like interesting ways to call what we do. It's, we're in, we're in doing that for people. So, so what made you realize that organizations need a, a structured way to think about the AI as opposed to just go to ChatGPT and upload your most valuable, you know, Excel spreadsheets and ask questions and God, where does it go? Nobody knows.

Speaker B: Happens. Yeah. Ah, well, it really struck me when I was in house at a digital consultancy and working with Fortune 500 companies, some of whom are in really high risk industries, financial services and healthcare. And I was in that agency when ChatGPT came onto the market. So it did feel like a something of a free for all. We're like, what the heck is this tool? We don't really understand but boy, we better be adopting it and putting it into stuff. That was kind of the mindset. But you know, I'm a user researcher, so I know just from regular tech that things do happen, harms do, and this isn't like a surprising thing. So when we're adopting a technology like Generative AI, which is so powerful and yet so unpredictable, it just makes sense that we need to do the work of making it safe, secure, understandable for people who are using it. So I had the opportunity to lean into that at my consult. I was designing our first ever executive AI governance group and I developed some frameworks that I led these Fortune 5000 through. And they appreciated it because they were like, I never thought about these things. And that was really the aha, uh, where I was like, oh, this is important and necessary and it complements what delivery teams are, you know, trying to do.

Speaker A: So let me ask you more practically, if a company is starting to use AI because everybody's already deep into it, what do you think they need to stop doing or slow down so they can get it right rather than just go all in and then they're like, oh crap, we just screwed up everything. How do we undo what we've done?

Speaker B: Yeah, yeah. Now I hate to say slow down because people get really antsy about that. So what I want to say is really more about like being intentional when you're looking at adopting or building AI. And so for me what I found is getting to the ground truth. Now that can mean you're actually sitting with your teams who are looking to use AI in their workflows to really understand what's happening there. Or you might be talking to teams adopting AI who are like, uh, oh, I don't know if I can put this information in the system. Cause I don't know, you know, if that's allowed or you know, it's going to go somewhere into like the training data or what have you. So that's the ground truth. That's where you need to reconcile what it is people are actually doing with where you want them to go. And then when you see the gaps, it's much easier to have a grounded governance conversation or implementation conversation around. Here's what people need to be prepared and equipped to use the tech in the right way.

Speaker A: And, and do you uh, have a preference on ChatGPT versus Claude versus you know, Microsoft has Copilot or is it all of the above that, that you know, you're recommending?

Speaker B: So I go, uh, there's a couple go tos and I think maybe everyone has. And then you pay subscriptions from. Right. So to some point you have to like make a uh, decision. So for me I use Claude. I do like Anthropic's at least stated commitment to safety. That resonates with me. It's very capable in particularly with like coding activities. It's of kind, kind of a favored tool. But I will say I've been using Gemini for market research. That's also a big part of what I do. And it does. It's just a workhorse of compiling information. So if you need, you know, something like that, I think it's very capable and it's kind of under, I don't know, rated under talked about.

Speaker A: It is what about Copilot? Because that's what a lot of corporations use because of maybe. I'm thinking it's more closed and they don't train with your documents if you have a subscription.

Speaker B: So I always warn people, you have to make sure you have the enterprise version of whatever. So in the case of Copilot, that's probably, um, built into your Microsoft subscription. So, yes, that is, you know, more secure. But I don't have. I don't use Microsoft. And in fact, most of the big companies do. But beyond that, like the long tail, it's mostly. We're like Google folks, folks out here, which includes myself, so. But I know it is also very capable. I have heard there's a lot of similarities.

Speaker A: That's amazing. So. So let's say they're already using AI and they didn't have a meeting with you. They don't know much about governance.

Speaker B: Mm. Huh.

Speaker A: Tell me more about how they could reduce their risk, you know, without stopping progress or slowing down progress, which I know it's a. It's a hard word. So what could they do? Because they're not waiting for it. No one's waiting for it. So they're already all in.

Speaker B: Yeah. Yeah. So they're off to the races. Yeah. So. Well, other than doing kind of the ground truth, like, let's assume you kind of have a good handle on where people are at, and now they're starting to adopt it. One of the things that I like to do is kind of. And this gets more into if you're delivering the software than adopting it, but to some extent, it still applies. I like to have folks think about, like, what could go wrong, like, what are you worried about? So I think people, as excited as they are, they always have a little something in the back of their head that they're not quite. Quite sure of. Right. So let's, like, actually talk about that and thank people for, like, bringing that to our attention. This is the kind of culture we want to, you know, support, where we're, you know, exposing our concerns and then proactively saying, well, you know, if your concern is where is data going? Well, we have enterprise plans. Here's how you access them, or these are the approved tools. Here's the kind of information you should and should not put into them, and you really kind of clarify the rules of the road, if you will, for people based on, again, kind of where they're at. So it doesn't have to be this huge overhead, but it does have to be, like, targeted help where they really need it, versus, like, here's 10 pages of, like, protocols that you need to follow to use these tools.

Speaker A: I love that. Uh, and I uh, want to switch gears to sort of the saddle Stitch framework. You have what's called the four AI horizons. Now tell us what that is.

Speaker B: Yeah.

Speaker A: What are the four AI horizons?

Speaker B: Yeah, I'm still kind of working through it, but I built it kind of by reflecting on what I was doing in practice at a more detailed level. So this is just a higher level version of that. And we've talked a lot about the key elements but which I will describe further. But I like the idea of Horizons because in the real world we don't always have these like critical decision points laid out really neatly in front of us. We're not always going through checklists, nor should we be when it comes to like responsible tech. So horizons give us a moment to, as it implies, like look ahead, look, you know, side to side, kind of expand what's in your vision and ask yourself curious questions along different phases of the AI adoption or build journey is basically what it is. So the first one is essentially, yeah, the ground truth. We talked a lot about that. Just know what's going on. Then it is about building your foundations. So if you need some approved tools, if you need to give folks the right guidance, those are the kinds of things you want to focus on at that point when you get to delivery, it's a little bit more of a fine grained pass about what are the risks inherent in the solution that we are trying to put together. And by the way, let's ask users about it, not just assume like actually test this out. And then finally whatever you've built in that infrastructure, you want to sustain that over time things are going to change. You want to make sure all that work you did to put your values into there, your intentions, your security protocols, your user input, that that's going to carry through for the long haul. So that's really the four horizons in a, in a quick nutshell.

Speaker A: And, and, and I think on our site you mentioned they break differently at different stages. Now I know you worked with a lot of Fortune 500 companies, tell us more about things breaking. I can see that from, you know, when you are a $3 million company, you have a different problem than when you're a $5 million company. So similarly I'm hearing that AI has, you know, AI adoption or the governance. There's challenges across different horizons. This episode is brought to you by IMS M. IMS helps companies use AI modern, uh, marketing and proven sales systems to create predictable growth. If you want better leads, stronger positioning and systems that can scale without complexity. Visit IMS m.net yeah, there are.

Speaker B: I think that's fair to say. And it also depends a lot on your risk profile. That's something we kind of glossed over in the foundation and governance setting. But you learn a lot about what kind of risks you're worried about or willing to take on and those that you're not kind of by doing that exploration. So for some folks it might be perfectly fine to have an internal tool for brainstorming. And yes, you've got your approved stuff. You're not putting IP into it. You don't really need to belabor that. But the kind of risks that you really want to think about are particularly those when you get into like a B2C situation where you're talking about vulnerable populations. You'll see a lot in, going on in, in the law and regular legal and regulatory environment with respect to children. I think for this reason, because we are seeing a lot of actualized harms, if you will, like breaking in real use context because younger folks in particular are using general purpose technologies like ChatGPT in ways that were not, it wasn't designed for.

Speaker A: Yes. And it doesn't understand that you're not supposed to be uploading these things. So whatever you train it, personal information, personally identifiable information, I can understand like, you know, medical people have hipaa and what if they're using it and they didn't realize that? So yeah, there's so many issues. Do you think that every organization should have some sort of a, uh, responsible AI sheet in their employment, you know, handbook? Yeah, because, you know, it seems to me like, you know, if sexual, uh, whatever it is, you know, you know, and then now you have this other thing and then now you have responsible AI. Like I think it has to be. Otherwise if they don't, uh, adhere to some sort of a rule, they're just going to do whatever they feel like doing. And it's all accessible online, so it's not like the business owner can stop them from going, going to chat.

Speaker B: G. Yeah, you get into a lot of problems with shadow AI as they call it. Folks are bringing in their own technologies for their own reasons. And you should be curious about why that's true. Because if there is a gap, then I would argue leadership could kind of, you know, help to understand and fill that need for folks in a way that is more secure. But I totally agree with you. I hear people talking about AI as kind of a utility like Internet or electricity, and if that's the case. It's just like a uh, you know, something that we all have to learn to deal with. You're not going to not be using it. I think is, is really key so having some rules of the road. But again I don't like to think of it as kind of a heavy handed thou shall not. It should really come from what your employees need to know, uh, to do the right thing very easily. And then over time as your use grows more sophisticated, so too can those you know, practices that you're putting into your governance. But it doesn't have to be all at once or super heavy.

Speaker A: Yeah. So tell me more about like if, if somebody is trying to make responsible AI part of their organization which I think every at this moment, what are they assuming incorrectly when they, when they're trying to do this versus what really is right. Like I wouldn't know what it is because I'm not living in that world so. And I don't have a lot of safeguards currently because we're all in on AI so we're like go figure out how to hundred extra productivity, you know, there you go. Simple enough, you're looking for trouble and so tell me more about what, what every, every leader is like, oh, you must use AI like me. Like go. And then. And they're assuming that it's automatically responsible. So Jill, walk the. I guess you know, what we don't

Speaker B: see, it's what you don't see.

Speaker A: Yeah, yeah, yeah. The blind spots.

Speaker B: Mhm. I think again, yeah, it does come back a little bit to the wrist. So I don't have any objection to the go, let's use AI because that's how you start to stress test how well it works in different scenarios. Gosh. I was just using Claude to clean up some language today and after a while I was like, you know what, I'm just gonna like kind of rewrite things. Like it helped me get started but I soon realized okay, there's some boundaries in terms of how I want to use it. In this case I wouldn't have understood those nuances if I'm not actively playing with the technology. But again when you get into scale and you get into the B2C context in particular, but could be your internal folks as users you do need to be a little bit more concerned about unintended consequences and we are just not creative enough in my opinion in envisioning those. It's not fun to think about risk and all the bad things that could go wrong. But there are databases online that collect AI harms or incidents that you can look through just to find, you know, what's the precedent for this technology in this scenario of something going astray. And if you invest just a little time in ideating in that way, all of a sudden you are empowered with so much more knowledge about what you can, you know, should be avoiding how you can avoid it. I mean, that work really pays off in terms of, you know, solidifying and making your systems more secure. So I think that's pretty amazing if you can look at risk as a opportunity to find advantage, uh, which is what I advocate. But you have to push yourself to think, how can this go wrong and go off of that happy path, which is where everyone wants to stay on. Because it's easy, it's frictionless, and we can go faster.

Speaker A: I love that. Probably want to ask you one question about leadership again, because I feel like people take AI and push it to the tech department, you know, the, the IT department is going to figure out AI because the CEO doesn't know or coo, which I've seen. Right. The CEO gets involved and says, well, I need so. And so here he, he deals with our it. What decision should never be delegated when AI is involved, even if they don't know what they're doing? Because like I said, the IT person is not thinking what Jill just said about responsible AI and governance and sensitive data and uh, B2C contacts, you know, anything that is vulnerable and that you shouldn't be putting online. What, what, what, what would you say to that?

Speaker B: Ooh, that's a really great question. And this is so true. Even in traditional software development. I remember one time talking to an engineering friend who was like, we can't put maybe into code. So for every day decision you offload onto the frontline devs, I mean, they're, they're embedding their judgment in there. And I don't think that that is right when it comes to things like organizational values or organizational risks that we have defined as off limits. So, I mean, what a burden that is to think about. And this, I've seen teams struggle with this. Like maybe they are looking at a, um, video that appears to be a deep fake video. Do we, what do we do about this? Or you know, should we, you know, even use tools to generate images that could be copyrighted? I mean, these are kind of we decisions. They involve legal input, you know, end user input. So those higher risk considerations should be defined well ahead of time and communicated to really everyone on the project team. So I think you're right. There are some things they should just never be burdened with having to figure out.

Speaker A: Right. And do you think that the CEOs should be the ones more involved in what is considered responsible and safeguarding the information? I think, rather than delegating?

Speaker B: Yeah. Oh, for sure. Particularly again, for high risk. But we have to figure out what that is. That is some effort unto itself. And once that is determined, then I think we need the communication line so that the CEO is informed, you know, proactively if things are, you know, on the cusp of these, you know, risk categorizations that we've established so that they can weigh in or delegate as needed. So the communication has to go two ways. I wouldn't say dump it all on the CEO, but everyone has some accountability that should be defined.

Speaker A: Agree. So there's probably thousands of founders like myself and like yourself. It's under pressure to add AI to their product, AI to their business. There's nobody I know that isn't building some AI on the side because they feel like they're missing out. Uh, what advice would you give that founder?

Speaker B: Yeah, well, first of all, you're not missing out. I mean, I think we need to take the temperature down just a little bit in terms of speed to market and how valuable that really is. I mean, I know you've probably heard the MIT study, about 95% of AI initiatives are failing. And the reasons that they are failing are because of things like people are targeting the wrong use cases, they're not going deep enough into their workflows to really question what's going to bring value. So they're targeting the wrong things. So what I would say is really figure out the impact that you want to make on the world with your, with your product, with your vision, and then back into talking to real people, your staff, your end users and just figuring out the right, you know, blending of your abilities and the AI. And don't let the AI drive know you, you, you know, you, you are the boss of the AI. Right. It should be tailored to incorporate your values, your mission, your goals and not have it kind of work the other way.

Speaker A: I one, uh, hundred percent agree. And I think uh, you're, you're taking more of a strategic role as opposed to just short term thinking. Sprinkle some AI into the product and never giving it the thought. And that's what we really do in, in our organization. We try to give vision to our customers. Like, hey, let's think long term. What do you want to have happen happened Right. So I like that approach because under pressure you kind of make short term decisions and never really think about what the impact could be long term.

Speaker B: Yeah.

Speaker A: Jill, you are an AI expert and you just blew my mind with this conversation. Where can they learn more about you?

Speaker B: Yeah. Well, you are welcome to find me on LinkedIn under Jill Stoverheims. I'm on there all the time. I'm also on YouTube and Instagram and Facebook. And with Jill, Saddlestitch is one way to find me. But also you can go to is saddlestitch consulting.com.

Speaker A: love it. Well, Jill, thank you so much for kind of making us sober about AI and responsible AI because like I said, it's, it's, it's a tool. It's going to help us do exponential things. But you have to do some, you have to have some safeguards, right? Would you agree that?

Speaker B: I'm glad to see you're doing that yourself? So that's.

Speaker A: Thank you so much. We'll see you next time.

Speaker B: Thank you. Bye.

Speaker A: Thanks for listening to Future Proof. If you enjoyed this episode, subscribe and leave us a review. It will help more leaders find this show. And if you want to see how IMS, uh, helps companies grow smarter with AI and sales systems, visit IMS m.net As always, thank you so much for listening and I'll see you next time.