Slow Down to Speed Up: Jonathan Spector on Establishing AI Guard Rails

Speaker A: What I've seen is it's not mainly a technology conversation, it's a leadership, governance and execution conversation. And I think the starting point for leaders is don't start with the tool, but start with the outcome. Get clear on what problem you're trying to solve, what risks you're trying to reduce, what decisions you're trying to support, and then put the right guardrails around it.

Speaker B: In a world filled with chaos and a myriad of risks, there is opportunity. You are listening to Riding the Wave Project Management for Emergency Managers, where we discuss how we adapt and rise above those rolling waves of hazards and threats we face and rise to the top. And now your host, the president of Pinnacle Performance Management, Andrew Boyarsky.

Speaker C: I'm speaking with Jonathan Spector, who is the founder and principal of Converged Strategic Partners, a boutique advisory firm that helps executive teams accelerate AI adoption, align around high stakes priorities, and translate strategy into execution. As always, a longer bio and links to contact Jonathan are in my show notes. Jonathan delivered a great webinar recently for the All Hazards Consortium titled Leading Through AI for Emergency Management Leaders. Uh, that was delivered in February where I met him. I, I would like to build on previous, uh, podcasts that I've done and articles where I discussed the opportunities and risks associated with adopting AI and emergency management. So first of all, uh, Jonathan, I want to thank you for joining us as a guest.

Speaker A: Uh, appreciate being here. Looking forward to the conversation.

Speaker C: So, in your recent webinar for the All Hazards Consortium, you stated, don't try to go from 0 to 100 mph. Go from 0 to 10, then from 10 to 20. If you manage to go from 0 to 10 mph, then you've increased your efficiency tenfold. I think that I can sum this up by saying go slow to go fast. Uh, which is in contrast to my previous episode where we I said go fast to go slow, uh, so folks can listen to that one. To frame this for our audience, who is a broad group of emergency managers. Most of them are a force of one person or a multi hatted emergency manager. What would you consider the minimum set of governance decisions, uh, that they must lock down before they begin to touch AI so they're not accidentally building some sort of unauthorized shadow program.

Speaker A: Sure. Uh, no, I appreciate you starting there and I did enjoy the conversation with, uh, with Tom in your last podcast. He's um, a great history, great guy and doing some exciting things in the AI world today. Um, so I think it is a powerful place to start in the conversation here versus Jumping right into a tools conversation right out of the gate like so many leaders want to do. Uh, what I've seen is this is really, it's not mainly a technology conversation. It's a leadership, governance and execution conversation. And I think the starting point for leaders is don't start with the tool, but start with the outcome. Get clear on what problem you're trying to solve, what risks you're trying to reduce, what decisions you're trying to support, and then put the right guardrails around it. And the reason the governance matters is not because policy is the point. It matters because it's what lets you pursue value without losing trust, judgment or control. So to your specific question around governance, those who have worked with me know that I'm not one for writing novels and creating shelfware, whether it's in strategic planning or otherwise. And the same thing I think really applies here. You don't need a 40 page policy before uh, you start scaling. You do need four or five clear rules that keep people out of trouble. Approved tools, uh, deciding what tools people can use and what's off limits. And as an aside to that, don't get hung up on the best tools. As Andrew, you well know, the capabilities are literally changing weekly. Uh, data categories, defining what can go into AI, what requires caution, what's prohibited thinking, you know, in simple terms. I had this in my webinar around the stoplight model, identifying green light, yellow light, red light uses and then related to that, uh, are use case boundaries similar to the stoplight model, a verification standard, deciding what must be checked before it can influence action and decision, ownership, being explicit about who owns the final call when AI is involved. This simple one page stoplight I think is really enough to start. Green for public information and formatting, yellow for internal drafts that require review. Red for life safety sensitive regulated information. That's the minimum model that really makes safe experimentation possible. And I'll say one more thing Andrew, on this is that, and I think this will resonate well with this audience. As important as establishing governance and guardrails are you also need the scenario plan for your worst case scenario. Exercise this in this AI world.

Speaker C: So on that note, okay, I'm a force of one. I'm a multi hattered em. I might be a deputy fire chief or the fire chief. And my um, other responsibility is also the emergency management portfolio. Who else do I need to bring into this picture so that I have a greater review and, and how do I enlist them?

Speaker A: I think there are co pieces to that. If you're A force of one or heavily multi hatted as many in this world are. Uh, it's less about having the perfect committee, but really building this small trusted circle to the best that you can. And that could be one operational peer who understands field consequences, uh, an IT or security contact. Even if it's a fractional person supporting your organization or somebody shared regional or countywide that you can tap. There's a legal and a privacy component when your use cases are going to touch. Contracts, public records, privacy, hr, procurement, any of those regulated areas. And if you're truly, you know, alone establishing this micro network that could be neighboring emergency management peers, a duty officer, mutual aid partner, regional consortium contact like where we originally met. And I think really tying this back to the governance discussion, I was working with a client where the challenge was a small leadership team that knew people were already experimenting with AI, but they didn't have the capacity for a major governance effort. So the move was to create a very lightweight operating model. Approved tools, simple data categories, review expectations, and a few clear red lines there. And that was enough to shift them from informal inconsistent use to something much more deliberate. So I think it's less about hierarchy. And the point is really distributed judgment around use, risk, consequences. Small agencies, very small. An individual have to think in terms of a lightweight review cell, not a big governance board. And your last comment about how do you enroll them in this? I think there's mutual learning, people want to share. Nobody has figured it all out. And so if I can help you and tell you what I've learned, then I'm also going to learn from you in the process. So I think there's a lot of value that is relatively easy to engage people in the process because you can get to the point you're comfortable with. We don't know, but we want to learn it together. That tends to help a lot in that process.

Speaker C: So co learning, so to speak, or teaching and learning in tandem? Uh, I want to change viewpoints here. At the end of the Cold War, President Reagan was famously quoted as saying, and this is when it came to arms control with the Soviet Union, a very sort of outdated entity, so to speak. Uh, his statement was trust, but verified. In fact, we should invert these to state verify before trust when it comes to AI. In reading your guide that you shared with the ahc, uh, it pushes a verification standard for consequential outputs like requiring sources of information, data, et cetera. What does a defensible verification workflow look like when the clock is running?

Speaker A: Yeah, uh, it's important because everything is not equally impactful. And so in a phrase I'd say it's consequence driven as it moves people, money, resources or public behavior that it needs independent confirmation. So in practice what does that look like? It's asking the tool to show sources and assumptions, checking your highest consequence responses first against authoritative systems or known sources, uh, and then clearly marking what has been verified, what's inferred, what remains unknown. Requiring human sign off before any output calls for taking action or changing planned action, any messaging or any resource movement that gets impacted. So you don't verify equally, you verify what would do the most harm if it were wrong. That and verification, uh, standards should rise as related to consequence.

Speaker C: So what outputs would you say are absolutely a no go without independent confirmation?

Speaker A: Uh, so similar to in the verification standard piece, you know, again if it, if it moves money, people, touches resources or impacts public behavior, it shouldn't go out based on AI alone. So that could be areas like shelter status, shelter capacity, road closures, route status, uh, evacuation triggers or any protective actions that are going to be recommended. Uh, briefings, EOC briefings that are going to drive decisions, resource availability, casualty, uh, damage estimates that are going to be used officially. Anything that's public facing that could affect cross safety or legal exposure. And what this makes me think of a little bit here Andrew, is that this also gets into some use case identification a little bit where in practice I'll uh, give you a simple example. In one emergency management department setting the issue was not a lack of information, it was too much fragmented information that was coming in from too many sources. Uh, so the first practical use case that we talked about was not some grand autonomous system, it was using AI to take their messy notes, their raw inputs, turning all that into a structured draft briefing, save them time, it reduced noise, give the leaders something they could react to faster. Um, but the important part is that nobody treated this draft as the final truth. The value was the accelerated sense making while a human still verified the facts and owned the decision. So AI can absolutely draft, structure, summarize and flag issues, but the final operational or the public facing claim really needs independent confirmation. So use the tools to reduce noise, speed routine work, but not outsourcing the final judgments.

Speaker C: I know you discussed before using peers and collaborators so to speak as part of the process and I'm assuming here again, if you're a one person shop or you're multi hatted, you want to have let's say two levels of review and ideally not always possible, but you know, ideal to have. And that would include possibly some partner, maybe a duty officer, mutual aid reviewer, some sort of. Would you also build that into your exercises and into the training as well?

Speaker A: I think so. I think that's an important part of it. And like if you're truly a solo shop, than maybe replacing the idea of two person integrity with second source integrity, you know, a second person is ideal. Like you mentioned a peer, em partner, duty officer, mutual aid reviewer. Uh, if that human isn't available, identifying a second trusted source that can be, you know, an authoritative dashboard, maybe uh, a GIS layer, a road system, a uh, shelter system. Uh, so for the highest risk outputs, slow down that release of information long enough to really confirm the key facts. So the principle is not two warm bodies, it's that no high consequence output should rest on a single unverified AI pass. And you know, and you're stepping up a level here. A small operation should really start small and practical in their AI adoption journey. Picking one high friction, low risk workflow, maybe sitrep, drafting, summarizing field notes, creating briefing materials, put a simple review standard around it and uh, measure the before and after. So scaling the thinking and the process here is certainly possible. And candidly, even in a large shop, I recommend starting very small and very focused.

Speaker C: So on that note, as with every aspect of work that we do in emergency management, it requires that it be auditable. You recommended logging AI assisted work, the prompt that's used, the sources of information and data, the reviewer, the decision making, where does that live operationally in emergency management? So that it's usable, it's auditable and it's not a whole lot of extra bureaucracy.

Speaker A: Uh, got three critical terms in there. The usability, the auditability and that bureaucracy issue. Um, my belief is that the audit trail should live with the work product, not in a separate bureaucratic library or museum as they often become. Uh, it should be an easy to use workflow and so attaching it to the briefing, the report, the incident file, the artifact, wherever that work already lives and then keeping the field simple purpose, the tools used, the source basis, the reviewer, final decisions, date and time for recurring use cases. Creating a lightweight template so documentation is easy, is repeatable. I strongly discourage from building a separate logging process that people are going to ignore. If the logging becomes too heavy, then people will route around it. Particularly in a realm where very mission focused and time sensitive in what we're doing. Uh, so for example sitrep, uh, or briefing if it was AI assisted, a small notation block at the bottom is Usually more useful than a separate compliance log that nobody's going to open later. And one of the things I've also seen is that the additional benefit here is that these audit trails, they're initially set up for defensibility, but they're also creating a learning loop about what worked, what failed, where the process may need redesign. So there's a unintended, uh, positive consequence of doing this for the learning of the organization, particularly in a realm like AI.

Speaker C: So in essence it just becomes another part of the workflow, if you will, so that we can go to it, we can look at it. I was recently introduced to the idea or the method, uh, of real time capture that a colleague told me about, which is kind of interesting. We'll talk about that at another time. I think this plays into it, right? Looking at what happened, how did things happen, so we have a record of it and we know what went into that, then we can look at how do we improve it. So one area that I have talked about recently are some of the dangers or risks associated with the use of AI. Uh, and recent, some recent research shows that passive reliance on AI, treating it as the easy button, to uh, quote Justin Snare, who's the founder of Prepper AI, reduces a practitioner's self efficacy, their sense of ownership and a sense of meaning, while active human AI collaboration preserves these critical capacities. And I can say, speaking for myself, I feel enlivened when I'm using AI. And I see the level of discussion and just as a little aside here, I think, um, I talked to you about this. I created the sort of the three whys, emergency managers using the three different agent models that I use, which is ChatGPT, Gemini and Quad, and having them, um, discuss between themselves, because this was sort of my idea and it's a very rich discussion. Of course it's a lot of information to sort of pour through, but it can be very helpful. Having said all that, how should emergency management deliberately design AI integration to ensure that responders remain capable of regulators of these complex systems rather than just becoming dependent reviewers of machine generated outputs?

Speaker A: Yeah, it's a hot topic. Um, and you know, think about a recent workshop I did a couple weeks ago here in the D.C. area was with regional leaders from 24 different jurisdictions. And one of the leaders asked, so how do we preserve critical thinking when everyone's using AI? That's a big question that's going to require a little more time in this conversation, but it's certainly something that's on top of mind for folks and as you were talking about your scenario there, I was thinking about, particularly with ChatGPT, that unless you set it up in a certain way, you get these long dissertation responses and topics and you find, even as a subject matter expert yourself scanning through it to find the key points. And so it comes back to this notion of so finding even in those of us that are comfortable in using it on a regular basis, this deskilling that I've seen it happen. When AI does become, as you said, an easy button. One step here is to utilize and really mentally and structurally set up AI as a drafting assistant, a thought partner or noise reduction tool, while humans still own the interpretation and the ultimate decisions. So use the tool for first drafts, for synthesis, pattern surfacing, not for final judgment. Training people to critique the outputs and not just accept what comes out and then require explanations from the tool. Even, you know, why is this recommendation sound and what might be missing from it? The other part is designing workflows where humans stay active regulators of the system and not passive regulators, reviewers. And this, Andrew really gets into this area that I feel pretty strongly about, which is the human side is not separate from the risk side. It's really a part of it. And it's become kind of cliche to say everybody's heard the term human in the loop. And I think that's a little shortsighted because in mission critical environments like emergency management, you need an expert on the loop. You know, experts who really understand context, engineering experts who can determine the credibility and the accuracy of the output. Uh, brings up the topic of training where we need more than just traditional responder skills. But when experts can really learn how to harness the power of the tools, that's when the ROI really shows up. I'm a little concerned about the human in the loop element because it can lead to these cursory reviews that determine good enough. And what I'm starting to see is that the standard of good is currently in danger of decreasing to good enough. And so it's not just a physical body or physical review, but having the right person in the process. The combination of the expert with the, uh, output is extremely powerful. Starting to shortchange, that is can get dangerous.

Speaker C: Yeah. And I think this is also the slippery slope that we see sometimes. It's like, oh, well, I guess we can do more with less, right? We can. Or I mean, I don't like to say less because they're not less people, they're fewer people or less time available. But if an organization looks at, well, we don't really need this many full time equivalents. We need fewer people that are here. There's a danger in that. And uh, let just, let's talk about that what those, that that sort of slippery slope is because that, that multi headed one person it's easy to sort of say well now we can increase their capacity, right? We can just insert AI into the picture and then they have more capability. Right.

Speaker A: It's very dangerous at a couple levels. You know at the, at the single person shop feel like I'm doing more. But how are you maintaining the quality and the credibility of what you're putting out? So at some level you actually can do more and it may take the same amount of time or even more time because there's more that you can, there's more value you can provide. So it's not just this automatic let's save time in this world. If the quality and accuracy isn't there then you start to lose credibility not just internally but with public trust, you know, then that's disastrous for an organization and this realm of the time savings and thinking about the optimal way to use the tools. It's just leaders really have to be thoughtful about this because the answer is not don't allow use. Uh and in fact I've explicitly started telling leaders that if you don't outfit your best and brightest what's the most powerful tools for them to do their job most powerfully they will go somewhere else and find an environment where they can use the tools. So it's not don't punish them from, or fear of people losing their jobs with the tools. Your best and brightest are going to find the most powerful ways, free up their time to spend more time doing the highest value work and using the tools to be able to augment. So think about augmentation versus automation. That's kind of a first step here.

Speaker C: So I want to circle back to the idea of working with vendors who offer solutions or working with the off the shelf solutions that are given out there and what could we be missing right from the picture in terms of data and then how is our data used within that? And that was a very good blog article that was put out by Justin Snare and I'll put that in the show notes for folks to see. What should a small agency demand from vendors contractually and operationally to prevent some model drift from changing decisions that they're currently, you know, how they make decisions or to enhance decisions as opposed to shifting in a way that was not intended?

Speaker A: Yeah it's and this is a really interesting space right now because even six months ago the, a lot of the message, and mine included was don't jump into a, you know, a specific use case tool because start off with the generative AI tools and understand capabilities because some of these tools are very locked into a particular model and you're going to have problems, you know, down the road when you haven't even worked on your cultural adoption of AI and now you're trying to go all in with, you know, with a big enterprise tool. That's, that's an issue. And at the same time, what's happened more recently is we're seeing this acceleration of the vertical market focused tools and I do believe that is where the future is going to be. Uh, we're hear the conversations of replacing some of the, you know, your longtime SaaS tool and some of the others in there. But coming back to. So it's we're at a moment or in a lot of moments, but certainly in a moment regarding the tools, what I would say to your question here about the uh, you know, the vendor interactions is you can't just buy the tool. You'll see the, you'll see the magic show and think that's awesome and let's bring it in. But you really have to know, you have to buy the right to know when it changes, be able to test when it changes, escalate, understand escalation when performance shifts. Without getting too much in the weeds here, a few considerations that joke for me are the contractual considerations around change notification, incident reporting, data use restrictions, uh, retesting rights when the model is updated. And then there's the operational side of periodic spot checks using known scenarios, uh, having a review cadence, particularly around hurricane and wildfire season in this realm, a, uh, clear escalation path if the outputs start to drift or degrade. And really talking to vendors about how they handle updates, how they handle monitoring rollbacks, how they issue communications about changes. So if you depend on a tool during a season of consequence, then you really need to know whether the tool you tested in June is still the tool same tool you're using in August when you're really in the thick of that season. And this drifts into some of the accountable procurement and operation disciplines which we won't get into here. But these conversations are really important with the vendor. And as a leader, it's enough that you're taking on how we're creating this sandbox in this environment, governance within our organization. And now I'm bringing in a third party tool beyond the general AI tool. And it's a whole different set of conversations but still need to be, you know, managed really thoughtfully.

Speaker C: One area I want to explore is one that I have seen, witnessed it myself, experienced it in terms of, as I like to say, asking the three wise emergency managers who are, have this been this uh, AI model that I've created for research and things of that sort. And it's what's called trend slop, uh, and that's defined as the AI's tendency to recommend popular sort of crowd pleasing or user pleasing answers over necessarily accurate or context specific ones. There was some recent research that tested seven large language models across 15,000 plus trials that found that AI tools consistently recommended trendy, consensus driven strategies over situationally appropriate ones. Now, in a resource constrained environment like in emergency management, that trend slop bias towards collaboration, differentiation and long term thinking regardless of context can mean that AI assisted planning may sound tailored while quietly sort of steering agencies away from some of the hard operationally necessary choices. At the same time, human factors, such as factors that research long documented time, critical command and control environments, high automation can increase that automation bias, right, that people ignore the contradictory information over trusted machine outputs. Given that we are dealing with resource constrained time constrained environments in emergency management, um, where we struggle with stuck in the middle syndrome, we try to be everything to everyone with a certain degree. How do we prevent those AI tools from reinforcing sort of that buzzword driven incrementalism that may be keeping us from making hard strategic decisions during disasters that are when we're in the moment, so to speak.

Speaker A: Well, first I appreciate you introducing me to the term of trend slop. I get a lot of conversations around AI slop. And so now we're getting to the next iteration there? Uh, yeah, for sure. Uh, now I've got a label for what we're talking about. Uh, the issue is not that AI has no ideas, it's that it often has the same polished ideas for everyone. And in emergency management that context really matters too much. So it's a concern. So my thinking on that is using AI to generate auctions, not choosing strategy, pushing it. I touched on this a little bit before, but really pushing the tools to force explicit trade offs, what would we stop doing? Not just what would we add asking the tools for disconfirming evidence for failure modes really, you know, getting into, you know, understanding other local constraints that might show up, challenge it to consider that its first response is inaccurate and push it hard with conditions. What if our staffing dropped by 20% or funding dropped or mutual aid is delayed. A lot of things that are playing out in emergency management world today with federal funding and grants and so forth, comparing the outputs against doctrine, local realities and the agencies, your agency's actual decision rights. So really pushing the tool. You know, one of the things that mentioned uh, the uh, some of my work is that it's one of the powers of thinking about how you treat these models. It's this endless energy assistant that uh, never gets frustrated, never gets tired of your questions. And so really shifting, there's that assistant conversation with that thought leader, really putting it next to you and saying challenging it with the responses that it's giving you. And really in all of this the hard work and of the strategy is still. And for leaders that's still deciding what not to do. AI can broaden the option set, but it still should not be making the final judgment for you. You ask it for more and more ideas, it'll keep giving you just say, give me 10 more, give me 10 more. It's going to keep giving you. So the discerning piece for the leader is really that's where that leadership and that expert that I talked before has to come into play. Bringing some of these other experiences or filters into the conversation and then also like I said, challenging the tool to push back in areas where it's gone too broad. I see.

Speaker C: So it's still an assistant as opposed to being a member of the command structure so to speak, for command, um, element and organization.

Speaker A: Yeah, I think there's an elevation from assistant to thought partner which is still powerful, but again it's a partner, it's not a final decider.

Speaker C: Got it. This has been very helpful, very informative. I appreciate your time to share your expertise with my audience here. I spoke with Jonathan Spector, who is a founder and principal of Converge Strategic Partners, a boutique advisory firm that helps executive teams accelerate their AI adoption, align around high stakes priorities and translate strategy into execution. Again, Jonathan, I want to thank you very much for coming on the podcast.

Speaker A: I preached the conversation and leave it. Sort of a final thought here as we're going through this is I'd encourage leaders to not start with the tool like I said earlier, but really start with the outcome. Put the guardrails in place, help people build confidence and then run your practical pilot to start your job as a leader. You heard me say this before, Andrew, that your job as a leader hasn't changed, but the way that work gets done is changing and the tools are changing fast. But as a leader you still own the impacts and the consequences, so don't lose sight of that as a leader.

Speaker C: Thank you for my audience. Just so that you're aware, I will include Jonathan's contact information if you'd like to reach out to him. And um, yep, look forward to seeing you, hearing you hearing your thoughts on the podcast. You've got mitigation projects, grant applications, training and exercises to deliver, but keeping all of it moving on time on budget with the team you have. That's where good intentions stall. Project Management for Emergency Managers Workshop gives you the practical tools to manage, scope, schedule, budget and deliver. Built specifically for how you work. Walk in with your real projects. Walk out with a real plan. Taught by practitioners who've led responses from 911 to Sandy to Covid workshop options options for one to four days in person or online, it's time to move from planning to done. Visit pm4em.com to learn more.

Speaker B: You've been listening to Riding the Wave, hosted by Andrew Boyarsky, President of Pinnacle Performance Management and Clinical Associate Professional professor in Emergency and Project Management at NYU and John Jay College.