Episode 431: Agent Governance Is the New App Governance

- Welcome to episode 431 - of the Microsoft Cloud IT Pro podcast recorded - live on 06/26/2026. - This is a show about Microsoft three sixty - five and Azure from the perspective of IT - pros and end users, where we discuss the - topic or recent news and how it relates - to you. In this episode, Ben and Scott - get into why agent governance is quickly becoming - one of the most pressing challenges for Microsoft - three sixty five administrators, - As agentic workflows become more common across Copilot, - Copilot Studio, and third party platforms, - organizations are dealing with a sprawl problem that - looks familiar but hits differently.

Let's dive in - as we talk about Copilot agents and the - similarities and differences to enterprise apps. - Can I give you a quick rant? I - don't know. Is it gonna make me start - ranting?

Is this gonna be the episode where - Scott rants and Ben gets ranting? Alright. What's - your rant? I happen to have an email - address, an Outlook dot com email address that - is a very short - handle, and it's a very common thing.

- But it's kinda like a a vanity handle. - Whatever. But it would be something that's pretty - common, - especially - if your name was Scott and your last - initial started with an h. - So - I have this thing - where - I don't understand - why - people - use this email address, which is my email - address, to be very clear.

Like, it's all - locked down to me. I've got all my - YubiKeys, - authenticator. - Like, I closely monitor the logins because I - do get these weird, like, who somebody's trying - to log in to your account kinds of - things, right, with password sprays and everything like - that. But so so it's a real email - address.

I use it. Like, it's a real - email address. Unfortunately, - I have this issue where other people - are also using it as a real email - address. Like, I had somebody the other day - went and signed up for a Walmart account.

- So they signed up for a Walmart account. - Somehow, they were able to sign up for - a Walmart account, authenticate with their phone number - to that, like, they didn't need to get, - like, the access code out of email or - things like that. And then now they've got - a Walmart account, and they're making purchases on - it. And I'm getting, like, the invoices and - the receipts.

- Normally, what I can do is when I - get these kinds of things, I go and - I - I say, hey. I forgot my password, and - I request the account to be deleted, or - I just change the password to something that's, - like, black holes. Like, I'm not a smart - person and use somebody else's email address to - sign up for this account. Like like, it's - just this big long string that can be - crammed in there.

But the problem I'm running - into lately is there are sites like Walmart. - I've got PayPal kicking around, a couple of - banks that people have signed up for, like, - stuff that I should not be getting. Right? - So but I can't go in, and I - can't request the deletion of the account.

So, - like, Walmart, you go in and you say, - hey. Here's my email address. Send me a - code. It says, great.

I sent you a - code, but you have to authenticate with your - phone. Well, I don't have my phone. Right? - Because it's not my phone.

It's somebody else's - somebody else's phone that that put in there. - So I have this email address that more - and more, I cannot use for anything personal - because there's so many other people in the - world with the same first name and last - initial - that are using it for all their stuff - and getting it pushed through. I I have, - like, Walmart account, bank accounts. - I have a PayPal account in The UK - that somebody signed up for, like, can't get - into.

I had this week, somebody did Instacart, - and another person did Grubhub and signed up - for Grubhub Plus. Cool. - It's - really weird. - And I would understand, like, common email address, - like, - first, last initial, blah blah blah.

Like, somebody's - just pumping it in to - sign in to Wi Fi somewhere randomly. Right? - And they don't care and, like, hey. I'll - go spam this other person kinda thing.

But - these are, like, legit - accounts. These are legitimate - things. People are buying stuff. They're getting bank - accounts.

- There's some guy in Iowa - who runs an entire survey business, and I'm - getting all his invoices - and all the surveys - that he's, like, sending back and forth to - the companies that he's working with. Like, I - don't understand - how - this is the state of the world. It's - so strange to me. You said you weren't - gonna get me ranting, but this is gonna - be an interesting one, I think, that's going - to come up.

Not because part you mentioned - it in here. It was like a sub - bullet of what you said when you talked - about you needed your phone for authentication. - Is - my problem is I have password less set - up a lot of my accounts because I - don't wanna type in a password. And I'm - curious if you start seeing this.

The amount - of prompts - that I get an authenticator - because - and I've had a client that had this - too with one of their email accounts because - nobody needs a password because it relies on - the push notification. - I get, like, I get spammed with authenticator - requests, and it's not approve or deny. - It's number matching once. And I'm like, okay.

- I don't think this was me. Sometimes it's - close to the same time I do it. - But now I started seeing it's, like, from - Germany, and it's from The US, and it's - from here, and it's from there as and - there's no way to stop it. Like, I - can't go change a password.

I can't go - change my authenticator - app. Like, in this world of passwordless, - I feel like these authenticator - spamming - number matching things - are just going to get annoying. Like They - are very annoying. So so I have the - same problem, again, on my Outlook account, but - mine aren't number matches.

- So the problem is the way Microsoft does - it, e even if somebody happens to find - a way in, like, Microsoft's rationale is, like, - hey. We're still sending you the note the - push notification. Right? So, therefore, it's up to - you to approve or deny.

But I don't - know if the push notification - like, I know which ones are from me - in the moment as a number match. Like, - that's very clear. Yep. You might have something - in the background.

So, like, for example, for - me, I sign into my outlook.com - account through Outlook on my desktop. Makes sense. - Right?

It might be Outlook just trying to - reauthenticate. It might be my iPhone trying to - reauthenticate. It might be my tablet or another - computer. Like, could be some service I signed - up for, and I gave permission - to read my But you don't know what - it is, and - Microsoft - in they they used to publish all the - security information.

So you could go in and - you go to, like, account.microsoft, - and you could see all the sign ins, - and you could see where they were coming - from. That got so noisy that they stopped - publishing those. So now I get all these - push notifications.

- I get all these authenticator prompts, and I - have no - way to even know where they're coming from. - Like, I don't even know if they're coming - from Russia or China or The United States - or anything else anywhere. Because, like, I'll I'll - tell you, like, if I go and look - I was just doing this before I hopped - on because I was trying to clean up - some of these emails out of that mailbox. - So I went and looked.

The only login - that shows up in when I go to - account.microsoft.com - for that - Microsoft account, the only login that I see - is one login from today. Okay.

I authenticate - to that account a ton of times today - and from my iPhone. And so, like, my - iPhone's running Defender, - and I have a VPN client on there. - So, theoretically, my iPhone should be showing up - as a separate login, like, being routed through, - like, Defender and, like, Seattle where the VPN - location is. Or if I hop on to - Tailscale and I happen to have an exit - node in another location, - shouldn't, like, the login thing when I get - reprompted for auths, shouldn't it show me that - I relogged in from a new location?

Like, - they they just, like, masked all that information - and hid it away. I think it was - I get why it was noise for customers. - I bet it was a ton of telemetry - for them to collect as well and have - to kinda store and push out and surface - and query and - and all those kinds of things. But in - this world of, like, passwordless, - you tie a phone number to your account, - it's no longer - just a password kinda thing.

It is it's - rough, especially when, at least in my case, - for people who haven't gone through this, it - stinks when somebody else is using your email - address. It's really annoying. Thanks for that because - I thought I was going crazy because I - did the same thing. When I started getting - error as I've been getting all these random - prompts, I wanted to go see, like, where - are they coming from?

Is there any way - I can report these? And I had the - same thing. I'm like, I thought I used - to see all the login attempts, and now - I don't see anything. And I was like, - maybe I was getting entry confused with my - personal account, but they did used to have - that, and they took it away.

They very - much cut it back, - and they removed a bunch of controls there. - Like, they used to have stuff for, like, - reporting things Yep. And all that. So it's - down to the point now where even if - you go into the docs for, like, reading - about, like, what recent activity is, - they go so far as to say, if - you get an email about unusual activity and - you're not sure - if it's from Microsoft, just go ahead and, - like, sign into your account.

And even if - you get one about unusual activity that is - from Microsoft, when you sign into your account, - the detail's not there. There there is no - detail there. It's just it is completely - opaque. I I don't know how they could - make it better versus going back to the - way it used to be.

Like, show me - every time you send me a push notification, - show me why. Like like, give me a - little bit of insight into where that was - coming from. Like, because if I could even - see, like, a user agent string and a - timestamp, like, if I could see, like, oh, - that was Outlook from macOS - at 03:25PM - on Friday. Oh, that was me.

Like, I - was just in Outlook. - And, oh, yeah. I did get a little - error pop up or things like that. So - yeah.

- Any anyhoo, that that's my rant. Somebody else - having your email address sucks. - Passwordless - is - super annoying in a world of opaqueness. - I think you should like, I think the - vendor should air on on on the side - of verbosity - and let folks know, but, unfortunately, - they do not do that.

Yes. And report, - like, if I report that I'm not logging - in from this particular one, I'm surprised how - many keep coming through that appear to be - coming from the same location. Mhmm. And I - don't know how you get around it.

Because, - ideally, I would say, you know what? I'm - not in Germany, and, frankly, I'm not in - any country but The US. Any push notification - from anything other than The US should be - denied. Now if I travel, - I'd want a way to open that back - up.

But for now, like, I should not - even be having to deal with those push - notifications. - It's interesting. It's weird. Actually, for my Microsoft - account, - I think I'm at the point where, like, - at least for my consumer account because of, - like, the state it's in.

I was gonna - say consumer accounts is where I have this - problem more than my business one. Definitely the - most there. But for that, I would actually - prefer to be able to turn off authenticator. - Like, if I have another passwordless - mechanism, - so if I've got biometrics just through my - phone with a passkey - or I have a YubiKey or something, give - me the option to turn off authenticator.

- You let me turn off my phone, like, - my phone number and OTP. - Let me turn off authenticator. Like, don't force - I'm sorry. Like, that piece of garbage application, - like, at down my throat and make it - the only the only pass in.

- If you're managing file storage today, you've probably - felt the pressure. Refresh cycles are harder to - plan, costs and complexity are increasing, especially as - file workloads scale across regions. That's why Microsoft - customers trust Nasuni. As an Azure certified software, - Nasuni provides a global file system designed for - enterprise scale environments, - keeping file data secure, governed, and accessible across - Azure environments without added infrastructure complexity.

- It's the foundation to keep your unstructured data - stable even as your infrastructure evolves. To see - how much you could save while keeping your - initiatives on track, visit nasuni.com/tco. - That's nasuni.

com/tco. - You know what else people are having to - deal with, Scott, in managing outside of authentication - requests? This was such a good transition to - our main topic for the day. It was.

- Good job. - And agents. Agents are everywhere. Agents are everywhere.

- So this is an interesting one. Like like, - my day job, - I think a lot about - SDKs and rest APIs - and things like that. And even in, like, - my day job, I spend more and more - of my time thinking about how agents interact - with those things over humans as more and - more stuff gets handed off. And I'm definitely, - like, in that ecosystem.

I think you're in - that ecosystem as well, kind of being tied - into even just, like, generic Microsoft three sixty - five. So if I'm just in Microsoft three - sixty five, let's say Copilot, I've got a - researcher agent. I've got an analyst agent. I - can connect to - to SAP.

I can connect to consumer services - like Canva, - all these different things. And now that Cowork - has g eight, I think you start to - see - a a little bit more probably usage - and uptick there. It's becoming, like, more prolific, - I think, to see - agentic workflows, - whether those are, like, true, like, bounded agents, - they're just kind of fluffy declarative agents in - m three sixty five Copilot, whatever happens to - be, but agentic workflows - augmenting - more and more of just the day to - day of what's going on there.

So that - raises an interesting question. In a world of - agents - and likely more agents than humans that are - in your organization because there's an amplification factor - of I'm one person, and I might use - 20 agents. - How do you kinda know what's going on - out there? How do you monitor?

- How do you respond? And I think, largely, - like, there's an argument to be made to - say is that - agentic governance or agent governance - is - the new - application governance space. So much like you might - have been in, like, a world of MAM - and managing to the applications - that your users could use, now you're in - the world of agents, and you also have - to manage to the agents that your users - can use. Or maybe you're in a fully - ungated world, and you don't know what agents - your users - are using out there.

And it's such a - wide landscape - and wide surface area for agents between Microsoft - three sixty five Copilot. - You got Copilot Studio. - There's the whole just broader - Microsoft - security stack, - and you kinda gotta get out there and - find a way to - govern agents - either as applications or as, - I guess, artifacts of your enterprise - and kinda part of your - core business. It's interesting because I in some - respects, I always thought about agent governance as - also - the new - almost people governance - to your point, but agents are also apps - and agents have to deal with sprawl.

It's - governing agents is turning into I feel like - a massive thing. Because to your point with - apps, you have, like, who can access this - agent? Kinda like you'd have to manage apps. - Who can log into it?

What can they - do with this agent? But then you also - have agents running - similar to people where it's the data issue - is what data can these agents access and - which agents can access which data, and how - do you protect data from agents. And then - you have - the sprawl aspect of it of, again, people - just going out and adding agents, and - you have, like, ghost agents now. The whole - in some respects, it's I would say it's - turning into the one of the more complicated - things to govern because there's so many different - aspects to it in terms of what agents - can do and how people access them and - how they do things and even how agents - manage agents.

You're getting into these things now - where you have agent orchestrators that manage - sub agents. - And agent governance, I feel like, is it's - already started, but I feel like it's just - gonna continue to explode - over the next, I don't know, months, years - as you try to figure out how to - govern this? So it's a very broad area. - I think in and it's not probably unfamiliar - to most administrators.

Like, you're used to governing - specific aspects of your platform. So let's say - maybe, like, app consent. - Hey. I created a new I needed a - new enterprise app registration created that that came - with an owner, came with a specific set - of permissions - on the graph, things like that.

Maybe you - had a Teams app, again that had its - own kind of construct. You had SharePoint that - had its own construct. So we're very used - to dealing with the individual constructs, - but when they start to get mishmashed into - this one - broader bucket, which has a whole bunch of - different subcomponents, - it becomes - really hard. And I think this is where, - like, Microsoft comes in with things like Agent - three sixty five and says, hey.

You need - more of a control plane here for agents - because it's an identity - problem. So it's not just who is the - agent, it's who and or what is the - agent. Like, who created it is one question. - Who can use it or who consume it - is another question.

What permissions does it have? - Does it have different permissions and different modes? - K. That that that's just identity.

Right? You - have five questions right there. - You've got this - data access issue - or kinda - lack of visibility or something that you need - to kinda figure out is, - what's the data that these agents can retrieve? - What are they allowed to do with that - data?

Do they transform it? Do they pass - it on to another agent or another process? - Are they just summarizing it? Like, when they - go and do the retrieval, - let's say you retrieved a bunch of sensitive - business data, and then you're telling that agent - to summarize it in a table.

Well, it - might be interacting with, like, an LLM, so - it could be chat GPT - or an anthropic model or deep seek or - whatever - out out on the sides. Now you gotta - worry about not only what data it can - retrieve, what it can summarize, what it can - transform, but you also have to worry about - leaks, like like, pretty big. - You have an application problem - because you don't always have visibility into - not only, like, what are the connectors and - tools and actions that these agents use, but, - also, like, what are the raw APIs?

- I've seen some things when I'm interacting with - them because the LLM guides them into it. - They'll say like, oh, hey. I don't see - a tool for blah blah blah, but I - read the documentation for this thing, and I - think it's got a REST API. I'm just - gonna, like, go wild and - see what I can get out of it - with, like, an external service call.

And, again, - you're back to data leaks, all that kind - of stuff. You've got a life cycle problem. - So - we had the identity problem of, like, who - created it, who can use it. There's also - a life cycle problem in who approves it, - who owns it, like, how do you attribute - ownership to an agent, - when and or if should it be retired, - what happens when the person who made the - agent, like, if it's a custom agent, what - happens when they leave your organization?

That's a - big problem with things like power automate flows - and everything else that's out there, but that - was just a single space. Now you're in - a multispace kinda thing. And then there's just - all the security stuff. So - bad actors, malicious prompts, - prompt injection, - risky connectors, - maybe there's, like, a mismatch in privileges and - configuration of an agent, so you get this, - like, creep aspect - that goes in there.

So so it's really - hard because - the question, - I think, used to be, like, what are - the apps that have access to my tenant? - And you could kinda go scope that down, - and you could run-in, - and you could start to wrap your head - around that. - But now the question - is - really different for administrators - in these environments where you have to ask - not only what are the apps, but now - it shifts to - what agents - can access my tenant, what agents can act - across my tenant.

So it's not just about - access in a single area, but it's also - about, like, across the spread of tools, Teams, - SharePoint, - Exchange, all those things. What are the tools - they expose? What are the tools they actually - use? - And what's the business data that they have - access to and that they can - reason over?

So it just makes it so - much more of, like, a heady thing. I - think Microsoft is approaching this in a manner - of saying, hey. We have things like Microsoft - Agent three sixty five, which start to lay - a groundwork - for a control plane for - management - of agents - to help you kinda get there. So it's - a path forward to observe, - have some measurable governance, - some - sense of - security with a agentic use in your organization.

- But even then, I think there's a lot - of room to grow and a lot of - space to - kinda pick things up because as much visibility - as things like Agent three sixty five give - you today, they don't give you a full - three sixty view yet. And I think some - of it is gonna be how do people - start - trying to get that view, and are people - going to adopt agent three sixty five? Because - to your point, it's going to pull in - a lot of things. Like, I go to - my agent three sixty five today.

Out of - the box, I haven't done anything special, and - I have 200 - to your point about exponential growth, I have - 295 - agents - in my organization. - These are not And your organization, - your tenant is tiny as far as, like, - the number of users. - It's not it's not an it's not like - a thousand person org. Mix of things.

Like, - I've done some stuff with Dynamics three sixty - five. So I have a bunch of Dynamics - three sixty five agents. - And then I do have a few custom - agents. I have the Microsoft agents, like the - learn agent - and - the planner agent - and the SharePoint admin agent.

- And then it does show and this is - an interesting thing that it shows us. These - are just agents that are registered with the - service. It shows things like Databricks - and - Mural and - Canva - and Mural, - where it's agents that are available, it doesn't - necessarily - mean they're connected, - but it means they're registered with the service. - But in order for agents to continue being - registered as people maybe do create ones that - they're using, - there's still - SDKs and different things that custom agents, - depending on where they're created, - there are certain things that have to be - done with those agents - to get them to show up in agent - three sixty five.

If someone goes off and - creates a certain agent somewhere and doesn't tie - create the hooks to tie it into agent - three sixty five, it may not show up. - There are some things that are still in - preview, like registry - sync, which does allow you to start - connecting supported - third party AI platforms and syncing - third party agents - in. So, like, if you have agents out - on Amazon Bedrock, - Google Vertex, - Salesforce, Agentforce, - Databricks Genie, - you can set those up to start syncing - those agents into agent three sixty five two.

- But there's there is still also a level - of adoption of here's the tool, here's, - not a framework, but a platform we've given - you to manage agents. There's still some of - that onus on organizations - to make sure all those agents are actually - getting in there. It's not the kinda go - click a bot and and set it and - forget it - kind of thing that you need to kinda - get in and figure out. Because it's I - think it was hard enough to manage the - tenant and the sprawl and everything else that - came along with it, and now it's just - that much more.

- This episode is brought to you by Trusted - Tech. If you're managing Microsoft three sixty five, - you already know licensing isn't simple. E three - versus e five, the brand new e seven - licenses, - Copilot security bundles, and with price changes coming - in July 2026, - the stakes are getting higher. Trusted tech helps - IT leaders make sense of their Microsoft three - sixty five environment with a free licensing consultation.

- Their engineers analyze what you're licensed for and - what you're actually using and where you can - optimize, whether that means consolidating security tools, preparing - for copilot, or eliminating wasted - spend. Plus, they offer proactive and reactive support, - which has been awarded yet another solution partner - designation from Microsoft - for support services. - If you want a clear data backed plan - that has helped over seven five hundred subscribing - customers save up to 20% on Microsoft three - sixty five before your next renewal, visit trustedtech.

team/mscloudprom365 - and schedule - basics - and maybe take a step back and say, - like, hey. Like, what can you start to - do with agent management - in the Microsoft - three sixty five admin center? So when you - go in for - kind of this govern - governance aspect of - agent three sixty five, which I don't know. - It's a weird tongue twister.

I keep wrapped - in my head and getting wrapped on it. - So for me, the first thing I do - when I'm going into - a - client or even in my own environment is, - I would say, where I think most people - should start, where I usually start, is actually - looking at - all agents and looking at that agent registry. - And I will say if you don't have - agent three sixty five licensed, I've compared this - across a couple platforms. - You get some of this.

You don't get - the full - you don't get the full report. So if - I go into agent three sixty five and - go look at all agents, it gives me - the name of the agent - if it's available. So am I actually making - this agent available to my users? The platform - that it came from, so is it Copilot - Studio?

- Is it third parties are just blank? Is - it Foundry? - Is it some of those other ones? But - then I do start seeing interesting things.

And - this is the part that's always fascinating to - me is I can see the risk of - the agent based on some criteria that Microsoft - has established. - I can see the active users, and I - can see the total sessions. - So one of those very interesting things with - me is I just go sort this agent - registry - by active users or by total sessions. So - I can go in and see that - I have - 50 people - using - this HubSpot - agent that are active users.

And across those - 50 users, maybe I have 500 - total sessions. It starts giving me just some - of that before I even do anything else, - it gives me some of that big picture - of - what agents - have people started - adding. Because - you can just go into - Copilot - and start adding agents. Maybe you have some - added, and you wanna know are people actually - using them.

It just starts to give you, - again, that picture - of here's what's out there. Here's what people - are using. I had a very different kinda - take when I had stood up a tenant - and was going through and looking at some - of this. So I kind of approached it - from the lens - of, - I I I guess, maybe, like, even - an internal, - like, rubric for - where does the greatest risk lie.

So it - wasn't so much about usage. It was more - around - what are the categories of these things. So - if you see, like, teams pop up in - the list, do I need to worry about - teams right now, or do I need to - go worry about - the third party - connector kind kind of thing? So if there - was stuff in there, let's say, that was, - like, published by your organization, - the the those might be a little bit - safer.

Right? Like, you've probably published them out - there for a reason. - They're, - like, predictable to you, so you see them - and you know what they are. - There's the whole category - of - custom agents.

So somebody in your organization - created a custom agent - in things like Copilot Studio, which there's a - weird new backdoor in the Copilot Studio. - It, like, even if you haven't licensed users - for it after they GA'd co work. So - so maybe there's some things that you wanna - look out for there, or you're just interested - in, like, what are the parts of my - organization that maybe have a little bit more - maturity - and are going and running with these things? - My my threat matrix for things that were, - like, Microsoft agents, so whether it was Dynamics - or - internal component kind of thing.

Hey. If it - was developed by Microsoft and it was integrated - with Microsoft, like, that might not be my - my primary concern. - I'd rather bubble up things like the external - agents, so stuff that was created by either - a third party - was maybe created by, like, a developer that - a random business unit of yours - went and spun up because, you know, they - were able to or you didn't have the - existing - governance constructs in place in your - in in your tenant already.

So and then - from there, dial into usage and see. So, - like, if I said, hey. - The external - or the, yeah, external partner agents and maybe - custom agents created, like, within my orgs. Those - are my two biggest, like, factors.

- Then I'll go sort by usage and see - if you - can kinda suss out who's doing what there. - And I guess maybe mine isn't necessarily usage, - but it's looking it's looking for similar types - of stuff as you based on usage. That's - how I would freight. That's what I'm looking - more at as again, if co work shows - up at the top of my list of - usage, big deal.

If Asana shows up and - I'm like, I don't even know anybody was - using Asana in the organization, and now there's - a bunch of sessions with the Asana agent, - it's why or - something that's risky. The other thing I think - that's fascinating, and I don't know how much - you played with this, is once you find - one of those agents, clicking on it and - looking at all of the details - inside - of each one of those agents. I don't - to me, I think I don't know how - much it's a governance topic as much as - it's a information as you're maybe assessing - the risk of it, or you're assessing - why is this agent there, what are people - doing with it, or if you need to - from the governance perspective, - if you deem need to go in and - start limiting - who can use that agent.

Again, it's that - weird mix of now I'm not worried about - just users. I'm worried about APIs - and what they consume. And so now you're - back to the identity, the data access, - the that that whole spread of of things, - and it's just it's a lot of, like, - cognitive load. So you can kinda see what - Microsoft's doing by trying to make it easy - to see.

So, hey. Cool. I got a - single pane of glass. - I've seen tenants where it's like, that single - pane of glass is overwhelming.

Like like, it's - not helpful. It's more like, oh my gosh. - I'm gonna have a heart attack just having - to, like, - consume this page and trying to figure out - what do - I need to do here, - especially when you're not - maybe - you you haven't read your way all the - way through the documentation or you haven't, like, - fully started to consume these offerings. I think - Microsoft is trying to, like, move the needle.

- I don't know if you've had a chance - to play with the the shadow AI component - and some of the stuff that comes out - of Frontier. - So you've got this if you're in the - Frontier - program, - okay, because, hey, - you're on the latest cutting edge of AI - innovation here. It also opens up the shadow - AI piece in the admin center there, and - then that starts to get you - information - about, - like, agentic harnesses in use within your organization. - So let's say you had somebody who went - out and installed, like, OpenClaw, things like that.

- I don't know if it can detect, like, - some of the other harnesses, like Hermes - or or things like that, but it's very - much like preview capability, but helps you start - to rationalize - it just a little bit so you're not - having to, like, do everything - your yourself and have some of that overwhelming - stuff. Because if you are in an established - tenant and you were licensed already for Copilot, - like, you had an e seven. Right? Like, - we we've talked about e sevens in the - past.

So great. You upgraded those. All of - a sudden, all your users are licensed for - this stuff, and they're just out there, like, - doing whatever. And they've been running that way - for three months, six months, a year if - you've never been in there, like, it's gonna - be a little bit of an overwhelming experience.

- I was hoping for more in Shadow AI. - Maybe more It's pretty bare bones today. It's - pretty mine has open claw in it, which - I can start detecting. And then it has - a coming soon.

And And under coming soon, - I have Ollama in Po desktop from Quora. - So those are the only I have one - it can detect too that are coming. I'm - surprised - there's not more in there. But, again, we'll - put our disclaimer on there of Frontier as - up and coming.

Think of Frontier as, like, - beta preview type stuff. Hopefully, we see more - coming in there soon. Or, like, where does - Scout show? Because Scout's a harness, but it's - not Shadow AI.

- But there's no other, like, dedicated - pillar category, - yeah, for - for harnesses - or or things like that. So because Scout - doesn't even show yeah. Scout shows up nowhere - right now in here. It's a weird world.

- So maybe we should talk a little bit, - in the couple minutes we have left, about - kind of visibility into agents - and what you can see about them. Because - we kinda talked about high level. You can - get in. You can see what agents are - in use, the categories, - things like that.

But then when you actually - have an agent that's out there, you can - get a little bit of high level detail - about what the agent is. You get insights - into the users, - potentially data and tools, permissions - for that agent, activity, things like that. So - so maybe we hop through a couple of - those just for folks who can get a - sense of what they can dig into. To - me, this - is super helpful when you're trying to dig - into it.

Like, the details initially - gives you things like when it was created. - Has it been updated? Agents get updated over - time as you build them. Who published them?

- Owners. What channels are they available in? So - I'll do Asana, for example. I use Asana.

- So it's available for me in Copilot. It's - available in Outlook. It's available in Teams or - the different channels. So where can people access - it?

Details really gives you a good overview. - It even gives you, like, if there's - instructions - with that agent, - if it's tied to an environment, this is - gonna be more applicable to, like, a Copilot - Studio agent, which environment is it in, just - a good general overview - of that agent. - Users - is more controlling it. So we talked about, - like, again, from an application perspective, what does - the agent have access to?

Users is who - is it installed for and who is it - available to or who is it shared with. - So you can like, if you're a developer - and you've created it, you can just share - a an agent with somebody. But then available - to is who can actually install Asana? Should - it be - shown to all your users or are there - just specific groups that should be allowed to - install Asana?

- And then installed for is do you actually - wanna pre install it? All my users are - gonna get Asana by default, So it's a - little bit for me, it relates very closely - to Intune when you have, like, required versus - available. Installed is this is required. Everybody gets - it.

Available is they don't have to have - it, but if they want it, they have - the ability to go add it to Copilot. - And that's back to the, like, analogy about - applications. Right? So so age agents are applications, - but they're applications - with, like, just a broader set of things - that that you need to go look at.

- The other interesting thing - about, like, the that that analogy of apps - to agents - to carry it forward - is you've also got options around, like, agent - availability and installation. - So there's a difference between - making an agent available - and letting users - discover it versus - pre installing or pushing an agent or agents - out to a security group or a set - of folks that that are out there. So - there's this kinda like, I made it available - in visibility play, but users have to go - self install, and if they install, great, they - needed it, versus - I've made this agent - or this agent set of agents, whatever it - is.

It's actually, like, part of the user's - job now. Right? Where, like, you you push - it out like you would push out - an application through Intune - or things like that. - You're out of time.

Copilot needs to be - deployed ASAP, but is your tenant really ready - for it? Years of data, permissions, and users - lurk in its shadows, ready to be exposed - by AI. - Sharegate Protect sees it all, so you can - find the exposure risks, fix them fast, and - deploy AI with confidence. - Microsoft - three sixty five governance.

- We got this. Learn more at sharegate.com/governance. - And then on the other side, again, going - into applications, I'm gonna jump over the permissions - tab.

After you've looked at users or you - wanna do that, one of the other things - you have is permissions that are very much - like, if you're looking at those enterprise apps, - where it gives you and, again, Asana as - an example has - channel read basic - all to read channel names and channel descriptions - in Teams. This is where you need to - know, like, your graph permissions because all of - a sudden, you're seeing permissions. Scopes? I'm amazed - they haven't, yeah, like, done a better job - with abstracting the scopes.

It's still, like, group - dot read dot all, teams activity dot send. - So So it's all this random stuff. And - it makes sense when you see it, but - does everybody actually know, like, what What that - means? Yeah.

Like, what is user dot read - dot all, like, versus - another read dot all scope or permission - set that's out there kinda thing. - So so you end up in this world - of, like, there there's probably - I - my experience has been, like, there's common - permissions or, like, they're broadly applicable. Like, there's - a bunch of stuff out there that has - things like user dot read dot all. So - is, like, is that okay in your organization - that this thing can read your entire orgs?

- This thing can? Like, yeah. Maybe. Maybe it's - a requirement of it.

Maybe it's not a - requirement of - it, - what whatever it happens to be. And then - you've also got the whole delegated permissions - aspect of sometimes these agents act autonomously, - so the agent is the agent, and sometimes - they act on behalf of user, and sometimes - they do both at the same time. - Yes. And that's one, like, I do there's - a few things I like in here.

This - does show delegated versus application there. The other - thing I like here, and I don't think - it shows it when you're dealing with enterprise - apps or app registrations in Entra, is the - last column does show privilege level. So if - I'm looking at channel.readbasic.

all, - like, to your point, what does that mean? - Should I be worried about it? Here it - says that's a low privilege level. Okay.

So - if it's a low privilege level, I maybe - don't know what it means. It does give - it details. It says it reads channel names - and channel descriptions on behalf of the signed - in user. But then it says privilege low - versus - chat dot read write, that's gonna be a - medium privilege.

- So as you're looking through this too, you - can get you still wanna understand them, but - you can get a nice - scan of, - are there a bunch of high privilege ones - in here, or is it primarily just low - privilege? - What are those different privilege levels of those - graph permissions? There's another weird tab. We should - talk about it while we have a couple - minutes left.

Agent certification - or certification. - I'm waiting for mine to load. Agent certification. - Publisher - attested.

- This is - yeah. This is - one I have some clients that are very - concerned about. Yes. - And it's super important to them - is this gives a whole bunch of details - on - how is this agent certified according to - different - compliance - regulations?

- So I have a sign up, so I'm - gonna keep going with it. Is - one you have, is it publisher attested? So - for Asana, is this attested - by - a self - assessment reported - by Asana? So so to be very clear, - self assessment - by the publisher.

Publisher. So this is Asana - telling Microsoft. - Even for Microsoft, you'll see Microsoft certified - versus, - yes, - an Asana or somebody else or SAP is - saying, yes, self attestation. - This is an area that I would love - to see more transparency - about - what's actually checked in the self attestation flow.

- Like, how does Microsoft verify that? Because there's - callouts in the docs when you go and - and read through this. So let let me - pull it up so I get it right. - So Microsoft three sixty five certification - reflects an independent - security - and compliance assessment that's valid for twelve months - and helps confirm that the agent meets Microsoft - standards for handling tenant data.

Great. - What's in that independent security and compliance - assessment? And what is, like like so and - it's self attested for twelve months. What happens - if Asana pushes an update and now they've - broken their compliance all the way?

Like like, - did you come back and review that update? - Like, it's a - that one's a little bit of a weird - situation. - And I get where Microsoft is. Like, there's - no, like, broad, like, agent security consortium or - something like that where you can say, here's - the open standards.

Here's the way this stuff - goes. But, hopefully, the industry does coalesce there - over time. Like, you'd think they'd have to - for regulated industry and all that. How would - they even go through - certifying it?

Because it says right in here - when it's publisher tested that Microsoft makes no - guarantees regarding accuracy. - I mean, I suppose Microsoft could require certain - documentation to be uploaded, but even - when I go in it's - if I'm looking at something, an agent, how - would I even go in and verify that - it meets - HIPAA, that it's - HITRUST, that it's SOC one or SOC two. - Other than just getting - documentation, - like, to get SOC two, I know that - when you have to go through a whole - bunch of stuff, you get those you get - the certification.

- I don't know if Microsoft makes them upload - it, but you do get things like HIPAA, - where based on everything I've understood about HIPAA - is - you don't there's no stamp of approval of - once you do a, b, c, d, e, - f, g, you're HIPAA compliant. It's a lot - of focused on you need to make your - best effort to protect this type of documentation, - and one person's best effort might differ from - another person's - best effort. And it's really hard to actually - stamp something.

Like, even Microsoft three sixty five. - Microsoft three sixty five is HIPAA compliant. - If you go in and do the right - stuff, - is a sign of the same way when - they say they're HIPAA compliant or there are - other certain configurations you need to make. It - it's to your point, it's kinda - vague.

It's some of these can be hard. - Some of them are a little bit more - straightforward. - I think I said, I hope it's one - of those areas where, like, the industry coalesces - over time. Because today, it's - it's a little bit of, like, a gut - feeling of, alright.

So I know who built - the agents. Good. I was able to identify - that. And, like, how much do I trust - that person - or that publisher?

- And a little bit of light, like, what's - it been validated - against kind of thing. It's I think it's - no different probably than or or maybe analogous - to, - like, app stores. Right? Like, Google Play - or Apple running the - iOS or Mac app stores.

Right? Like, hey. - Customer signed it. They came in.

They pushed - it out, but that doesn't always - prevent malicious apps or other things - that that are coming through. That might be - a fun day, right, when the very first, - like, malicious, like, broad malicious agent comes through - And somebody's like, oh, hey. I gotta, like, - yank this agent out of every tenant, kinda - like publishers do with app stores and - and things like that. Conditional access for agents, - risky agent access policies and conditional access, Scott.

- All sorts of things. So, yeah, so I - I think this is, like, just to wrap - up, like, this is an area for folks - to - go and look at, especially if you're in - licensed environments and maybe you haven't had the - time to dig into it. Like, you should - have access, I believe, to all this if - you're on e sevens and kind kinda latest - and greatest there. There are ways to license - and get in.

So if your tenants are - configured for things like Copilot, - for Cowork, - and these things like that, it's worth at - least browsing through - and seeing what's out there if you haven't - been in there. And, hopefully, no, like, big, - like, red flags or overreactive - decisions - to take, but having some of that visibility - and starting to - kinda build some of that confidence because all - like you said, you're going from managing, like, - these individual constructs to managing a whole bunch - of bundled things at once.

- It's a lot more to - wrap your head around - and stay on top of, and then you're - very much trusting in vendors - and folks like Microsoft - to kinda give you the tools with the - insights - and visibility that you need. Yep. And really - starting to think about what does agent governance - look like at your organization. What is - how are you gonna start not just looking - through this, but how are you gonna start - managing those agents?

- What are people allowed to do? I feel - like it's also one of those topics that - a lot of people have even put much - thought into yet is what is agent governance - look like for our organization? - It's a rapidly - evolving thing. And probably even more important, - and I'm sure we'll wrap back around on - this topic sometime in the future, - but as all this stuff's transitioning - to - consumption based billing versus kinda like the broader - the broader unlimited usage.

- Now there's also the aspect of, hey. That - thing that you deployed to your users six - months ago that, I don't know, let's say - it costs $20 in usage per user per - month, like GitHub Copilot, for example, - all of a - sudden could be costing, - like, thousands, - tens of thousands. I've seen some really wonky - bills and and threads flying around on the - Reddits of the world and and things like - that where some of this stuff is going - off the rails for folks. So there there's - also a little bit of, like, a cost - management play for organizations - as to, like, who's using what, when are - they using it, things - in the world of certainly more in the - world of consumption billing than it kinda was - in the past a couple of months building - up to this.

For sure. Cost management is - one that'll be another topic for another day. - I started playing with it, though, because - there's a bunch of stuff that's going to - change. We'll have to fit that oh, by - the time we come out with a new - episode, it will have already been implemented.

There's - a bunch of stuff that changes on July - 1 in terms of availability if you don't - have some of that set up. The world - continues to change. Another topic, another day, Scott. - But for now Another topic, another day.

All - good. I'm gonna go to a graduation, open - house, and hang out with my family up - in Michigan. Alright. Good luck.

Have fun on - the Peninsula, Ben. Alright. Thanks. I will do - that, and enjoy your weekend, and we'll talk - to you soon.

- If you enjoyed the podcast, go leave us - a five star rating in iTunes. It helps - to get the word out so more IT - pros can learn about Office three sixty five - and Azure. - If you have any questions you want us - to address on the show, or feedback about - the show, feel free to reach out via - our website, Twitter, or Facebook. - Thanks again for listening, and have a great - day.