Why CIOs Need to Break Something: AI Leadership with Parul Saini of AI Ally Works

Speaker A: Foreign. Every executive must, must have hands on experience in building these tools, building a scalable, reliable system. Not just sitting on replit and building a poc, but really understanding how difficult it is to automate something using an agent if the data is not clean. Unless you understand what your teams are working with, you won't be able to make the right decisions or make the sort of strategic move. It's absolutely critical for us to have that hands on experience right now. Welcome to CIO Classified where you'll find candid conversations with the world's leading CIOs. This week we were joined by Parul Sani, founder of AI Ally Works. Parul spent years leading it at some of the most demanding companies in tech before stepping away to found AI AllyWorks. Parul and Yousef dig into why every executive needs to personally build an agent and watch it break before making another AI strategy call. And why when she asks her peers about agent security, nobody has a clean answer yet. Now here is your host, Yousef Khan.

Speaker B: It's 7am M on a Friday and of all the things you could be doing right now, you decided. I would like to speak to Yusuf Khan. Let me uh, start my first question. What were you thinking?

Speaker A: You are my most favorite person in the world. So when I decided who will I wake up for? Oh, that's very 5:30 in the morning on a Friday morning. After doing a dinner last night.

Speaker B: Mortified and only use mortified. You're very, very, very kind. Great, great to have you really appreciate you make the time for us. I think there's lots to cover, lots to kind of understand a little bit about both your career but also kind of what you're doing now and of course the craziness that we see in the industry specifically with respect to AI. So let's, let's just start off at the very basics. You've been and built a phenomenal career, you know, rising up through the ranks. You started in the consulting world, you then worked for software companies, small company called Zora, uh, a slightly another smaller company called Splunk where you were there for several years and actually really took the ropes on a number of major initiatives. And then you know, Uber and you were there leading it and you know all that operation at 30,000 person company across multiple countries. What was that like? Tell us a little bit about the lessons learned.

Speaker A: Yes, I think the primary lesson learned at Uber. Well, I joined the company because I was a big proponent of Uber and I've always used the product and it was a category defining product and it was really exciting to be part of the company. And uh, innovation never stopped at Uber. So when I joined we were moving from a, uh, B2C first company to a B2B first company. And it was really amazing to see the pace of innovation and the way we did innovation at Uber because like, nobody had any ego and we will decide to go do something and somehow it would magically happen. Yousef it was still a surprise to me, but because people are so confident, so determined that things just happen. So we were from an IT perspective, we were moving from a B2C to B2B first company. And there was a lot of foundational stuff that we had to do because when I started, people didn't know what it was or what we can do for them. There were some really good foundations in place that the leaders had put before me. But from a business perspective, go to market functions, how to support sales, basic things were missing. And it was a tremendous opportunity for the IT organization to be, to move from reactive to proactive and truly become that operational intelligence layer for the company. And that is something I'm really proud of because I challenged my teams quite a bit to operate completely differently, think outside the box, be product leaders and not IT managers, if you will. And they all stood up to the challenge. And where we were able to create this beautiful relationship with the stakeholders wherein the GMs would come to us first going, we were trying to do this in our features. Tell us what you need on the IT side, what do we need to be ready for? So it was just really amazing to see that change when I left. And it really makes me happy to be able to put that foundation in place.

Speaker B: So I guess just one, one more question on that, which is CIO's need to kind of learn. And I talk about the fact that you need to build deep partnerships, understanding the business, you know, getting to understand the customer experience of the very, very, very center. Of course being a consumer app, you could have firsthand experience of that. How did you navigate a company that's growing at the scale that it was with a lot of the executive leaders, et cetera. Like some advice that you would give to CIOs and IT leaders were looking to try and replicate that scale product,

Speaker A: much like sales is a, uh, people first work or game if you will. For us to develop the right understanding of our stakeholders, we have to get to know them as human beings first. We have to understand what are the constraints that they're working with, what are the challenges? Because I always thought My job was to take that cognitive overload that they have with regards to technology and take that on mic sell so they can focus on their primary jobs and be absolutely 100% successful. So I spent a lot of time building that relationship, building that trust. Of course I have to show quick wins and build technology. All that is secondary as far as I'm concerned. I need to build that trust with them so they know that I have their back 100% and then they can ping me at 9pm on Friday night and I will answer that question and I will have, I will find out the answer to the question if I don't know, but like I am there for them. So with all of my stakeholders, that's how I worked. And I would say a lot of people struggle within Uber, but I was uh, people would come and tell me there's a lot of politics at Uber. I'm like where I m don't see it. I'm able to get everything done. I, whatever I decided if I wanted new budget, even like getting headcut, which is very difficult at Uber, I was able to figure it out because people knew that I was coming at a place where I understand where the business is going and mapping what the stakeholders want, what their goals are with where the business is going and how technology will enable them. That's where I focus and it is a lot of work. I will say you can't do a 9 to 5. 9 to 5 was my primary job. Then I actually spend time with people, be it my team, be it my stakeholder, be it anything else wherein you have to build this personal deep relationship to be successful. So I would say to summarize that product thinking, you can't have point in time creation. You have to be thinking forward. So if your stakeholders are focusing on year one, you have to know where the technology is going three years from now for you to make the right decisions. Today that is just table stakes. And third, build those trusted relationships and put people first is the primary thing.

Speaker B: Fantastic. Thank. I mean I think a lot of takeaways there which I think people will really absorb and think about, especially in what they're looking to do, uh, how fast the industry is moving. You've got to basically adapt. So let's move on. Now it's a new chapter for you. You're building AI Ally works. Let's talk a little bit about what made you think about, you know, both make that decision to start this and tell us a little bit more about what you're looking to do.

Speaker A: Yeah. And you and I have talked about it in the past and you know, you've had front row seats to the health challenges and personal challenges that I've been dealing with last few years. And you know, in my mind I'm a very determined and strong minded person. So in my uh, I wanted to do everything at the same time taking care of a pretty big, you know, personal health challenge along with taking care of a pretty big responsibility, work wide. And I realized that I was prioritizing the wrong thing, which happens often when you're, when you're working on something. We tend to deprioritize life and deprioritize health. So despite the big wake up call that I had received, I continued to deprioritize my health. And I got to a point I realized that you know, as an engineer I really need to look at data and I need to do predictive analysis to understand that I'm going to go off a cliff if I do not take a step back to fix the root cause and really take the time that I need to take to build a strong foundation for health so that I can have the next 50 years of my life. So that was the decision where I stepped away from full time job, wanted to focus on health. And as I took that time off I realized that society is fundamentally shifting, which I learned being in the room with other C level executives over the last few years. How rapidly it was shifting is something that I understood mid last year. The thing that concerned me was we at the enterprise level were struggling to understand how AI is going to change society, shape our world, how we should implement that in the business, how will the business models evolve? We at the enterprise level were struggling and I'm very connected with the small business community in Oakland just because a lot of my friends are small business owners and I saw how much they were struggling because there's a lot of misinformation, there is a lot of information. Most of them are non technologists. So there was a lot of confusion and they were really worried about what is to come. And I could see that big ripple effect. And most of them depend on enterprises for their businesses. So I could see the ripple effect. If enterprises struggle, they cut cost, they cut staff. It is a direct impact to this backbone of our economy which is small and mid sized business owners which make 53% of US GDP. So my point was who has their back? So. And a lot of people would come to me and ask me to decode AI for them. Like a friend of mine who ran A law firm for a long time. He wanted me to explain what AI is. And another friend of mine who is an artist, he works with enterprises, takes industrial wastes and converts that into art. He wanted to understand how he can apply AI for productivity. So I saw this opportunity and this need that they had wherein I could take my CIO hat, put that back on, but then become CIO for these small mid sized businesses and help them understand how technology is evolving, where the world is going, what type of choices they should make, what constraints they should be thinking about in terms of security and compliance, and then just how they can augment their life with AI. So this is more for solo entrepreneurs. I gave the example, but I'm doing something similar for businesses that are 5 million and up to 200 million in revenue is what I'm focusing on. So I call it AI Ally Works because I wanted an ally to be there in helping navigate AI for small and mid sized businesses.

Speaker B: So I guess the other question is, if you think about AI budgets and the way they're growing, especially with more IT responsibility for that and involvement, how should companies be thinking about, uh, sort of budget management in this world of kind of the exploding use of AI? I mean it's, I would say that we saw uh, you know, glimpse of this with SaaS, of course, and you had, you know, what we used to term the shadow it, which we just now just call it applications, uh, sort of coming into place and now, but now it's a little bit different. You've got people who are not just saying I would like to use this application, but are literally looking to, you know, vibe code or build agents and sort of, you know, sort of put that in place. So how should, how should companies and how should CIOs think about that, that sort of change that's happening in the industry?

Speaker A: I, uh, think that's a really good question. Forget about AI for a second. Even when I was doing my role as an IT executive, there was a lot of wasted money when it came to application, right? It was again because, you know, the, the challenges of society show up in the technology, the way the technology is built, emotions show up in the way products are built. So the anxiety around which SaaS product understands my work showed up in all of these SaaS. Applications that came in an organization because people wanted to buy what resonated the most for them. So we ended up having hundreds of applications. And like in some startups I talked to, they have 200 applications, which is, which is a staggering number of applications to have for a new company. But we had many more at Uber. So I introduced a framework and I had to do this framework and you know, have my team use this framing, my stakeholders use this, this framing. Finance teams use this framing. Where I said think of your application stack as a three tier structure. One is enterprise level wherein everyone gets the same application for doing their job across the entire organization, which is the enterprise level application. So Gmail would be one such application. I'm not going to give you a different email and I will have a different email. It's not going to work. Everyone gets the same chapters, enterprise level applications. So when you're making a decision on introducing an application that affect all the employees, think about which one would make sense considering different factors. But we choose one and we choose one only then think about that is tier one. Think about tier two which is business specific application. Now uh, not everyone needs Salesforce, but all the business units that are selling would need Salesforce per CRM. So which CRM would be the best fit based on the type of businesses we support? We can have different offerings of the CRM, but one platform we shouldn't have Salesforce plus HubSpot plus something else. And then the tier 3 is niche offerings which is HR made something very specific and then uh, legal needs something very specific. So in the world of AI we still have the tier one offering. They don't go anywhere. Tier two to some extent will change. Tier three, the niche offerings that we had, those go away because those were the tools that you know, engineering teams would need for doing project management that they the way they wanted because legal project management was different. Those tools can go away very easily. And just in my career at Uber especially I did not get new budget every year but I was able to extract renegotiate contracts, sunset some of the application. I was able to reclaim at least 30 to 40% of my budget every year. And that's how I reinvested in new technologies. AI is just opening that gap and showing us that the gap already existed. We have to do something about it now. Uh, so I just funded from my savings into new technology and then you know, the executives will, will have a different AI pool. But it was more, more, more about shifting from SaaS, applications that were nice to have to AI infrastructure. That is a must have right now.

Speaker B: You know, I've spoken to a number of CIOs, CTOs, engineering leaders and of course over the last couple of years a topic of conversation always comes up is is AI first and foremost they're Receiving no shortage of demands from the business to be able to integrate AI in and build AI products. But ultimately, they've also asked the question, where does AI belong in the engineering stack? You know, one of our wonderful sponsors for this episode is Blitzy. They're able to provide an enterprise autonomous software development platform. And one of the things that really stands out for them is the infinite code context that they bring. It's very, very clear that Blitzy is one of those solutions that is able to really uplevel an engineering function, something that I'm very, very excited about. So I would encourage people to visit Blitzi and book a demo and look how Blitzy transforms your enterprise operation. Let's actually talk about specific workflows you were involved in, being able to roll out agents and looking at AI initiatives across a number of core functions. You've got CIOs who need to kind of do this on a regular basis. Just talk a little bit about how you've encapsulated that thinking in your existing company in terms of operationalizing some of these workflows. And how should people be thinking about this? How does this come together? Sure.

Speaker A: We started actually quite early in Uber, in 2023, when I think it was ChatGPT version three. Five came out and Google, just before even they launched Gemini, ChatGPT 3.5 had come out and it was very clear that it's going to become a core infrastructure for an organization. So I did two tasks. One was the build path and one was the buy path. For the buy path, wherein I it was a matter of time. Every platform have some type of AI offering. And I wanted to make sure that there is a task force in place to be able to evaluate that. And I get the voice of the organization, not the voice of the IT team. Because we are not here to solve, we are not here to make decisions for everyone. We need to enable them, but they need to have a voice. So I reached out to all the executives, um, Dara's direct. And I asked them to nominate a pair of people from their team that I can put in my CoE center of Excellence for evaluating AI solutions. And that created both some excitement and some FOMO. So because I had the executives handpick five people and five people only, because I had a subset of very, very few licenses in the beginning, it forced the executives to pick the best people in their teams and the most influential people in their team, which was important. And then the people that were not part of my COE were very interested in what the COE is doing so then they would come to me and be like, can we become a part? And I would add them only if they commit to spending time on that tool and giving me data that I could use effectively. So we ended up having 250 people in the COA and my team did a great job of managing all the militias. The second thing that I did was I created Runbook. So if you are a TPM or a uh, program manager and I give you ChatGPT and access to a few other tools, I need to be able to tell you based on your day in the life of how can you start using these tools right away as opposed to. Because you never have time to go learn it. So how can I make it easy then? Uh, my program manager, really good. So they will sit down with the people going, let me show you how to do it and then you do it. And then we'll run surveys, see how it's going to. So that was first, uh, first track and I was able to make decision on buying the tools based on the data I was receiving from this track. So that I know like I knew if I bought something the usage is guaranteed more or less and then we can only go up from there. The second track was where we had opportunities to become more productive for the organization. And my favorite use case was sales. Because so far if I would go to a GM and I would ask them how are you trying to or uh, what is your strategy to meet revenue goal? They would always hire more salespeople. That was fascinating. I'm sure there is something else we can do. So my point was what about if we just supercharged the salespeople? Um, and this was end of 2023, AgentForce was not what it is today. We didn't have sales a, we didn't have AI, sdr, nothing. Right. We only had GONG at the time and, and some, some expectations that people had. So then my team actually built a version of agentforce within Salesforce and we surfaced it and of course there was like um, code in the background. But we surfaced it in Salesforce and that was more for the outreach and pitch type use cases. And that's just one example. Took the approach of fde, if you will, in some, in some aspects my team would go sit down with the salespeople, we'll watch them work, and then we will come up with an idea of where we can use AI in that entire workflow.

Speaker B: And that, like I said, is. I think that's the key thing. It's like you've got to actually understand the customer experience, whether it's internal or external. You've got to get into a layer of depth of how that's broken down and then see how you can base each of the ways. Is that fair to say?

Speaker A: Exactly, exactly. Yeah. And uh, we, we did like all of the use case determination and then we would prioritize them. Then the, the thing that I don't want to forget is ROI analysis. We built models to do ROI analysis working with the finance team so they will have our bags when we go and talk to the CFO and potentially ask for funding to then showcase what the ROI is going to be. But the finance team can speak to IT and not the IT team. So that is another thing that I thought was successful. As far as we're concerned, the sales use cases were a, uh, hit.

Speaker B: Let's go in a few final questions to sort of go into. You've had a, uh, career across multiple companies, you know, both in kind of enterprise software, of course, massive incumbent, big brands. Let's assume for argument's sake that somebody convinced you to go back and become a CIO again. What are some of the things in your playbook, one or two things that you would do very differently, uh, in a new CI role versus what you've done in the past?

Speaker A: There is a philosophy that we were living by when I left, which is called strengthen the core and innovate at the edge. Because I have done this at a very large scale. I know what decisions we need to make to strengthen the core, which means primarily data and integration and essentially governance around data as well. Relationships, organization, maps and such. That is strengthen the core. But then when we'll innovate at the edge rather than going a SaaS application first. And I'm talking more about productivity, I'm, um, not talking about erp, ucr. There's no way I'm going to build an ERP or CRM in Binaural. Uh, yet. How people experience work, that must change in today's world. We have to take a new view of how you are experiencing work. And what do I give you as an IT executive to make your life, your productivity higher than somebody else, make your life more efficient. So it's more about personalized productivity in an organization and that innovation is something I will start right away. There is an interesting point that I want to bring up that we were, um, at a discussion, ah, some time ago and Bris at the table was sharing that they're hiring an intern that they had last year to manage agents now and they're talking about.

Speaker B: Yes, I've been, uh, I've, I heard about this. Yeah, totally.

Speaker A: Yes. So that's an interesting change and that there are new startups that are coming up in this space. One is we found AI, if you haven't heard of it. Tatiana introduced this concept of Homo sapiens and AI sapiens. So her team is built of, uh, AI sapiens and Homo sapiens. And I think it'd be a matter of time. So that is another learning that I would like to take into my new world as I go as to how do I think about staffing people that will manage agents.

Speaker B: Great. Okay, that's definitely a new paradigm. Let's go into one other topic. So what is the number one concern you have about AI deployments from a security, uh, lens or governance lens? Like, you know, one that's literally at the top of your agenda either as something you would operationalize to be able to make sure you secure, or just a concern that you have that you want to keep tabs on.

Speaker A: The concern is it's so easy to build right now, but it's not very easy to maintain still. So the challenges that existed in the SaaS application world where we didn't even know how, like who bought what SaaS application using their credit card and put their enterprise credential in it and, uh, what type of data they exposed. And that challenge, we barely got a handle on that with all of the fancy security platforms that we had. And here comes agents. For agents to work, they need to get access to these baseline platforms, which has been the nightmare scenario two years ago that kept me up because we were. The things that will prevent catastrophe are not moving as fast as these agents are moving or the technology that enables these agents is moving. So that's what really concerns me. And even now when I talk to my peers, I don't think the answer is clear. Um, I don't have an answer to it. I just feel that we have to think about security and compliance differently than we have in the past. Much like how we are thinking about our interaction with software. We're thinking about that differently, but we're not thinking about how to secure that differently. We can't use biometrics, we can't use old sort of metadata to make those decisions as we've in the past. I just feel like we're not moving fast enough on that space.

Speaker B: Okay, so final question. So if you think about the fact that boards are now kind of mandating both AI fluency, people are just kind of nodding along and, but doing the actual change requires actual work to make happen. You've been a leader of large teams. Now let's assume you're back in the CIO role. If other CIOs were listening to this, what advice would you give them to say, here's the one thing or two things you should definitely do to be able to sort of move the needle on that?

Speaker A: I always go back to this book that I read a few years ago. It's called Brave New Work. It's written by Aaron Dignan. And that book introduced the concept of something called chaortic organization. Not chaotic chaortic, right? Yeah. And the primary thesis of this book was that the organization structures, the corporate organization structures were created during the industrial revolution or that era when industrial revolution was coming up and we were building industries. So a lot of the structures have existed and the hierarchy has existed in service of the industrial era. When we moved into software, we really didn't update our organization structures. So that gap in the way work is managed or the challenges in the way the work is managed, with the hierarchies that we've created, the gap existed. And you would notice that oftentimes executives are removed from the hands on work and what's going on in the ground. They're very good with executive functions, but they would not have the details that they need to be able to make decisions effectively. They rely on their teams and the teams rely on the teams. That world is completely changing. So the first thing all executives, all leaders, um, need to understand is that the organization structures as we know of, they don't exist. The skill sets as we've thought of, don't exist or shouldn't exist. That still exists. The world is evolving. We have to think about again what happens three years from now. So, uh, and this also came up in the discussion that I was at more recently and I've also also thought about it. Always when I was hiring people, we had these very clear competencies, job descriptions, requirements. But we always missed asking people to have critical thinking or just think outside the box. Those were not the skills we prioritized, but those are the people that make a difference, right? Not, not the people that would have. And of course you need experience, but not the people that know exactly how to configure, uh, salesforce really, really well. But the people that can sit with the sales team and understand what needs to be done and let's figure out how to do it, which is, you know, engineering skill set, if you will. So engineering mindset, critical thinking, thinking Outside the box, these three things have become really, really important now. So for leader they need to understand the organization structures have changed, the way they should hire for skills has changed. And the skills have changed because how do you accommodate for agents in your workforce, uh, going forward and how to get rid of that hierarchy. So I would say that's one thing because the flatter the organization, the fat threat will move. The second thing that we need to be mindful of is hands on experience. Every executive must, must have hands on experience in building these tools, building a scalable, reliable system. Not just sitting on replit and building a poc, but really understanding how difficult it is to automate something using an agent if the data is not clean. And what does it mean to have clean data? Because it means different thing to different people. But uh, yet the agent that you've built break and you have no idea what broke it. And you look at the code, you understand the chaos that it has created. Unless you understand what your teams are working with, you won't be able to make the right decisions or make the right sort of strategic move. It's absolutely critical for us to have that hands on experience right now.

Speaker B: As you know, uh, it's an area that I'm working very heavily on so. Because I believe that's going to be one of the biggest issues that sort of come up in CIOs leading AI projects so far. Thank you so much for being on the show. Thank you very, very much for your insights. We appreciate you doing this early morning out of your schedule. I couldn't be more prouder of this next chapter for you. And of course you know that I and a whole bunch of other uh, of our peers and friends are here to support you any which way we can. So thank you for making time. I just really, really appreciate it.

Speaker A: Thank you. Fantastic to be here.

Speaker B: Thanks for listening to CIO Classified. This episode was brought to you by Blizzy. Want to accelerate software development by 500%? Meet Blizzy, the only autonomous code generation platform with infinite code context. Purpose built for large complex enterprise scale code bases. While other AI coding tools provide snippets of code and struggle with context. Glitzy ingests millions of lines of code and orchestrates thousands of agents that reason for hours to map every line level dependency. With a complete contextual understanding of your code base, Glitzy is ready to be deployed at the beginning of every sprint. Glitzy handles the heavy lifting, delivering over 80% of the work autonomously. The platform plans, builds and validates premium quality code at the speed of compute, turning months of engineering into a matter of days. It's a secret weapon for Fortune 500 companies globally. To hear how engineering leaders are transforming the way they deliver software, visit blitzi.com that's B L I-T Z Y.com schedule a meeting with their consultants to enable an AI native SDLC in your organization today.

Speaker A: SA.