The Biggest Mistake Companies Make With AI Right Now | Hugo Fragoso

Speaker A: The winners, they will definitely have governor AI. So that's, that's the only possible outcome, whatever form, whatever vendor. But they need to have a governance layer to AI. The losers will do, will have a lot of technical depth or a lot of AI slop because again AI is great but it, it just generates things and they will end up with a lot of disconnected kind of prototypes glued together in a way that they will have to manage uh, technical debt.

Speaker B: Hi Hugo, welcome to Drag and Drop.

Speaker A: Hi Ferris. Uh, thank you for inviting me and I hope that I can do my best to clarify whatever you have in mind.

Speaker B: Thanks, I appreciate you speaking to me. You know before we kind of jump into things it'd be good for those who don't know you to get a bit of introduction to yourself and your background.

Speaker A: So I've been quite a while in our Systems now almost 1112 years but I came from the ecosystem so I was myself node system architect for uh a large insurance in Portugal so Fidlidad and I was the one responsible for evaluating the platform and getting to know and setting this, the, the, the first kind of governance model for this low code that at that time, this was 17, 18 years ago there wasn't like a benchmark for what to do with low code. So struggling, struggling and trying to fit in that on a very typical traditional kind of uh, high code house that was fiblizard at that time. Uh then I, after a couple of years after I transitioned into OutSystem4 from a startup, uh on that startup I have applied some of the concepts that the platform brought at a high expense. So a lot of hours getting the DevOps correct and all the governance correct. But once I transitioned to Outsystem within my outsystem in terminal I did a lot of, I uh, had a lot of roles so I was working uh in operations so managing the cloud, setting up everything related to our cloud offer, uh, AWS partnerships and so on. I worked briefly for around two years as a csm. CS was starting back then in outsystems and I was working as kind of a technical account manager for top uh, uh customers in the US and then I transitioned to solution uh architect. So I was for four or five years in London managing the SI team there and then uh, yet again I transitioned and that system offers a lot of opportunities for that. And in the last two years I'm working uh in an EMEA scope with strategic customers. Internally we call this function Business Value Consultant so it's trying to translate and have top customers uh, uh, extract the most value out of the platform. And I think that it's a bit of the topic of the conversation today.

Speaker B: That's what I wanted to ask about. You know, what, you know, with this strategic, you know, kind of customer title, what puts an organization into that category of being a strategic customer?

Speaker A: So there's um, there's always a vendor perspective and the customer's perspective. Right. But the ones that click and um, get into these kind of strategic buckets are the ones that see the platform not as a technical capability for it, but as a business capability. So the ones that have kind of the business goals align with it and that's unfortunately rare. And there's a change process to get there. Also it's very important to have the right executive sponsorship. Uh, without executive sponsorship is very difficult just to have um, high tech capability that is not aligned with the outcomes that uh, our customers want. But I would say that really this kind of alignment and extracting and having clarity on the business goals before doing the first step and then being able to have these executive sponsorships so that they scale, it's one of the most important things for an organization to be considered strategic for us.

Speaker B: And you know, what do these organizations have to really have in place before they start with the platform?

Speaker A: So actually that's a curious uh, kind of question because again uh, a lot of customers start or prospects start evaluating these kind of technologies looking for an IT capability without they want a quick fix so they have a problem and they want to fix that problem or that gap as fast as possible. But in hard reality if they do not have clear data lake or uh, a uh, data structure that you can lay the foundations if they do not have uh, a good integration, uh, capability or uh, integration mapping. It's very hard to bring these technologies because ALT system comes in and typically plugs into data and plugs into existing integrations and builds on top of that. And the last thing is really the change management process. And then this is really important because we'll get the speed and agility that the customers want. But without clear access to data, uh, without clear integration capabilities and a solid change management uh, process or an operating process, it's very hard to bring these technologies. And it puts us in kind of a strange place where we do offer the agility and the speed to get the outcomes. But then the lag, uh, uh, factors are really data and integration access, uh, to get that going.

Speaker B: Yeah, and you know, and you touched on, you know, what are those warning signs, you know, that you see that, okay, maybe this company isn't quite ready.

Speaker A: So having access to a strong champion. And a strong champion is not someone that wears the outsystem T shirts. He's someone that actually is able to understand how to map the technical capabilities to what did the outcomes that, that you want. Uh, without that strong champion it's very hard to be ready, right? It's very hard to start. And then yet again it's the same principle of technology only go so far. And this applies to all vendors, right? Uh, without having uh, that strong champion, with a full breadth of uh, capacity mapping to whatever uh, portfolio of vendors or technologies the customer has in place, it's very hard to really expect a vendor to solve a problem. Right? So it's really ownership. This is still a build platform out systems. What that means is even though we are sometimes trapped in between the build and the buy kind of cycle, you still have to have ownership of the outcomes. And without having this, ownership of the outcomes is not the technology that is going to solve that. So having, making sure that you have a strong champion with ownership of the outcomes is uh, I think one of the most important aspects.

Speaker B: And you know, with these organizations if they could only get you know, one thing right, what do you think would that would be? Would it be you know, governance architecture, you know, their operating model, what is the kind of, the main thing they could get right to kind of set them up for success?

Speaker A: People and process, uh, everything will follow. What that means is that so when you start to do the first steps in outsystems it's um, sometimes it's unclear who owns the outcomes and then these kind and uh, so we have a very strong partner ecosystems. And what that means is that sometimes customers just bring a partner to solve that problem without wanting themselves to have ownership of that problem. And what that means is that this creates a huge skill gap uh from what the partner brings and actually uh, the outcomes that the internal it has to have. So it's really important to understand that people and processes come first. And I think this is the definition of uh, ready to adopt these kind of technologies before really even looking at the technical capabilities and onboarding developers. For example, I do think that having also a certification level of developers so that you have a common kind of baseline of knowledge. It's also very important so people processes and with that people in process scope, how can you train and onboard the right people to uh, opsystems? Also the change management is important on these kind of people and processes because right now it's kind of different because right now low code is kind of a space uh uh. And opsystem is already a brand name within this space. But beforehand what happens is that we were actually fighting against developers with no particular reason. That means that these mindset of developers needing to understand that they are a mean to the end. So they will need to fulfill a business capability or deliver on business capability without going into the bit and the bite immediately. So developers are passionate for. I was myself a developer for a particular set of technologies uh and changing these mindset of uh you uh are fulfilling a function and just choose the best tools to do uh uh that that use case or build that use case is rather important.

Speaker B: Yeah I guess it's not you know it's not important of how you get like you know what you use to get there. It's you know it's kind of you know the way you do it. It's not that important. It's kind of getting the end product and delivering what you need to deliver.

Speaker A: Yeah, correct.

Speaker B: You know m moving on from you know the early wins to kind of more enterprise wide adoption. You know where do enterprises typically you know underinvest in the first year.

Speaker A: So like I said no en I think is the major underinvestment making sure that their teams are ready uh to execute. I think that's the point. And then uh at a higher maturity level as soon as you have the team enabled. Sorry. It's really how do you get to a CoE stage? So how can you create the operating model? That means that you are reusing as much as possible. You are creating the foundations to get the demand of uh the it. So this is a uh demand problem and not a supply problem. The technology will serve the supply part. Now the demand getting uh so that the backlog is clear ownership and the team enablement. I think it's the major underinvestment uh within this kind of space.

Speaker B: Yeah. And you know what do you feel and this might follow on from the last answer but you know what separates organizations that scale from those that.

Speaker A: So they are not building use cases, they are building a factory. So that's, that's the kind of the shift that needs to happen. And it, it needs a special type of structure and mentality to identify reusable patterns to also not over complicate because sometimes there's a lot of legacy in terms of the governance model that these customers bring from other technologies. And then when they are starting without system they also need to understand the outsystem platform brings a lot of guardrails uh, in the way it generates code and runs code. And that means that these kind of gap needs to be understood in a way that they maximize the output of this factory instead of just trying to build a use case. So that's, I think it's the most important paradigm shift is think about the output in the factory and less about what you're building at the end of the day.

Speaker B: And you touched on obviously that internal capability is really important and obviously partners are also there to support. How should each enterprise think about internal capability versus partners? Because like you said, obviously in the space wherein we come across many who don't have that internal capability, how should they be balancing that?

Speaker A: So partners bring speed. So what that means is that on day zero they are actually able to hit the ground running and just deliver. Now the most important thing is for you to make sure or a customer to make sure that their knowledge is translated into their internal teams. So having uh, at least one architect to be able to translate that and also to clear roadblocks because there will be uh, a couple of roadblocks. It's important in order for you to kind of get the show going right. So partners are very important in the first kind of stages. On a later stage partners are also important. And some of our customers actually realize that um, they see kind of partners as uh, uh, ah, an extendable arm of them. So if they have the COE in place, they have the ground foundations and they are able to plug multiple partners. So some partners are very niche to specific technologies, say partners in SAP for example that ah, bring that expertise. But some partners are more tailored to unique UX UI kind of B2C kind of use cases. So if you have your COE and your governance model well defined it's fairly easy on the later stage to just get the partners on the right, framing on the right and with the contract well understood they are not just delivering, they need to deliver the outcomes with a certain level of quality that is asked by kind of a frame contract that these kind of very mature customers have with partners and typically they go to market. I see customers that have basically year tenders that they just go to market to our partners and just say look for the next year we are choosing three partners, please bid and uh, realize that this is the interface and the level of quality that we need. So there's multiple kind of uh, from getting the first step in speed with the partners transition then the knowledge to an internal team and then creating the interfaces so that an internal team can be augmented for specific reasons. For A specific project. I think it's very important.

Speaker B: Yeah. And talking about those internal kind of roles as a client or customer scales, which one's the most important roles they need to kind of have that you touched on it as they kind of scale and grow and want to build that cae, which roles are the most kind of critical.

Speaker A: So solution architects that don't need to be solution architects within the autism scope are one of the most important roles. And these are the people that understand the build versus buy kind of dilemma. They can kind of understand the reusable patterns that get the most out of the technology. And then uh, after those I would think that having a clear platform ownership on the business side. So if you are looking at the platform that can fulfill uh, a series of domains. So imagine that you start with a customer with a supply chain kind of use case. So you have a business owner and the stakeholder for that, that uh, that uh, uh, uh domain. And this is very important because the agility and its speed that then the autism projects run put a lot of pressure within the business to take decisions. Right. So you cannot be waiting for two weeks on a decision when you, you in two weeks you can get the, the, the, the use case built or the new uh cycle um, or the new release uh done. So it's very important to have these like strong solution architects kind of baseline that understand the repeatable patterns then full business accountability on a business owner. And sometimes you need the, between these. So um, even though we typically on autism projects we do not have a specific business analyst to translate these. Some customers might need them depending on the maturity that they are writing requirements against.

Speaker B: Yeah. Uh, do you feel you know, with customers and other partners and this building internal capability that you know. Because some countries have a shortage of experienced talent in those areas so they feel they're more against hiring traditional and training versus they'd rather just use a partner who already have those skills.

Speaker A: Uh, well, so you need to certify developers, ah, this is the kind of the first uh, baseline like I said in the beginning there's always kind of a pushback by developers of using these kind of technologies. And it's important to have someone that has the knowledge to transition them from a technical developer to a business developer. And this is kind of the middle ground where outsystem and local generally sits. So people that can actually have enough technical knowledge that they can go deep within the platform stack and take the advantage of the platform capabilities but while still maintaining kind of a uh, relationship with uh, the business this creates over time kind of the platform standards that you need to operate to. And really uh, certification is a uh, cornerstone. You can certify any developer coming from traditional technologies to alt systems in a fairly easy way. So typically the rule of thumb is a professional developer can get certified in like two to three weeks and get proficient within a month. Uh, an end developer will uh, get there but will take typically uh, twice as much. And it needs to have that kind of mindset, uh, programmatic brain to be able to do it. But uh, without the certifications you are just running loose within your teams basically.

Speaker B: And what advice would you give to leaders who are really dependent on hero developers?

Speaker A: We have a lot of those. So every time you have a neuro development you have a single point of failure. And I have some funny stories along these times where I uh, was starting with work with customers and realizing and spending time. So this was pre Covid everyone in the office and that was I was spending time with the customers and once I was in a UK customer banking customer and it was kind of interesting because I realized that that person that we were talking with which was a uh, to be champion was actually a hero developer. And what that means is that even though he feels realized by that, so everyone was going to his desk asking questions, he wasn't blocking a lot of things. But uh, customers need to realize once that person is gone on a very competitive job market so he can go away or he can just be absent on vacation. So you have a very strong dependency with that particular hero uh developer. So how can you build the guardrails, how can you build the operating model and how can you be um, clear to all the team that the technology is transparent and with a good baseline uh, of architecture. So AI mentor helps a lot on this stage. Everyone has this clarity on what to do next and there's not a single point of contact uh, or failure within the team. So they are important, uh, they are important because they hold a lot of context typically but at the end of the day it's really a liability that the customers have if they depend on these hero developers. We love them but also from uh, a customer perspective they, they need to realize that uh, it's a short term gain for a medium to long term loss if they just that all in one single person.

Speaker B: And I think that uh, it's more important in the countries like the UK where there is a shortage of talent. Right, you know, experienced talent. So if you're in one of those countries like the UK there's probably Recruiters tapping that person up on a weekly basis. So the chances of them moving on are very high.

Speaker A: The UK and the US are very competitive markets on the job market and that means that uh, developers and these top tier kind of developers have a lot of choices and people are, are still uh, uh, sentimental. What that means is that uh, sometimes unfortunately we see these kind of people that are uh, they would be the most amazing champion but they are actually uh, not able to change their mentality and just understand that they are business function uh, at the end of the day. And AI actually is transforming this because code is right now a commodity. Everyone generates code and these people need to, or these kind of developers need to realize that their edge that they had in the past in terms of the job market is being eroded over time as AI comes.

Speaker B: You touched on it, you know, switching to the most talked about topic, you know, for the last few years, probably for you on a daily basis. You know, it consumes everything. You know, AI and agentic AI, you know, I'm interested to know, you know, what is the first reality check you know, you're giving regarding agentic AI quite an interesting.

Speaker A: So there's a lot of buzz within AI. There are very few customers that are actually using AI at scale. And I do think that the first most important take is uh, do not use AI just as a suggestion. So generating context or even code is quite easy. Doing it in a way that AI can act with the specific guardrails. I think it's the key to this kind of industrial transition. Uh, it's very easy nowadays to uh, use dllms and uh, everything about code generation actually giving that the correct frame so that that piece of non uh, deterministic code can run within a deterministic kind of enterprise in it's key. And I think Agent Workbench does quite a lot of these which is all the tracing and all the guardrails that is around. Agent Workbench actually allow this kind of transition from just suggestion uh, to actually acting, acting with the human in the loop and having some control of the process. But I do think that it's a unique uh, uh, place and time in the industry that when I retire in a couple of years I will look back and uh, take my findings out of that. So no one knows what's coming but everyone is excited.

Speaker B: Yeah, yeah. And then you know, what prerequisites are needed, you know, before agents are uh, you know, you touched on it, you know, they're trusted for you to take action because obviously you know there's obviously this big hype and like you said, there's not that many actually doing much with it, you know. So, you know, uh, what are they?

Speaker A: Um, so first off trying to get the guardrails. Right. So we released things. Not very connected at this stage, uh, with all the roadmap and the releases. But uh, we released I think one month ago. The evals, evals are quite, quite important because uh, it's really a safeguard that the customers have around the LLM output. This is the utmost importance. Otherwise you are just uh, relying blindly on agents. And when you are also relying on agents, you need to understand that you have a life cycle in terms of uh, as soon as you do the first agent and you are kind of understanding from all the trail logging and all of the execution logging what the agent is doing, you are putting these safeguards. But then if you want to transition for the agent to act actually you first have to have a human in the loop to control the output and uh, of that agent. So how can you build these guardrails? You put the human in the loop and what is your decision in terms of threshold of quality so that you can then remove for some low risk kind of uh, uh, actions the human in the loops. This transition and this life cycle of an agent, I think it's very important so that the agents can act actually. Um, and also the same way in connecting to what we were uh, talking, this is still development. It's the same thing. You are just accelerating um, some of the domains that you actually could write in code a couple of years back, but with hundreds and hundreds of lines of code to get the logic, the rule engines and so on, you are delegating those to an LLM. But at the end of the day the output still needs to be owned by business. So business needs to be the one deciding which agents can act, which agents do need. The, the, the human in the loop kind of process. So it's looking at this kind of abstraction that the agents allow so that you are still under control, uh, in the outcomes that you delivered by the, the use case that you are wanting to build. So I think this transition. Interesting.

Speaker B: Yeah, yeah. And you know there's, there's obviously, you know, most organizations are very kind of cautious and not kind of rushing, but there is some who, you know, they just want to kind of rush into it and get things going. You know, what mistakes are these companies

Speaker A: making, uh, choosing the LLM first instead of the platform? Because um, if you look at the last just 12 months or even six months of AI evolution you see the battle in between the main players. Gemini, Entropic, OpenAI and so on. All of these models will evolve with time. And that means that if you are choosing at this stage and point in time a given LLM to ground all of your AI work that means that you are missing a trick. So how do you understand how to choose the platform and the governance on top so that these LLMs can be pluggable replaced over time? Uh, I think that's the key kind of uh, uh point. You cannot bypass uh, uh governance in this kind of process. So platform over LLMs at the first, I think it's the, the, the most important uh step in terms of uh, decisions uh to enter this AI world. And if you look at what everyone is doing, they are loving either they choose one and it's like football, they choose that kind of. And they are all for that club. And I do think that uh, in a capability kind of perspective you need to be able to plug and unplug the same way you look at previous 10 years ago integration. So you have a CRM. How can you make sure that you are not grounded to that CRM&M, you can replace the CRM if that CRM doesn't uh, fit your needs. So um, platform over LLMs, I think it's the most important decision.

Speaker B: Yeah. And you know, when kind of you know, advising these strategic customers, how are you advising them around, you know, how they should think about experimentation versus production.

Speaker A: It's, it's interesting because so our customers or even prospects at a large scale do have access to a lot of technologies. And what that means is that they are in love with AI because right now they can produce uh, an output or an application. I do think that we offer the same kind of capabilities in terms of, of ODC and Mentor and Vibe coding right now. Um, the point here is that they are really just looking at two silos. Either the experimentation silo, either the production silo. Right. And what we are trying to get here in outsystem is yes you can experiment with outsystems in odc. It's even free. So it's not a licensing metric. Generation of applications. You can have as many applications in non production environments for example, but then how can you then transition that pilot to the end result so without having just uh, dummy uh data with plugging with the right integrations with the right LLMs and the right data, uh, uh, so that you can transition a pilot to actually a production application. So that's I Think it's uh, uh a bit of uh, a mixed world right now with so many vendors on the experiment side.

Speaker B: Yeah. How early do you think you know security and risk teams need to be involved in this process from day zero.

Speaker A: Uh the lighter you bring them the more pain you'll suffer at the end. And this is especially important and this is not new. So even though AI puts a specific spin into that, you need to have a security team understand what is outsystems and this is very important. Not only that but beforehand uh, I had a lot of uh, interactions with security teams uh that they want to control the output of the generated code. They do not understand the system generates the code pretty much in a systematic way. So once they scan or do a static code analysis under the code or pen test, any finding, you fix one and all the applications are fixed. So that mentality of having the security teams also understand this kind of agentic world, the safeguards, the guardrails, the evals, the traceability that they can shift, uh, they can shift to uh, their own tool of chic to understand the logging and the tracings and the execution is very important because if you try to plug these at the end uh, it's just, just derailing your project and it's putting a lot of pressure on the go live instead of just making sure that security is from day zero, uh knowing that what is out systems are the AI, agentic AI capabilities of out systems and how all of these works so that they have the right level of control in order to give the go ahead to go live of an agentic application.

Speaker B: I'm interested, what is the kind of obviously with you know, agent, workbench, mentor as you speak to CISOs, what is the kind of consensus, what's the feedback when they see these features in the

Speaker A: platform they are right now comparing, Basically they are feature comparing with other vendors, uh I think the long term out system customers that are strategic enough they understand kind of the noise that there is on the market, the new ones are actually just comparing and sometimes they are not comparing the same things again. They are comparing an output on the result versus a platform or the governance of out of that platform. The real problem is that when people try to do shortcuts so for a short uh gain they abdicate a lot of uh, the governance, the security and so on in order to go live uh, with uh, the use case that they have in mind to apply agent case. So I do think that there's a bit of um, perfect storm uh within the C Suite in terms of AI, some CIOs are just wanting to tick a box. Some of them are thinking strategically and, and understanding kind of the governance of how does AI change what I already did and what is this translation layer that needs to be put in place so that they can still deliver the same outcomes and taking advantage of these

Speaker B: new technologies and you know, ahead, you know, to the future. And obviously things are changing rapidly. It could go in a total different direction. You know, what do you feel will differentiate, you know, the winners from the losers over the next three years?

Speaker A: Let's say the winners, they will definitely have governor AI. So that's, that's the only possible outcome, whatever form, whatever vendor, but they need to have a governance layer to AI. The losers will do, will have a lot of technical depth or a lot of slop, AI slop because uh, again AI is great but uh, it just generates things and they will end up with a lot of disconnected kind of prototypes glued together, uh, in a way that they will have to manage technical debt. There are also tools to do that. But if you are starting to lay tool on top of tool and tool, uh, that completes all your full stack kind of capability and governance model, you are actually having a lot of tools. So then you need to, to strike kind of a balance in between, uh, what is the least amount of tools that I can bring into my business that creates the right outcome and the right governance model to be able to achieve my business outcomes.

Speaker B: And you know, with you speaking to many leaders, you know, you know, what are the, what are the kind of top three priorities for you know, not the ones who want to tick a box, but the others, right, who kind of, you know, what's their kind of priorities.

Speaker A: You know, today dietary is number number one.

Speaker B: Ah.

Speaker A: So making sure data, uh, so when you point an LLM to data you really understand that you have a lot of power. And if you don't get the data right, uh, that means that you are just generating weak uh, outcomes or weak outputs out of uh, your data. Then the orchestration layer. So orchestration is important because if you think as how AI works, AI is just a piece of automation, right? And automation without orchestration, uh, without the guardrails and everything that we spoke about is just creates a we a weaker or a more uh, subjective uh, outcome. So how do you, you put these together, how do people workflows? The correct data lake works together in a way that is orchestrated to fulfill uh, uh, a use case. And then going back right to the beginning of the interview, um, People are still important. So that operating model, within all of this frame we are generating business outcomes. The mentality needs to be right, the guardrails, the foundation is not around AI but around how you deliver code need to be in place and only if you have the data, the orchestration layer and then this operating model in place you can actually get the right outcomes.

Speaker B: And you know, obviously we spoke about customers, you know, and you know, strategic accounts. But you know, personally for you, what are you kind of most excited about or uh, looking forward to seeing how things evolve with like you know, as we're in this kind of agentic era let's say.

Speaker A: So I, it's a tricky question because I don't know at what level I can speak about but I do think that the enterprise brain of specific functions is a very interesting thing. It's kind of strange because it's the way conversations are morphing within these kind of strategic uh, customers. Because agents, I do think that they are just a middle ground agent. Put an LLM with context, with tools to do execution or to execute uh, and to act on. But if you think about the abstraction that uh, some of the hype tools on AI are bringing and the kind of capabilities that the outsystem platform brings, you definitely realize that outsystems can be the glue in between the tools that are right now being used by small teams, startups to actually plug into the enterprise space where kind of the platform orchestrates uh, a business function with AI. Uh, what that means is that is pulling up uh, architecture patterns, integrations necessary at the right place so that you can execute a business function being that sales, supply, uh, uh, marketing, whatever uh, uh, business function. So how can you make sure that, that you are um, within that context and you do not pollute the next context. So how do these business functions work together so that AI or the platform outsystem allows you to kind of capture uh, depth business uh, function and accelerate the plug into AI of the platform capabilities. Uh, regarding uh, this kind of the way that the industry is going around, autonomous agents and so on.

Speaker B: I appreciate you speaking to me Hugo. It's great to talk to you and hear what you're hearing and seeing currently in this crazy time that we're in. You think there's not been one like it, keen to keep in touch and see how things evolve in the future.

Speaker A: Yeah, thank you so much Feros for the time and for your work in the low code space.

Speaker B: Appreciate it. Thanks Hugo.

Speaker A: Uh,