Beyond RAG: Building Production-Grade AI Coworkers for the Enterprise (feat. Karl Simon)

Speaker A: Hey folks, welcome to Snowpal podcast. Our uh, guest today is Carl Simon. Carl is the CTO of subatomic building AI co workers that help wealth management firms automate workflows and scale without adding headcount. Carl, thanks for taking the time to chat.

Speaker B: Yeah, thank you Krish. I'm happy to be here and talking with your great audience.

Speaker A: Thank you for that. Uh, before we get into the topics, we have quite a few of those Carl, if you can give a quick introduction about yourself and your company and what you folks do.

Speaker B: Yeah, so hey, I'm Carl Simon. I'm co founder CTO Subatomic. Now before I joined Subatomic, I've had a long history, a long experience that crossing 30 years of helping companies modernize based upon the latest technologies that are introduced to the market. From the early days when data warehousing was tip of the iceberg opportunity to modernizing for the digital world, social media, mobile, big data, um, machine learning. And now with general vi democratizing the accessibility of doing things with AI. I'm helping companies now get on board with that with sophisticated capabilities to subatomic. We believe at subatomic you hire AI, as Chris just noted, not buy software. More on that in a bit.

Speaker A: Thank you for that. Um, it's a lot of experience and there's plenty to learn from you the next 60 minutes. So without further ado, let me jump right in. Um, for starters, let me would like to understand a little bit more about subatomic if you could just take, I know you have a few different products but if you could take one, uh, you know, it doesn't matter which one and sort of tell us a little bit about what problems specifically it aims to solve. Who are your customers like the ideal customer and what problems you expect for them to have and how does your solution help them address those issues?

Speaker B: Yeah, you know a fantastic example is in the financial services vertical, specifically in wealth management domain where we have a client who originally was a pessimistic prospect, turned excited, heavily satisfied customer, uh, and then became our lead investor in our seed round that closed last October. So we saw many opportunities in the back office so that they can, you know, any wealth management firm with their advisors can focus in the front office growing your book of your aum, um, your revenue, your profit. So examples there included things that are really labor intensive. You know currently for many wealth management firms, whether you're preparing for a client meeting and perhaps your top tier ones you meet with monthly or at least quarterly and there's information spread out and Fragmented across many different systems. You know about the custodial systems, risk management, retirement estate, uh, planning, tax planning. All that information along with your email, your calendar, your CRM, your ERP and your SharePoint or other, um, document storage devices, you need to bring all that information together, both structured, unstructured and unified data model to be able to perform impactful use case execution. And so when you think about that client meeting prep, you need the full visibility across the information that spans all that information with a 360 view, with your cognitive intelligence through our AI co workers, because you should hire AI to do your work and work with you. We actually, you know, we basically eliminated 8,000 hours of labor across, you know, the entire advisor team there and now they can actually grow their business and they're doing just that.

Speaker A: Okay, that's a good start. I have a bunch of questions there to unpack slowly but surely. Um, so let's say the, a lot of the data that you mentioned, you said structured and unstructured data. So I'm imagining this and maybe our audience will as well as we go along. What the customer that you mentioned, Right, one of your customers, they have data in disparate places, in disparate forms and shapes and they need to make sense of the data in the context of whatever it is they're going to be doing. Perhaps the interaction that they're going to have with their client, for instance. So at the uh, 50,000 foot call, are we talking about a RAG based system that goes into all of their data repositories and finds information that that person actually needs to get onto those conversations?

Speaker B: It's RAG plus and the plus is significant. So start with the rag. You can find semantic similarity, for sure. Semantic relevance, you know, is improved if you do it in a hybrid RAG way with both semantic search and BM25, you know, indexing. But contextually relevant really is more fully accurate when you understand your clients and the different services you provide, the different requirements that client may have, the profile of the client, everything that is required across all your services that again begin with your financial management all the way through tax management, estate planning, how they all interconnect in a true complex, cognitive, interrelated understanding. So then when you perform rag, it's contextually guard rail and bounded within the contextual relevance of those interrelatedness, um, of those different entities. And we capture that in a knowledge graph that's critically important. That's where the plus comes in. And to do it right is tricky. But we handle the tricky, we handle the complex.

Speaker A: So, so how does your architecture look like? Say what are some tools like to get your piece of the puzzle working? What are some tools and technologies that you use? And also if you could give us to the extent that you're able to share sort of a high level architecture because you know, like I mentioned, our audience is predominantly an engineering audience. So everyone's generally curious to know how companies and individuals are using the latest and greatest technologies to help solve their problems. And so you could speak to maybe uh, any LLMs that you're specifically using, uh, maybe the languages that your dev team is generally working on and also some of the AI tools, perhaps you're using some services from one of the hyperscalers, possibly maybe one of those, or maybe using some new cloud for instance. So just a good high level overview of some of the tools in your repertoire, so to speak.

Speaker B: Yeah, absolutely. Before I begin on the tools within the stack, I'll state that for security, which we take very seriously, it's subatomic and we embed a lot of our own rigor into the defensibility of your systems and your data we deploy in your cloud provider with you as the tenant of the account or if you want an on prem so that we'll adapt and adopt into your security M measures as minimum we bring value add. But then the tech stack of building includes everything from having react components in the UI that can be dynamically displayed, even generated at runtime based upon the nature of a request. We use Python as a primary language because in case the customer wants a co management and we find that once in a while where there's uh, a hybrid support model between cloud, client and subatomic that we actually uh, make sure that the languages that we choose are ones that are predominantly available in most of our clients. For the stack, everything from the database, we choose the database that is best for you. Quite often we use Postgres because it's a fantastic database at a much lower cost but Snowflake databricks, you name it, know we support and build on top. The uh, the engineering framework is Lang LangChain and as a predominant one we also use others, you know, for different types of um, different types of agentic deployments as necessary. But Lang chain mostly with lang graph in support of agent, uh, graphical building or building the graphs and then Lang Smith for basic tracing which we augment more fully through extended auditing logs and bring it to fruition in a highly visible, observable way for our clients.

Speaker A: Okay, a couple of things to unpack There when you said you are able to deploy uh, your products in the client's infrastructure, that's quite similar to say we do a few things. One of the things we do is sell APIs and we exactly provision have support for something of that nature. But what I want to understand is, so yours is um, let's say if I for a new customer, maybe it's a small customer, do you provide, Is it a SaaS application where people can just go online self onboard, sign up and start leveraging or do they need to find a place for them to deploy this, be it on prem or their cloud service?

Speaker B: Essentially at the moment predominantly we deploy it in their cloud provider or on their premises. Never say never. But right now we're not a SaaS player, we are actually a custom build, define it for you in your location.

Speaker A: Okay, so then I need to understand a little bit more. So you also mentioned uh, a few different databases of the client's choice. In other words, I was imagining a product. Our product is we have a set of dependencies and when we deploy them in the client's infrastructure, be it AWS or Azure or Google Cloud, doesn't matter. It's essentially using our code, our dependencies, our libraries and our database. We use Mongo, we use others as well. So that stack does not change, it's just the deployment. And where this code lives and runs happens to be in one or the other servers. But in your example you mentioned, people can pick whether they want postgres, whether they want maybe some other database, for instance. How does that work? Because your code I imagine, is it written for, I mean for a variety of systems. And how does that work? Like why? One, is it written for multiple systems? And two, why would you want to write it for those? Because why not say hey, your dependencies are Postgres or VV8 or Oracle for instance and that's where you're going to be storing your data.

Speaker B: Yeah, well I'm glad you asked that question Krish. And there are two important reasons of why Subatomic does it your way. And if we were using my company backdrop, you would see AI your way. I'd be able to point into my back intimate, um, background screen. Basically two things. One, we don't want to actually complicate your architecture. We want to make it easy to adopt and we'll talk more about how you interface or how you interact with your chief of staff that is a part of Subatomic more at the application level in a moment. But even when it comes to adopting Technology, we want to make it easy and not overcomplicate the number of different platforms you may have. So when I say users have a choice, if they don't have a platform of choice, we typically use Postgres, but they may already be building a data lake or a data warehouse that might be on Databricks or Snowflake for example, or SQL Server or Oracle. And so we're not going to tell them you have to rip it out. Maybe they're already paying for licenses. We're trying to reduce the cost of entry into working with subatomic. And so in other words, working with subatomic means easy adoption from a technology perspective, from a user adoption perspective. And I hope we talk about that more. And that's part of our process. Now the second part of that answer is that we're only a 13 human being team, but we have over 100 AI co workers, including our own AI coworker, data engineers and software engineers. We're able to port into any platform rapidly core code that can actually just understand the very, the specific dialects of different systems and how to interact.

Speaker A: Which means, does it mean when you go work with a customer that has a slightly different setup from your previous one, do you have different adapters that connect to a database of that choice? In other words, let's say if you work with Snowpal for instance, and one of our next APIs, we're looking to use uh, a vector database like VV8 as for example.

Speaker B: Right.

Speaker A: Let's say that happens to be a data store. And, and we're like, hey Carl, we know that's what you guys need to use. So would you actually have adapters for a variety of these data stores for you to be able to support what the client actually currently does use?

Speaker B: Uh, when the database has a different structure, whether it's key value versus relational versus graph, we have uh, adapters that convert data as uh, necessary. But generally speaking it's less about predefined adapters and more the, we have conversion code conversion capabilities so that when we are working with a different solution, we have that specific dialect of the code that should be used.

Speaker A: Okay, I think I have an idea and I'll probably hold that for now and jump into something.

Speaker B: Let me make it more concrete, Chris. In other words, if you're using, maybe use AWS redshift, right? But you may want to move or you know, you want to use Snowflake, which we're offering, we can actually convert from a redshift to a snowflake dialect with Minimum impact. But if you're changing the nature of the structure of the information, maybe it's more object relational, maybe it's again, some of the other examples I gave you, from key value data stores to graphical representations of the same. We convert the code to adapt and adopt to what systems you have, if you want to preserve those systems and not add more. Now, if you're okay with adding more, that's also an easy option for us to fulfill too.

Speaker A: Makes sense. And is this, uh, you know, you talk, let's talk quickly about the ability. You know, every customer is going to be unique in their own way. And your model particularly relies on the fact that, hey, we're not going to ask you to change what you're possibly using or doing, but we'll sort of seamlessly find our way into your ecosystem. So you'll have to make the least amount of changes to be able to leverage your products and services. Which makes sense because, you know, people are going to the least friction there is to integrate, the more chances people are going to integrate. But is the reason that you're able to support a variety of these variations and adapters of, uh, this kind, Carl, is it because now code does not have to be written, it's sort of a segue. But I want to still understand that it's because your dev team now does not have to write all of the code like every other dev team on planet Earth these days, that you're able to generate a chunk, a good significant percentage of that code. So you have a version, you have a second version, you go work with a third client, you slightly have a modified third version. Now if your fourth client comes along, wants something different, is it because you're able to use these three code, uh, bases or adapters as examples, feed it into your AI tool, so to speak, and have that generate the fourth version that's unique to that client is that I'm assuming a bunch of things here, so it could be completely off. But I just want to start the conversations here. There is that a possible reason that you're able to be this flexible, that the code, a lot of it is being generated these days.

Speaker B: It is, but something that I want to articulate as a boundary. And what I'm going to further define based upon your question is that we don't reuse custom code across clients, again, AI your way. We want customers to feel like they're getting a differentiated version that actually represents the way they think about their data, the way they execute their workflows through standard operating procedures. The way they reason and make decisions from a cognitive layer perspective. But yes Krish, I mean for things that are more standard, industry standard components that everyone shares as a foundational understanding of how to uh, you know, reflect their domain and actually bring it to life through technology, there is a foundational floor and whether it's fulfilled by agents from our factory floor that are skilled up on domain specific uh, skill sets and then applying the custom specific SOPs and cognitive knowledge, those first two layers are reusable modules. Up until we deliver something that's specific to a client at that highest level, whether it's agents that are being built up that way or full feature mo features that we build into modules that are standard for the industry, we use that as starting points and then we can build off the custom layer on top of it. So that allows us, allows basically an assembly line, you know, to get to the point of okay, now that we're going to run discovery with you as, as our client will know exactly what your way of doing things is and we'll apply that at the highest level. That will only be for you.

Speaker A: Can you give us another example of perhaps even a second financial client or a non financial client that uses your product right now Carl, just so I can get a broader perspective and then ask something because I have a few questions but I want to make sure those are in the context of what you're doing. So maybe a second example might help my understanding a little bit more.

Speaker B: Yeah, uh, so in manufacturing we work with uh, a high end oven convection manufacturer who also provides maintenance and support. One of the first use cases that they want to achieve was auto generating a report. After the field service tech went into the field, evaluated a broken oven for example and identified the root cause and documented the whole diagnostic process before arriving at uh, the root cause and the resolution applied. And so this could be a multi day visit and the whole report had to be well defined and executed but you had a team of tech, you know, field service techs that did it very differently and the quality varied and that's important both for information capture by he tech, the actual um provider and making sure there's a level of quality always provided as that report is sent out to the customer as a part of the overall. Here's what we discovered, here's what was done. So we used AI to basically apply not just best practices in forming an entire report, the type of content that goes in but also the cognition that he tech uses to basically diagnose and support actually the field service itself. The Diagnostics and root cause resolution, pulling up the related documents through hybrid rag and then that is contextually bounded within the knowledge graph tree of thoughts and then basically auto generate a quality that is now shared across the entire service team.

Speaker A: Okay, so how is that to get deeper into that experience? So let's say the field technician goes out to the field and they capturing data. Are they capturing data onto your system directly or I mean when they go write these notes or capture what are they capturing and how are they capturing that information?

Speaker B: Yeah, they're capturing it within our system, directly within subatomic basically we built a ui, that's what they preferred. But it could have been headless where it could have still self guided them through the steps of uh, collecting information, diagnosing, pulling up the right information to identify root cause. But it was through a UI that stepped them through a workflow and depending upon the nature of the service.

Speaker A: So is it like an iPad app for instance or is it like a

Speaker B: uh yeah, we made it mobile ready from day one and they use tablets, you know, sometimes mobile phones, but often tablets as well. And so the intent was to be universally form factor ready and to do it their way, which not only was based upon that initial cognition that led them through standard uh, workflows, but depending upon the nature of the situation it would self guide them dynamically depending upon that category.

Speaker A: So they're capturing data into, let's say they go into your mobile app on iPhone or the iPad, they're entering data and details, maybe they're taking photos, maybe they're capturing some videos, they stick it and that essentially since you're running it in the customer's infrastructure, that data is going into their data stores. Whether it's postgres, whether it's Data Lake, whether it's Snowflake, doesn't matter. That data lives in their own world essentially. Now once the data is submitted, what, what are some next things that happen there in the workflow?

Speaker B: So once all the information is collected, including you know all the, I mean I'm glad you alluded to the multimodal aspect of this. Pictures, videos and descriptive uh, text for again problem diagnosis readings, uh root cause analysis, resolution applied on a, in a chronological way, the RAI then auto generates a uh report that is based upon their preferred practices. And uh, there were a few of their technicians that guided exactly what should be uh, adopted company wide that helped drive that AI synthesis. Again we use agentic AI that can determine even at a uh, step by step level of what needs to be done with the Very specific guidelines and guardrails in place and then the final assembly of everything into a format of standard report is done as a final step within that chain, uh, chains event.

Speaker A: Makes sense. I'm going to ask you a rather silly question. I want to ask this because I've also been asked this many a times, many people. I don't necessarily think it's silly but, but it might not be the most technical question someone's asked you. So here I go. Uh, let's say I. We get, we get a sense, you know, thank you for sharing a lot of those details and we get 20 minutes in, have some sense for what you, your product offering is. So hopefully this question is not completely off the mark. So let's say we collected this data, we entered into the app that was provided by your company and then it stores this data and it's got rich data because the more fields go into those, more agents go into those physical human agents go into the fields and get more data. You have richer and richer data so you're able to make better decisions. Now the silly question I want to ask Carl is a lot of times we have these tools like Claude for instance that have, you know, you can pick the data model or uh, the LLM for instance that you can work with. Right. Some more expensive, some not so much. So there's a variety of plans that they do provide. What is the fundamental difference between that agent, right, going to the field, capturing the data. Sure. They don't have an app, so they're going to have to capture it in text format for instance. And let's say they stick it into something like cloud and asking it to tell that person what needs to happen, be it diagnosis, be it something else, be it what next steps are, be it create a visual representation of this or be create a workflow. It does a lot of things as we all know. So the reason I'm asking this question is not specifically to your product because I haven't seen it, but in general because I do see a lot of the product offering come into play these days, which are sort of wrappers if you will, on top of these existing models that exist. So there is this inconvenience of going into say ChatGPT or Cloud, but if you are willing to do that, you actually did not have to use these other products, for instance. So just that's the context of the question. If you could just answer that space either specifically from what you offer or even generically, it doesn't have to be yours to how you see this, this being a problem or a non issue?

Speaker B: Yeah, no, it's not a silly question. A lot of people get this confused, you know, when they don't really understand how if they don't just go directly into Claude and develop a prompt, you know, that is even rigorously defined, you know, what is the big difference? There's, I mean this is where the agentic harness becomes critically important. A lot of the responses, even if you provide a lot of instruction, you know, basically gets uh, you know, much lower accuracy levels. I mean if you ever look at the benchmarks, you'll notice that even the best ones aren't like 80% or 90% and above. And so having a good agentic harness which can break down your overall structures across a set of steps and actually reference knowledge or cognition on the fly, which may be too complicated to type in within a prompt. Uh, the difference is if you have a complex challenge and we at subatomic focus on the complex, the multi step workflows that cross systems, that cross organizational departments and the teams within and we help bring, you know, eliminate all the friction that passes between information, uh, gathering uh, across the teams and distributing it across the right people, it becomes automatic with the cognition that span that entire workflow. That's something that's not easy to build in, into a structured step by step and then within a step dive into uh, branches of potential logic depending upon what the situation is. So that agentic harness is then further fortified with observability and security. I mean at least from subatomic, these are things that are first class citizens within the harness that is part of, you know, the overall execution to ensure top performance and the observability into the same that it used your workflow steps, it used your reasoning, it came up with interim answers along to a final answer. And it actually is done for both the clients and for subatomic. And when I say subatomic, it's both for the deep lens AI coworker that we call it deep lens that actually evaluates, it performs all that auditing and itself corrects in the initial execution. And then you as a human being can actually also provide your feedback. And it course corrects both the cognitive reasoning in the knowledge graph and anything related to RAG retrieval processes. It's constantly optimizing. So then at the aggregate level, because that was just at the single request level or workflow execution level, we identify those patterns where there could be optimization that uh, is across any of the things like accuracy, performance, faithfulness, uh, consistency, standardization, costs. We use multiple Models, you asked that earlier and my apologies for never answering that. Whether it's the commercial ones like OpenAI and they have multiple ones that are big and small even for inference, let alone separate ones for embedding. Same thing with cloud. But we also use open source models where applicable. We define and uh, actually at runtime determine the appropriate model for purpose fit, uh, assignment given the task that's needed and also to ensure that we're not overspending or sending a small focus task to a big uh, model unnecessarily where it may not be as good as a small model that has been trained to very specifically address that singular task. So there's a lot there. Uh, sorry, I'll just uh, wrap up this statement by saying sending it into one model is so self constraining, not just for the results of uh, the responses that come back, but the way you want to manage costs and make sure it's done in a standard way every time across your organization.

Speaker A: Fair enough. You know, the next thing I want to ask you is, you know, there's many ways software engineering as I see it as somebody much like yourself, been doing this for multiple decades, changed. But I want to ask you about your thoughts and here's my reasonably specific question, say about, I don't know, a decade ago, right? You're building software, uh, you approach the problem, you understood what your team or the client, if you're building, if you're a managed service company, what they wanted to do, what their budget was, uh, what skill sets they add in the team. If not, you brought your own resources, sort of had a hybrid mix of engineers and other folks who played different roles in the team and you know, went through a very traditional software development life cycle that I don't think had changed for a fairly long period of time. I mean the technology has changed, the language has changed, the frameworks changed, but the core essence of how do you go about building software at least, you know, uh, I'd not seen it fundamentally change up until not too long ago, right. Maybe the last two years, three years or a year ago. I know it's changed quite a bit now, but that is less defined because I see differently adapted by different companies based on the size of the company, based on the type of teams that they have, the geographical locations that they possibly happen to be in. So I feel, my opinion is that we are in a state of flux to some extent. Uh, I mean Bay Area is probably a slightly different world, but if you just get out of the Bay Area and you look at the rest of the world, I see a fair bit of inconsistencies even when we talk about software development life cycle. So I have a few things to ask you there. But at the offset, are you seeing that like in other words, for your team, for instance, has the SDLC changed from how it may have been maybe like two, three years ago? Or do you think that core tenets of it have not stayed, uh, have not changed much?

Speaker B: The core tenets have not changed much. You know, very specifically, when we hire for engineers within our team, we make sure they have the software engineering practices and disciplines they need to know design patterns, data, uh, engineering patterns at its core. And then you can know better how to introduce AI into the overall flow. You know, the practices are still the same, the optimizations you're trying to achieve through these patterns that are tried and true remain the same. But now you're just better understanding AI as a component within the overall end to end, uh, engineered flow.

Speaker A: You know Carl, I'm going to challenge you on that simply because I want to one, play the devil's advocate and two, I've actually had people tell me quite the opposite many times. We live in a world where uh, the separation between roles is becoming very thin. Right. It's grayer. Meaning, uh, I've had people on the podcast who've actually launched their products, not just gone to prototyping or proof of concept, but they've actually gone live or raised money and have paying customers and they never wrote a line of code themselves, right? They've never written as non technical founders, they never ever wrote code in their careers and they only, they don't even have a technical co founder. And for instance a lot of people tell me that they don't have actual developers in their teams and they've wipe coded their way all the way through to production. Now I don't know what that means because I don't come from that world, but I can tell you based on, you know, what I'm hearing, reading and speaking to people. But that is happening. So my question to you would be, why do we need to know the fundamentals of computing? Why do you need a computer science graduate these days? Because if somebody is a good problem solver who knows how to take a problem, it doesn't matter what the problem is. Could be an engineering problem, it could be a food problem, it could be an insurance problem, if they're able to think, if they have the cognitive ability to dissect a problem in an industry that they might be unaware of. Using a variety of these tools. Why do we even need for them to have that experience in that space?

Speaker B: Well that certainly is an ideal Krish. Um, ultimately there are plenty of Vibe, uh, coding success stories, but in reality very few of them are, can actually hold off the rigor of true production scenarios. They don't often catch all the different possibilities that reflect the true company's situation. And they're not often built with security and observability in mind. In fact, the Vibe coded examples are quite often the most fragile out there. Now the reason why I believe now you need the software development discipline is because while we are an AI co worker engineering team mostly, again I said at the Beginning, we're only 13 human beings. We have over 100 AI co workers, predominantly in engineering as we have two different internal products. Nexus for data engineering, Nucleus for the workloads that run the top, they get the entire code. Those AI co workers that live in those solutions 80% of the way there and then it's fundamentally a way of getting to the finish line. Now in the engineering team, what that means is not just understanding the business problems that you're fulfilling and ensuring that you're delivering according to that, but it's also making sure that the uh, AI coworkers were built in a way that ensured those best practices in the discipline because otherwise they become fragile in the real world in production setting. So our individual contributors that were just core developers of the past become the managers of their AI co workers. They need to make sure that the code is, is uh, durable for all conditions and then that assurance in that time frame is short. So I do recommend you have engineering, you know, strong engineers that can ensure that if you Vibe code, you should at least have someone check over that code for you. So vive. Coding may eventually get to a point where the software discipline is already embedded in the, in the uh, autocoders. But you do need someone to ensure before you roll something out for a client, otherwise you put yourself at big risk.

Speaker A: Thank you for that. It is quite reassuring. I wanted to hear that because I agree with everything you've said there, but I just want to sort of assess both sides of it because I, I read something the other day where, you know, programming in the next maybe couple of years is going to become like assembly language programming, right? I had never done that in my career because I started doing my first job at, I think it was a C programming job before I eventually moved to Java and whatnot. But I'd never, not even in college. I had done assembly lang assembly, uh, level programming, for instance, in the next couple of years it's possible, I don't know, we'll have to wait and see. Maybe people never write code anymore. Right? Maybe people just assess, review, fix and adjust, uh, and tweak code that is being generated. But perhaps they're not going to write lines uh, of code themselves. Right. Till we come to a point that, where maybe code becomes transparent. I also read something, I don't remember not too long ago that said there could come a time, not code, there will come a time, just we don't know how long, it's six months out or six years out that you don't ever get to see the code that actually is functioning, you know, uh, behind the scenes, uh, not even as a programmer. Perhaps your job situation, you know, maybe you do something differently. So there, I mean you also called out for something there. You said 80% of the code, if I'm not wrong, is actually that is a slight change or a big change to the SDLC to a large extent because you're not now getting business requirements from a business team and writing the first line of code like the hello world and getting started from there before you create your draft PRs and whatnot. It sounds like, you know, a lot of the cases you have 80% of the code generated and then you start from there as an engineer. Now the question also becomes Carl, as to the 80% of the code that's being generated, that certainly does not have to be done by developers or architects. Uh, a business team can do it totally. Right. So what does it mean to the composition of engineering teams as you see them change, how are you seeing the shape of those teams be? I'm guessing it's not traditional developers and one tester, a part time product manager and DevOps engineer, a team structure that we've had for a long time probably is going away. What are some um, new structures that you're currently seeing in an engineering team?

Speaker B: Yeah, so I believe actually for the developers to also be the testers as an example. But I'm going to share a few different things that I want to specify as an overall response. But if we start there, I believe in the idea that again we flipped the ratio of humans to AI coworkers in the engineering space. Again even subatomic a year ago had many more humans than AI coworkers and now the ratio got flipped and therefore we went from 10%, uh, or 80% human code generation to 80% AI co worker generation. And when you have fewer humans they need to actually perform multiple roles because you're only going to be allocating a small set of your budget to human beings at that point. You're going to be spending eventually more on compute than human labor when it comes to engineering. But, but also I believe in the idea of test driven coding. And if you make sure that you embed that into the AI coworkers that perform that, you specify the test conditions in which, you know, upfront in which the code should be able to handle, then it's built the right way from day one. And holding engineers, the humans, the human engineers, as well as the AI coworkers that execute these guidelines and guardrails accountable is critically important. When you separate testing into a different person as a sole role, you actually remove accountability and responsibility from the actual coder. And I don't believe in that. But there's a lot more going on. Again, I think today you're right. Business coders, uh, business users can define the features in logic and the test conditions functionally. You know, that should be met. But you do need the technical conditions to also be met. And so the inputs that go into the AI, uh, coworkers out of discovery is both functional and technical in nature. And they both feed in to the first team in that AI coworker assembly line, which starts with uh, the planning and design aspects before the build begins by the next AI coworker team and then eventually testing on that test driven designed approach of engineering the code.

Speaker A: How are. You mentioned an interface, you mentioned react, you mentioned the agent, uh, actually going in and collecting data from when they go on site. How are these interfaces changing? Uh, one of our products is the project management software Snowpalm. And people have asked me, Krish, how would the design look like if you were to have designed it today, would it be different? And how different? And my answer was like I didn't even have to think for a second before I said it would be 180 degrees different. Right. We built it like a four, four and a half years ago, but the world was a different place. We would do it quite differently today. So are you seeing like in your example of that app, is it a traditional type of interface, Carl, where people go in, there's dropdowns, all kinds of objects, UI objects where they interact with, or is it conversational where they just have a prompt? Maybe they write what they found, what they've not found, what they're thinking and then it actually auto populates into fields that's transparent to them. They are quite radically different approaches I want to know where you've seen people lean towards these days.

Speaker B: Yeah, you know what's interesting is we began with a headless chat interface only and a lot of people love that, including our flagship client in wealth management that actually became a lead investor and then we had ran into clients that in addition to that they wanted a dashboard for whatever reason it brings comfort because of the familiarity. Now where we differ on this is we don't just provide a hybrid approach where you get the, the true headless experience of chat, which can actually be not just directly accessed through subatomic but through your chat. Typical chat interfaces like Slack Teams, you know, Google Chat, whatever, you know, text, email, you can access, access what we call the subatomic chief of staff that knows all the AI co worker teams that exist for your different use cases, anywhere you like to do work, which improves the adoption rate. But yet when people want to see visualizations of things, we wanted to not just deliver what they felt comfortable knowing. You give me all this, you know for sure out of the box, you know, you build it for me like this. But we wanted it to be dynamic, that was important and so you can ask any question in a chat, it will dynamically give you a breakdown of everything you need sexually and auto generate a visual dashboard for key metrics that support the answer that you got. So that is a key difference and again always with a uh, cross cognitive understanding across your different services. For example that uh, is a complex relationship that needs to be carefully built but that is what we bring to surface. Again, wherever you like to work.

Speaker A: You know I've worked with a number of wonderful UI UX engineers in my career, right? Some really good smart people who design, design these interfaces like I could never imagine doing. But I uh, even if the, you know the outcome of end result of what they did was remarkable, sure. But the approach they all took to getting to the destination was not necessarily very different. Meaning they followed good practices, the best practices in their own industries as to, hey, how do I take the problem? How do I understand the clientele, demographic and whatnot and the personalities and the styles of the end users, humans and let me build something that makes the most sense for that team or that set of people. Now it's changed because the consumers are not just human. There's more and more agents going to exist. So there's this conversation around, okay, who are you building your sites for? And you know Google's conference. I think one of the things they mentioned yesterday was how their search is changing after like I Think quarter of a century and how maybe the way they're going to index is probably going to change. And there's talks about SEO being dead or not being dead, much like SaaS being dead and not being dead and so on. But the thing I want to sort of understand is how do you think UI and UX should be approached given the fact that it's not going to be built just for humans, that you're going to be having to design them because you know, these websites. I was talking to a podcast guest the other day and he mentioned these markdowns that you need to have for each of your website pages because agents are not going to look at noisy HTML ah, pages. They would rather look at markdowns like dot MD files so they can actually make a judgment really quickly when a question is being asked. So how do you see user experience and user interface change and adapt to a world that's a combination of both humans and agents?

Speaker B: Yeah, well I'm going to first finish out a clear understanding of the human part and then extend out to the human and agents both as potential consumers. Again, I'm going to echo intentionally. We believe that whether it's an agent or a human, you dynamically need to actually adjust to the specific audience member, whether it's business or technical, you know, and uh, build it specific to the request itself. And so we do believe in the idea of dynamic display and independent of how you initiate it and again from where you may have initiated it from different communication channels of human slack teams like we talked about, you know, it will actually reliably give you a dynamic response contextually relevant to what you just asked. It doesn't need to be static anymore, that's key. But then when we extend to agents, I think you hit on it Krish. I mean the markdown representation is highly consumable for agents as well. I mean I think, you know, early on information was being passed using JSON, but ultimately the models didn't necessarily be able to infer and properly synthesize with JSON, the outputs, ah, coming back were unreliable as well. Large language models do much better with markdown as the format. And so when you send past things across agents, whether it's an internal agentic scenario or you're working across maybe a supply chain and you're doing agent to agent, uh, communications, you need to actually structure the information in the best way that the, the audience member can consume. But again it should work dynamically across a series of functions that the agents, just like a human being would have Access to. So there is still a nature of whether it's human or agent to do role, to do RBAC based security and have a registry of tools or functions that are accessible for that particular caller.

Speaker A: I talked, I asked you about team structure earlier, right, Carl, Just touching a little bit upon that one more time. Uh, you know, as we have this conversation, Meta started announcing the layoffs. I don't know which layoff is this for the year for them, but it's end number, right? They're letting off 8,000 people and then you hear this all the time. Amazon, I think AWS announced that not too long ago. So we are seeing this happen in real time. Right. And you're in the Bay Area, so I don't know how the mood is different there, uh, because you know, typically all of these end up coming from there, right. A lot of the direction happens to be coming off of that part of the world. So, uh, and I've talked to people who've said they had 50 people in the team, they have like eight people now. Two sides to the argument. One is I can have the same number of people in the team, get more stuff done. That makes sense. But maybe if that's the case, AWS and Meta don't have dearth of stuff to do. If anybody has money and a lot of things to do, it's those companies I have to reckon, right? I have to believe. So how do you see this pan out, do you think? I mean, what do you think is the genuine number of people you need in terms of percentages compared to say three years ago, if you had a team of hundred people, how many do you need today to get that same stuff done?

Speaker B: That's a hard question to answer, Chris, because it's always a function of if you over hire to begin with, right? Everything is really an outcome. I mean, it should be a function of how much work you have to get done today. And so even before considering the impact that AI has on how many human beings should be doing it, you need to understand how many individuals need to perform. Now this conversation ladders back to the idea of higher AI, not software, where AI is now replacing human labor. There's no doubt that if for today you had the right number of human beings before the introduction of AI, replacing, yeah, you may be reducing the uh, number of human beings in your group to a fraction of what it was for us in the engineering space. We actually probably cut at least 3/5 of our human need of what we needed to do by now. Um, we never hired beyond what we knew we were going to need when we used AI and adopted into Nexus and Nucleus for other departments. It depends. I mean we know coding is, you know, engineering is one of the hardest hit ones, the managers across different teams. Because now AI can actually bring together unified understanding through data and cognition. You don't need to actually run things up. Different managerial levels, which has always been interestingly analogous to the system level friction of passing information that isn't unified. It was always a point of friction that impacted both accuracy and timeliness of getting the information to the right people. Now those are getting. The friction across those levels are getting eliminated. So I think managerial levels, especially mid manager, is hit probably just as hard as coders, which means that maybe 80% of mid managers like director levels, maybe even senior manager levels, may be removed for eight out of every 10. If not today, maybe as soon as 2028.

Speaker A: And at a personal level or uh, maybe even at a professional level. Carl, what are some tools that you had used in the past maybe three years ago? Let's take three years because it's a good window of time that you don't seem to have a need for anymore because of maybe Claude or ChatGPT or Gemini or whatever it is that's helping you do stuff that you somehow decided that, you know what, these are tools that I used to use. I have some like that. I'm just curious, are there some tools that you would have used quite consistently three years ago that you don't actually ever find value for today?

Speaker B: Well, it's funny because I'm going to use an example which is almost counterintuitive to someone who's performed data engineering for a good part of his career. The whole spreadsheets usage is just never. Decades ago we were, you know, there was, uh, there are future predictions of the demise of the spreadsheet and people will use databases and interact through a simple unit user interface. And now maybe, uh, actually preparing spreadsheets is one of the key things that if you work directly with a model like Cloud OpenAI, uh, or you know, even Google Gemini, you can have them auto, you know, infer an existing spreadsheet and then augment or evolve that spreadsheet for you. So presentations, all the typical office productivity software, which is really singular, task focused, you can just use models and gain productivity and eliminate your need to use those, uh, or at least update these types of information. Docs, Excel, presentations, you name it. But um, for complex workflows, I'm finding it at the overall system level, those individual SaaS players across CRM, ERP, uh, even Domain specific applications become less necessary because you can actually through an AI solution again subatomic, we offer it accessible in many different ways. You don't have to go into those systems. At minimum you're reducing your licenses and therefore the number of users because that's always typically usage based price or sorry, user seat licensing. Right. You can reduce your costs, you can potentially even eliminate a system. So for example, we at subatomic evaluate CRM systems. We decided why do we need it? We have AI co workers who can build one for ourselves and we do. We're an AI first company so we actually utilize very few external SaaS systems that are domain specific or function specific and just build it ourselves. So that's my experience and I'm sure many other companies are going to have that same experience too. You referred to the SAS apocalypse earlier and whether or not that will come to fruition, we believe it will and we see already evidence for our personal use and other companies that are becoming very interested in the same.

Speaker A: So Carl, as we get to the tail end, uh, I have one or two questions and I'll let you ask me anything if you have something to ask me and then I'll have a final question for you. So one of the next things we are doing as a company is building a fintech API. We are an API store. We publish APIs, help other companies build software quicker. And in the context of that fintech API, some of us spend a fair bit of time in the stock market. So this is not financial advice, just so I tell our audience, or just for educational purposes. But I'm curious if you had $100, somebody gave you $100 for free and they gave you a list of companies that you could actually invest in, meaning buy a stock. Right? Uh, I'm going to give you a few companies and you can only pick one. No diversification, right? Just you pick one off these, whatever comes to my mind right now. Um, I want to see which one it might be. Right? So here I go. I'm giving you $100 and you're going to have to pick one. And I'm going to run through a list. Bear with me until I finish that list. Micron Workday. Salesforce. Um, Intuit. Atlassian Team. The ticker symbol. Um, Nvidia. Microsoft. I can think of some more. Maybe I'll add one more to the list. Shake Shack for fun. These are the list of companies. You've got a hundred bucks, you can pick one. Which of these you might say none of these. But uh, if you were to choose one of these, which one might it be?

Speaker B: If I had to choose one, I would either bet on those who are, you know, selling AI at Strong Clip. So Nvidia. I'll be Nvidia and Microsoft's involved but I'll be honest, I don't, I wouldn't know who I would pick because I haven't done an evaluation of their current stock price. I don't know how much of forward looking earnings and revenue is already baked into the price. But the companies that I believe will be around and surviving. I'll answer it more in that context without knowing the right price to pay with my 100 Shake Shack

Speaker A: because. Is it because you think there's got to be. People have to eat no matter what happens. There's need, there's a need for humans to consume food.

Speaker B: It's a sure thing. And I believe there'll be through robotics AI that you know, makes the operational aspects of you know, top tier fast foods accessible. Now I'm assuming Shake Shack is a top tier still. I know it's a very, it's a well known brand but honestly I don't mean to say that instead of the key AI companies by the way, I, I'm pretty sure Nvidia and Microsoft will definitely be around just as as well. But I'm m assuming they are way overpriced without knowing for sure.

Speaker A: Fair enough. The last question for you and then you can ask me anything you want to ask me is let's say you're going out for dinner. What would be your order of appetizer, entree and dessert? Be it all does not have to be part of the same cuisine. Just one dish from each of these three categories.

Speaker B: Wow. All uh. Right. So for a starter and I'm not always a starter eater by the way, I'm usually a single course entree person. But if I did the three courses, probably octopus, you know, some sort of grilled octopus would be my starter. It could be a really good arugula salad with a light dressing, but something like that. The entree would probably be a sea bass, you know, maybe again grilled with some sort of Greek, Italian, you know, or other Mediterranean type of uh, preparation. And dessert? Yeah, I think it would have to be a delicious um, molten lava chocolate cake.

Speaker A: That's good. I have a feeling there are not enough Indian people in the Bay Area because you didn't pick any Indian dishes there. No, I'm just kidding.

Speaker B: There's a lot of Fantastic Indian cuisine, by the way. I, uh, you know, I love all the curries, so.

Speaker A: No, no worries. I'm just messing with you. Thank you. Do you have anything to ask me? I know I asked you a bunch. Is there anything you want to ask me? Yeah.

Speaker B: What would you choose for the three course meal?

Speaker A: Yeah, you could ask me anything else as well, but I'm happy to answer that one because nobody's asked me that before. I, uh, come from the southern part of India. I've come, uh, live most of my life in the US now, but I, uh, my formative years were in Chennai. Born in Chennai. Um, so my favorite food is basically South Indian food. Um, I don't know if you've seen it. It's called rice cakes. It's those little white rice cakes that you have with different chutneys. Um, the chutney is the selling point, essentially, that can be had for breakfast, lunch, or dinner, uh, as an appetizer or an entree. Just how many of those you end up eating? And then there's a different variation to that similar batter, but it's a crepe called dosay, where, you know, you might have seen it's a, it's like a pancake, but it's flat and it's much bigger. I think they make a billion of those every day in the world. And it's, it's, it's like French fries or pizza, uh, extremely popular. So those would be my choices any day. I could eat them all day long. Except, uh, it would have to be made either in India or in Singapore, or if I don't go that far, at least in Ukraine. The Indian food in the U.S. in my opinion, is horrible. So I can't pick any here. Uh, but, but it's, but that, that'll be. And then for dessert, there's, uh, uh, something. It's like a round, donutty things called gulab jamun. So I would go with that.

Speaker B: Yeah. Sounds like I need to make a trip to get real Indian food.

Speaker A: Yeah. At least to Singapore. If you don't go all the way or you go that far, might as well go to India. So Singapore's got some phenomenal South Indian food. I think almost the best outside of India. I think the best South Indian food you could have is in Singapore.

Speaker B: Fantastic. I'll ask you one more question, and you can maybe you could be as selective or as comprehensive as you want in the response for all the topics that we talked about. Which ones do you have an interesting different perspective on?

Speaker A: Oh, That's a great question. Okay. Do you have another three minutes or so for me to answer that?

Speaker B: Yeah.

Speaker A: Yeah, happy to say so. I don't know if it's a different perspective, and I'm not even a question of agreement or disagreement, but I'll just tell you from what I've learned by having hundreds of these conversations with other distinguished guests such as yourself, from different parts of the world. Right. Mostly the US and some in Europe, but. But I've also talked to people, not as many at this point, but in South Asia and in Australia as well. Um, so the answers sometimes tend to vary, again, by geographical locations. There's similarities when talk to folks in the U.S. but again, between the west coast, particularly the Bay Area and the rest of the country, I mean, I can sense some differences, some remarkable differences as well. Right. So here is what I would say. Uh, I have my biases because I grew up as an engineer. My whole career has been doing engineering work. So when somebody says, do we need developers anymore? Part of me is in denial. It's just natural human instinct. Uh, but part of me, as somebody who's running a company, I'm not in denial. And I know that as we look to build new software, Carl, uh, the whole approach to building it has changed, more so than I've ever known in my life. And, and I've never. I've only been building products in my career. So I've mostly, uh, been at the cutting edge of technology for the most part. I used to work for Oracle in my first job. Mostly 90% of my work is product work, per se, so. And I've always picked the right set of tools and technologies. Right. Even if I'm saying so myself. So it's not like I felt like things were moving slowly, but things have started moving a lot quicker now. And things have started changing because the number of people I talked to who've never written code, who are actually in the business of building software, is an incredible number. And it's kind of amazing too, because software has certainly been democratized. It's not, you don't need to know algorithms, you don't need to know programming, you don't need to understand N tier architectures. Right. To get into this business of engineering. Which is kind of cool because I didn't think it was possible, but it is possible. But at the same time, I don't know where that line is between, hey, I've gone live without really understanding what code is being run. Cause AWS said they had outages like a month or two ago. And they attributed some of those outages, if not all of them, to it being AI generated, if I'm not wrong. And they required that some senior engineer or somebody in the team has to approve of what is being generated. But as Claude becomes smarter and smarter, and I love the Cloud co founder, you know he speaks beautifully, right? He speaks to both sides. You have to believe that these tools are improving dramatically, meaning the skills that a lot of us believe we have as engineers. I strongly feel like it's going to become a commodity in a year or two from now. I strongly believe that even as somebody who's in that business, which means what you end up wanting to build has to be different from what you might have wanted to build in the past. How you want to approach the building of that software has to be quite different too. And how you differentiate yourself from other competitors, what your actual motives is going to have to be redefined. Because writing a lot of good code is not a moat anymore. I mean, having good architecture is not a moat anymore because a lot of tools can do it for cheaper and much quicker. Um, and I've myself not seen me use certain tools that I'd used in the past. Like Canva is a tool that I'd use quite a bit. I like the company like the founder for all the struggles she had as an Australian. So the whole story was beautiful and I like the product as well. But I can tell you, in the last 12 to 16 months, I, uh, found I'm paying for some. We are paying for subscription, but I don't even know how many times we've actually gone to Canva because we don't seem to find the need there. Similarly, there are a lot of tools that we had used in the past which we never seem to want anymore. And there's a new set of tools we end up needing to go to. So I think there's a fundamental shift in, in the world. Right. It's like an inflection point, either whether you agree or disagree. The reason I asked you about those companies is Micron is one of those companies that's gone up like a thousand percent in the last eight months. Right. And I've seen that founder a couple of times in D.C. actually, Sanjay Mehrotra. It's amazing. And people are saying it's going to probably even go up even more so because the need for memory is increasing. From what Jensen Huang had mentioned, he was at a chat, I think, with the. Was the ServiceNow founder or somebody else who's talked with him? No, the Dell. Michael Dell and him were in a chat like a couple of days back and he mentioned about memory. So there's a. There's one part of the world that believes in AI going to be changing things fundamentally, where you can put every hundred dollars you have and borrow a hundred dollars and put that into AI. And I also talk to people who believe in this, but are rather skeptical. They're worried about these crazy valuations. Some people say the forward P does not matter. You know, it's growing so much. It's like Palantir or, I don't know, SanDisk has gone up 3,000% in a year. You don't care. Borrow your money, put it in these companies. You're going to retire soon. I think each part of the world is split, as I see the world is split there and each camp feels very strongly, right. For every Michael Burry who says I'm sharding Palentia or Nvidia, there are tons of people buying that as well. And, you know, sometimes you're wondering where you want to be. Uh, and that's why I talk to my guests and try to figure out how comfortable they are in the business they are in. Are they going to be changing careers and things of the nature. So not so much to agree or disagree, but here are the bunch of learnings I essentially have had and people telling me that, yep, Karish have gone live. I don't even have a developer in the team, My product's in production and my first reaction call was, OMG M. How is that, uh, is that even safe? What happens? You find a bug in production, how are you going to find somebody fix it? How are you going to be answerable to your customers? Because when GitHub can have a hack or somebody hacked GitHub and it's scary, right? You probably saw the news where, I don't know, it's, you know, they were compromised. So you talked about security, you talked about governance. A lot of those things are important, but just the list of those things that are important. I feel like it's going to change. We're going to worry about tokens, token management, how much money you're spending on engineers versus tokens. Maybe you want to spend more money on tokens versus developers. So those are the things that I think are beginning to change and the way we solve problems is changing. So I think we're all going to exist, hopefully in this space. But who ends up being successful and who ends up being needing to look for a different career? Purely Depends on some huge gambles and risks that I think each of us are going to have to take. I think being playing it safe, in my opinion, Carl is, is going to be the riskiest thing to do. I think that's just my two cents.

Speaker B: Yeah, actually fundamentally you and I are 100 seeing it the same way. I uh. But your reaction to someone putting in a production and you know, being the accountable one if something is broken to their clients is the view I possess for today. It may be different in a year or two. I think it'll take time for that. Fine. That final 20% to be already built in for best practices of software architecture and design. You know, right now it's not, it's not um, versatile enough to handle different situations and map it to the right, you know, selections of architecting and, and uh, patterns that need to be present within the overall, you know, flow of information. So. But we'll get there. I have no doubt it's going to happen, you know, eventually and it's sooner than later. Meaning within five years I uh, could put a, a high confidence level on. Right. And that could be a small handful of that years. But right now you do still, I really strongly believe you need someone to check the code and make sure you can be accountable. Especially because I mean you're in Fintech. You know, we're focused on fintech as well as other high compliance, you know, high accuracy required outcomes and consistency durability for all the scenarios. You have to be ready. You have to be accountable. Your data processor for them quite often so for maybe some of the ones that variability and accuracy can survive imperfection. Yeah, it could be easier to buy code your way and handle down times because your system's not mission critical. Maybe it's a marketing application, you know, but yeah, we are fundamentally aligned. Actually it's gonna have, it's the change rate is actually going parabolic and it's only a matter of time.

Speaker A: Well, thank you for that. I'm gonna let you have the closing comment, but uh, and also please stay on on the call for a minute after we are done here. Uh, again folks, thanks to Carl Simon. Carl's the CTO of Subatomic. We'll include the link to, you know, Carl's LinkedIn profile and to get subatomic AI the company homepage so you can definitely check that out. That said, Carl, take it away. Your closing comment.

Speaker B: Yeah, no, thank you. For everyone who listened into the podcast, feel free to you know, DM M me through LinkedIn if you want to connect, talk shop, explore opportunities that maybe we could be extremely helpful for. And again, if it weren't, our URL is get subatomic AI. So there's a get in front. Uh, we're call to action oriented here.

Speaker A: Absolutely. Thank you very much, Kal. Much appreciate your time.

Speaker B: Likewise. Thank you, Krishna.