The B2B Podcast Index
Index
All categories
MarketingSalesSaaSFinanceHROpsLeadershipCustomer SuccessAI & DataProductStartups & FoundersRevOpsEngineering & DevTools
MethodologySubmit
Best of:MarketingSalesSaaSFinanceHROpsLeadershipCustomer SuccessAI & DataProductStartups & FoundersRevOpsEngineering & DevTools
An independent project byFame
SearchBest episodesGuestsInsightsMethodologySubmit a podcast
Index/SaaS/SaaS Scaling Secrets
SaaS Scaling Secrets artwork

Tackling AI's Blind Spots with Abhishek Jha, CEO of Elucidata

SaaS Scaling Secrets · 2026-02-24 · 36 min

0:00--:--

Key moments - from our scoring

Substance score

63 / 100

Five dimensions, 20 points each

Insight Density14 / 20
Originality12 / 20
Guest Caliber13 / 20
Specificity & Evidence11 / 20
Conversational Craft13 / 20

Elucidata builds AI solutions for biomedical data, but Jha challenges the SaaS label that venture capital prefers. The real value, he argues, comes from the last 15-20% of work - the domain-specific customization that requires solutions architects and forward-deployed engineers to collaborate directly with customers on context, data ontology, and processing specifications. This model mirrors companies like Palantir and Waymo rather than horizontal SaaS platforms. Jha spends significant time on out-of-distribution data - a critical blind spot where AI models encounter scenarios absent from training data. In healthcare, this matters profoundly: is a patient's non-response to a drug an outlier to ignore, or a signal for billion-dollar discovery? Jha distinguishes between detection and resolution of out-of-distribution events, advocating for data-centric AI that invests heavily in diverse, structured, well-integrated datasets across multiple modalities rather than relying on larger volumes of lower-quality data. He frames AI's confident hallucinations as a feature matching human behavior rather than a pure bug, while announcing Elucidata's new AI Labs to tackle these five-to-ten-year challenges.

Key takeaways

  • →SaaS is a misleading label for complex B2B solutions; the real value lies in the last 15-20% of work done by solutions architects and forward-deployed engineers working directly with customers on domain-specific customization.
  • →Out-of-distribution data - inputs absent from training data - poses fatal risks in high-stakes domains like healthcare and autonomous vehicles, yet most AI approaches prioritize data volume over quality and diversity.
  • →Detection of out-of-distribution events alone has significant ROI, enabling safer system behavior (like human takeover) without requiring full model resolution or generalization.
  • →Multi-modal data sources (like lidar plus camera in autonomous vehicles) help reinforce confidence in ground truth by cross-validating signals across different sensor types and reducing artifacts.
  • →Data-centric AI focusing on structured metadata, diversity, and federated learning across modalities outperforms traditional approaches that scale model size without understanding data quality or contextual relevance.

In this episode

  1. 1SaaS Label Limitations and Managed Services Approach
  2. 2Understanding Out-of-Distribution Data in AI Systems
  3. 3High-Stakes Consequences: Healthcare, Self-Driving Cars, and Real-World Examples
  4. 4Multi-Modal Approaches to Reinforcing AI Signals
  5. 5Detection vs. Resolution of Out-of-Distribution Events
  6. 6Why LLMs Hallucinate and Temperature Settings
  7. 7Biology and Drug Discovery as High-Stakes Use Cases
  8. 8Data-Centric AI and Building Reliable Systems

Mentioned

ElucidataAbhishek JhaDan BakowskiPalantirWaymoTeslaMonday.comAsanaChatGPTClaudeAlphaGoOpenAI

Guests

Abhishek Jha

Topics in this episode

Forward-deployed engineersData-centric AIManaged Services modelWaymo autonomous vehiclesOut-of-distribution data detectionElucidata AI LabsMulti-modal learningFederated learningBiomedical data integrationSolutions architects

Questions this episode answers

What does out-of-distribution data mean and why does it matter for AI systems?

Out-of-distribution data refers to inputs that fall outside the distribution of a model's training data. In low-stakes scenarios like Netflix search, the consequences are benign, but in high-stakes domains like healthcare or autonomous vehicles, encountering out-of-distribution events can be fatal - such as a Waymo encountering an emergency jet landing on a highway.

Why doesn't AI just say 'I don't know' instead of confidently giving wrong answers?

LLM hallucinations are partly by design; the models are incentivized to generate answers because deterministic uncertainty would be less useful than querying a database. Additionally, out-of-distribution detection is a separate, solvable problem from resolution - detecting uncertainty and triggering human takeover (as in Waymo) is valuable even without fully resolving the model.

How is Elucidata different from a typical SaaS company?

Elucidata operates as managed services with tools and technology accelerators rather than turnkey SaaS. The core value comes from the last 15-20% of work - forward-deployed engineers and solutions architects collaborating directly with customers on data ontology, security, cloud infrastructure, and domain expertise in drug discovery.

What role do multiple data modalities play in improving AI reliability?

Using multiple data modalities (like lidar and camera in autonomous vehicles) helps reinforce signals and build confidence in ground truth by cross-validating observations, reducing false signals from individual sensors and improving model generalization to out-of-distribution scenarios.

What is data-centric AI and how does it differ from traditional approaches?

Data-centric AI prioritizes diverse, structured, well-integrated datasets with rich metadata over simply scaling model size and training data volume. It focuses on data quality and contextual relevance, which Jha argues delivers better ROI and reduces performance drops when models encounter real-world out-of-distribution events.

What our scoring noted

Our reviewer’s read on each dimension, with quotes from the episode.

Insight Density

14 / 20

The episode contains several substantive technical concepts (out-of-distribution data, data-centric AI, federated learning, physics-based learning) explained with reasonable depth and practical examples. However, substantial portions are devoted to general business philosophy, company positioning, and soft advice that most B2B operators would already understand. The most densely insightful segment discusses OOD detection mechanisms and the eval framework gap, but these represent perhaps 30-40% of the conversation.

If you encounter an event that is out of distribution of your training data. The consequences can be profound and fatal in some cases.
The few aspects to that. One is, you know, the data should be structured ready for ai. Uh, the metadata context should be captured, right, and that. Allows your bottles to get more out of the same

Originality

12 / 20

The out-of-distribution problem framing is not novel in AI circles, though its application to drug discovery and the concrete healthcare examples provide some freshness. The data-centric AI approach (vs. model-centric) reflects existing academic consensus from researchers like Andrew Ng. The guest retreads standard SaaS-to-services positioning arguments and offers derivative wisdom about moving fast and listening to customers. Limited contrarian positioning or truly first-principles thinking emerges.

we talk about, you know, data centric ai, right? Where you invest more on the data side, making sure it's diverse enough, making sure it's structured enough
SaaS is a very convenient label that has been used. We have embraced it but when rubber meets the road

Guest Caliber

13 / 20

Abhishek is a real practitioner - founder and CEO of an operating company building AI in healthcare - which is materially stronger than pure thought leaders or consultants. However, he's a first-time founder admittedly learning on the job, not a battle-hardened operator with multiple exits or decades of domain expertise. He has credibility in biotech/life sciences AI specifically but lacks the gravitas of, say, a seasoned pharma executive or AI researcher with major publications. Solid practitioner, not exceptional pedigree.

I'm a first time founder, right? Uh, so a lot of my learnings have been on the job.
Trained as a scientist. Abhishek has spent the past decades scaling Elucidata and building best in class AI solutions on messy biomedical data.

Specificity & Evidence

11 / 20

The episode provides some concrete examples (Waymo jet engine, AlphaGo move 37, Netflix search, Waymo lidar vs. Tesla camera debate, MIT/Broad Institute hackathons, Nvidia/ARC virtual cell challenge) but lacks specificity on Elucidata's own metrics, financial performance, customer outcomes, or technical results. The company's internal playbook is described at a high level (data structuring, diversity, federated learning, physics-based learning) without concrete performance deltas, timelines, or dollar figures. No customer case studies with results are provided. Vague on impact of announced AI labs.

We are seeing that in all of those problems, right? Which are very AI heavy, a lot of data, different types of data, different quality data where we can execute on this playbook to show a very clear, performance improvement of the model.
One of them was organized by MIT in, uh, broad Institute, uh, earlier last year. One of them was most recently organized by. Broad Nvidia and ARC Institute

Conversational Craft

13 / 20

The host asks competent, reasonably well-researched follow-up questions (Waymo examples, multimodal sensor fusion, why LLMs confidently hallucinate, evals gap). However, the conversation lacks genuine pushback or productive disagreement. The host rarely challenges vague claims, doesn't press on business metrics or customer acquisition friction, and accepts high-level answers without drilling deeper. The guest is rarely put in a position to defend a position rigorously. Solid interview craft but insufficient rigor for a substantive B2B podcast.

I wanna just kind of, maybe to tie it to the help the audience explain I think I. Processed what you said and maybe want to put out some examples and you can tell me if this is what you relate to.
Is there a way to think about diversity of data that attacks the long tail directly?

Conversation analysis

Computed from the transcript - who did the talking, and the words that came up most.

Most-used words

data40problem28model23example21distribution20back19different15waymo15high14quality14move13valuable12saas11clear11first11question11

Episode notes

Dan Balcauski speaks with Abhishek Jha, Co-founder and CEO of Elucidata, an AI solutions company focused on drug discovery and biomedical data. Abhishek shares his journey scaling Elucidata, the challenges of fitting into the SaaS label, and the importance of listening to the market. They discuss the complexities of handling out-of-distribution data in AI, the importance of data quality and diversity, and the challenges of building reliable AI systems in healthcare and life sciences. Abhishek also highlights the need for clarity and first-principles approaches amid the AI hype, as well as the value of strong partnerships and rigorous evaluation frameworks. 01:18 Introducing Abhishek Jha and Elucidata 02:11 The SaaS Label and Market Realities 09:15 AI and Out of Distribution Data 20:27 The Universality of AI Challenges 22:08 Building Reliable AI Systems at Elucidata 23:18 The Importance of Data Diversity 25:46 Addressing the Long Tail Problem 28:16 The Role of Evaluation Frameworks 32:14 Core Values and Market Realities Guest Links elucidata.io Abhishek on LinkedIn Abhishek's Substack

Full transcript

36 min

Transcribed and scored by The B2B Podcast Index.

SaaS is a convenient label that we have embraced as a founder, I have learned, to prioritize what my market tells me, what my customers tell me, more than what, anyone else for that matter, right? If you encounter an event that is out of distribution of your training data. The consequences can be profound and fatal in some cases. If you have a patient that is not responding to a drug, is that an outlier that you shrug and move on?

Or is that a real biological signal to, discover and create a new billion dollar company? Listen to the market. Things are changing very rapidly. And I also say any lesson that we have had has a very short expiry date right now.

Things are changing so rapidly, go to market lessons, your tech stack. move fast with strong, clear moral authority. Otherwise there's a very clear threat that you'll be left behind. Welcome to SaaS Scaling Secrets, the podcast that brings you the inside stories from the leaders of the best scale up, B2B SaaS companies.

I'm your host, Dan Bakowski, founder of Product Tranquility Today I'm excited to welcome Abhishek Jha. Co-founder and CEO of Elucidata, an AI solutions company focused on accelerating drug discovery and improving patient outcomes. Trained as a scientist. Abhishek has spent the past decades scaling Elucidata and building best in class AI solutions on messy biomedical data.

He prioritizes clarity and real world impact over AI hype, especially when it comes to deploying models in high stake settings. Abhishek, welcome to the show. Hey Dan, thank you so much. Really looking forward to this conversation And.

Before we dive into your scaling journey, give us the elevator pitch. What does Elucidata do? Who do you serve? Wonderful.

Yeah. So what we do is, we essentially build very high quality AI solutions on real life, messy biomedical data of different modalities. And we serve scientists like me who are working really hard with a lot of, passion. And it's a challenging problem to bring.

High quality drugs, therapeutic options to patients who are desperately in need. One thing that out from our pre-show chat was you mentioned feeling almost cheated by the SaaS label. And I would I believe you would consider Elucidata a SaaS company. Could you unpack that for me and our audience?

Oh, yeah. How much time have you, you got? So, uh, I'm a first time founder, right? Uh, so a lot of my learnings have been on the job.

Uh, LinkedIn, uh, has been a good, or, debatable, but in important source. Nonetheless uh, a lot of, you know, fellow founders, investors I have a pretty good support system all around me. And SaaS is a very convenient label that has been used. We have embraced it but when rubber meets the road, I think, you know, uh, as a founder, I have learned, uh, to prioritize what my market tells me, what my customers tell me, uh, more than what, anyone else for that matter, right?

And that does not, is perceived as valuable as, uh, we would think would be right as a business model SaaS, right? And that's a lesson that, we have learned again on the job and just by following some very basic first principles approach of listening to your customers, most importantly, right? And that's number one. Number two is the times have, changed as well, right?

The landscape, the stack. Has changed quite dramatically thanks to AI and the way Chad GPD has captured the popular imagination and there are very few such singular moments in the tech world, right? A handful of them, if you go back in time, so com, a combination of both of those things has really forced us to go back to the table, think about our identity as to what is that we do that creates value for our customers. And just to wrap up my thoughts on that.

Like, I think the most honest label for us is managed services where we have a lot of tools technology accelerators, right? Which we can bring and make it, purpose-built solution for you. And that I think, is. Far more valuable.

You can call it SaaS in some context or not, but to me that's a distinction that we have really seen on the road, on the ground. It's definitely exciting times to be in technology, and you mentioned AI and I wanna talk about that in a minute, but just to unpack that a little bit more, what you said about LinkedIn. I saw a good tweet the other day, said, if I ever become a multimillionaire, I'm gonna buy LinkedIn and then turn it off. So I totally understand you're debatable, given what you saw, LinkedIn or Twitter or otherwise about what SaaS was.

You mentioned. Hey, maybe there's not as much focus on, listening to your customers, but what did you expect versus what you it turned out to be? I guess given that situation. Yes.

Yeah. So what I expected, and again LinkedIn, as much as we can be cheeky about it. It's a great source, but. The most credible source that I find is fellow founders who have been on this journey perhaps a few years ahead of me, that sounds to be the most credible source of real insights.

Uh, but coming back to your question, Dan, uh, I think, uh, my very naive expectation was that, you know, a very true classical product story, right? Where you have something off the shelf, DIY on day one, right? And what we learned on the job is that the value that our customers derive from. Is that last mile, you know, the journey of 80 to 95%, right?

Where you have to, uh, respect that context, that understanding, that specification that they have in their mind about what data sources are valuable, what ontology makes sense. Those how does the data need to be processed, right? And that universe is not in finite. So it's not like, you know, you're going back to the drawing board starting all over, scratch, but.

It is important, right? That is where the value is perceived and derived from for our customers, right? And to us that journey of going from 80 to 95%, uh, we have taken a very clear stance. Now we'll do it only with a customer, right?

We'll stop at 80. So it's more like pass than SaaS again we can get into that. So that to me, is it more honest label for the company? And I understand it's not the most success label for investors and whatnot.

But again, if I go back to the first principles this is what we do and this is what we learned on the job. Let me just unpack that a little bit Yeah. Yeah. you said a couple things in there, so you know, you thought maybe, hey, we'll put a SaaS out there, turnkey DIY, and that maybe gets you to 80%, but in your market space challenge, that doesn't work.

I know that. You're out in the Bay area normally, and one of the hot new phrases out there is a forward deployed engineer. Is this what you're hinting at of Hey, we wanna actually get our engineers on site Yep. maybe a little bit more sort of custom Yep, yep.

Yep. work to get them to that last mile. Yeah you got it. And that's what I think we lead with, right?

You call it forward deployed engineer. At some point we'll call it to we call them solutions architect, right? These are subject matter experts. And, uh, if you think about it, right, like, we don't have a very simple offering like for example, like, let's say Zoom, right?

Uh, we have a very complex offering and our customers are very nuanced very, uh, particular about the details, right? So they want someone who can talk to them about ai, about security, about cloud, about matics, about drug discovery. That's a pretty broad range, right? And that's what creates value for them.

So having subject matter experts who can almost, a customer representative, right, uh, on our side is very important of how we can create value. Okay. And that is what we are trying to talk about, right? And that does not fit very well with a very simplistic, naive notion for a complex stack that we have to offer.

Now, again, that's a very key variable, right? Like for some other founder who's building something else. This may or may not be relevant at all, right? Like the motivational model might just work fine.

From what I am hearing from, uh, other founders and on social media, right, that model is breaking down for other sectors as well. But my conversation, my comments, my learnings have been around our offering. Which is quite complex, larger, for businesses B2B a particular kind of B2B, which is, scientists and engineers at healthcare and life sciences companies, right? That makes perfect sense.

And I don't know. Yeah, I think maybe the most famous example right now is maybe like Palantir and they've Palantir or And they. c ct ai, right? Yeah.

and it seems like at least the market is not overly punishing, them if their valuation is any indication, Yeah. TBDI could be totally wrong by this time this podcast comes out, depending upon how the market No and I don't think that you're in more as rare a company as maybe you laid out. There's definitely, those turnkey solutions that are maybe more horizontal. Platforms, ala like project management software, the Monday.

com, the Asana As yep. world. But there's definitely a lot of folks in your shoes. So I think your perspective would be super valuable.

I you hinted at AI a couple of times. I wanna start our conversation. There. One of the things that you mentioned to me offline was the problem of dealing with out of distribution data Yep.

Yep. systems and especially how it pertains to your world of working in the bleeding edge of AI and biology. For listeners who haven't this term, break down what out of data distribution means and why it matters. Yeah, no, that's something that we have been talking a lot about off late, and you will hear us talk a lot more in days to come.

And this is again, a lesson from the field, right? We have been deploying AI solutions. For quite some time, very high quality solutions in high stake environments. And a consistent pattern that we have seen is one of the key tenets of any AI or MLO supervised learning model, is that the data that it has been trained on and the data that it'll be tested on in the wild is coming from the same distribution.

And, that is a key assumption and often in real life in the, while it breaks down. Right? So what are the consequences? So let's consider an example, right?

Uh, let's say, you know, you are on a nice evening you're class of wine and you're watching Netflix, and, uh, you have some documented in mind, which is not something that commonly people search for. It's on the third page. That's a good example of what. Your search query was out of distribution, right?

It's in the tail. It's not in the mean of the distribution, And the consequences are quite benign. You shrug, you complain, you move on, right? But if you're in a high stake environment like healthcare and life sciences, or let's say self-driving cars, right?

If you encounter an event that is out of distribution of your training data. The consequences can be quite profound and fatal in some cases. Right? Uh, let's say the car does not know when to give the control back to the human driver.

If there's a human driver on the seat, right, Waymo does not have it. Uh, just recognizing an event which is out of distribution from the data is very important, or in our industry, in our vertical, that we are focused on. If you have a patient that is not responding to a drug, is that an outlier that you shrug and move on? Or is that a real biological signal there that you want to, discover and create a new billion dollar company?

That is what I mean by out of distribution problem faced with. Now, if you go beyond the problem statement, what we are arguing is that the traditional AI approaches, which fly on a lot of data of, perhaps no, not so. Dub quality, maybe the quality is good or not, it's not quite well known, is not going to translate very well to solve these out distribution problems. That requires a rethinking a shift in how we approach this problem.

And that's where we talk about, you know, data centric ai, right? Where you invest more on the data side, making sure it's diverse enough, making sure it's structured enough, making sure it's integrated with other types of, signal to reinforce. Or the signal that you get from one modality with other and that combined with a good enough model gives you a better ROI in terms of model performance the model performance does not drop very significantly when you go out in the wild.

So that's, what we have been talking about. And I think that requires a shift from traditional AI approaches. And in an acknowledgement of that, we also did a press release yesterday to announce. Our AI labs at Luc Data, which is focused on this problem and committing to this problem, right?

It's not something that we will solve in a quarter or a year. It's more like a long term view of what is going to happen in the industry in the five to 10 year timeframe. I wanna just kind of, maybe to tie it to the help the audience explain I think I. Processed what you said and maybe want to put out some examples and you can tell me if this is what you relate to.

So as it pertains to seeing. Out of distribution situations. There was a story I'd read about Waymo where the Waymo car was driving and I guess there was an emergency landing by a jet plane onto the highway, and the Waymo had never seen a jet plane in its training data and just ran right into the jet. Is that a good example of what you're referring That, that's that, that's a good example where, those consequences are quite fatal.

That's a very good example. I'll give you another example where the AI model kind of surprise, which we still don't understand. So there's a very wonderful documentary on YouTube, uh, that I recommend to anyone who would listen to me. My co-founder suggests that to me.

We also had this documentary as a, uh, watch session at our offsite last year. Uh, on AlphaGo, right? This is the model that Google DeepMind published, and in one of the games when AlphaGo is playing, uh, this world champion of go, uh, it discovers a move. That move is move 47 or 37.

I, I forget, right? That move was by almost all the commentators in real time was thought of as a mistake, but that move was the turning point of, uh, AlphaGo winning the game again, right? And that's an example of a behavior where, you know, outta distribution. Activities, events do happen from the model that you're, it's not clear if it was not ever clear that if that move, move, move was part of the training and how do you deduce is not quite understood.

But your example is a very, you know, good example of what an auto distribution is like and, uh, on the, not so good side. And there's also example of auto distribution that models can perform in emergent behavior that we don't fully understand. How did that come about? To me it's a, problem at the bleeding edge of AI that we still don't understand very well, but it's very important in some high stakes setting like Waymo or healthcare life sciences.

Yeah. And you mentioned something else around, trying to use different modalities to assess what the ground truth is. There's another example I'd read about. There was a battle, at least in the public sphere between how Tesla and Waymo using Sensors where Tesla said, Hey, we're not gonna use lidar.

We're only gonna use yes. because, we're trying to manufacture it scale. And that adds extra complexity. And Elon's super focused on, on making sure the manufacturing process doesn't get overly complex.

Waymo, I believe they're. Argument is to the effect that you're saying, which is, Hey if we have these bolt, lidar, and camera and they show us two different things, that gives us a better sense of Is the, is this an artifact that is just Picked up by one of these sensors Or is this actually happening in reality Yeah. is that a good example of what you're referring to? That's a great example.

And if I translate that to, for my vertical, right? As a scientist, you always are circling with not enough data, right? That's always a perpetual question, but anytime you can use different modalities, different observations, right? That helps a lot, right?

To reinforce that signal, to build that confidence. And I'll go back to, a very classic fairy story, right? Where seven blind men are grouping different parts of the elephant and they have different realities. And if you unbind them, they see the whole picture, right?

That, uh, has its limitations. But I think that drives the point home, right? Like where multi modalities help. Yep.

I'm gonna ask a very naive question just because I, one, I'm curious about the answer and I just don't know, and you may, so is the blind spot that the AI just doesn't know it's out of its depth, guess, if I think about that jet engine Waymo example, it's like, Hey, I've never seen this before. Maybe I should just stop. I don't understand in that situation why that, like, it doesn't say, Hey, like I'm gonna go revert to a safe like situation. And this is just, and people run into this maybe more commonly with something like chat GBT or Claude, where you ask it a question and it confidently Yep.

Yep. and it doesn't tell you, I don't know Yeah. And so I think there's a big question, I have not being an expert in this space, why doesn't it just be like, I don't know, why Yep, just yep. does it just proceed?

What could you shed some light on? What's going on there Experience? Dan, that's a great question. And we, uh, internally in the AI lab, discuss and talk about such bleeding edge problems a lot.

So for one the outta decision problem is not a monolithic problem. There, you know, subclasses of this problem, And the first prob first class is what you're talking about is just detection of out of event out distribution events. You don't do anything, just detect it. And then it's much easier to figure out what to do, right?

Like for example, in Waymo's case, Hey, this is out of my depth, I'm being honest. Human takeover, right? That's a very pragmatic solution to a very complex problem, right? So just detection, not resolution, not generalization of the model, right?

There are different class of ion problems. Just detection is valuable enough, right? For example, uh, there are five different models that came out last week, right? And I have to compare them.

Now, if there was a metric which shows that, hey, how generalizable this model is for my data, I would save a lot of time. And, effort, right? So just detection of and out of distribution even, or data is valuable enough. Uh, point number one, it's not a monolithic problem.

There are many, classes of this problem, right? And. The, uh, second comment that I would have, like, there's a really interesting paper that came out of OpenAI team where there's, uh, you know, parameter about the temperature of different layers if that is set to zero, uh, or there's an incentive that you have set up to give some answer even if the answer is wrong. And if that is said to if you can tweak or play with it to.

How say it less or penalize the model to come up with any answer, not the correct answer. And I think there will be a lot of innovation in days and years to come. Um, hopefully, uh, we do some of that. A lot of the, you know, tech world is really pushing the boundaries on that.

But I think, uh, fundamentally the way you get the istic answer out of any LLM right, that's also by design, right? Otherwise, you'd, you're better off. Like, or you're back to the. Whole world of querying your databases, right?

That's a very deterministic way to get an answer. So I think it's also should be considered as a feature, not as a bug, right? That's how the, uh, solutions are designed. Uh, and the istic part is also the valuable part, right?

Uh, I mean, as humans, right? Like, you know, we do that all the time. If we had the same bar of accuracy, consistency every single time with every human. You'd have very few friends, right?

Like, not many of them would be hallucinate as well. So it's a it's a mirror in that sense, right? Yeah. I might frame that slightly differently 'cause I think the problem people have with LLM hallucinations is that they're confident about their answer.

Where I would, humans, no, a lot of if you go to this Delhi in India, right? The joke is that even if you don't know directions, the way you tell the directions is as if you know it really well. And very people would say that, I don't know where that place is. They would volunteer to, share very detailed directions.

Yeah. Yeah. Maybe there's some cultural implications there as well. As it pertains to this out distribution data problem what is it about biology specifically that makes this so visible?

Is it unique to your field or is this just that the stakes are high enough and you can't ignore it? I think first question, I don't think it's unique to this field, right? The, it's, it is common to a lot of field. Like I give you an example of Netflix, right?

So the problem is there, the stakes are not very high, right? And there are also a lot of high stakes environment, right? Like, self-driving cars is a good one, right? Where the stakes are high and this problem is, so the problem is everywhere in every any AI solution has this problem.

Period. There should be no confusion about that. In some verticals, in some industries it's worth solving for in some it's not. Right?

And healthcare and life science is definitely one, right? Like, uh, if you think about any valuable biotech company in Cambridge, right? All of them are based on some discovery that is, that did not make sense on day one. That did not match the pattern, right?

That was an outlier. And I, I talked about. One of the co-founders of my last company Lou Ley, very famous, very celebrated scientist, right? And he always said that, Hey, that is what intrigued me.

That is what gets me, you know, my juices flowing. That that's what excites me, right? That why did it not make sense? And is this noise that, someone did not do the experiment well or whatnot?

Once you rule that out, right, often you're sitting on a huge opportunity of pushing the boundaries of what is known to humans at large, right? And that's a, in some cases that transfers into a company that is valuable, great outcome for patients and stakeholders and such. So they're rare. They could be rare, but they're very valuable.

And it is definitely, you know, a high stakes environment because you know, at the end of the day, you have patient lives to worry about. So it is definitely worth solving, right? That's the push. But it by no means it's unique or specific to healthcare and life sciences.

So take me through what this means in practice for you and your teams in building Elucidata. What does this mean in terms of design? Like to build around guardrails. Is there a way to make AI know what it doesn't know?

You mentioned there's a distinction between detection and resolution. Take me through that process, help me help you know, a lay person help understand what the trade offs are like. I imagine there's, additional, expense to make them more reliable. Do you need to make, pull them back from claiming like ca what does that look like as you build these systems knowing this problem exists.

That's a great question. And again, that's a perhaps the answer will emerge in the next many months and years. What we are doing right now is the playbook, the recipe that we have come up with is to focus more on the data part. And what does that mean?

The few aspects to that. One is, you know, the data should be structured ready for ai. Uh, the metadata context should be captured, right, and that. Allows your bottles to get more out of the same, you know, architecture and same all things being equal, that helps, right?

That's number one. Uh, I think anything that you can do to, uh, increase the diversity of data, that helps a lot. Any understanding of use case where the model will be used, right? Uh, that helps a lot in choosing the most relevant data to train on.

So federated learning becomes part of your stack. If you want to increase the diversity and do that securely and respect the privacy concerns that, all the data stakeholders would have. There, there are a couple of other tools in your repertoire that we are deploying as well. Any way that you could induce some physics based learning, some Abhishek rules of.

The problem that you want to solve, right, uh, helps a lot, right? That really teaches the model how to go to the basics and really reduce from there, uh, and alpha fold. Uh, the model that was published from Google DeepMind, which also won the Noble Prize two years back, right? Uh, does that quite effectively where it teaches the model, uh, some very basic rules of how proteins fall into a structure.

And then it all, you know, it often goes back to that to deduce what a new sequence would. Get into that. Right. So there, there is a, uh, recipe, a playbook of sorts that we are, you know, building and we are deploying that in, uh, real life situations by participating in hackathons.

One of them was organized by MIT in, uh, broad Institute, uh, earlier last year. One of them was most recently organized by. Broad Nvidia and ARC Institute, uh, about a virtual cell challenge. And we are seeing that in all of those problems, right?

Which are very AI heavy, a lot of data, different types of data, different quality data where we can execute on this playbook to show a very clear, performance improvement of the model. And we don't tweak the model much at all, right? We compare them, but we keep the model architecture, the representation models the same. Unchanged.

Not to say that there is no room for implement there. There's a lot of room for implement there, but our thesis is that you get bigger bang of the bang for the buck by focusing on the data issues first. Okay. Does that add some color?

Yeah. oh, yeah, no. So you mentioned so clean data, diversity of data. Those make sense.

I guess the, with diversity of data where the challenge is, from my understanding is that, will go back just because, layman example, go back to Waymo and a jet engine, Yep. Yep. So it's like, okay, we could add a bunch of more Waymo driving around neighborhoods and different, okay, there's a pedestrian here, not a pedestrian here, right? Maybe put 'em in a different country, right?

But oh, there's a jet engine on the road. Is there, I guess because those problems, outer distribution is. Almost entirely defined by the long tail Yeah, yeah, yeah, exactly. is is there a way to think about diversity of data that attacks the long tail directly?

I, we, I live in Austin. We have Waymo way. I literally sometimes will see four Waymo's in a row driving Yes, in front of my house. yes.

like they're just doing loops Yeah. you've been down the street a bunch of times and I'm sure it's all going back to train data, but I'm like, I'm not really sure what new you're learning, but I guess this is more gig gigabytes or terabytes yeah. of data to Yeah. So I guess what you're thinking about diversity of data, does it, how, does it really help with that long tail or like how do you think about it?

Like, as it, especially maybe as it pertains to your area, like Yeah, no Dan, this is a great question and that's why I go back to, one of your earlier questions, right? That's, this is not specific to healthcare and life sciences alone, right? I think that's the bleeding edge of AI period, right? So the long tail problem is.

Uh, a problem. You know, it's, it is true in ads. It's true in for Waymo, it's true for us. Um, now in increasing the diversity of data, does it solve the problem completely?

No, it does not because you cannot possibly ever, you know, do this right? And if you did this, then perhaps you don't need a predictive problem to begin with. So there's always, uh, that balance of like, how much can need to prove? Will it completely solid?

No, right. And that's where, uh, you know, the fourth thing that I was talking about it's part of our recipe playbook as well, right? Building some very ab initial first principle physics-based rules into the model helps, right? Like, oh, there's a flying object that is coming my way.

Unusual. I've never seen this. Let me stop or revert to a plan B. So even a detection of such a model based on some very, even if you've not encountered that data, like that you have not encountered that, right?

Or does it is outside of that distribution, most of the models that are published today do not have any such mechanism, right? So just detection of od even which is one of the classes of OD problem is valuable. A lot of value is there, right? It's not an easy problem by any means, but.

You cannot possibly ever exhaust all the diverse data that you will ever encounter. If you did that, then you'd not need any predictive model to begin with. You can always go back and you pick it, that it, it always, it does make me. Always marvel, advanced these systems are how advanced the human mind is because, I'm sure most human drivers have never seen a jet Yeah, yeah, yeah, yeah.

Exactly. Exactly. of them would've ran Yeah. Exactly right.

And not to pick on Waymo, they've Yeah. Yeah. But it is quite an interesting thing. I know that one of the hot topics in the world of people building AI systems are evals Yes.

they're looking at, Hey, these are we're monitoring what our systems are Yep. Like a customer chat bot Yep. Hey, what were. What were the conversations?

Did the AI chatbot, stop the conversation when it should have handed off to a human? Did it issue a refund when it Yep. Is this something that you has value in, in, in your world, in your systems? How has that affected, how you guys look at maybe rounding out this long tail of out distribution problems?

Dan again a very sharp question. I think we, uh, believe, uh. Like, you know, the bigger models, uh, often come with this kind of news or someone has worked to show that it can pass MCATs or LSATs, right? Like there's some, or even for, um, natural language, there's some tasks benchmark to benchmark models.

But if you think about, uh, more vertically focused models or more custom LLM applications, they often do not have any benchmark. It's just not there. So you're relying on anecdotal evidence that, oh, it works, it is impressive. But it's not quite production.

Right? So one of the things that is very, um, uh, a difficult conversation to be had, it's a boring conversation to be had, but it's an important one that we often very strongly recommend that, hey, the first level of engagement should be to create that eval framework, right? And even more hairy. And manual for a custom LLM application.

'cause it's your data, you have a particular sense of quality, right? So we push for that and we are very prescriptive about that. For example, we are very clear that prioritize quality, don't prioritize saving time or anything because if you don't have a good quality response doesn't matter. does, it, doesn't matter if it's cheaper or, you know, faster.

Right? It's all moot. Right. And again, um.

Uh, it's not obvious and it's not very easy argument to be had in real life settings, right? Because everyone is trying to show the ROI story about how, how much time it saved and whatnot. And our approach has been to like, Hey, look, uh, let's do this methodically in a very scientifically rigorous way to build very high quality solutions. So, coming back to your question, that eval is a big gap, a big opportunity for a company like us, others as well.

Uh, more so in custom applications that B two Bs are doing it across different verticals. And we have spent months and, uh, like with some of the partners where we are really pushing the boundary, right? Uh, I mean, I would not take names without their permission. Is essentially taking, you know, three or five models that we think will be most relevant for their use case and do the most boring evaluation framework there.

Right. That's very rigorous, very high quality, but you have a baseline, you know what you are standing on. Right. And then you know how to improve.

You have a particular thesis, you test it out, rinse and repeat. Right? And that has served us well very well. Yeah.

I, just to close out this thread has been super helpful. I think the, you, you hinted at it, right? Some of these conversations where you're talking about the realities of AI are, maybe quote unquote boring. I have not found this conversation boring.

I found this conversation very fascinating, but I'm also weird. And but I imagine that causes. Some amount of friction when potentially you're in a market where maybe you have competitors or just the general zeitgeist of AI hype where everyone's promising, you're gonna have, PhD level support Yeah. Yeah.

zero Yeah. So I guess, tell, could you just walk me through and I guess putting yourself in the shoes of our listeners who maybe try to navigate that themselves. Like what have you learned about having those conversations and trying to, sell your product to a customer, but also having to have these very real. It's not everything you may have heard on the internet Yeah, yeah, gonna have to walk slowly.

You learned about having that conversation in a way that's productive but also gets you the business result you're looking for. yeah. So, Dan, that goes back to one of the core values of Elucidata, right? We, our promise is to give you clarity right in, in this environment, not this internet, but even published papers, right?

Peer reviewed papers from very good journals, right, are adding to that, noisy environment. Right. My LinkedIn posts are adding to that, uh, noisy environment, right? The papers are adding, company launches are adding funding, news are adding.

So my job and elicit takes, this is a core value for us, right? To provide and deliver on the promise of giving you clarity. And if I'm part of. Clarity to give you a solution.

Great. If I'm not, that's fine too. So if you want to skip the violation framework, I believe, uh, sooner or later, you know the quality issue will hurt you. It's only a matter of when, not if.

And at the end of the day, you know, we will all be around, right? Google will be round and Pfizer will be round and will be round. The quality will. Be very clear to our customers, right?

So there's a lot of solutions out there. Um, some of them will just not stand the test of quality, okay? And, uh, in a vertically focused or vertically focused solution that, you know, we are trying to push for in one vertical, right? That's a hill that I'm willing to die on.

I don't expect everyone to agree on that on day one. Um, that's, that will be nice. I've lived long enough to know that does not happen. Doesn't matter what your idea is, slowly and steadily we'll do that.

And we have been quite successful in doing that so far. Coming back to the foundational values of the company abject. This has been great. There's so much more I wanted to cover, but we're running outta time, so I want to pivot us to some rapid fire closeout questions.

Is that okay with you? Sweet. Yeah, let's do that. Awesome.

Well, look, I don't believe anyone of any success has got there on their own. When you think about all the spectacular people that you've had a chance to work with, learn from, is there anyone that just pops to mind that's had a disproportionate effect of the way you think about building, leading, growing companies now? I'll not turn this into an Oscar speech, but there's one person that I owe a lot to, uh, that's my wife. Uh, she has been you know, a wonderful partner, a great advisor, a great counselor.

I, I lean on her very heavily. Uh, she's ananya, she is, you know, also has a very aggressive career at Meta. But to me, uh, that's a name that I cannot, that's the first name on the list. Yeah.

Maybe you can go give the meta team some advice on building AI products they might need at the moment. Yeah. if I give you a billboard, you put any advice on there For other B2B Sass EI was trying to scale the companies, what would it say? Listen to the market.

Yeah, go. Go. First principles. Things are changing very rapidly.

And I also say this to myself and anyone who would listen to me, right? Any lesson that we have had has a very short expiry date right now. Things are changing so rapidly, right? Go to market lessons, your tech stack.

It's a continuous, so, move fast with strong, clear moral authority. Otherwise it's a very clear, there's a very clear threat that you'll be left behind. Absolutely. Abhishek, this has been great.

If listeners wanna connect with you, learn more about Elucidata, how can they do that? Hey, so I'm quite the social media of choice for me is LinkedIn. I am there almost more than what I should be. I'm not on any other platform Substack and LinkedIn.

If you want to reach out to me, I'm very responsive there. Awesome. Yeah, put links to Abex LinkedIn that listening homepage in the sub his substack in the show notes for listeners. Thank you yeah.

I write up something there quite often. Um, So. to Abhishek for sharing his journey and insights. For our listeners, you found Abex insights valuable.

Please leave a review and share this episode with your network. Really helps the podcast grow. Thank you so much, Dan. Thank you.

It was wonderful chatting with you. Thanks for having us.

Related episodes across the Index

Other episodes covering the same guests and topics, from across The B2B Podcast Index.

  • Building the Internet of Medicine: How AI Could Transform Children's Healthcare | Dr. Timothy Chou - Founder, Pediatric MoonshotProgress, Potential, and Possibilities Podcast / Show · on Federated learning92 / 100
  • VC: How Benchmark Picks AI Winners - Max 10 Bets a Year, 5 Partners | Chetan Puttagunta (GP)The GTMnow Podcast · on Forward-deployed engineers89 / 100
  • Cutting the cost of Customer Success by 70% (without layoffs), with Jim Richmond, CCO of SmartlingThis is Growth! · on Forward-deployed engineers87 / 100
  • Public Intelligence Limited: A Punk Rock Guide to Sovereign AIData-powered Innovation Jam · on Federated learning87 / 100
  • From SaaS to Systems of Work: The Vertical AI Opportunity with Nick TippmannInvesting in Startups · on Forward-deployed engineers81 / 100
  • The Recruiter Who Says Resumes Don't Matter & Beat the Giants with a 2-Person Firm | Bryan LeeUnlocked Professional: AI and Future of Work · on Forward-deployed engineers80 / 100

More from SaaS Scaling Secrets

All episodes →
  • The Unexpected Ways AI is Shaping SaaS Companies with Tom Car, CEO of Productive65 / 100
  • From HBS Dropout to Successful SaaS Exit with Palash Soni, CEO of Goldcast85 / 100
  • Conquering the Desperation Mindset in SaaS Pricing with Apurv Bansal, CEO of Zenskar82 / 100
  • Scaling a SaaS Selling to Skeptical Scientists with Mike Klein, CEO of Genomenon86 / 100
  • Balancing a Risk-Reward Model in Real Estate Tech with Alvaro Erize, CEO of CINC86 / 100
Explore the best B2B SaaS podcasts →
All SaaS Scaling Secrets episodes →