How AI Learns to Smell with Alex Wiltschko - #771

Speaker A: AI has advanced primarily by learning from the digital world. Text, images, audio, and increasingly, video. But many of the problems people want AI, uh, to solve live outside these modalities. In the physical world. Smell is one of the most interesting examples. It's how animals detect disease, identify food, navigate environments, and communicate through chemistry. Yet scent has remained largely outside the reach of computing because, unlike language or images, there has never been a practical way to digitize it at scale. Alex Wolchko, founder and CEO of Osmo and a former Google DeepMind researcher, is working to change this. His team is building what they call olfactory intelligence, AI systems that can model, predict, and design sense, while creating the data sets and infrastructure needed to bring smell to the digital world. In this conversation, we explore what it takes to give computers a sense of smell, why. Why scent is such a difficult AI problem and what it teaches us about the next generation of foundation models. Here's Alex.

Speaker B: 99% of species on this planet can only speak with chemistry, right? Thinking of bacteria and fungi and plants and insects, like, they just only can talk with molecules. And I think that it's really worth adding other kind of alien forms of intelligence to our AI models. And the way to do that is to train it on the intellectual output of those other intellects, which is that's chemistry. That's the sense that's in the air. They're produced by living things for reasons to talk to each other.

Speaker A: I'm Sam Charrington, and this is the TWIML AI podcast. For over a decade, I've been exploring the ideas and innovations shaping the future of AI through conversations like this one that help you understand what's real, what's next, and what matters.

Speaker C: Let's jump in. When I think about giving computers a sense of smell, uh, there's kind of two angles to this. One is, you know, there's some scent out in the world, and I want my computer to be able to recognize it the same way I do. Uh, and the other, which is, I think more along the lines of what you're working on at osmo, at least initially, is to have the computer kind of grok the idea of scent so that it can create new ones.

Speaker B: Any scent that's been given to computers, there's three kind of broad steps. You got to read the world, so, like, turn atoms into bits and information. You have to map it, like, understand it, so, you know, be able to manipulate it, digitally encode it, send it. And that's like JPEG and rgb, right? Um, and then you have to be able to write it back out again. Right. So a printer or a display or a speaker. Um, and so the thing we focused on at Google Brain was the missing piece, which is for scent, is the map. So color has had a map.

Speaker C: So kind of representation of, well, what to what, though?

Speaker B: Exactly, exactly. So, like, let's, let's approach it from the side. Like, how did this work for vision? How did this work for hearing? Right. We've had maps for a long time, right? So the map for sound is just one dimension. It's low to high frequency. Really simple to say. Uh, and then for color, it's three numbers. It's rgb. Right. Or whatever your preferred color space is. But those three numbers tell you how to deal with color. There's three channels of color information in our eye.

Speaker C: But of course you're simplifying a lot because for each of those other modalities, there's lots of different maps.

Speaker B: Totally.

Speaker C: Those are just examples of.

Speaker B: They're examples of simplified examples of. And they can kind of be translated into each other. But I'm like, I'm papering over like centuries of psychophysics here, and anybody who knows anything about those things is going to come screaming at me. Um, but you'll have to forgive the simplifications. I'm going to simplify sense stuff too. And if people talked the way that I'm going to talk, I would come after them too. Um, so, yeah, there's cmyk, there's lab, there's hsv, there's many different maps. And then there's more complex maps.

Speaker C: I was scarred by a DSP class in grad school. Okay, so you came back in.

Speaker B: Exactly. You know, I'm like, there's filter sets, there's Gabor filter sets. There's, you know, all kind. There's Huff Trans. Like, there's all kinds of ways of representing images. And I'm super simplifying it. Right. But like, maps, they.

Speaker C: For.

Speaker B: Certainly they're there and we know them. We've known them for a while. And the notion that we can map color has been instrumental in building, like, CCDs and CMOs and therefore, like digital imaging. Exactly. And then also the printers. Right. So like the ink in the inkjet cartridges, we know we can combine them to make, you know, millions of colors. There's three channels of color information, roughly, roughly RGB, uh, in the eye, but there's over 300 channels of olfactory information in the nose. And it's still a mystery exactly what they code for, but it's certainly much Higher dimensional, at least in terms of channel count.

Speaker C: So when you say channel count in the nose, is that mapping to physical structures? I'm going to butcher this. But for the eye, I'm like, is it like rods and cones and stuff like that?

Speaker B: Exactly, yeah. So there's rods and cones, and together there's like four channels. They're like roughly RGB grayscale. Again, there's a lot of details there. Um, those are specific receptors that are encoded in your genes, right? Expressed in cells in your eye that are sensitive to light. So the equivalent for smell are those cells are called olfactory, uh, uh, sensory neurons. And those cells actually start in the brain and they poke through your skull and they actually touch the world. It's one of the two parts of your brain that actually leaves the skull. Um, and so when you smell something, your brain is physically touching another living thing that's like let off a little bit of some of itself for you to smell. Um, so there are millions and millions of olfactory sensory neurons in the part of your nose, the inside that actually does the smelling that's sensitive to smell, called the olfactory epithelium.

Speaker C: Does the nomenclature here that these are sensory neurons imply that they're more fundamental than a rod or a cone, which I'm imagining are more like superstructures, like bigger things.

Speaker B: I'm, um, probably gonna mess this up. But rods and cones are cell types, and so those are names for types of cells. And there's many other types of cells in the retina that'll help kind of compute the raw information that these cells get. So the equivalent of those like, primary sensory cells, like without these cells, light doesn't turn into awareness, right? So it's the front line, right? So the frontline cells for smell are called OSNs, or olfactory sensory neurons. And these cells at the tip, they basically shove part of their cell membranes into a little mucous membrane that then touches the world. And those little tips are just chock full of proteins called olfactory receptors. And those are the things that actually sense the chemical world. And so how many of those olfactory receptor types are there? Uh, over 300. So that's where that number comes from is, is each of those receptor types are differently sensitive to the chemical world, right? Just like some, uh, some rhodopsins, some, those are the proteins that actually sense light. Some are sensitive more bluey and some are sensitive more greeny. Some are more reddy. Um, and there's, you know, hundreds times, there's uh, a hundred more, um, uh, different types uh, in olfaction.

Speaker C: So what does that say about the resolution of the human olfactory system? Like, is there like a number of smells that the typical person can smell?

Speaker B: Like, I don't think we have a good estimate for that. There is a very famous paper which has a trillion which has been pretty thoroughly debunked. Um, but uh, I don't know if we can really estimate that. Um, there's a few things though, when you talk about sensitivity, right? So think of, um, so I had laser eye surgery. So I had glasses for most of my life. Um, I couldn't really see that well. So I didn't, I couldn't resolve really subtle differences, but I could, I wasn't blind. Like I, I, I could, I could perceive all the light. So I didn't have, I light was getting in, but I just wasn't being focused properly. So with smell, um, if as long as your airways are working you can smell and detect things, you might not detect subtle differences. Um, but, but you definitely are sensitive to it and some people aren't and they have their, you know, airways closed for various reasons like anat or an injury or something like that. But like, you know, there's amazing experiment by Noam Sobel, um, who showed that actually people can do scent tracking. So if you, if you put on a blindfold and you get somebody down on the ground and you leave a little trail of like chocolate or cinnamon or whatever, people if they concentrate, can actually scent track just like a dog. They're super slow, but they can do it. Um, so just that just goes to show that like there's this, there's this, I think a myth that like we're not good at smelling, we're freaking amazing at smelling. Like we can smell the equivalent for some molecules of like a one little teardrop in an Olympic sized swimming pool. We are so sensitive to some things. Um, I mean that's how like natural gas has a smell. It's just the tiniest amount of these molecules called mercaptans are added to it and we can smell it like super sensitively. Like parts per billion or trillion is crazy. So I mean we're good, like humans are good at smelling.

Speaker C: So we're trying to build this map. We have this structure that we know about from biology. Do we know, like how do we, you know, what's next? I guess, like how do we represent that structure or what's the next step from.

Speaker B: I'll walk you through how we thought, thought about doing this or how we have done it. Um, step Number one for us was there's a really basic version of this problem, which is like, assume you're only smelling one molecule and you know its structure. Like, you can draw it on the board like you're in high school chemistry class with, like, atoms and bonds and all that, and you know what it smells like. So this particular sets of, you know, combinations of carbon and nitrogen and oxygen smells like sweet, smells like vanilla, smells like phenol. Like, smells like chocolate. Like, those are maybe four descriptors drawn from a set of a hundred that apply to that molecule. Well, okay, go get thousands of those pairs and then train a neural network to predict that relationship from structure to odor. Um, that problem called the structure odor relation problem, had been unsolved for a hundred years. And actually, in some cases, people thought it was unsolvable. Um, and so what we did as one of our first, um, kind of outputs at, ah, Google Brain, was we just trained a relatively new, at the time kind of neural network called a graph neural network, um, which was specialized for chemistry. Um, it's actually very related to the transformer. It's kind of been subsumed by the transformer in the intervening years. And we were able to predict what things smell like very well. In fact, so well that we said, why don't we set up an odor Turing test? Let's go find molecules that nobody's ever smelled before. Some have never been made before, like, nature has not seen them. Let's predict what they smell like ahead of time and keep our prediction secret. Let's go get those molecules, physically send them to another location. I have a great collaborator that I worked with on this. His name is Joel Mainland at Monell. Let's train people to smell and describe smell. It doesn't take a ton of training, maybe like, eight hours to do okay at, like, hey, this smells grassy or phenolic or vanilla or cucumber, you know, and maybe 50 terms, uh, you can be trained on. Let's double blind, um, have these people smell and describe completely new molecules and then compare how well this panel of people does to, first of all, some individual panelist, because one person is always worse than the average of the panel. So let's compare that to our model. So the question is basically, you know, the oder Turing test is if you want to make your panel better, would you rather add another person or would you rather add the predictions of a model? Right. Um, and it turned out our model predictions were better than any one individual panelist on average in the panel, meaning that we've Passed an Odor Turing test. Like, our model predictions were human quality, which was pretty cool. Um, that was the first thing. And what we did with the neural network is we cracked it open and we looked at what's called the embedding layer, which is a part of the neural network that basically, uh, turns the inputs into a vector. That's. That is the map. Right? And then that map is what we kind of can slice up in regions and use for classification. So this region of the map is vanilla, this region of the map is Red Barry, et cetera. Um, without that, you actually can't do that classification problem. So that embedding turned out, if you do the engineering right, it just kind of needs to be around 300 dimensions to work really well, which is, like, suspicious, but, you know, just suggestive. Right. I can't make any claims, um, but, uh, that embedding had a ton of beautiful structure in it, and that seemed to be a first candidate for a continuous predictive map of smell. And we called it the principal odor map. And that's been foundational to what we've built at the company since then.

Speaker C: Before we get to that structure, you mentioned a graph neural net was the fundamental architecture here. What did the nodes and the edges in the graph represent?

Speaker B: Yeah, great question. So, like, if you were doing machine learning on a social network graph, the nodes would be people and the edges would be relationships between people, like friendships. And then it might be one very large graph of, like, Facebook or Twitter. Uh, in our case, uh, every, uh, graph is a molecule, and the nodes of the molecule are the atoms. So it might be a carbon, it might be a nitrogen, it might be a sulfur or an oxygen. Um, and the edges in the graph are the bonds, and that might be a single bond, a double bond, a triple bond. And the graphs aren't very big because molecules that have a smell aren't very big. If they were huge, they actually wouldn't make it into the air and fly away. Um, and if they were also huge, they wouldn't fit inside the binding pockets of the olfactory receptors that we talked about. So they're not too small, they're not too big. Um, they tend to be less than 20 atoms, but more than like three or four. Um, and so those are the graphs, those are the inputs. And, uh, the graph neural network is able to basically process that and propagate information and basically gather the macro structures in that molecule. Um, and then eventually you, uh, turn that into a fixed length vector that describes how that molecule actually might smell. And that's what the principal odor map is.

Speaker C: And so now the structure that you observed in the embedding space. Talk a little bit about that.

Speaker B: Sure. So we took that almost 300 dimensional vector and we took a two dimensional shadow of it so we could look at it. Um, and we used something called principal components analysis to do that or pca. And what we did is when we plotted this two dimensional map, we made every dot on that plot be a molecule. And so there were about 5,000 dots on that plot. What we did on that plot is we circled regions of molecules that all had the same smell.

Speaker C: So the sweet neighborhood, the ah, cucumber neighborhood, that kind of thing.

Speaker B: Exactly. And they didn't have to be neighborhoods, Right. All these, all the cucumber molecules could be completely spread out, in which case every region might actually be all overlapping with each other. But what was really beautiful is if we circled the floral region, it was this pretty big part on the left. Uh, we can, you know, share the images with the listeners if you'd like. But the floral region was pretty big and was on a side of the left. But then if you draw like jasmine or rose or um, you know, violet, they actually ended up being sub regions inside of floral. And we didn't tell it that, like, we didn't tell it that there was this nested relationship of nature there, there. And similarly the, the region for, um, for fermented and alcoholic was actually shaped like a bottle during our first model training. We haven't touched it since. Cause it's so funny. Um, but things like fermented and whiny and like rum, uh, those, those all fermented alcoholic, um, scents were all in the same region as well. And that this, this pattern continued for basically all of scent. I mean that's, I thought that was very beautiful. Um, one thing that really struck me with that map, which we're still working out frankly, is these scents. Actually, if you blur your eyes, it almost looks like how. It's a story of how nature makes those scents, right? So fermented, uh, scents like from wine or rum, et cetera, those are made by yeast that are ferment. Actually it's a biological process that's producing these molecules, right? So there's a story of how the molecule's produced. And same, same with flowers. Those are all genetically much more related to each other than they are to say, um, other species that produce nuts or you know, bark or whatever. Um, and uh, they're all clustered together. So in a way, like there's this story of like actual biological life and scent that seems to be very tightly woven together.

Speaker C: And are there like confounding examples there? I'm imagining, you know, both fermentation and florals are kind of these natural scents. But are there other scents that are like completely unnatural? Maybe the thing that we put in natural gas to give it a smell. Like, and what are the shapes, the shapes of these non organic smells, like, differ in some fundamental way.

Speaker B: So actually that was our first line of business that we started was like, can we somehow make new molecules that smell great, um, and that are safe and that we can produce at scale and that are affordable all that, um, and turns out like, that's a very interesting business to be in because if you can make some molecules, um, smell great but they're not safe or they're being removed because of new, uh, regulatory action. Um, so we need replacements because we want our products to smell great, like our laundry to smell great, our home to smell great. Uh, and so we actually use these models which we've developed over many years to actually find new fragrance molecules that have never existed in nature before. And then we make them, um, and uh, we kind of, we could bring them to market, um, which we're in the process of doing right now.

Speaker C: And so when you're doing that is your objective. Like I'm imagining that somewhere in some prior step you've got a classifier of ah, like smells great, doesn't smell great. Uh, and you're trying to map to that as opposed to, you know, or you could be, well, I want it to be like floral or I want it to be fermented or I want it to be lemony or something like that. Like how do you guide.

Speaker B: It's always really specific, right? So in, in, in, in our, in our industry, um, it's pretty clear what the molecules are that need to be made. Um, so hey, we're missing uh, a citrus molecule that is long lasting, or we're missing a vanilla note that uh, is optically clear because vanilla is typically brown. And so you don't want to, you know, people want clear fragrances so that they can change the color of the product. And there's many others like this. But you know, we kind of know what we need to make. Um, and so we typically focus the team because it's ultimately a team that's doing all this with, you know, physical infrastructure, all these models and a lot more chemoinformatics, um, on the kind of highest priority items. And of course we discover some interesting things by accident. Along the way, um, and we, we don't ignore those.

Speaker C: The initial set of molecules were these, um, these are just kind of known, like written down. You didn't have to collect anything and like, I don't know, like, put it through like PCR or something. Like, we just, we knew what these are.

Speaker B: Yeah, we, we didn't have a, we didn't have any labs at that point in time. And so the data that we collected partly through we like did some creative data licensing. We also found some stuff on the Internet. I mean, I had been, uh, in the world of send for like 15 years when we started that project. So I was like, okay, I know where I'm going to go to get this stuff. Um, and we were successful. Um, and since then, uh, we've, we've dramatically scaled up our data collection abilities. So, you know, we, we had I think 5,000 molecules in our first dataset. We've digitized 6 billion molecules at this point.

Speaker C: Um, what does that mean? Walk us through that process.

Speaker B: So we've enumerated all the molecules that could possibly be made. So, as in the real, actually realizable physical molecules, um, that could possibly have a smell. Um, and uh, we've basically ran predictive models on all of them and we've made a huge number of them. And so we've made more Newton.

Speaker C: So this is the initial part of that statement is like, there's some physics that governs, you know, what, uh, the ways that molecules can form. And you can like filter that based on some set of criteria. The three atoms, the 20 atoms. And maybe like these bonds work. These bonds don't work. And then you, you get some like, starting place, this list of potential. And there's a manufacturability filter as well. So you, you ultimately end up with this list that you. Big list, a big list that you kind of derive like from first principles, like the way that these things work together.

Speaker B: Is that the idea? That's right, exactly. So those are like, those are. We, we, those are real molecules. Um, we can make any of them, uh, and they're more likely than not to be able to have a smell. And then the rest of like whether they're useful or interesting or beautiful. We have AI models to predict all that stuff. Um, as well as safety. Safety is really critical. Um, and uh, that's kind of one of our core data sets. The other thing is we've smelled a lot. Like, so we train people to smell. There's many different protocols for how to like, smell something. Hey, does this Smell good or bad or intense or not intense. Is this better than the other one? There's like many different ways of doing this. And we've gotten really good in, ah, dialing that in. And we have people basically internationally that smell. And so we've, uh. I think I want to get the number right. It's probably shifted since I last looked this up like a week ago. But we've digitized five 43 million sniffs. Um, so that's like the largest olfactory data set for the purposes of training AI models I think, ever. Um, and uh, we had to make all of that from scratch. There's no scale AI or there's no mechanical turk for smell. We've had to make that internally, consume it ourselves and generate huge amounts of olfactory data for olfactory intelligence.

Speaker C: And that data set ultimately looks like, ah, a molecule, however you want to represent that. And a set of labels that the human might label that smell.

Speaker B: It can be broader than that. Right? So, um, that's a part of our data set is like we know the molecular structure and then we smell it and label what it smells like. But it also might be like a cucumber you buy from the grocery store that we smell. Right. Or we might, uh. And that's the case of analytical annotations. It might be a product, like a market product.

Speaker C: Okay, so a thing and a smell, necessarily a market, exactly.

Speaker B: And sometimes we don't just smell it, we put it through analytical machinery. Right. So that gets to how you actually. Where does the real world data come from? Like, you need to use chemical sensors. And so we've put huge amounts of data through chemical sensors. And then we also can align that with human sensory labels so that we can begin to actually relate sensors to human perception, which. And like, that's kind of core to what we do.

Speaker C: When you're in this part of the process where you're manufacturing these molecules. Um, like, is there. Are there known toxicity screens that you can. Oh, yeah.

Speaker B: You have to go through a very rigorous process in Europe, in the US and worldwide. And there's a binder of, um, tests you have to submit. And they're really thorough and they're the right test. Right. So is this safe on your skin? Is this safe to breathe in? Is this safe for your eyes? Is it safe, uh, for fish? Because you might flush some of it down the toilet or in the shower drain right after you wash yourself with a shampoo.

Speaker C: The question that I'm curious about is can you derive that from molecular structure or do you have to do it empirically.

Speaker B: You can predict it, which is super important for how we're so efficient at what we do. Um, but then you have to test it physically. It's just the law. Um, and it's the right thing to do. Like you just, just check, right, do the experiment. And we do, um, over and over for all the, for all the products that we're taking through regulatory, um, review.

Speaker C: This is maybe a digression from the kind of technical conversation I want to have is like the business side of this, things like, are you validating that you know the scent is what the client wants and then you license it to them and they find someone to manufacture it? Are you, like, making the sense at

Speaker B: scale or like, where we do most of our business is we actually blend, uh, molecules and ingredients that already exist and are already approved. And so like, if somebody comes to us and they want to launch an air freshener or a shampoo or a fine fragrance, we need to, in order to get that out quickly, we need to be able to blend molecules together that we can get. Um, and, uh, we, we actually stock many of these ingredients and we've taught, uh, olfactory intelligence, which is really a fleet of different models, how to convert, like a customer's request. Like, I want a scent that smells really fresh and clean and you know, is going to be useful or liked by Gen Z, uh, men. Um, like that's enough of a specification to kind of begin to make a scent against that. And then we have Master perfumers and perfumers 2/3 ax and just mix them together. I mean, you're not too far from the truth. Like, generally new scents are like existing scents, right? Because scent is art and like, art evolves. It doesn't like, take these crazy jumps, right? So you can always tell what the lineage of Ascent is. And like, yeah, Axe is one of those, right? Actually, Axe is actually really famous because it was the first time that fine fragrance perfumes were actually brought to that mass market price point. So some of the scents in Axe Body Spray had never been like, accessible to the average person because those.

Speaker C: I never associated Axe with Fine anything, so.

Speaker B: Yeah, exactly. It's like it's built its own rep over the years. But, like, the way those scents were actually chosen into design were super interesting from like a marketing and kind of scent development perspective. Um, but yeah, we, we've taught OI to collaborate with perfumers and also just work directly with our customers, where you basically prompt it and you're like, hey, I'd like to design a Scent, our perfumers use it as inspiration and our customers sometimes design their own scents for themselves. Right. In fact, I had conversation earlier today where I sent somebody an OI design scent and they loved it, they picked it, they're going to launch it.

Speaker C: Right.

Speaker B: Like, just right out of like our software system. So you can think of that as like a canva, right, Or a figma percent where you know, you, you and me can, can make, sent that, that, that wins in the market. Um, and so that's what we've put it all together with.

Speaker C: So the foundation of this is the, you know, this embedding space that you've trained and I'm envisioning the, the task that you described as being akin a little bit to the war 2 vec, uh, embedding space math. Like King, uh, minus. I always get the.

Speaker B: Yeah, it's like King Min Queen minus woman or something. There's arithmetic right in the text space. Yeah, exactly. So no it's, it's really similar. Right. So the core business for us, um, is, you know, you want to launch a brand that has some smell in it, like a shampoo or a fine fragrance, you got to get the scent made. Um, so we actually have a factory where we make it. But the, the core algorithmic pipeline is we take your text or image or audio description and then we embed it into a perceptual space. And there's lots of steps involved in getting that right and having that be commercially like viable and acceptable. Um, but then we then have to decode from that space back to a formula. And that formula is a set of instructions for a, ah, formulation team in the factory behind me and a big robot that's the size of a school bus that can make a new fragrance every hundred seconds. Um, and uh, that's basically how all this is strung together. And the algorithms are of course driven by data. And every single time we make a new scent for a customer, like, yes, they're happy, we're happy because we get paid, they're happy because their brand is working. Uh, but we also get data, right? So we're always sniffing those scents, we're always putting them through our chemical sensors. And so it creates this really nice like self perpetuating machine where we found a business where we can serve the world and like actually help people be happy and like become wealthy and grow their businesses. Um, but also like it helps us on our mission. Right. So we, we, the last thing that we do is not going to be fragrance design, but it is certainly the first and so it's teaching us and it's also funding us. And I think that's a really powerful, like, I'm glad we found it because I think it's really, really powerful for what we're trying to do.

Speaker C: But there are a couple things in there, uh, that I wanted to dig into. You know, essentially the encode and decode parts, like you mentioned multimodal. And that's like super interesting. Like, so I can describe this thing but also say like, I want, you know, I want this scent to evoke this image, like, or this image to evoke the scent, whatever the directionality is. Like, um. And so the first question is like, how did you go from like this kind of molecule map thing to this multimodal embedding space? Right. And then on the decode side, okay, you know, you found the neighborhood of some, you know, something. If it's, if we're still like in the regime where this embedding space is fundamentally embedding like representations of molecules, like, how do you get from that, from that, like a single point to. Well, that's a mix of these five things. Is that also AI or is that more deterministic or experience based thing?

Speaker B: Great questions. There's like a few things buried in. So the first is um, the map that we discover. Like we've evolved the system a lot. So um, what I'm going to say is like the gist, um, but the implementation and engineering details have kind of diverged from what, like the simplest way to describe it. But like here goes that map, that roughly 300 dimensional principal order map. The first model that we trained does indeed take single, uh, molecules as inputs. But there's no reason that you can't target that embedding with other inputs.

Speaker C: Right?

Speaker B: There's no reason you can't target it with a. So like, what about a chemical sensor? Could you just like chop off the head of that neural network and like, you know, put it on another neural network that took in like chemical sensor readings? And the answer is like, yeah, you can for sure do that. So the question is not so much like how do you crunch everything down to the original input space, it's more like how do you map new inputs into that same space? And we've been able to do that. Um, and then there's a question of how do you deal with mixtures? And like that is the holy grail. And we've spent a ton of energy and time and science on how to do that. And I'm super proud of what we've done what we're doing, what's still ahead. Um, but like, the special sauce is really there. It's like, how do you reason about how these molecules interact? Um, that's a good question, but I'm

Speaker C: not going to get the answer.

Speaker B: Not a detailed answer.

Speaker C: Yeah.

Speaker B: Ah, come work at OSMO and push back the frontiers. But no, it's like it is the special sauce. Um, ultimately, truly the data. Data that drives it is actually the moat. Um, and so I think we have the largest olfactory data sets in the world ever. Um, but more importantly, the rate at which we're generating data just far outstrips anybody. Right. Like, even the companies have been around for a hundred years. Like, they say they have data. What they have is a lot of Excel spreadsheets that don't actually kind of add up to much. Um, and so everything that we do from the very beginning has been designed to be collected for analysis and for model training. Uh, and that makes all the difference. Yeah.

Speaker C: So data and model training suggest that that mapping is indeed a learned thing. It's another model. As opposed to the relationship's not that complex or it's business rules or Definitely not business rules.

Speaker B: I mean, as with any AIML project, you always start with the dumbest models first and see how far it gets to. Um, and then, uh, in the most interesting cases, you're like, that's actually not Millennial equals. Exactly. Ex. Yeah, just, you know, keep, keep pushing the same sense. Like still Polo Blue, it's still Abercrombie and Fitch fierce. Um, and like, that's not, that's not that wrong. Right. It's just you'd like to be a little bit more nuanced with it. Um, and those are great sensitives to the test of time, truly. Um, but, uh, uh, it's. Yeah, it's really an exercise in collecting very, very large data sets tailored for AI and then what you do algorithmically on top of that, like when we started the company, like, whatever, almost four years ago, knowing how to build specific kinds of models on this data modality was super important. And that's actually like, been an accelerant for us. Um, but really, like a lot, if you just have great data, a lot of stuff kind of falls out from it. Um, but you have to be careful with every step of the way and have great, great people who are thinking about every step too. Yeah.

Speaker C: And does graph models still figure significantly into what you're doing? Or have you evolved to transformers or

Speaker B: in the right applications? I mean, we're really not dogmatic about the modeling approach that we take. Um, we're really dogmatic about data size and data quality.

Speaker C: There's an implication then that you spin up new models for individual projects as opposed to what I envisioned was that you had the olfactory foundation model and that thing changes infrequently and you just use it for a new task.

Speaker B: So when we say olfactory intelligence, we kind of mean like the suite or the fleet of predictive models on all aspects of smell. And there's dozens, there's dozens of models. And if you go look at like Neolabs or life sciences companies, some of them are pretty upfront that like their like core model or foundation model is actually a fleet of models. And that's how self driving cars work too. Self driving cars are driven by like a fleet of models that all connect together along a spine to coordinate sensing with action with planning and all that stuff. So we're, I think we're close. The way we think about it is closer to autonomous vehicles, which is actually where my CTOs kind of background is. So Rich ran the data architecture for Nvidia's autonomous vehicles program. We met at Twitter when we were working there and then he went to Spotify and did Rexis. Um, but uh, you know, that in

Speaker C: and of itself is interesting and opens up this question which is like in AV there's this tension that I've explored quite a bit through interviews between models that you know, are somehow faithful to the way we understand at least the physical world or the physical process or like classical control in the case of AV versus end to end. I don't care about any of that stuff. I'm not going to like human tinker or subsystems, whatever. I'm just going to end train on data. It sounds like you're more in the, like we've got this physical understanding of you know, olfactory systems or some process for developing these scents and we're gonna like have sub components or sub models.

Speaker B: Yeah, I would say one part of that's true and one, one's not. Um, so the one that is, that is not true is like we don't necessarily strive for physical understanding. We strive for predictive accuracy and like, like helping our customers and like you know, building like being able to predict the right thing to keep the organization going. Um, and sometimes we use physics. That's just a modeling choice. It's like using a physics based model. Um, and I would say we actually cannot do one fully unified model because there's regulatory work that has to be done. So you kind of need one prediction for. Is this safe in this specific way? So you gotta have at least it could be one model with like 12 heads, but you do have to have those specific outputs and there are specific data sets that need to be used to train those and they um, effectively become independent models in that regime. Yeah, got it.

Speaker C: So the sub models are, is it fair to say the core is predicting a, uh, smell, but then you've got these other heads or attributes that you're also trying to predict, which are toxicity, maybe manufacturability, maybe cost of manufacturer, maybe

Speaker B: whatever regulatory safety, you know, all that stuff. And like, you can't really skip any steps. Right. They're all required. But yeah, the core is like, what does it smell like and does it smell good and does it smell strong enough? Like those are kind of the core things that you need and then everything else you can figure out.

Speaker C: Does it smell, does, does it smell good? Is that a, uh, a, ah, derived property of what does it smell like? Or do you also predict that independent of what it smells like, it's a

Speaker B: derived property of the SC with the target consumer? So, you know, an example is like, uh, you could either choose parmesan cheese or kimchi or strawberry. Like depending on who you show it to, they may or may not like it. And in the case of strawberry, actually the kind of strawberries that people think of or want in Japan actually are pretty different from the ones that we think of or want in the U.S. and so, well, those are like the gift ones, but just like the flavor. Right. Of them, um, it's a different sweetness profile, it's like almost borderline different fruit in terms of how you construct it in a product. Um, but what you like is heavily, heavily driven by what you've been exposed to before. And what did you feel like when you got exposed to it? Um, and if you're in a Korean household being exposed to kimchi is you do that under like warm family conditions. Um, and if you're exposed to kimchi, it could be like literally whatever. If you're, you know, a non Korean person who's never experienced before, you're like, what did I just open? I happen to love kimchi, um, but it was an acquired taste. Um, and similarly, like from my culture, there's all kinds of like, you know, horseradish and gefilte fish and noodle kugel and all this stuff. So like, I love those things. But that's like my people's food. Like that's what we eat. And not everybody likes it, but I'm into it.

Speaker C: Which raises the question about, um, taste, like, which is kind of. I guess it's more adjacent to smell than it is to physically.

Speaker B: It's pretty close. Yeah.

Speaker C: And so does that, um, you know, is that something that you dabble in? Is it far enough away that you don't, you know, think about it?

Speaker B: We think about it a lot. We don't work in taste today because we're really, really focused on our kind of first vertical. And, like, we gotta. We gotta focus. Um, so there's kind of three aspects to this. There's smell, which is for sure, just like, what comes in your nose, but also, like, you can smell things that go the other way. It's called retronasal olfaction. Um, so when you're eating something, you're actually creating this chimney effect that, like, kind of has scent. Basically vent the reverse way. Um, and so you. This is how flavor is produced, right? So if you ever eat a jelly bean and hold your nose, you actually can't. It just tastes sweet. You can't tell if it's like lemon or lime or grapefruit or whatever. Um, so flavor is 90% smell, 10% taste. And taste is just what happens on your tongue. Like, sweet, savory, sour, salty, umami, all that. Um, bitter. Ah. But it's in terms of dimensionality and richness. Uh, again, like, I'm sure the taste neuroscientist would kill me for saying this, but, like, it's just not as complex. Like, there's just. There's less diverse.

Speaker C: They've only got six. We've got three. Sorry.

Speaker B: You can have a taste person on your podcast next and they can defend themselves, but, like, they just. There's fewer channels of information. And for sure, the experience of flavor is, like, destroyed if you cannot smell. Just ask people who lost their sense of smell in Covid or who've ever tried the experiment of just, like, holding your nose when you eat a bite of anything. So, um, all that to say that flavor is super related to fragrance, and it's kind of there waiting for us when we're ready, as it were.

Speaker C: Um, so we've talked a lot about kind of where this is all going and what's possible, but any additional thoughts on that?

Speaker B: Like, I want to give computers a sense of smell, and that means reading and mapping and writing smell. And, you know, we've been on our journey super pragmatic about, like, building the scientific capabilities, which is the first two years of the company and then finding a great business to go put this to use in, which is the fragrance industry. But like we, like, we're going to go further than that. Um, and in fact we're talking with ah, a number of organizations. Like I think the thing that's needed now for not just design of scent, but the detection of scent is exactly what you're talking about from before, which is we have to build a foundation model. And the thing that's been missing and that we've been building is you have to collect a whole bunch of data, right? And like, you know, for instance, if you wanted to do the very, very noble work of smelling someone with cancer, uh, early detecting their disease early, or if you wanted to uh, detect uh, early infection with malaria or another infectious disease that claims the lives of mostly

Speaker C: kids, um, that's super interesting thinking about those as data collection problems.

Speaker B: But so here's the deal. You can go directly after that and collect sent uh, data from people with or without those conditions and try to build models. You're never going to get enough people to build a great model. Right. And like there's just, it's just hard to go get that much data meaning

Speaker C: because there's not enough signal. And the correlation between scent and disease or some other factor.

Speaker B: Are you speaking just low numbers? It's just from a pure statistics and machine learning problem. You know, I think basically there's not an obvious, obvious, obvious signal that says this person has, you know, a disease and then doesn't. There are subtle changes across like many hundreds or potentially thousands of molecular signals. We just don't know. Um, but we know dogs can do it. Animals can actually detect these patterns. So there's something.

Speaker A: Right.

Speaker C: That's what I was thinking of when you raised the.

Speaker B: It's for sure there, it's for sure there. Um, but we can get computers to do it. We just need to go get a ton of data, right? Like we need to like band together and build a huge effort where we just like, let's go sniff 10,000 people. I don't care if they're healthy or sick. I don't care how old they are. I don't care. I mean, let's record all that information, of course, but like, like, let's just go get a ton of data. Let's go to the grocery store and get cucumbers and bananas, flowers and like steaks and whatever. Um, and let's just go smell everything and then build a huge olfactory data set of what the world smells like. And if we can do that, then. Then it sounds a lot harder than

Speaker C: scraping Reddit at one time.

Speaker B: It is a whole lot harder than scraping Reddit, but it's worth it because literally nobody's going to do it unless we pull ourselves together. So, like, but this, it's, like, worth doing, right? Like, you know, this is the thing that maybe isn't a sign, but, like, I have a lot of friends in the world of AI and ML, and, you know, I came in through this as a biologist, and as a biologist, like, you go do experiments and it's, like, hard and, like, you gotta go to the bench and get your own data. And, you know, it's gritty. Um, and in the world of AI, like, you kind of just assume the data's out there, and if the data's not out there, you kind of assume you can give somebody a credit card to create the data. Um, and, like, I think that's amazing, but, like, the amount of infrastructure that you have access to is just astoundingly huge. And, like, you just have to understand that if you want to, like, continue to expand the realm of, like, what AI can do and compute can do, like, it might get hard. Like, you might reach a point where there's not a vendor that can handle everything for you and, you know, shine your shoes or whatever, like, you might have to go do the work yourself. And so we're very much in that regime. And I love, like, it's. It's so hard. It's just. It's beyond brutally hard. Um, but I, uh, love it because nobody else is doing it, right? And, like, that's very much my preference for how I want to spend my life is I want to do weird things that nobody else is doing that matter. Right? And so that's. That's kind of my selection criteria. Like, there's plenty of people that can go build, you know, AI voice agents for customer service. I think we're going to need that to, like, make commerce better. Like, I'm not the guy to do that. Like, I want to do this weird thing, right? I want to give computers a sense of smell. Um, and so, you know, I think where we're going is, like, we are collecting huge amounts of data, but importantly, we have this platform that allows us to store all this data, organize it, like, massive efficiencies of scale digitally, and then increasingly physically, we can make a huge number of unique sense. Like, we have all this infrastructure that we've built painstakingly over a few years, and now let's go get all the Smell data. Right. And let's build this chemical, um, slice of reality, uh, into AI, which. Let's build a true olfaction, uh, foundation model. And, but, but the data comes first, right? Like, we need to go get all the data. And then, like, you know, when, whenever these models come out, like Opus 4. 7 or when, when Mythos comes out or Codex 55, they talk about, uh, the model card in terms of all these benchmarks. And so what I, what I imagine for the future is that we'll collect all this foundational data and malaria detection will be a benchmark, all right? And cancer detection will be benchmark. And, you know, building a great market, um, product for a new shampoo launch in Malaysia will be a benchmark. Um, but.

Speaker C: Or even the idea that the generic. This is where I thought you were going, that the generic foundation model that you are using by, by generic, I mean kind of general purpose part of a better term, um, is not just text and, you know, multi, kind of the common modalities. But is now has. There's some kind of benchma or model card statement around, you know, smell and taste.

Speaker B: And yeah, it's like, let's add these weird new benchmarks. Which is like, how well can we predict, you know, what something smells like or how some smell that we detect in the world, which is just a combination of molecules, is predictive of something that we, that we care about. You know, like one. One other way I like to think about this work is like foundation models for text and for, uh, images. They are approaching, in some cases exceeding human intelligence, certainly mine. In certain regimes where I've been pushing it, I'm like, holy crap, this is smarter than me now. Um, uh, it's human intelligence, and that's because it's trained on human intellectual output. But 99% of species on this planet can only speak with chemistry, right? Thinking of bacteria and fungi and plants and insects, they just only can talk with molecules. And I think that it's really worth adding other kind of alien forms of intelligence to our AI models. And the way to do that is to train it on the intellectual output of those other intellects, which is. That's chemistry. That's the sense that's in the air. They're produced by living things for reasons to talk to each other. And, uh, Terry Tao, who is this famous mathematician, he has this one paragraph in a paper he just wrote where he's reflecting on these new forms of AI and the kind of weird mathematics that they can do. And he says, I think increasingly we need to have a Copernican view of intelligence. Meaning for a long time in astronomy, we had the Earth at the center, but then we had to dislodge it to actually face facts. Um, and Earth's a great place to be, but it's not the center of the universe. Right. Um, and similarly, we've placed human intelligence at the center of. Of the debate about artificial intelligence. And, like, pretty cool to be human. It's awesome.

Speaker C: I love it.

Speaker B: Um, but, like, why should we be at the center? Like, what other forms of intellect are there? And so when he. When he. When I read what he wrote, I've kind of instantly, like, it instantly resonated with me. And I've very much taken on this view that, like, we should have a Copernican view of intelligence. There's so many forms of it, and there's so much, uh, richness out there, um, of things that are intelligent, some of which we might not even recognize as such. Uh, and I think if we're building AI for our species and for the planet going forward, like, we should have those forms of intelligence built in.

Speaker C: Thinking about all the progress that we've been making in, you know, broadly physical AI, you know, robots, autonomous vehicles, all that. Um, and thinking as well about, like, how good our sensors are for, you know, this approximation of visual stimuli. Like, do you have a sense for. I'm imagining you do. Like, where are we with, like, the, you know, olfactory CMOS or whatever that, uh, you know, knows on a chip?

Speaker B: I would say the devices that we use to detect chemistry are truly phenomenal, but they're. They've been very specialized to live in a laboratory. So think about computation in, like, the 1970s big mainframes. Right. Stuck in a back office or like, a data center. Right. So we're still in the data center era of chemical sensing and of olfactory intelligence. Um, so, like, our scent printer is the size of a school bus. It's just not. We're not gonna. That's not gonna leave. That's bolted into the concrete, right?

Speaker C: Yeah.

Speaker B: Um, but eventually all the.

Speaker C: We just need those wheels on the

Speaker B: school bus, not wheeling that out, I promise you. Um, uh, but all this should fit in our phone, right? Over time. And that's the story of technology, is something that works, that's valuable, and now we just work really hard to make it small. So that will come.

Speaker C: The cancer example is a good one. What are other examples of, like, if we had the auxiliary olfactory apparatus chip on the phone, like, what would it let us do that our noses don't do.

Speaker B: So super excited by is just like the creativity that I think would come if we opened that up to everybody.

Speaker C: Right.

Speaker B: So like how, how could we have predicted all the apps on the app store, all the uses of camera phones and all that stuff, like, it's just incredible. Um, I think that there's definitely things we can contemplate. Everything that a dog's nose does. Right.

Speaker C: I'm thinking of the app that you download that tells you who actually dealt it.

Speaker B: Exactly. All right, well the fact that definitely Sam today, um, I've got data to prove it. Um, so there'll definitely be fart apps for sure in the. I just, I, it's human nature. I've told you. I worked at Twitter and we cleaned up. Like I just, I know human nature. It's just like, you know, we've got a fascination with many things including the scatological. But um, I, I, I think that um, anything that a dog's nose can do, and there's like 80 things more or less that you can train a dog to do, like detects, so tracking

Speaker C: for example on your phone.

Speaker B: Exactly.

Speaker C: That's an amazing spoiled produce, spoiled meat.

Speaker B: And these are, these are things you want in your fridge, right? You want to know, can I eat this? Should I eat this? Right. Is it done? Is it cooked? That's useful for a robo kitchens. There's so many things that the human noses and dogs noses can do. None of the examples alone I think are so far that we've seen are huge businesses or necessarily accessible businesses. Smelling cancer is important, but like the business there is really rough. Right. Medical diagnostics are like not a great, unfortunately not a great business to be in. Um, and uh, it's just figuring out how to actually like, like, like not only build it but then make it self sustainable. Like that's one of the harder questions that I, I try to wrestle with and I don't have the answer, frankly.

Speaker C: Do you feel like that the difficulty of that question is, I mean clearly like it's suppressing the development of the technology. But do you think that it's like if the business model was there, we could easily figure out the nose on a chip or is it just really, really hard and therefore the, the bar, the uh, economic bar is really, really high.

Speaker B: I think it's a blend. So if, if the business model was figured out, I think we would have been sprinting at this for the last like four years exclusively as opposed to building a really great business in the fragrance industry which Seems unrelated, but when you really get down to the, you know, the science and the technology, it's super related. Um, but we have devices that you can bring people to or that you can wheel around that can do this.

Speaker C: Right.

Speaker B: Um, it's just a matter of, uh, how do you throw an appropriate amount of resources behind, like scaling that, miniaturizing that. Very possible. Right. I can easily, easily envision a device that's the size of like, what do we call this? Like, I don't know, like an 80s cell phone. So bigger than the cell phone, but like kind of a bigger box. Right. I can imagine a box that's like, you know, about that big. Like, um, I don't know, a small baby's shoe. Shoe box that can smell as well as a human nose. In fact, we're pretty close. Ours is about the size of two shoeboxes. We have a deployable sensor that's as good as our nose as you and me. Um, and it's the size of two shoeboxes. It works. Um, so getting that smaller, very straightforward engineering. Not easy, but like, straightforward. Going smaller than that and like putting it into a, into something the size of this phone and then putting it into a chip the size of this phone, that's going to require some creativity and some work. And I actually, I couldn't tell you exactly how that's going to happen. I can tell you that we have proof. It lives between our eyes and our nose. Right. Like, we're walking proof. But, um, yeah, there's a lot, there's work to be done to actually, like, make that, uh, affordable and pervasive. Yeah,

Speaker C: yeah. Something you said maybe think about, you know, is there some inherent limit in the application of MLAI or the way you've done it? And that like, what's important isn't really mapping to like, you know, single label scents, like, you know, cucumber, cinnamon, whatever, but like mapping to emotion, which is a much richer thing. But certainly if you're like, you must be already doing that. That's what people are trying to get at anyway.

Speaker B: Yeah, we think about that a lot. Yeah, so. So I think m. First of all, there's more labels than just cinnamon and cucumber. Uh, no, no, no. I don't mean like we didn't name enough. I mean, like, there's other ways of labeling. Like these two are really similar or they're dissimilar. And here's a scale. So the richness of sensory information once you start to not use human language and use like numeric Scales, which you can do. It just takes some fine tuning. You actually, you learn a lot about the chemical world when you start to label things more carefully that way.

Speaker C: Um, I mean, in some sense that's the original observation from the embedding work

Speaker B: at Google practice is that proximity actually means perceptual proximity. So things that are nearby on the map smell similar, right? Yeah, exactly.

Speaker C: Um, and so then that like, I don't know how you would do this, but I'm envisioning like jointly embedding emotional, something, uh, emotional valences with uh, you know, olfactory information.

Speaker B: And I, I think that's very, very much possible. Um, I think the challenge. So there, there's a trend now called neurosense, which is largely bullshit, if I may be frank. Um, and you know, people use EEGs. I, I like, I used to use EEGs like in my training and also I know people that run EEG companies and they're very upfront. EEGs are useful for telling if you're asleep or if you're having a seizure. And it's very valuable to quantify sleep and quantify seizures. But it's not going to tell you if you're feeling up or low. Can't do that. There's no signal there. Um, and so there's a lot of crap out there. And so we've not waded into that space yet. Because I have to do it. Right. I can't live with myself if we don't do this. Right.

Speaker C: Right.

Speaker B: But there's so many examples of this being true, of sense really being able to unlock emotions in one way or the other. And um, we will get there and I think we will do it. Right. Um, but we're gonna have to be very careful about how we quantify emotion. Um, uh, which is its own thorny problem. But, uh, this is something I used to work on too. I love the topic.

Speaker C: Um, and uh, I'm imagining the real estate agent going and spraying fresh baked chocolate chip cookie in the open house.

Speaker B: They already do that. Right?

Speaker C: I know. They bake them. Is there an actual can?

Speaker B: Oh yeah, there's candles for sure. Yeah.

Speaker A: Ah.

Speaker B: Um, new car smell is like invented. Right? Um, it's a, it's a thing. Right, Right. But uh, no, a lot of this is also association. Right. So same with the kimchi example. It's like, well, that might be good. Or it might, it might be the hearth. Right. It's like, well, if. Have you ever been in a home where a fire was lit? Like, does it smell like a Burning building or does it smell like comfort and like, you know, marshmallows and hot chocolate? Right, right. It just depends on what you're exposed to. But for sure, like uh, the evidence I think is pretty clear that there are some scents that do things that are positive to your mood and it's almost physiological and can increase your focus or can increase your awareness or reduce your anxiety. Like it's pretty clear to me. But I think we need to be really like aromatherapy. Yeah, I think aromatherapy, uh, has elements of really deep truth in. And same with Ayurveda. Like, I think that these are really old traditions. Um, they're, first of all they're really appealing and they're beautiful to experience. It just smells nice. But like I, in kind of reading the research, there's nothing that's super clear cut. But like I just am of the conviction that there's some kernel of deep truth there. In the same way that like acupuncture now has this Western equivalent called dry needling that's like actually therapeutic. Like acupuncture got there way earlier. They just talk about it differently. It's the same stuff. And maybe some ways there's still things that haven't even been appreciated in the Western adaptations of it. Um, but I think that that correspondence will also happen for Ayurveda and for aromatherapy and then also probably for the um, aromatic aspects of uh, herbal Chinese medicine as well. Plants are medicine and poison factories. They make all the molecules that do good and bad things. And all of our drugs, many of our drugs has some natur, natural origin to them or they were inspired by a natural origin. So like there's, there's gotta be some. I just believe it. I think there's gotta be something there where ascent has, has real like powerful, uh, harnessable impact to uplift our mood and to make us feel better or, or, or perform better or whatever our desire might be that's actually, you know, realizable. And uh, we just have to be, we have to be serious about it and we have to be thorough. That's all.

Speaker C: Well, Alex, it's been great catching up with you and uh, getting the download on olfactory intelligence.

Speaker B: Super fun to talk about it. It's obviously wide ranging, but it's like we're, we're pushing back the frontiers and so like all the things you normally take for granted, like we have to think all the way through. So I, I, I, I'm just thrilled to talk about this stuff all the time because I live it and breathe it, but it's fun to share with you, Sam. Thanks. Thanks for having me on, Sam.