Approximately Correct: An AI Podcast from Amii · 2026-07-21 · 38 min
Key moments - from our scoring
Substance score
66 / 100
Five dimensions, 20 points each
Quinn Lee, a behavioral neuroscientist, and Marlos C. Machado, a reinforcement learning researcher, have discovered that when you train a neural network to navigate through visual observations, it naturally develops representations that look remarkably similar to place cells and grid cells - the actual neurons in the hippocampus and entorhinal cortex that animals use for spatial navigation. Their work was motivated by the Stackenfeld paper on successor representations, which proposed that the brain represents not just where you are but where you might be in the future. Rather than pre-programming their model to produce these cell types, they let the neural network learn an objective function based on eigenvectors derived from the environment's dynamics. The key insight: when they collected actual neural recordings from freely moving mice and compared them to their model's predictions, the model explained the data without having been fit to it. This cross-disciplinary effort suggests that temporal proximity and predictive representation may be fundamental principles underlying how both artificial and biological systems learn to represent space.
Place cells are neurons in the hippocampus that fire when an animal is at a specific location in an environment, while grid cells in the entorhinal cortex fire in a hexagonal or triangular grid-like pattern as the animal moves through space. Together they were discovered by O'Keeffe and the Mosers and won the Nobel Prize in 2014.
The successor representation is a computational principle where the brain represents not just where you are but where you're likely to be in the future given your current state. This principle, from reinforcement learning, can explain how place and grid cells encode spatial information based on predictive expectations rather than just immediate location.
No - they developed the model before Quinn had neural data, and when the model was later compared to actual recordings from freely moving mice, it explained the biological data without ever being fit to it, suggesting the approach captures fundamental principles of spatial representation.
The Stackenfeld paper proposed that the hippocampus learns successor representations for prediction; this new work extends that by showing how to train a neural network end-to-end from raw visual observations to learn eigenvectors that produce place-cell and grid-cell-like representations in a way that matches recorded neurons.
The convergence between what emerges naturally in trained neural networks and what neuroscientists observe in real brains suggests that representing things close in time similarly, while keeping them separable, may be a fundamental principle of how biological intelligence encodes space and time.
Our reviewer’s read on each dimension, with quotes from the episode.
The episode delivers substantive insights about representation learning, the successor representation framework, and how neural networks spontaneously develop structures analogous to biological place and grid cells. However, much of the early conversation involves introductory material and friendly rapport-building rather than dense insight delivery. The core technical contributions - temporal proximity principles, eigenvector learning, emergent hierarchical spatial representations - are explained but often at a conceptual level without deep mechanistic detail.
The general principle is that we want to represent things that happen close in time in a similar way, and while we still being able to separate them.
And because of that we said, oh wait, now there's an opportunity because I really like that line of work...Now we could implicitly be doing this with this method.
The work itself is original - training neural networks to learn grid-like representations from sensory observations without explicit spatial input is a novel approach that bridges AI and neuroscience. However, the podcast discussion doesn't venture far beyond explaining existing literature (Stackenfeld, Gershman) and the paper's findings. The hosts don't push toward contrarian positions or unexplored implications; the framing is largely confirmatory rather than challenging.
it's not that I had done any of that, but it's just like, I think that what I did is real...it's actually invented something that actually happens to some extent.
They thought that place cells came from grid cells...But the directionality that was spoken about in this paper seemed flipped around.
Marlos Machado is an AI reinforcement learning researcher and recently-hired AI professor with demonstrated track record in representation learning; Quinn Lee is a trained behavioral neuroscientist and neurophysiologist who records from neurons in behaving animals and has genuine domain expertise. Both are active practitioners who have done the work at scale rather than commentators. However, the episode is academic research discussion rather than industry operator perspective, which limits some B2B relevance.
I work, uh, until this work, let's say, I would say that I was working with very boring computational reinforcement learning...with a very big emphasis on relatively big problems.
So I am trained as a behavioral neuroscientist and, uh, neurophysiologist...I do things like record from neurons when animals are learning and remembering to perform things like navigation tasks.
The episode references specific papers (Stackenfeld, Gershman 2018, Tony Zador neuroai work) and concrete neural structures (hippocampus, medial entorhinal cortex, place cells, grid cells) but largely avoids quantitative metrics, experimental parameters, or numerical results. The description of methods is conceptual rather than detailed. No specific success rates, accuracy numbers, or comparative performance data are provided.
the medial temporal lobe is really important for this. So specifically, areas like the hippocampus, um, which for a human, if you hold your fingers above your ears, they'd be about, uh, an inch on both sides.
The discovery of grid cells and place cells won the Nobel Prize for O' Keeffe and the Mosers in. I think it was 2014.
The host (Alana Fish) asks reasonable clarifying questions and demonstrates genuine enthusiasm, but rarely pushes back or probe deeper on claims. Follow-ups tend to be soft and confirmatory (e.g., 'Cool' or requests to explain rather than challenge). There's little productive disagreement or probing of limitations. The conversation is collaborative and pleasant but lacks the sharpness needed for extracting maximum insight.
And you say that in your blog post like that?
Did any of that feel like it rings true?
Computed from the transcript - who did the talking, and the words that came up most.
How do we know where we are, and where we are going? While we use our eyes to take in the information, it is our brains that are doing the heavy computing as we navigate the world. Now, new research using machine learning models is perhaps giving us a glimpse into what is going on inside our minds, and how the cells needed to navigate the world might form. On this episode of Approximately Correct, we are joined by two Amii Fellows and Canada CIFAR AI Chairs - Marlos C. Machado and Quinn Lee - who combined their expertise in machine learning and neuroscience to better understand the mysteries of navigation.
Transcribed and scored by The B2B Podcast Index.
Speaker A: I don't know if I would characterize it as my unhappiness is what led me to do this project. Uh, I think that this has been something that, like, I do write a lot of papers out of spite. This was not one of those.
Speaker B: Hey, welcome back to Approximately Correct. I'm Alana Fish.
Speaker C: And I'm Scott Lilwall.
Speaker B: And this month on the podcast, we have brains.
Speaker C: You're so excited for this.
Speaker B: I'm so excited, yeah.
Speaker C: Why are you so excited?
Speaker B: Uh, because it's a thing I work on, too. I work on brains, specifically language in the brain. But this time the podcast is actually about navigation and how the brain works with navigation.
Speaker C: That's right. We talked to Quin Lee and Marlos Machado. They are Amy researchers, Canada cifar, AI chairs. And Marlos works a lot in the AI reinforcement learning area. And Quinn is in neuroscience.
Speaker B: Yeah. So he records from actual brain brains, which is so cool, like capital N neuroscientists. And he actually records place cells and grid cells inside of, uh, performing mice actually moving around in space. And place cells and grid cells are the kinds of cells in your brain that help you figure out where you are in space and as you move around, keep track of that. Right.
Speaker C: So if I'm moving around in a room, those are what are kind of telling where I'm in the room. Don't hit a wall, that kind of
Speaker B: stuff and, like, how far you've traveled in.
Speaker C: Yeah.
Speaker B: So it's so neat that, um, they were able to do this project where they trained up a neural network with reinforcement learning sort of around it, and they found that the things that, uh, the neurons that emerged recorded something like, uh, place cells and grid cells. Right.
Speaker C: And so these weren't cells that they had, like, designed to do.
Speaker B: They didn't pre program. Yeah, they just sort of fell out just the way they trained it up. So it's not. It's not proof that this is how brains work, but it is a way that if you set up a system that looks a little bit like this, you get things that look like brain good cells and place cells, which is pretty cool.
Speaker C: Yeah. So it's interesting how we can study the way that artificial intelligence works to maybe understand a bit more about how our own brains work.
Speaker B: Yep. This is a great example of how that synergy can go both ways.
Speaker C: Yeah. So this was an interview that, uh, you took the lead on. I wasn't, I wasn't in there because you, uh, are the brain exper. Uh, listening back on it. I mean, the work that they're doing is just absolutely fascinating. So we're glad to be able to share them with you. So this is our conversation with Quinn and Marlos.
Speaker B: Welcome to approximately Correct M. Marlos and Quinn. Thank you.
Speaker D: Thank you for having us.
Speaker A: Thank you so much. Very excited that you'll be here.
Speaker B: This is a work that I have been excited to hear about ever since Quinn interviewed at the U of A. Um, because it was such a wonderful sort of overlap of interests I knew that you guys had. And so I'm so excited that you're working together. Uh, so maybe just to get the audience up to speed, can you tell us, um, what general areas you both work in?
Speaker A: Uh, yeah, so I work, uh, until this work, let's say, I would say that I was working with very boring computational reinforcement learning. Very classical, traditional. The question of if we have virtual agents in a world, how do they learn to achieve goals, but by trial and error, uh, and with a very big emphasis on relatively big problems. So a big chunk of that has to be about representation, learning, neural networks, and, uh, how those things interact with sequential decision making problems.
Speaker B: And so you're saying a lot of words and none of them are navigation. And I know what we're going to talk about today is mostly navigation. So is navigation a part of that?
Speaker A: We'll get there. I think that, uh, I don't think that navigation is special. Uh, I think that from a reinforcement learning perspective, uh, we can talk about navigation in physical spaces like we did, uh, but we can also talk about navigation in imaginary spaces, uh, abstract spaces. And in that sense, it's all about navigation. Yeah.
Speaker B: Okay, cool. And Quinn, what's your area?
Speaker D: So I am trained as a behavioral neuroscientist and, uh, neurophysiologist. And I also do some, some computational modeling of how neurons do what they do in brains and how animals do what they do in real environments. Um, so I mostly work with animal models. I don't do human, uh, research to date really. But, um, yes, I do things like record from neurons when animals are learning and remembering to perform things like navigation tasks, but other kinds of memory tasks usually as well.
Speaker B: Yeah. So before we get too far down this road, I just want to get a high level sense of what is the paper about?
Speaker D: The paper is about how do you know where you are and where you're going? Um, and how do you know that from observations you can make with your two eyeballs? Maybe that's what I would say.
Speaker A: Yeah. And I think that maybe I want to say why. I think it matters. And it's because when we talk about intelligence, one of the things that terms that are thrown around is representation learning, which is, how do we learn to represent where we are? I mean, there's a million answers to this question. And every time that a new paper is written, uh, it's like, oh, we did something. And I think that what this is showing is that maybe it's additional evidence that trying to do the things that we did is not a bad idea. M We're seeing this in biological beings. It seems to be a good explanation. So maybe it's a good general principle first to think about how we go about representing where we are in the world.
Speaker B: And what's a general principle?
Speaker A: The general principle is that we want to represent things that happen close in time in a similar way, and while we still being able to separate them. And there are technical ways of talking about what the separation means. But, uh, the way that we do this in the paper and this line of work is to talk about things being orthogonal to each other so that they don't interact too much, but then we represent this. Basically, this temporal proximity seems to be a good underlying principle, uh, on how we should structure how we represent where we are. And in general, in a very biased perspective. That's what my lab has been focusing on, uh, for the last couple of years.
Speaker B: And if you were going to say why it's important, what would you throw in?
Speaker D: Um, one big reason for me that it's important is, um, I'm interested in what kind of frameworks can explain things that we observe in brains to try and say something about what the brain is learning to do. Um, and so stuff that approximates things that we can observe in real brains is something that I'm just motivated by because I think brains are interesting, um, and I want to know how they work. So, um, what matters to me is coming up with good descriptions of something that you can observe in brains that generalizes well to many cases, um, and really measuring that, using the brain as your measuring stick, um, for how good an idea is.
Speaker B: Right. So we're training a model that is learning from the experience, the visual experience it's having, and it's creating sort of a set of representations in the machine learning sense that look like something that the brain is doing.
Speaker D: Yeah.
Speaker B: And that wasn't on purpose.
Speaker D: It just really happened. Yeah. And I think that's important for me because I want to know how brains work. And my hope is that, uh, that will be useful for the machine intelligence case as well.
Speaker B: So how does the brain do sort of tracking of stuff in order to help us do navigation?
Speaker D: Um, so we think that the medial temporal lobe is really important for this. So specifically, areas like the hippocampus, um, which for a human, if you hold your fingers above your ears, they'd be about, uh, an inch on both sides. You have hippocampi. Most people don't know that because they always say the singular version. But, um, you know, inch from your finger on both sides above your ears. Um, and in that brain area, there's lots of cells that seem to respond to you standing in a specific location in other areas nearby, maybe facing in a particular direction. Um, so there's all these, like, spatially tuned neurons, spatially responsive neurons in this part of the brain. Um, and they also seem to be really important for memory. Um, so that's why I often tell people I study learning and memory and navigation.
Speaker B: Um, and before we get too far, I want to make sure we have described what a place cell is.
Speaker D: In a good cell place cells are classically thought of as together representing what we call a cognitive map. So if an animal was navigating through an environment, every time it traverses a specific location in that environment, that neuron would become active.
Speaker B: There's like, sort of like one neuron per place.
Speaker D: Yeah. The reality is that they're actually much messier, um, than that. So one neuron can have multiple place fields. So locations that it would respond to. This has kind of interesting statistics to how it happens. But, um, that's the core idea, is that one neuron represents a place in a specific environment. Yeah.
Speaker B: And then grid cells.
Speaker D: So grid cells, um, they're maybe more exciting for a lot of systems neuroscientists. Uh, but, um, they were discovered by the Mosers, um, in the early 2000s. And you find them not in the hippocampus, but, uh, in the medial entorhinal cortex. So this is just adjacent to the hippocampus and, um, the sort of outside and back of the temporal lobe. Um, and so grid cells respond in this grid like pattern. So as an animal is running through a space, it would look like points on a triangular grid.
Speaker B: So it would fire at points as a m. Mouse or animal moves through space.
Speaker D: Yeah, exactly. Yeah. And, um, they form this hexagonal or actually triangular, uh, grid like pattern in their firing properties. So together, the discovery of grid cells and place cells won the Nobel Prize for O' Keeffe and the Mosers in. I think it was 2014. Yeah.
Speaker B: Yeah. Cool. And I mean, it's hard to describe those pictures, but I think if you just Google place cells and grid skills, you'll end up with some really pretty pictures that make it pretty clear. Mhm. So what about that, about that makes you excited, Marlos?
Speaker A: Uh, oh, so much. I, uh, think that part of that has to do with uh, my motivation to the research. You know, like, I think that uh, as a, uh, researcher we can have all sorts of motivations. So some people want to build systems that are going to be useful to other people. Some people want to prove theorem so that we understand something about these algorithms. I, uh, am maybe, maybe in an arrogant way. I am very interested in trying to use that to understand intelligence. Uh, and of course we have those biological beings are the ultimate proof of intelligence that we have. So very inspired by some of the early success that we had in reinforcement learning as a field, that some of those models were eventually used to explain natural phenomena. The most famous one, of course, temporal defense learning and dopamine. Um, I, I've always been like, I always been excited about like, well, are some of the things that we are doing actually meaningful beyond simply like, oh, I can, I can play this video game or I can do something like that. So it starts like that. And, but now, of course I don't have any training from a neuroscience perspective. I'm actually very intimidated to be hearing from tough you and Quinn. Uh, but then you can all correct me when I say wrong things. Uh, but the point is that, um, over the last decade or so it became more common to see a lot of that overlap between researching neural networks, computational reinforcement learning, and some of those modeling, uh, systems that we get both from bigger availability of data, uh, and also better, more general methods in computer science. And suddenly someone is publishing a paper that say that, hey, look, this thing like seems to happen to some extent in the natural world. And I say that this was, I think that the moment that I read that paper, it was one of the first times that I got really, really excited about my own research because just like, it's not that I had done any of that, but it's just like, I think that what I did is real. You know, like, it's like it's actually invented something that actually happens to some extent.
Speaker B: Huh.
Speaker A: So I, I like to use that example as one of the papers that I really wish I had written myself.
Speaker B: And you say that in your blog post like that?
Speaker A: Yes, I think it's a, it was very inspiring and I think it's uh, I really like Some of the work that, that that group has been doing. And, and then because of that, um, it has always been at the back of my mind. But like, how would I even approach this? Right? Like, and so these questions and these things, they have been around from different angles that have been on the top of my mind. And then I think that the moment that I then decide to, like, okay, I get to be a full time professor. I'm going to have a research group. Uh, one of the many benefits of our job is that I think that it allow us to actually ask long term questions. And then I was like, you know what? I'm going to do it. Uh, and then we started working on this, uh, somewhat naively at first, uh, because it's like, okay, I'm the one embodying a neuroscience in the group. And like, oh, I've read this paper, it says this and so on. Uh, but then of course, serendipitously, um, I get to see that Quinn was interviewing and I read the abstracts like, oh, I really like that. So I found a way, uh, of meeting with him while he was interviewing. For the people who don't know, because he's in psychology, it's not that common that people from other departments are going to actually do this. Like, no, no, I want to have breakfast with him. Um, and, and I remember that leaving like, uh, Lee, after leaving his talk, I was just like, when are we making him an offer? Because he does everything I care about.
Speaker B: He emailed me like within a day.
Speaker A: I was like, I was like, Marlowe's,
Speaker B: uh, cool your desk.
Speaker A: Like, when are we making him an offer? Because everything that I care about is he does, but he does in neuroscience and why I'm doing this in the computational side. So, yeah, so I think that there has, like these research questions have always been motivating me, but like from an expectator side for a very long time. And I think that eventually I decided to bite the bullet and try to do it. And m. Things naturally happened.
Speaker B: And so maybe you could just explain the high level ideas that came out of. I say Stockenfeld, but how is it.
Speaker D: I think it's Stackenfeld.
Speaker B: Stackenfeldenfeld.
Speaker A: We should have asked her first.
Speaker D: Should have asked him.
Speaker B: We can. Yes.
Speaker A: Um, yeah. So, uh, I think that, um, so the idea of that, uh, paper, um, and maybe, uh, Quinn should be the one answering that question. But, uh, the idea of that paper is that we can look at, if we look at the recordings that people have made of, uh, specific locations in the brain like the hippocampus, the entorhinal cortex. Um, there's this question about, well, is there a computational model? Is there a set of equations? Is there something that will explain, uh, the activations that we see and the emergence. Yes. Uh, and then what uh, they have shown in that paper is that if you were to look at the hippocampus as a predictive machine, meaning that the hippocampus is trying to make predictions about the future, uh, the prediction of where you're going to be in the future, uh, which is something that it's called the success representation is a very good computational, um, model that explains the activations that we have for place cells.
Speaker B: Yeah. It's funny that you said Quinn should explain that because to me that's a paper about reinforcement learning. Yeah. So do you have thoughts on like, what came out of that paper? What was important?
Speaker D: Um, yeah. So I actually read that paper after it was suggested to me by my friend, ah, Alexandra Kynith, um, to check it out because she said that um, she had not seen another single paper in neuroscience at that time that explained the breadth of observations in navigation and learning and memory literature that this paper had to offer. So I was super excited to learn about this. Then I opened up the paper and um, couldn't read any of the notation when it came to the reinforcement learning stuff. So I had to go read Rich's book and um, get the background to actually understand the paper and eventually start programming that stuff myself. But um, yeah, so I was initially excited because it was explaining as well with another paper that Sam, uh, Gershman wrote, I think in 2018, um, separately there was just this breadth of observations from animal context learning to human visual transition matrix like representation learning to um, rodent place cells that this paper had something to say about. And I had never seen a paper in that area of learning and memory and navigation that brought together that many concepts and had something to explain, uh, with a single learning objective, um, or theory. So that was really exciting to me. Um, the thing that the paper does really well is it kind of shows if you measure something as similarly as you can, um, with a reinforcement learning agent to what you observe in this task space. It's like, look, they look similar. Um, what I thought was missing was we have the ability now to um, record lots and lots of neurons in the brain simultaneously. What happens if you produce a bunch of predictions about how the neurons should behave or what they should do, uh, and you compare that directly to what you'd observe in an analogous Task. Um, so that was sort of the next step that we wanted to take.
Speaker B: What is the successor representation?
Speaker A: So the success representation is going to say, like, okay, let's say that I am. The question is, how do we represent where we are? Right. We have to represent this, uh, for, like, in a computer. So the way we generally represent this is going to be a vector, a sequence of numbers. So, for example, imagine that I give you an image. What am I going to give you? Well, I'm going to literally give you the numerical values of each one of the colors in that image. And you can say, this is what I'm going to consider to be where I am, my state. Now, the thing about reinforcement, learning about how we behave is that the future matters. Right. Like, so the future matters in a way that maybe I shouldn't be representing something based on only where I am. I should be representing something where I'm going to be in the future.
Speaker B: Yeah. Given where I am, where could I be?
Speaker A: Exactly. It's as if the brain first got place cells, and from the place cells, the brain got grid cells.
Speaker D: Yeah. And so the reason why neuroscientists don't really like this is because you. If you look at the anatomy of how these systems are wired together, the entorhinal cortex, where you find grid cells, projects into the hippocampus, where you find place cells. But the hippocampus also projects back to the entorhinal cortex as a loop. Um, but in the early 2000s around, when grid cells were discovered, we thought that place cells came from grid cells, that you could kind of add up these gritty inputs and then produce something that looks like a place cell? The directionality that was spoken about in this paper seemed flipped around.
Speaker B: I had heard that also that, like, the. The Stackenfeld paper was backwards.
Speaker D: Yeah, that was a. A point of contention with, um, the amount of explanation this. This idea had to offer to neuroscience. Because the anatomy was flipped.
Speaker B: Right.
Speaker D: Um, but maybe this wouldn't be an issue if you're trying to learn grid cells. M. First and foremost.
Speaker B: Right.
Speaker D: So, um, which you could do.
Speaker A: So in that sense, uh, motivated by that, we're asking the question, like, how can we learn? Uh, how can we learn? Like, how can we design an objective so that I can have a neural network and I can say, hey, this is what you should do, and I can train a neural network, that the neural network would receive photons, would, uh, receive, uh, audio. I don't really care what it would receive. Neural networks can receive all sorts of things. But out of doing this, it would then be able to spit out these eigenvectors. So like, can I train it so that it receives observations and then it learns to estimate these eigenvectors? And we were motivated by the computational reinforcement learning questions we had. Uh, but then once, uh, by the time my master student, uh, with Michael Bolling, Diego Gomez and all his, uh, co, supervised by Mike and I as a PhD student, but by the time that we uh, were doing this, Diego came and said, well, I think I actually have a way of doing this. Uh, I can improve upon some previous methods that were out there and there is a way that we can do this. So now finally we had a very robust method that would receive observations, images and so on and would eventually spit out the eigenvectors. And because of that we said, oh wait, now there's an opportunity because I really like that line of work. But that line of work wouldn't do things in high dimensional spaces because they're not trying to go from this abstract high dimensional space to something very concrete. Now we could implicitly be doing this with this method. And beyond that, the objective that this neural network is spitting out is are going to be the eigenvectors. So it's also learning the first thing first.
Speaker B: Right.
Speaker A: Instead of like this, this, this dependency. So when we saw this, like, oh, there is actually an opportunity here that knowing this literature, maybe we can revisit some of that and ask the question, is this a more general way of doing this? And it ended up leading to even more than we, than we initially set up. Church.
Speaker B: Okay, so in this work you're comparing what you see in a neural network to neurons. But, and so you must have been recording from neurons. Where did data come from?
Speaker D: Yeah, so I actually recorded, um, that data set with my friend and collaborator, Alexandra Kyneth. Um, we were both postdocs in Mark Brandon's lab at McGill at the time. So we recorded these data with mini microscopes in freely navigating animals together.
Speaker B: Oh, awesome. Cool.
Speaker A: And one thing that I want to say that we don't get to write in a paper is that the data came after.
Speaker B: Right.
Speaker A: So we developed the model, uh, and as we were describing, like, oh, that matches what other people have published. But then when Queen joined the project, he had the data, the model was developed, and it was very sobering to see that the model, that, the model that had was actually explaining that data.
Speaker B: Right. So it wasn't, we were trying to, we weren't trying to fit this model to the Data.
Speaker A: Yes. I wish we had recorded that somehow before. Yeah.
Speaker B: Right. So when Marlo talked to you about this model, what were you, what were your thoughts?
Speaker D: Well, I was really excited that um, the point was to learn grids, um, and not necessarily to learn place cells first. Um, and I thought it was really, um, thrilling to try and learn grids from a, ah, sensory observation, um, rather than. It feels a bit like cheating to say top down, the input is going to be where I'm currently located. Um, and it felt much closer to the real deal to say I'm just going to give you a sensory observation, you're looking some direction in the environment, you're going to wander around and then I want you to learn grid like representations, um, from this, so to learn the eigenvectors from that directly. So that's what he told me, um, they were doing and I hadn't seen anything like that yet where that was the objective and it was being learned from a high dimensional, you know, naturally, like sort of biologically plausible observations. So, um, that solved some of the problems that I had and others had. I think as a neuroscientist, um, looking at the earlier versions of the paper which provided the foundation for this, um, but I felt were kind of missing. So it felt like real progress. And we could give something that looked like what a real animal might see doing the exact same experiment, um, as an artificial agent.
Speaker B: Mhm.
Speaker D: So we could get closer to matching the types of experiences that an animal might have or a human observer might have, uh, in m. The model case.
Speaker B: Right. And so when you were looking inside the model, what did you find?
Speaker D: Yeah, so the next really exciting thing that happened was although, um, Diego and Marlos and Mike were trying to learn grids, um, first and foremost there's a few layers in this, uh, multi layer, um, perceptron. And if you look in the spatial activation of layers upstream of where you get grids output, um, just before the layer where you get grid cells, um, you observe things that look like place cells. Um, and just before that, uh, what was really interesting to me specifically was you observe other kinds of spatial representations, um, that you're not trying to learn with the learning objective necessarily or directly. Uh, they just emerge naturally. Naturally isn't the best word to use
Speaker B: for a computer, but whatever.
Speaker D: Yeah, but the other interesting detail in this is you get things that look like representations of borders and like how close are you to boundaries in the environment. And some of the modeling that I had done and others had done, like uh, William Dacoty and Caswell Berry show that you can get things that look a lot like place cells if you use these other kinds of spatially responsive neurons that just kind of fell out of the model one step upstream of where these place cells emerged. Um, so I thought that that was incredible, um, that this just kind of fell out of trying to learn grids and doing it from this really high dimensional, biologically plausible sensory observation.
Speaker A: But it's interesting, like even if you look at just the output of the neuronatrical, we get more than grid cells, some of the outputs. There are other cells that, from what I understand, are also thought to be in the medial entorhinal cortex. And they also emerged and they emerge in order.
Speaker B: Right? That's another pivotal.
Speaker D: Yeah, yeah.
Speaker B: So I shouldn't say they emerge in order, but they appear in order in the stream of the neural network.
Speaker D: Yeah, yeah, they appear in a specific sequence, uh, in the network. Whether that's the sequence in the brain is a matter of heated debate.
Speaker A: But I think that one thing that it's undisputable is that this is a mathematical, uh, there is a mathematical, uh, proof. Now let's put this way, or like a proof of existence that composing those cells, the composition of those cells is enough to get the behavior. Now is this what the brain is doing? I don't know. But there is a way. Like, uh, is it the only way? No, but it's sufficient if you have, it's a sufficiency proof. If you were to have those, you could compose them and their composition allows us to get that output.
Speaker B: Did any of that feel like it rings true?
Speaker D: M. Well, I think the brain probably does predictive coding, like things that happen close in time should be shaped by my expectations about what I expect to happen next, uh, based on what's happening right now. So I, um, think that there's some biological plausibility to that. Um, that's part of the equation. Um, and there's people that have been modeling things just in that type of way for a long time. I guess that goes all the way back to temporal difference learning and how dopamine works in the brain. But also people are modeling in that type of way in sensory prediction errors and things like that too. Um, yeah, I think there's a semblance in terms of the brain tries to make predictions about what's going to happen, uh, in the future. So, um, it makes sense from an efficient coding standpoint in the brain. Um, it makes sense for the temporal contiguity of experience and behavior so um, yeah, I think that it has some semblance to how we might naturally do things.
Speaker B: Yeah. Cool. And so what's coming up? Are you guys going to keep working together? Do you have future plans?
Speaker A: I mean that was my whole pitch. I was like, can we hire this guy so I can work together with him for the next decades? Um, we've been talking about lots of ideas about not only how we exploit this expertise from learning this type of representations, what are the limitations that we have right now? Uh, things like how do we get to be, how do we make this go condition, for example, how do we incorporate now rewards into this process? Um, and how do we go about dealing with different, um, for example, what about partial observability? Do we want to introduce recurrence? So these are some of the things that seem like very natural extensions of uh, what we have been doing so far. But then we also have been talking about ideas uh, that are not necessarily related to this but are still connected. And maybe Quinn should talk a little bit about some of those.
Speaker D: Yeah, uh, I mean there's definitely ah, ambition to continue working on this topic. Just things that naturally spin off of it. Um, so I'm interested in how also the hippocampus might represent things that aren't space per se, but any sort of abstract task space. And I think this one thing that's exciting for me about this particular modeling case is that you can take any input you want. It doesn't need to be visual observations. It could be some sound or other type of stimulus that maybe informs the animal or agent about what's happening in the world and what it should pay attention to. In my new research group at um, the University of Alberta in psychology, we're trying to build tasks right now where animals learn to navigate to goals in a very similar environment or similar task space to um, what was described in this paper. Um, and then sort of iterations and variations of that. So as the goal sensitive versions of this um, model are being built, we're trying to have tasks kind of at the ready where we've already collected data in real behaving organisms. We can compare the behavior even of artificial agents trying to achieve a goal with animals trying to achieve a goal so that it's not just about the representation of the space in the brain, but also what do you do with that to achieve something uh, in the environment. So there's definitely lots of directions that we uh, want to explore. The issues may be constraint.
Speaker B: I think this is such a lovely example of how AI can be combined with another field and how interdisciplinary science can be the most fun. And so do you have anything to say about what that interaction was like, this AI plus X idea?
Speaker A: Uh, I mean it has been extremely um, pleasant, uh, for me in the sense of like, I don't know. I wrote a blog post about this project and one of the terms that I used maybe to upset some of my friends is that I felt like a real scientist doing this. And I think that what happens in computer science, right, like is that we can have philosophical debates, it's math discovered, is math invented, and things like that. But I think that there's something sobering about uh, you being grounded by a natural phenomena. You know, like, and when we get to develop something and then we go talk and we see like, oh wait, this is, this is um, something that actually happens and then we can use some of the models. I think that it's one, of course it's validating for us. It's like, oh yes, uh, we keep telling ourselves that we're doing something useful, but it's actually useful. You know, like eventually we get to show this. Um, and I think that there's so much learning. I think that uh, getting myself out of the comfort zone and then being like, oh, how do other people do things? Oh what a surprise. We don't know how to do everything or the way that we do things are not optimal and then maybe someone else has something, a different take. So I think that it's very valuable for both learning, for both impact. I think that, I do believe that uh, science is just like, it's not a zero sum game, right? So in a sense one plus one can be more than two when you actually get together those expertises. And I think that this is something that uh, I think that this worked out in this paper. We are hoping to do more, but I think that in general we are seeing this happening more and more around us.
Speaker B: And you've been parachuted into Amy and you're surrounded by AI. What are the benefits? Do you see?
Speaker D: Um, I think that uh, for me a lot of the benefits have been um, starting to develop another language. Um, and this is alluded to in some neuroai papers, there was a big one with lots of co authors that Tony Zader, um, wrote a few years ago. And a big part of I um, think building intelligent machines that are intelligent in similar ways to brains, um, is having language and ideas that span both of these disciplines. So being surrounded by a group of highly collaborative um, and open computing scientists, being a Neuroscientist. I, um, get to learn to speak that language, um, and they also get to learn to speak mine, um, because often we're saying the same thing just with different terms. So that's been really enjoyable as an experience for me. Um, and we care about a lot of similar fundamental ideas about what, what is required for intelligence. Um, and I think the intellectual curiosity amongst us is highly overlapping. So, um, it's been really fun for me to participate in this process of translation, um, but also with common goal setting, um, and having this crosstalk between the disciplines, we have to have just these conversations for that to happen and regular interaction. And I think this environment has been really fostering that successfully.
Speaker B: So I love that. This has been so much fun. Seriously. Thank you so much for joining us, Marlos and Quinn.
Speaker D: It's been a pleasure.
Speaker A: Thank you so much for having us.
Speaker C: So that was Marlos and Quinn. That's just such a great example of what we like to call AI X. Like, AI being used with another scientific discipline and some really amazing advances, and always a really cool story.
Speaker B: Yeah, I love AI plus X. I also work in AI plus X because I do psychology and a little bit of neuroscience also. And we have so much AI plus X at Amy now. We have physicists, astrophysicists, chemists. Chemists. Doctors, of course.
Speaker A: Yeah.
Speaker B: And, uh, material science biologists.
Speaker A: Yes.
Speaker B: So we've got tons more episodes on AI X. This will not be the last one.
Speaker C: Yeah.
Speaker B: Yeah.
Speaker C: And if you want to stay up to date on all of those, the best way to do it would be to subscribe to Approximately Correct. Uh, you can find the podcasts on Spotify or Apple Podcasts, and we also put a YouTube video out for every episode, so you can find us there.
Speaker B: Yeah. So if you're watching us right now on YouTube, you can jump down in the comments and leave us a comment about maybe what kind of AI X you would want to hear about.
Speaker C: Yeah. Uh, we always love to hear what people are interested in, and it gives us an idea of where to explore next. So please do.
Speaker B: We'd love to hear from you.
Speaker C: Yeah.
Speaker B: And so with that, uh, I'm Alana Fish.
Speaker C: I'm Scott Lilwall.
Speaker B: And this has been Approximately Correct.
Speaker A: Sam.
Other episodes covering the same guests and topics, from across The B2B Podcast Index.
Maximizing GPU Utilization: Heterogeneous Pipelines with Ray and KubernetesData Engineering Podcast · on Reinforcement learning88 / 100
Shipping Practical AI: How to Build Real-World ML for 2D Drawings (with Marina Petzel)ShipTalk · on Reinforcement learning88 / 100
The Benchmark With No Instructions - ARC-AGI-3 (winning team!)Machine Learning Street Talk · on Reinforcement learning85 / 100
AI Meets Data Infrastructure: Cost, Performance, and What’s Coming Next - with Barzan MozafariWhat's New In Data · on Reinforcement learning84 / 100
AI Pricing & Demand Forecasting Today: Insights from a Decade in Applied AI with Jayadeep ShitolePricing Heroes · on Reinforcement learning81 / 100
Crafting Data Solutions: Shrinking Pie and Leveraging Insights for Optimal Data Learning - ML 176Adventures in Machine Learning · on Reinforcement learning81 / 100
AI and Economics: Twins Separated at Birth? with Kevin Leyton-Brown (Live from Upper Bound 2026)79 / 100Behind the Scenes of Upper Bound With Stephanie Enders | Approximately Correct Podcast47 / 100Why AI Needs to Stay Weird with Kate Compton | Approximately Correct Podcast75 / 100Should AI be your Study Buddy? with Kowalchuk | Approximately Correct Podcast77 / 100Amii Rewind: Can AI Predict Survival? With Russ Greiner - Dec. 2024