Sebastian Raschka: Learning ML, Responsible AI, AGI | Learning from Machine Learning #4

Speaker A: How did the best machine learning practitioners get involved in the field? What challenges have they faced? What has helped them flourish? Let's ask them. Welcome to Learning from Machine Learning. I'm your host, Seth Levine. Welcome to Learning from Machine Learning, uh, on m this episode. It's incredible to have Sebastian Raschke here, lead AI educator at Lightning, former statistics professor at University of Wisconsin, the author of Python Machine Learning Book and Machine Learning with Pytorch and Scikit Learn and overall just an amazing force making AI and deep learning more accessible and teaching people how to use AI and deep learning at scale. Uh, welcome.

Speaker B: Yeah, uh, thank you for the kind invitation to be here. Very exciting, Seth, uh, to have me here on your podcast. I think it's uh, relatively new podcast. I'm especially honored to be one of the first people on this podcast. So I hope we will have a lot of fun, hopefully a lot of stuff to talk about because yeah, we work both in machine learning and have a lot of overlapping interests.

Speaker A: It's awesome to have you here to uh, get things kicked off. Do you want to give us a little bit of a career background? Uh, your journey? How did you get to where you are today in the machine learning field?

Speaker B: Yeah, that's um. How far do you want me to go back when I.

Speaker A: So maybe as far back as you want.

Speaker B: Yeah. So I think what uh, how I basically started was um, during. It was during my undergrad I got into um, yes, statistics, r. Python programming eventually. And um, I've always been a tinkerer. I kind of, I must say I always liked the coding more let's say than the math. Nonetheless, I always was like somewhere in between the two. So I was never really uh, let's say a software engine engineer. But I was also never really a mathematician, um, if that makes sense. So I was like uh, more like an applied researcher or scientist. Um, and yeah, my background is essentially during my PhD, I worked on computational computational biology problems where um, it was usually centered around some prediction task, let's say, uh, virtual screening where we were interested in finding small molecules that inhibit some biological response, let's say related to diseases or other, um, types of biological, um, say systems. Uh, in the same way we were also modeling uh, protein structures and these types of things. And yeah, we did a. Well we had to do a lot of coding coming up with rules to let's say classify things. And um, there was this class when I was in grad school that is, it is more than 10 years ago now I think. So it was called Statistical Pattern, ah recognition. And I was, my advisor back then. She recommended me taking that course because well it was something where you can maybe automate this prediction type of problem that we had instead of hand coding things, going through things. And I must say I wrote a lot of brute force for loops in Python to optimize things using or even using very simple um optimization libraries. And that was uh, kind of like eye opening. So that uh, course was mostly focused on Bayesian methods, um, let's say base optimal classifiers and then naive bayes to make that uh, more feasible and these types of things. But that kind of like introduced me I would say to machine learning. Like the concept of, I mean it was more statistical learning but the concept of learning from data essentially. And then I took another class, data uh, mining. And that was also the time where Andrew Engs class was launched on Coursera, the machine learning class. And I got totally hooked. Uh, it was I mean in two ways, revolutionary at first, like working with data, um, like letting computers learn from data automatically. That was super fascinating. At the same time Coursera as an online uh learning platform was also super cool as a student. Like wow, I can do this at home. I can uh, I mean I like going to classes in person. But this was just like also very revolutionary where you had everything at home. You could take the class whenever you wanted and it was just addicting to take that class. Andrew Ng was such a good teacher. Um, I got really hooked and yeah from there eventually I joined the Statistics Department at UW Madison in 2018 where I uh, focused on machine learning and deep learning research. And then uh, in 2022 I joined Lightning AI. Um, I liked my time as an assistant profess but uh, things change in machine learning where um, the problems become more challenging and bigger. If you are let's say a small team it's more challenging to keep up um with let's say technology and resources. And like I mentioned before, I'm not let's say the typical mathematician type of person. So I like computing. So I looking for an opportunity where let's say I have a team of people and infrastructure to work on different type of problems and also like how to extend uh my educational let's say passion from just in classroom teaching to also maybe developing an online course which is what I'm for example among other things doing right now. So yeah I joined Lightning AI, long story short. Um, and yeah, and yeah so since then I've been really happily uh working there like my time at UW Madison as well. But uh, yeah, you can't do everything all at once. I guess so.

Speaker A: Yeah. Yeah. Um, no, that's a great journey. I too was captivated by Andrew Ng's Coursera class as I think a lot of people in machine learning, uh, so having the experience being a, ah, professor and now working in industry, how would you compare, Working, uh, in academia compared to industry?

Speaker B: Yeah, so I would say, um, I wouldn't say one is necessarily better than the other, just very different. Um, I think as an academia, what I especially like was, uh, this academic, um, thing in the air where you have freedom to do whatever you want. And it's very exciting to be in academia in that sense. You, uh, get to design your own research projects. But with that, um, there are also a lot of responsibilities. So you have to write grants, you have to make sure every. You are becoming basically a manager. You are managing your small lab. You have research students, you have to make sure, let's say your research students get paid. You have to then reapply for grants and these types of things, which is if, uh, things go well, it's, well, very satisfying. But, um, I must say, as a person who likes doing things, uh, I would like to focus more on the research and let's say also the teaching rather than, let's say writing grants and these types of things. So in that sense it's very different. You have these responsibilities where you have to do a little bit here and a little bit there. So you're getting drawn into different directions, which I would say is not a bad thing. It's just depending on your personality, whether you prefer that or just to focus on one thing and doing one thing. Well, I must say I really like doing research. But, uh, one thing I didn't like was a bit, um, let's say the reviewing system. That's, I think, something everyone complains about. Peer reviewing. Um, there's a lot of work to do. If you are a peer reviewer, you get a lot of papers for conferences to review. But then also as an author, it can sometimes be a little bit demotivating because, uh, reviewers are, I would say, sometimes very critical and not always in a constructive way. So sometimes you get this, uh, almost mean or hostile comments. And this was something where I was like, I don't know if I want to do that for the rest of my life. Same with grant reviews, um, where sometimes you get these very, um, I don't know, no apparent reason, because sometimes even someone misunderstood your report. You get very mean responses. And I was like, maybe let me focus More on the good things, building things, um, teaching, um, and less on these types of things in industry. Uh, I mean there are of course other trade offs. But I would say what changed for me is that um, I basically get to focus more on certain things without having to worry about um, other things I liked less. I'm not like a manager basically. So I like to build things and I like also to teach people. So yeah, I'm glad that I found something where I can focus more on that.

Speaker A: That's great. Um, speaking of building things and tinkering, do you remember one of your first projects uh, in machine learning and what attracted to you initially?

Speaker B: Yeah, my first project in machine learning I think, um, besides, let's take. Oh no, I think one was be a fun one. That was back then when I took this data mining class that I mentioned. Um, that was a side project because we had to come up with a class project for that class. Um, and by the way, that is also something I um, took inspiration from from that class. I also always emphasized in my courses to include little class projects. It's always something that students uh, found very exciting. And back then. So there were two things I was uh, working on as a student. I was working on um, fantasy sports predictions. Back then I was a um, soccer fan, um, and there was like a website where it was called. I forgot the website but it was a daily fantasy sports where uh, you basically assembled a team of players and they got scores based on how well they performed in the Premier League games on the weekend. And so they were. It was basically a constraint optimization problem where you had certain budget and you wanted to basically maximize. You wanted to predict how many score or what the best players are, um, based on the budget basically. So you couldn't. When there were also other constraints like the formation. You couldn't have 10 strikers. You could only have I think maximum three strikers. So it was very interesting. Um, and based on that I built machine Learning classifiers with SciKit learn very simple ones to basically predict what the promising players were. And that was very uh, interesting as an exercise because that's how I taught myself Pandas. The um, um, data array library or data data uh, frame library. And I tried to automate as much as possible. So I was also trying to do some simple nlp, um, going through news articles, basically predicting the sentiment and extracting names from players who are injured and these types of things. It was very challenging but it was a very good exercise to learn data, uh, processing and implementing simple things. But yeah, that was maybe One of my first projects not related to my PhD at all. It was more like a side project. And also I built something called um, I think it was called Music Mood. I called it Music Mood, which was for this class project where it was about predicting um, the mood of music in terms of. Is this a positive negative song? And um, originally it was the Happy Rock Song project where we had um, also the genre. So it was genre and the mood. And yeah, I turned this into an open source project. I think I shared, um, I built like a simple website with flask where people could enter the movie, um, sorry, the music, uh, lyrics and then get a predicted label whether it's positive or negative. And yeah, that was like a nice little project because it was also like almost like an end to end project where we had to collect our own data. So it was with two, uh, other classmates, we collected our own data, cleaned the data, built a classifiers and then built that website, um, also on top of that. So it was kind of like a, yeah, um, pretty comprehensive project. Um, the machine learning was pretty simple. With scikit Learn, I think we used the random forest classifier. But yeah, a lot of fun. M A good exercise I think.

Speaker A: Yeah, that's awesome. I think the best way uh, to get involved is just to find something that you're interested in. You know, create a project, find some data. Uh, you learn a lot of the skills doing it that way, solving problems that you're interested in. Right.

Speaker B: Um, if I may ask you before we go to. Sorry. M. What was your first, uh, machine learning project? Um, if you can remember, like on the spot.

Speaker A: It's a really good question. Um, well, one of the first ones that I worked on was a. Basically it was a computer vision project where we wanted to use face recognition or face detection actually to control, uh, a media player. So if you looked away, the media player would stop. If you looked at it, then the media player would play. And then we started to get into different, um, hand recognition. So like if you put your hand up like this, then it would stop. If like doing it like that, it would raise the volume. So it was really interesting. I got to learn about all of the, um, you know, different algorithms that are used to do face detection. Um, and I learned so much about computer vision. Um, for me the amazing part of that was just I've always had a really strong background in math. Um, so being able to take images and converting them into, you know, all numbers was kind of mind boggling.

Speaker B: Yeah. And then you can do a lot

Speaker A: of things with them.

Speaker B: But now that you mentioned that, uh, where I think this type um, of system still lives is uh, if you use for example an iPhone. And I think they encode or they hide the text messages until you look at them for like privacy reasons. So I think they are only visible when you look at them. It's kind of like reminded me of your system basically where um, it's basically all the time detecting um, whether that your face is pointing towards the camera, if you are looking. And I think the next level is if it's you who is looking into the camera versus like someone else basically. Um, very interesting. Yeah, right.

Speaker A: Yeah, it was cool. It was also really interesting to see when it worked and when it didn't work. Um, you know, we trained it on like perfect conditions. Right. The lighting was perfect, uh, you know, and things like that. And then as soon as things got, you know, dimmer, it was much harder to detect faces obviously or like different, you know, types. Different types of people. Um, so yeah, we, we ended up creating like our own training data set and you know, it ended up being a lot of fun.

Speaker B: Um, I think that's, I can imagine

Speaker A: the best way to get involved. Yeah, just to find something that you're really interested in. Like we didn't need to do the uh, you know, recognition um, of our, you know, fingers and hands for that project. But we were just, so, we were so interested in it uh, that we decided to take it like, you know, one step further. I find that to be the most rewarding when you're doing it, you know, not just for a class or for a grade. Like you actually are very interested in the project that you're working on.

Speaker B: Super cool.

Speaker A: Yeah, yeah, you, yeah, you mentioned like sports and. Yeah, fantasy sports. That's something that I've been very interested in in the past. And then music also is one of my interests too. So it's like, it's awesome to hear that you worked on projects in those areas. Um, speaking of, you know, those, those sorts of projects, are there any other open source projects that uh, that you've been a contributor for?

Speaker B: I would say, um, back then I was using a lot of Scikit Learn and I also contributed a lot to Scikit Learn, um, in the recent years. Maybe not as much, um, because I got busier uh, with other things. But yeah, back then we had um, the ensemble, um, voting classifier, the feature selection, the sequential feature selection and some other things, um, where I got to contribute and that was a lot of fun. Um, besides that I uh, build like my own little hobby, uh, library called ML Extend, which is I think, uh, used by a lot of people now because it has this frequent pattern, um, mining sub module that a lot of people at companies use. Uh, I always see on the discussion board, a lot of companies, they have some um, proprietary data set about, I don't know, some customer item sets, data, ah, um, stuff where they have some questions and it's, I think it's very widely used, uh, not for machine learning, although it has machine learning um, capabilities mostly for the frequent uh, pattern mining. Um, but yeah, this was a library essentially because I built a lot of stuff, um, that I needed for my work. Uh, like little, let's say functions here and there for normalizing things and also some other classifiers and so forth where I just thought, okay, instead of just hiding them on my computer, I can make them a little bit more general and then I can share them with the world and then others might find them useful basically. And um, yeah, I just grew that library over the years, just adding and adding to it. And the other major one I would say was um, Biopendas, where in computational biology we worked with these, um, protein structure files and also small molecule structure files. And we were building back then our virtual screening library where we were um, making predictions on millions of molecules. And um, for that you had to parse these molecules in a way, uh, that you could process them. And there were a lot of libraries out there that did something like that. They basically had some proprietary, or not proprietary, but some API where they read in these molecule files and then you access the objects in Python, let's say with a, um, custom API and so forth, which is fine, but it's like, yeah, you have to learn that. It's like a specific library and you have to learn how do you get the number of carbon atoms, how do you, I don't know, get the position, the coordinates of that atom? And it is, I think, yeah, it is a bit, uh, steep in terms of the learning curve. And I thought, okay, why making that so complicated? Um, if we just had a way we can load that protein structure file into a Pandas data frame. I can just use everything that's already in a, that's already there in Pandas. I don't have to reinvent a function to compute the, let's say the center of mass using the coordinates. I can use all the function standard deviations mean everything that is in Pandas and to make that more convenient. So it's essentially a library where you can convert protein structure files into a Pandas data frame and then you can do machine learning, you can do statistics, everything on top of that without having to relearn um, let's say a custom API. It's basically all in a Pandas data frame. Um, and other than that I would say yeah these were my main libraries where I contributed to um, or that I built basically from scratch back then. But then uh, over the years I um, did a lot of open source stuff but not necessarily libraries. Uh, what I did more was um, education I would say like writing blog posts, um, explaining things, um, Pytorch, uh, and scikit Learn related tutorials or things like hey, let's implement a principal component analysis from scratch or let's say uh, implement a self attention mechanism from scratch and like writing the code. Uh, but not necessarily as a library because I think there are already a lot of efficient implementations out there so it doesn't really make sense to reinvent the wheel. But it's more about like let's peel back a few layers, make a very simple um, implementation of that so that people can read them. Because that's one thing. Uh, deep learning libraries are becoming more powerful if we look at Pytorch, um, for example. But they are also becoming much, much much harder to read. So if I would ask you to take a, the convolution operation in Pytorch, I wouldn't even understand. I wouldn't even know where to look in Pytorch to start with. It's like.

Speaker A: Right.

Speaker B: I mean for good reason because they implemented it very efficiently and then there's Cuda on top of that and stuff like that. But as a user if I want to customize or even understand things, it's very hard to look at the code. So in that case I think there's value in peeling back the layers, making a simple implementation for educational purposes to understand how things work. Um, so that's something I, yeah, I've also liked doing in recent years which is why I maybe didn't contribute so much to the core libraries. I was more like um, focusing on, on the coding for education essentially.

Speaker A: Right, yeah, no, that, that makes a lot of sense. I um, appreciate a lot of the writing that you've done. Uh, I really enjoy your blog. Um, I think you have a newsletter that I'm following now too. I'm looking forward to your, your new book that's coming out. Um, Q&AI.

Speaker B: What's the title? Machine learning Q&AI. So I can maybe say a few words about that. So this Is essentially, um, it started because uh, what I do is when I read or learn things I have for myself I have flashcards. Basically I write down uh, questions and answers for myself. Uh, so just, I mean usually when you write them down, that process helps you learn these things. And maybe you rarely have to go back to your flashcards because it's not about the memorization necessary, it's more about making the question. But then also it kind of feels good when um, you feel like you have read a paper or a book and then you made these questions for future use. So you know you have them written down somewhere just in case you forget they are there as flashcards in my software so I can look them up. And uh, people on the Internet, um, they ask me sometimes to share these um, flashcards. And what I did is I thought, okay, why not? But let me polish them a little bit up. Because when I write things for myself they are usually not that nice. They are also, I mean containing grammar errors or typos and was like, let me polish them, make them a bit more clear so that someone else can read them. And in that process, um, these notes became longer and longer. So they became like fully fledged, um, answers. Some of them like, I don't know, I just was in the mood of writing. And then some of them were like four or five pages long. Um, and yeah, so, so one question would be for examp, what's the difference between uh, an embedding a latent space and um, things like that essentially? Or when are fully connected layers and convolution layers equivalent and all types of questions? Or what is the difference between self attention and the traditional attention mechanism in RNNs? What are the multiple GPU training paradigms like um, Tensor parallelism, data parallelism and so forth. And the answers, they tended to become longer and longer and longer. And I was like, okay, instead of just um, I mean these are not flashcards any. These are basically book chapters. So I thought okay, I could just basically turn that into a book. Um, and yeah, it's basically machine learning Q and A Because um, it's like a Q and A, it's a question and an answer. But then also it was interesting that is um, chatgpt now. So an AI doing the answers. And as a little gimmick, um, I thought hm, uh, because it just came out, why don't I include also the answers by ChatGPT, uh, as an. So I have my own answer followed by the ChatGPT answer and a short discussion and readers can tell or can, let's say, judge for themselves which answer is, let's say, more um, appropriate or accurate. So what I, but so one thing, of course, ChatGPT, um, cannot create figures and these types of things. So it's kind of a little bit unfair. But I must say for my comparison, what was very interesting is that when I wrote the answer, I had usually a very long answer. ChatGPT was way, way shorter sometimes. Or yeah, I would say if you have 10, 10 items, I would say three items are wrong. ChatGPT answers, uh, contain sometimes factually incorrect things. It's easy for a, uh, domain expert to weed them out. However, what, what was nice about ChatGPT is it sometimes came up with things I didn't think about. When I, for example, asked about what are some ways we can deal or can improve or reduce, let's say overfitting. What are some techniques for reducing overfitting? I had quite the long list, explained everything asked, um, ch. It had some, let's say wrong answers, but some of them I didn't even think about. Um, and so that was nice. It's essentially creating false um, positives but it's also having these uh, true positives, let's say that you missed. So it's in a sense actually pretty good for brainstorming. I would say it's actually a pretty good uh, writing companion. You still have to know a bit about the field because, yeah, these errors, if I wouldn't know about, let's say machine learning, it could be dangerous because it would give me wrong information. But uh, if you, yeah, if you look for inspiration, I do think it's a valuable tool essentially.

Speaker A: Yeah, yeah, definitely. Um, I was about to say that I've been used. I, I use CHAT GPT as like a brainstorm assistant. Um, you know, it can help you with drafts, it can help you, you know, write outlines and things like that. But yeah, there, there is that danger, right? You're a, uh, machine learning expert, you know, reading about it and you're able to quickly pick out, you know, say whatever, 20, 30% of this information might not be factually correct. And it does become dangerous when there's someone looking at it and looking at it as an authority, right. Seeing, seeing the output and thinking that it's like it's, it's probably going to be correct. Um, yes. So I mean talking to someone in NLP and machine learning, we brought up CHAT GPT. Uh, it took us a little bit but uh, I guess we could dive into it now.

Speaker B: Yeah, there's no way to avoid it nowadays.

Speaker A: No, can't, can't avoid it. I um, know you've been in the field and you know you've seen uh, the progression. You know it's this, it seems as if it's like this overnight success right? You know, going to a million users in a couple of days. But obviously this has been years, years in the making. Um, where I want to start off with is how do you view the gap between the hype of something uh, like ChatGPT and the generative models now and the reality of AI?

Speaker B: Yeah so it's interesting I would say ChatGPT, um, did a good job in terms of closing the gap because honestly I must say it works pretty well and it is impressive. I don't know like how far it scales in terms of would be. I mean we can always improve things but I don't know what um, let's say the, the rate is of how we can make it better. I guess related to the hype. Um, I think there's a lot of it's like a sale. The same with um, self driving cars I guess where 5 years ago they already had pretty uh, impressive demos I haven't seen to be honest. Um, I mean the thing that they don't show you is what they have right now that is not released yet. But I do think it's usually the last few percent that are crucial. I think with self driving cars we have been, it's just a number, I don't know for sure but I would say we have been there for like 95% now like in five years ago it was almost, let's say 95% there almost let's say ready. Now five years later we are maybe there at 97% or 98% but can we get the two last remaining percent points to really nail it basically to have them on the roads reliably and so forth. And that is hard to say with um, large language models as well. I think we can reduce um, the factually incorrect information, make them more useful and so forth. I just don't know how, let's say how much work it takes to get just a few more percent more better performance. We will see with uh, the next generation let's say the UM GPT4 models and so forth if they apply then also the reinforcement learning with human feedback on in the loop on top of it if it's substantially better um like the same like from GPT2 to GPT3 maybe it's the same from 3 to 4 where we get again mind blown. Um, but yeah, that is one thing. Um, the other thing is I think people are chasing like hype wise. They see chatgpt and they're chasing um, AGI, uh, like artificial general intelligence. Yeah, that is an interesting question. I think uh, no one knows how far we are from AGI. Um with ChatGPT, I think there's a lot more hype around AGI, it appears closer than before of course because we have these models. There are people though who say okay, this is uh, the totally wrong approach. Um, we need something completely different if we want to get AGI. No one knows what that approach looks like. So it's really hard to say. That's the thing. If, if something hasn't been there before or it doesn't even ex. Exist it's hard to predict when it will exist. It's like um, it's very hard basically to make any reliable or any, any statement about that. I would say the, the thing though what I always find interesting is do we need AGI? More like a philosophical question. I think AGI is useful as a motivation. I think it motivates uh, a lot of people to work on AI, uh to make that progress. I think with, with, without AGI we wouldn't have maybe things like I don't like um, what was it called the alphago where they uh, had breakthrough. They basically beat the best player at go, maybe chess even back then. Chess. And how is that useful? I would say maybe AlphaGo and chess engines are not useful but I think it ultimately lead led to um, AlphaFold, uh the first version for protein structure prediction and then AlphaFold2 which is now based on nudge language models what uses large language models. So in that case I think without large language models and without um, the desire maybe to develop AGI, we wouldn't have also all the, let's say very useful things in the natural sciences. And so my question is like do we need AGI or do we really just need good models for special purposes? For example if I want to, I mean there was like a paper the other day, uh, accurate weather prediction, um, with deep learning, like more accurate than the best physics based simulations that run on supercomputers with a smaller, let's say more, um, not smaller but with a more energy efficient neural network and more accurate. So maybe that is sufficient. Maybe we don't need an AGI that can also predict the weather. Maybe it's better to just focus on improving that weather prediction engine and separately improving the protein structure prediction model AlphaFold. Maybe we don't need to chase something that can do all the things at once. However, I do think AGI is useful as a motivator to find better algorithms. So it's um, in terms of hype I think, uh, I'm, personally I don't see the purpose of AGI. Maybe I'm um, uh, too short sighted here, sighted here. Um, I would say what would we do with AGI besides what people say about replacing humans? I don't know how that, that really benefits compared to special purpose, um, applications of machine learning.

Speaker A: Yeah, right. I mean you brought up so many interesting points, I don't even know where to go next. Um, but um, let's talk about the use cases for um, generative models. So you were mentioning basically which I love this point where we're able to get these models up to a certain level of performance. Right. Say you can get a model to 90% or 95%, but it's that last 5% that are, that's so hard. And it's like either the closer you're getting to that 100%, it's like even, it's even harder. It's, it's even, it's even harder. It makes me think about, you know, like when you're training a machine learning model, any model, say like even like a text classifier and you have your F1 score at say like, you know, 0.85, you know, how much work can you really do to get it that, that much higher? But I wanted to take a step back and I wanted to talk about basically, um, generative models. It's a, uh, I think there's a lower threshold. So like error can be okay depending on your use case. Right. So if you're, if you're using it for something like just to make a draft, it doesn't need to be 100% correct. Because if you're making marketing content, let's say, you know, that could be the product I'm seeing now. Wix, uh, is offering like a complete generative, you know, using generative models to create your whole website. That's amazing. Right? That solves the cold start problem. It gives you so many options. You can build, you can build off of it. But then there's the other part. There's, you know, there's predictive models where you're say you're categorizing something, um, and you need it to be very close to you know, 100 correct, you know, depending on your use case. Um, yeah. And then you bring up AGI, um, artificial General intelligence. I think everybody thinks about it a little bit differently. You know, everybody has, like, a different sense of it. Everyone has a different definition. Like, are we trying to replicate humans? Are we trying to make things, uh, are we trying to replicate human intelligence? If that's the case, then I, I personally don't think that large language models is the way to go. There are certain things that I think about, like from GPT2 to GPT3. One thing that's very interesting are when you buy orders of magnitude, add all these parameters. Uh, there are these emergent capabilities, which is like, you know, really interesting. Um, I think in one of them, like, like you're reading so much of the English language, so you're gonna learn, you know, how to make grammatically correct sentences, and then you're gonna learn, you know, different relationships between things. And all of that stuff is amazing. But there's more to it, I think, than that. Just being able to predict, you know, the next word, the reinforcement and human in the loop piece of it is definitely gonna, as you were saying, minimize the amount of factually incorrect responses. Um, but, yeah, what, what do you think? Do you. Do you think that, um, that our goal should be to try to replicate human intelligence, or do you think we should be specializing in certain systems or certain use cases?

Speaker B: I personally, I think, um, I mean, for the sake of, uh, developing more efficient, um, learning algorithms or alternative learning algorithm algorithms, I do think it makes sense to, um, get inspired by, let's say, replicating human intelligence. But I would say if it doesn't work, that's fine too. I mean, this. I mean, the classic example is really, uh, airplanes or, um, submarines, where airplanes are inspired by birds. Like, hey, birds can fly. They have wings. Um, can we build something similar? Turns out the airplane, yeah. Is very different. It doesn't flap the wings, but it is. It gets the job done. Right? So in the case, we don't need to make a whole bird's fly. And in the same sense, we probably don't have to mimic, um, how, let's say humans learn and think. Although I do think it would help understanding that because there might be more inspiration that we can use for these models. One thing is also related to that. Um, ensemble methods are, uh. So building an ensemble of different methods is usually something to improve how you can, let's say, make more robust and accurate predictions. And ensemble methods usually work best if you have an ensemble of different methods if there's no correlation in terms of how they work. So they, um, are not redundant, basically. So that is also one argument why it makes sense to maybe approach the problem from different angles to produce totally different systems that we can then combine. I think that's also interesting from the perspective of how people try to implement large language models as part of a search engine, because I feel like, yeah, we don't. So it's kind of like related to artificial intelligence, general, uh, intelligence where maybe we don't need one system that solves it all. Because for example, with um, Chat GPT, it can do math. Uh, it's some of the emergent capabilities that you mentioned. Uh, but it's not useful for simple math. Like if you say multiply 13 by 100 by, let's say 13 by 117 or something like that. It's maybe not useful to use ChatGPT for that. We have a calculator that can do that accurately, that doesn't need to be trained. There are uh, simple rules. Yeah. So in that case what we need is more like identification of what we need to get the job done. So maybe having um, you know, like Siri, um, What Siri is doing is um, it's parsing the language. I mean besides the fact that it doesn't work well. But let's say it would work better in parsing your input. What it does, it's reroute routing your input to the appropriate application on your phone. I think if you say set a timer, it will use the timer app on your phone or if, I don't know, if you do a calculation, it will use the calculator app. So it's not trying to do everything itself, it's uh, trying to delegate. And I think with AI, I think that's the same thing. If we ask it to maybe compose text, the AI itself might be the best way, uh, to do that. If we want factual information, maybe sometimes just um, extracting information from an existing Wikipedia page might be more efficient than um, having itself, um, answering that. So it's not, I'm not saying it's um, not necessary to use an LLM, but the uh, LLM here would be more efficient at going to that website and summarizing the text rather than rewriting the text, basically if you are looking for an answer. Um, so, and I think that is one thing we could focus on, on how to basically delegate more efficient, efficiently and building an, that um, let's say delegates rather than tries to solve everything. Um, in my opinion, and also to your point, the AI doesn't even have to be correct all the time when creating um, text as long as we use it for, as uh, a template basically, not as the end product. So I think chatgpt, the main uh, use is for me. How I use it is to help me write texts. But I'm filling in the blanks. I'm not like if I want to text about something I usually write the text myself before then I say hey chatgpt, rewrite this. And I see if I like it more or less. I take certain sentences and then I even tweak them afterwards. I'm not really literally copy and pasting anything or in the same way with information. So there was another LLM I think was called Galaxy something where. Oh, Galactica. Yeah, Galactica, where they had um, an AI or LLM that was writing research papers. I think there was like this misconception that um, you let it write the whole research paper. I see it more as something that writes the template for a research paper. It's more like, I would say a sophisticated template builder. Um, I think it would have been better if it wouldn't fill in numbers or any factual information. It would leave blanks, you know, like, so that it's more clear to a human. Like hey, you have to fill in the numbers and the details and they're not provided by the machine learning AI system basically. So I think yeah, having these models, it's essentially about using them responsibly. Essentially. Yeah,

Speaker A: yeah. You, you bring up so many interesting points I, I to, to talk about like the different tasks that you want to complete. I see a future where um, yeah, depending on what prompt basically you, you, you, you know are asking, you could use something that's rule based or it could pull up the correct tool, you know, the correct tool. The I don't know, sibling or our predecessor of Chat GPT instruct GPT sort of was going into that how you can take an initial prompt and then have some follow ups. That's what's really nice about Chat GPT as well that you can sort of take the output and you can say make it longer, you know, you know, make, make it shorter. Um, I saw another recent paper tool former, basically showing some examples of how to use tools, tools, you know that you can basically combine the power of large language models and um, you know, using third party tools. I think it's this ability to sort of find that hybrid approach, right when, when can, when are rules the right approach and when should you be using more advanced systems? Which is kind of like always a quite, you know, always a question, right. Can you make, can you make it simpler.

Speaker B: M. Yeah, it's like this saying, like if you have a hammer, everything looks like a nail. And I think this is right now a little bit true with um, ChatGpt because we just have fun with it. It's like, oh, let me see if it can do this and that. But it doesn't mean we should be using it for everything. And yeah, now it's, the question is basically the next level would be how to uh, basically when to use AI and when not to use AI basically. So yeah, because right now we are using AI for a lot of things because it's exciting and we want to see how far we can push it, um, until it, let's say breaks or doesn't work. But yeah, sometimes we have nonsensical applications of AI because of that. Um, like, yeah, like training a calculator, a neural network that can do calculation. That doesn't really make sense, but there are examples where uh, I think reinforcement learning found a more efficient matrix, uh, multiplication algorithm, more like the algorithm itself, finding that that makes sense, uh, something you assume and wouldn't think about. But we wouldn't let it do the M matrix multiplication itself because you know, it's not deterministic in a sense. So you don't know if it's um, going to be correct or not depending on your inputs. And there are ah, definite rules that we can use. So why making it, let's say approximate when we can have it accurate?

Speaker A: M. Yeah, I um, think that that's something in the machine learning field that's really such an interesting area that deserves more research. Um, understanding machine learning models are going to make predictions, right? Like, like there are systems where it might not. It doesn't have high enough confidence to make a prediction. But when it makes a prediction usually it's like it's usually binary. Like it's usually like it's, it's, it made a prediction, this is what it thinks the answer is. But it doesn't give you that like that confidence level. You know how like when you're talking with a human, you can kind of tell how confident someone is when they're, when they're saying something, when they're saying it, or they might validate it, they might say, oh, I think I heard about this, that that's lost when you are talking with ChatGPT. Yeah, yeah.

Speaker B: And on top of that, uh, one thing is also there's a whole branch of research on um, that neural networks ah, are typically overconfident on out of distribution data. So what Happens is if you have data that is slightly different from your training data or um, let's say out of the distribution, the network will, if, if you program it to give a confidence score as uh, part of the output. This score for the data where it's especially wrong is usually overconfident. It's over, let's say estimated estimating its confidence, which makes it even more dangerous. So um, even the confidence score, let's say it's not, it's misleading if it's uh, a tricky problem which is kind of like ironic or paradoxical even. I mean it's, it's kind of an interesting research problem. I mean there are methods that try to address that. But yeah, it's not out of the box. It's a lot of extra effort to. I mean it's an ongoing research field. So yeah, like you said, even. But even if we had the over the confidence scores, it would be hard to use them or trust them. But also you bring up a good point. Um, so ChatGPT doesn't give us any confidence, uh, about anything. But then there's also, I mean an even better example. I think what's more clear is this um, classifier they developed to classify whether uh, chatgpt or no, whether text is written by an AI or a human, where they have different labels like uh, likely or not likely generated by an AI or something like that. And yeah, it's just a label so you trust it or not. And for example, when I used uh, Shakespeare, Macbeth texts in there, it predicted it was likely generated by AI. It's just a label. And well, what do you do with that? It's like totally wrong. But because Shakespeare um, was around when. Yeah, way before, uh, AI was a thing. But yeah, so there's another approach. It's called GPT0 where um, the researcher who developed that just gives you a score. It's only the um, perplexity score. And then you as a human, you have to compare it and think about it, which is maybe a better approach than just giving a label. But yeah, you bring up a good point. It's um, we just take it for granted or we just take a score and yeah, um, we use it. And it's maybe out of convenience because that's the simplest user interface. But with um, things like machine learning. Yeah, it is, depending on application, tricky.

Speaker A: Yeah. And I think that's definitely a problem that machine learning practitioners should try to address. But it's extremely difficult. Right. Especially as humans, we're trying to interpret these very complex machine Learning deep learning models and something like something that's out of distribution and it's trying to make a prediction on it and you get a prediction and the predictions high confidence. And it's like that, that doesn't even, you know, it's like why. It doesn't even make sense. It's that it's a little, it's a, it's a, it's a little scary because sometimes like, so take an active learning system where you're gonna label samples that have low confidence and then like those are, those high confidence ones are just gonna slip through.

Speaker B: Yeah. In that case, yeah, it would be achieving totally the opposite of what you want. Um, because m. It will give you the high confidence for the ones that you actually need to label because they are so different. It's. Yeah, it's um. Yeah, it's essentially um, antagonistic or adversarial. Uh, yeah,

Speaker A: yeah. I mean it makes you think about just how, I mean how many moving parts there are with machine learning and just trying to understand. And it's so important to understand every aspect of it. It's not just the algorithm. It's not just um, you know, the newest language model. Sometimes it's like common sense things. Understanding the data, understanding the output. You know, why are you making this? How is it going to be used? Those sort, those sorts of things.

Speaker B: And I want to say we are complaining here. Oh, yes, sorry. I'm just wanted to say we are complaining about this here. Um, that machine learning, uh, systems make these mist and we don't get the scores and we don't interpret them. Uh, it is, uh, I mean it's something to think about, I wanted to say. But it is challenging. It is not that I, um, would say people who are working on this, um, they are trying their best. Um, they put a lot of effort into improving that and make getting the best out of it as possible. It is just such a hard problem that um, I think, yeah, it needs more time and work. We are trying to do the best we can or most researchers are doing the best they can when they release the products. It's just such a hard problem. Uh, so I would say we, there's no one to blame about that. It's just how hard this problem is and. Yeah, um, so, yeah, yeah, of course.

Speaker A: I didn't mean to say, to say in that sense. There's an interesting, there's an interesting trend that I've found actually with machine learning practitioners. After they work in the field for a certain amount of time, many then shift Their focus into like AI ethics, which is exactly trying to address these type, these types of problems, uh, which, which I, which I find that to be, you know, very, very interesting. And the more I work in, in this field, you know, you have to think, you have to think about those. You have to think about those things.

Speaker B: Yeah, actually a good point, because I think it makes a lot of sense to um, let's say start with machine learning and then go into AI ethics because then you basically get exposed to all the problems that exist. But you also notice that it's maybe not so trivial because I think it's easier to say, well, this is not good and this is a problem. Uh, fixing it is the more difficult problem really. Um, and I think, yeah, experiencing the maybe frustration around machine learning, that's a good way to also be prepared for what's possible and what not and what could we do? And I think, yeah, it is frustrating sometimes to work with machine learning systems because we train these classifiers and then we see exactly, okay, this gets this, uh, input wrong. And, and, but we don't know why, why this particular input. We can maybe include more training examples of this particular input. We improve the system. It doesn't get this one any wrong anymore, but then it gets something else wrong instead. And it's like really like, uh, you're trying to fix one thing, the other thing breaks and it's. Yeah, it is very, very challenging. It's like, yeah,

Speaker A: yeah, um, yeah, it's very, very tricky problems. And uh, it's nice to, you know, have the chance to discuss this with somebody that's kind of dealt with these problems. And yeah, it makes sense after you are applying machine learning and understanding maybe some of the pitfalls, to then transition into some more of like the AI ethics sort of, sorts of questions, um, to change things up. Not really though. Um, but in the spirit of learning from machine learning, um, let's zoom back to someone who's just starting out in the field. Um, what advice would you give to someone that's just starting out in machine learning?

Speaker B: I would say, uh, yeah, that's um, tricky. I don't want to give anyone wrong advice, but I would say, um, machine learning is a big field. I think, uh, even like what we just covered, there are so many moving, uh, parts that are involved. And I mean, even zooming back, we have, have predictions, we have generative models, we have computer vision, we have natural language processing and all kinds of different fields. And then for each approach, for each field we have different approaches for generative Modeling, we have, let's say just for images, we have autoencoders, um, diffusion models, generative adversarial networks and so forth. And they are all ah, kind of like almost fundamentally different in terms of how they work. And it can be very, very, very overwhelming I think when, um, when you start out, out. So I would say, honestly I would start with the book or a course and just work through that with, I would say almost with a blindness on not getting distracted by um, other, let's say resources at that point, just working through that, um, because I think that's uh, happens to me all the time. I get distracted by something else. I look it up and then it's like a rabbit hole. And then you feel like, wow, there's much to learn. And then you get frustrated and overwhelmed because it's like, oh, the day only has 24 hours. I can't possibly ever learn it all. So I think really doing one thing at a time, like step by step, it's uh, it's a marathon, not a sprint. I would say. Um, yeah, so I think I would say yeah, take it slowly, enjoy it. Make, make sure you have fun. Um, try not to do all at once. Um, yeah. And maybe also finding a balance between trying things out or maybe implementing some ideas in a project after reading about them, um, and then going back to reading about more things, trying them out. So like having a balance between soaking up knowledge also and um, trying out um, things you learned about.

Speaker A: Yeah, I think that's really good advice. Uh, it's interesting when some, when someone asks me like, oh, how can I get, get, how can I learn about machine learning? There's no shortage of resources out there. Right. There's no shortage of new material coming out. Um, but it's sort of like hacking through the weeds and staying on a path to get yourself to a point where you can understand a certain level of the basics. You don't need to know every paper that's coming out daily. Right. It's not necessary. It's much more important to understand the basics. So you're setting yourself up for a future, um, of success basically. In a similar vein, um, if you have anything, what's one piece of advice that you've received that has helped you along your machine learning journey?

Speaker B: Uh, that's a good question. Um, top of my head. Uh, I wouldn't have a good, let's say advice. Someone, let's say gave particular to like to me. But I would say going back to the end range class that we talked about in the beginning. Um, I think something Andrew Ang always said in his classes was, uh, if you don't understand this part, don't worry about it. And I think that's ah, a good thing. It's like, um, maybe, um, if we don't understand a certain thing, maybe let's not worry about it just yet. Just that, I don't know, some, some things are more important than others. Um, also when we specialize, I think like letting go of some things to make room for other things. It's for me, I worked on some more mathematical papers where we had, uh, where we proved theorems and so forth. Um, like the ordinal regression papers we worked on, which was fun. But I, for example, I know that I'm not that good at proving theorems because I'm more like a person who enjoys coding and for proving theorems you have to sometimes sit there for days or weeks and stare at it until you get some inspiration. Um, and this is not for me and I think that's okay. Um, I would say not getting frustrated, I guess saying, okay, this is not for me. Recognizing that, um, focusing on my other strengths and yeah, that would be something like, uh, don't worry about it. Oh, sorry, I almost knocked off this thing here. Let's say what Andrew said, not worry about it. That is like something I think that

Speaker A: kind of relieved me.

Speaker B: I would say it's a small.

Speaker A: Yeah, it's a small thing. That's really nice because when he was, when Andrew Ng was going through say a proof for something or showing all the mathematics behind gradient descent or you know, how changing the weights or back propagation and things like that, you don't need to know every single detail right then and there. You might not ever really need to know every detail, but understand it. The getting and gaining an intuition. And that's what, that's what Andrew Ang always used to say. Gaining that intuition and getting that gut feeling and things like that, that's what's going to help you, um, along the way. Um, yeah, that is other than Andrew Ang.

Speaker B: Oh, yes. So I wanted to say exactly what you said. Uh, I wanted to say what you brought up, a very good point is. Yeah, you should of course make sure you understand the bigger picture and intuition, uh, in a certain way. But the details are sometimes implementation details I would say. But like you said, yeah, recognizing, um, when it's time to focus on the big picture and when it's time to dive in and really making sure you don't have to dive into everything basically also very good Exercises to implement things from scratch. Um, like reading about, let's say, decision trees and then implementing decision trees from scratch. Uh, for example, that's one homework I usually give where, um, students have to code a cart decision tree or C 4.5 tree from scratch, which is a good learning exercise. But I wouldn't say do that for every algorithm because if you do that. Yeah, you would get stuck. You would never really, um, move forward because it takes a lot of time. It takes weeks to do that. And knife is also, in a way, um, short. If you spend your whole time reimplementing old algorithms. Um, yeah, that's also not a good way of spending time. I think it's like being selective, I think also focusing on the big picture, sometimes diving in, but not diving into the details of everything.

Speaker A: Right. Um, yeah. One of my professors, during my masters, he had us by hand, you know, step by step going through back propagation for neural networks.

Speaker B: That sounds fun.

Speaker A: Like, you know, you're. You're beating your head against the wall and, you know, it's. It's very frustrating and it's not like you ever need to do that. Right. But there's something about even just doing it once that you do just kind of gain, you know, a better sense of it and. Yeah. You know, at first the details aren't that important. You know, future. Future. You know, when you're in industry and you're trying to get a model into production, I mean, sometimes things are so abstracted that you don't necessarily, like, need to. Which is. Could be a good thing or a bad thing. Right. Because it's fine if there's no problems, but it quickly becomes a bad thing when, uh, when you start to run into some issues and you're not even. And you don't really understand, you know, what's, what's going on with your model. But yeah, I mean, at first it's, it's much more important to get the. In broad strokes, just sort of get a handle of what's going on, building up that foundation so you can understand, you know, everything. You can't, you can't learn recurrent neural networks, you know, without understanding what, what a decision tree is. Right. Like, it's just, like it's, it's not, uh, you just, you can't. There's certain things. It just, it wouldn't, it wouldn't make sense. Like, you should start with logistic regression. You know, just do it. Right.

Speaker B: And good, good advice. Yeah. I would say always start with. Even if, you know, more sophisticated techniques um, if we go back to what we talked about with large language models, even if it makes more sense even for a classification problem to fine tune a large language model for that I would start like you said with a uh, simple logistic regression classifier, maybe back of words model to just get a baseline. Like something where you are confident it's very simple and it works. Let's say using Scikit learn before trying the more complicated things. It's not only because we don't want to use the complicated things because the simple ones are efficient. It's more about also even checking our solutions. Like if our fine tuned model or let's say Bert LLM performs worse than the logistic regression class flyer, maybe we have a bug in our code, maybe we didn't process the input correctly, tokenized it correctly. It's usually always a good idea to I think to really start simple and then um, increasingly get complicated or improve, let's say improve by adding things instead of um, starting complicated and then trying to debug the complicated solution to find out where the error is essentially right.

Speaker A: Even if worst case scenario if you use a very simple model, you just got it, you just have a basic, right, just a sanity baseline to work off of. Um, so other than Andrew Ang, who we both obviously admire, um, who are, who are some other people in the machine learning field um, that you gain inspiration from or that have influenced you?

Speaker B: Good question. Um, I would say because I also recently enjoyed uh, some of the educational material by Andrew Capacity. Uh, what he reminds me always is that it's fun to code things and it's like very contagious. Uh, if you see someone um, having fun coding things up. So that's something I did very early on in my blog where I implemented a principal component analysis from scratch or linear discriminant analysis. Other things I always usually used to implement things from scratch. But over the years I've become more, I would say conceptual. Um, because things got more complicated I was focusing more let's say on implementing an end to end system and then not let's say doing the step by step coding. And his recent um, stuff here reminded me of how much fun it actually is to do things um, from scratch. So that's like one inspiration. I would say M. Or other people, I would say M, maybe Paige Bailey because she always has so much fun on let's say social media. It's like also, also to remind you, I don't know, whatever you do, have fun like enjoy, share the, share the joy. You know, like that is, I think also important to keep in mind that, well, uh, things are sometimes complicated and I don't know, work can be intense. We want to get things done. But don't forget also maybe just to stop and enjoy sometimes, like to you know, share the successes, have spread some fun stuff, you know.

Speaker A: Yeah, definitely. Um, speaking of starting things from scratch, well, I think of it, I was able to read your uh, recent blog, understanding and coding self attention mechanisms of large language models from scratch. And yeah, just uh, I mean, yeah, we were talking about some of that going into it and understanding, you know, the similarities between cross attention and self attention. Um, it's really interesting to go down to the more basic principles and to see things from the code and how do I say it, it's like in production and when you're deploying models, you don't want to reinvent the wheel. Right, exactly right. Battle tested. You want battle tested things. But when you're trying to understand something conceptually, it's really nice to understand it from scratch.

Speaker B: Yeah, excellent point.

Speaker A: Second point.

Speaker B: Yeah, yeah, we really want to emphasize that like I think for real world applications, don't try to reinvent the wheel. I think, yeah, that is a lot of work and also risky. Uh, but it is like you said, it is good for learning. It's especially good for learning actually. One thing I like is also um, so I built sometimes things both ways. So when I want to implement something, I um, do the most naive implementation ever, like where I just use very plain simple Python code, write some unit tests to know because I want this and this output and then I try to make it more efficient. So like adding more efficiency to that to see if I can improve things. That's what I do usually for things that don't exist yet, but for things that exist, you can actually use what is already out there and then kind of like use that as a unit test almost and then try to make your implementation similar to that. But yeah, like you said, don't maybe use from scratch implementations if there's an existing solution. Um, only it's for learning purposes essentially.

Speaker A: Yeah, definitely. And then, yeah, to your second point, uh, from before, it's important, you know, to have fun, right? And to realize that, you know, learning, learning can be, you know, it should be enjoyable and expanding your knowledge is, is so important. Um, so to conclude, uh, learning from machine learning, the last real meaty question, um, what has a career in machine learning taught you about life?

Speaker B: I would say, I would say it's like, yeah, uh, being patient because there's so much out there. So it's like can't learn it all at once. Once take uh, take it one step at a time. But like what we just talked about, uh, making sure we enjoy um, what we're doing. But then also what I think uh, what machine learning taught me especially in the last couple of years is things are changing quickly. Um, so in that sense it's kind of like counter to what we just said. Like taking things slowly but it's also be open to change. Um, you know like be open to new experiences. Like it could be, be anything like from job related things to location wise where we live, uh, what our hobbies are. And that is something like related to machine learning in the sense that there are so many new methods coming out there. Um, things change completely. We were using GANs two years ago. Now we're using uh, using diffusion models. It's like being open to things and open to change and yeah, I don't know, like trying it out, um, making sure maybe we don't like it, we don't have to use it. Um, it's the same with life, like trying new experiences. I think that's great.

Speaker A: Yeah. I think being patient when, when you need to be patient but also just sort of accepting that we are living in a very fast, fast moving world where, where things are changing. So being open to change and like

Speaker B: machine learning everything gets better with time, with more training epochs essentially. So maybe hopefully uh, when we like with life experiences and stuff like that things get better usually. I hope so. Yeah.

Speaker A: Yeah. Sebastian, it's been such a pleasure uh, talking to you. If there are some listeners out there who want to learn more about your work, um, where could they go to reach out or to find out more about you?

Speaker B: Um, I think my website would be the best um, uh place because there I have links to everything else. So yeah, my website uh, is essentially my first name, last name.com Sebastian Rushka. It's maybe a little bit difficult to spell um, in in the sense of uh. It's easier if you maybe see a link. So it's uh, my first name, last name dot com.

Speaker A: I'll have it in the show.

Speaker B: Yes. Yeah, exactly. Yeah. So and uh, because it's a very long name otherwise, um, I'm very active on social media. Um, most of them basically like Twitter, Mastodon, LinkedIn. Um, so on most platforms I uh. RASBT. So that is actually uh, back. It's weird because it's back then uh, on Twitter there was a character limit. Ah, where the Twitter handle was cutting into that character limit. So I try to keep it as short as possible. Five letters. It's basically the first two letters of my last name, Ra and then SBT as my S in Sebastian. So rasp team. So I'm that on GitHub, uh, Twitter and some other platforms. So, yeah, if you, you want to reach out on social media, I'm pretty much everywhere. Maybe too much, but I must say that is also one thing. Over the years, I've been on social media over, like, maybe 10 years, and if you use it responsibly, you, uh, can learn a lot of things. Uh, we're always having good discussions where we discuss recent papers. There's always someone who knows more than you do. So it's always nice to have always these comments where someone points something out or follow up material or, hey, have you thought about this and that? And yeah, I think it's basically, basically, if you use it responsibly, um, it can be a very effective way for learning too.

Speaker A: Yeah, for sure. Um, Sebastian. Yeah? It has been so nice chatting with you. You're like a fountain of knowledge. I feel like there's so much that we could chat about more. We could do a whole other episode maybe sometime in the future. But thank you so much for your time. Uh, I really appreciate.

Speaker B: Yeah, that was fun. Thanks for having me on your podcast. Um, it was like a really fun hour to spend today, so, yeah, thanks for inviting me. I had a lot of fun and, um, yeah, anytime. Again,

Speaker A: Thank you for tuning in to this episode of Learning from Machine Learning. I hope you enjoy the insights and knowledge shared by Sebastian Raschke, a renowned author and machine learning expert. Don't forget to check out the show notes for links to Sebastian's work and resources discussed in this episode. If you enjoyed this episode, please leave a review and share with your friends and colleagues. Until next time, keep on learning.