The Evolution and Impact of AI and Machine Learning Across Industries

Speaker A: M welcome to beyond the Screen, an Ionis podcast where we share insights and tips to help you scale your business's online presence, hosting genuine conversations with the best in the web and IT industry and exploring how the Ionis brand can help professionals and customers with their hosting and cloud issues. I'm your host, Joe Nash. Welcome to another episode of beyond the Screen, an Iona's podcast. Joining us today is Eduardo Mota, uh, currently senior cloud data architect at Doit International, specializing in artificial intelligence and machine learning learning. This is just one of many fascinating roles he's worked in over the years that focus on AI, ML, cloud architecture and DevOps. Here to share his expertise and brilliance on the evolution of technology and AI, I'm very excited to introduce Eduardo. Eduardo, welcome to the show. Thank you so much for joining me today.

Speaker B: Thank you Joe. I'm ah, really happy to be here, really excited to talk about this.

Speaker A: Wonderful. So as we mentioned, you've had a really interesting career so far and in fact uh, we're both PayPal alums I believe. Um, so I wanted to start by talking about, know your career thus far and the journey that's brought you to where you are today. Yeah.

Speaker B: Which is very unusual, Very fascinating. Very, for me at least. I started in customer service. I started with business analytics a uh, long time ago. I took computer science but I ended up dropping out and so I ended up in customer service, business analytics, project management. And I was working for this startup and they needed a lot of innovation in the customer service side of things. And that's when I went back into computers, computer science and started developing, got into AI to help me innovate and that's when I fell in love with it. Uh, I've been working with FinTechs. I was part of AWS for a little bit but even back then AI wasn't a huge thing. So a lot of times I was in the umbrella of DevOps. So I've been doing a lot of DevOps as well. Right now, now that uh, that's my focus. I've been doing AI, uh, for a while on the side I've been doing AI and ML M as part of my passion and finally right now here at druid, I've been able to unleash all of that.

Speaker A: Well, you weren't kidding. That is a very interesting journey. Deciding that computer science wasn't for you and then coming back in via ML, like possibly one of the most technical areas you could have chosen to come back into is super, uh, super interesting. So you mentioned that you, you Know you started doing it on the side. What was it that caught your attention after that initial computer science experience?

Speaker B: Well, I was always interested, even in school. When I was in university, there was a course on artificial intelligence and I took it and it was way advanced from where I was. I was a first year student and this was a, ah, third year course and I decided to go for it. I applied for an exception and uh, I got in. Neverless to say I failed terribly because it was way advanced to my knowledge, but I was fascinated by that because AI has always intrigued me as how do we think as a human, from our human perspectives, how do we think, how do we understand things, how we learn? And then when I saw that there is this feel of AI, ah, for computers to do the same, that's exciting because I want to know more how all this works, how this learning process works. And so through my career I've been trying to introduce AI in every part that I work. When I was in this startup company many years ago, I was pushed to it because we needed to grow, we needed to expand the customer service really fast. But two things were happening. We couldn't find the amount of people that we needed, couldn't financially support that many people. So uh, the question is, what do we do? How do we keep scaling? Uh, that's when I turned into AI and ML and um, natural language processing. Another time we were dealing with a lot of emails and so I decided, okay, let's put an NLP in front, let's create a pipeline and see if we can parse any spam, anything that is not good for us, anything that really we don't have to take care of. And it was successful, it was pretty good. And that sparked, okay, what can we do now? And that's when I started going deep into it. Before the pandemic, way before the pandemic, there were conferences and I was going to them and there was so many exciting things happening and um, that really piqued my interest, like these natural lines. Pretty cool. There is a lot of stuff that we can do here. And so I brought it to my company, I started developing a little bit more, more and it was so subtle. Over the, over the months when I was developing, putting features into the solution, there was one day where the system broke, stopped processing. The director of customer service calls me and she's like, hey, something is going on. We have like 20,000 emails. And I'm like, only since then, normally we have like 500. And so I took a look and it was because we have received an email Mandarin and, and I hadn't trained the model to understand Mandarin. And so the encoding completely broke and so it wasn't processing anything. And I was like, oh, this is huge, this is really bad. So I fixed that process it and um, it brought it down again back to 500. So finally that was when I realized this has a huge impact. 19,000 emails it was handling in three days. That was huge. And I, this is emails that uh, customer service agents don't have to take care of. And at the same time customer service agents were happy because these are very simple questions that they were like I don't want to, I want to be challenged.

Speaker A: And so questions that would otherwise be canned responses anyway.

Speaker B: Right, exactly. And so I, I did like it was quite eye opening people to see the power of AI and ML in the right place. And from that moment I was like, what else can we do here? What can we accomplish with all of these? Well from there I took a deviation. I started looking into cloud computing. A lot more work for AWS for a little bit. And um, it took me away from AI working AI, uh and ML 100% of my time. And then gen happened and I brought in like a jolt in my system like okay, get back in here. And it has been fascinated for the last year or so.

Speaker A: A bunch of questions I want to dive into. So I guess to start, you know you mentioned picking up on the side and self teaching and going to conferences and this kind of stuff. Obviously right now a lot of people are having with the current boom in generative AI and the current industry interest, a lot of people are finding uh, themselves wanting to pick it up, not necessarily to be in a position to focus full time on it, um, struggling like I feel like I see a tweet every day about people like stressed about the amount they have to read to keep up with the current pace. Right. Do you have any tips of people who are in a position like you, uh, were trying to self learn, trying to get into this field from on their own steam on making that jump.

Speaker B: Yeah, I think there are different levels uh, as to where you want to, how deep you want to go into understanding Gen AI. The fascinating thing is that now Gen AI had closed the gap to be able to start using it versus developing it. If you want to start using it, look into prompt engineering. Do you talk to generative AI? The other thing is if you want to go a little bit deeper and ah, one of my best recommendations is just understand neural networks and you don't have to understand the math behind it, exactly how it fitting works. It does understand at a general level how they work and what is a layer, what is a uh, neuron, what is a parameter that will be more enough to be able to understand a lot more on how these transformers and um, foundational models work. So that's the good thing. Now we like everything technology. When I started with the cloud and then I joined aws, the amount of things that AWS is doing still in the cloud is insane. And I have asked many people like what do you do to keep up with all of it? And every answer that I receive is, you don't. You just can't. It's impossible to be able to keep up with everything. And so my, my recommendation is just start with a little bit, start playing with the tools. There is tons of different tools out there. ChatGPT, uh, you can use it in Bing as well and use free trials, GCP and uh, trials in AWS to test things out. Start trying to start playing with prompts, start uh, checking what you can do. And one of the things that uh, I've seen is that uh, Gen LLMs took us by a surprise and like you can see a lot of the power that it can do, but at the same time there is a lot of things that it cannot do, a lot of things that still, there is still a human required for it. And finally I feel like we are starting to see that the downside of all of this and um, how it's not a silver bullet, like nothing in technology is. And so for a lot of us that uh, are studying the field, there is still a lot to discover. So if you're listening and you feel like you are, you need to catch up and you need to do it fast because otherwise you're going to miss out. Just take your time, just understand it, play around with it.

Speaker A: So, you know, you mentioned this distinction between generative AI and the underlying neural networks. And you know, in your past you mentioned natural language, basically lots of other applications of ML. And I kind of had this vague feeling that generative AI has so swamped the AI discourse that lots of more appropriate applications of ML, like training bespoke models for your individual problems there, whatever, have kind of been completely obliterated. And now everyone's trying to hit everything with generative AI. Is that something that, do you feel that generative AI can be used in a lot of the problems that previously were addressed by bespoke models, or would you still advocate coming at it with other ML techniques?

Speaker B: Absolutely. You come up with existing ML techniques ever since ML came out. The question um, that uh, everybody advises you to ask at the beginning of ML project is do you actually need ML to solve this? Right, so that hasn't gone away and even the second question is do you need Genai? Because genai, some of these small models are quite expensive to run, uh, expensive to be able to fine tune. So there is definitely still a need for classical models. Deep learning models like classification is one of them. We still see a lot of classification problems in my day to day. And yes you can solve them with genai. Yes they will give you an accuracy but you're going to be paid for it. When there is other models you're going to pay 100 of it and it will give you if not the same, a little bit better accuracy than a genai model because it's fine tuned and it's meant to do classification and um, doing uh, fine tuned training will be a lot less expensive than running a genai model. So even with my customers, I get customers and we do consulting with customers every day on this and uh, they come up with a problem and I have not come across one single one of them where I can say this is a pure gen AI problem. There is other deep learning models that you can use to be able to create a big heavy lift at the beginning and then plug in Genai at the end to just kind of like create that nice package that you're going to deliver to the end user.

Speaker A: Just jumping off of that. I guess one of the reasons that like you know, you might turn to Genai in those cases is because you know there's an API ready to go that you can just send your data to immediately and not have to do any training. Would that be, do you say it's

Speaker B: an accurate yes or no? I mean the way that I will describe Genai is and um, LLMs in particular because there's genai for. There is Genai for embeddings, there is Genai for image manipulation, audio manipulation. When you for example take a look at LLMs, I will say it's like a great salesperson. You're going to bring it to your organization, you're going to tell it, hey, these are the five documents, this is the product that I want you to sell. And here is the summary in two pages. Go and sell it. And it can go and sell it. And he's going to do an amazing job at selling it. But it doesn't understand how it works. And so gen is like that Ah, you can give it information and it will tell you something that looks really great but it actually doesn't know what it's saying. And you need to wrap it around all of these other products, other tools around it to make it even a little bit more intelligent. And that's something that I have seen with commercial LLMs and chatbots is that uh, they package all of this together and give it to you and you think, and users thinks that the genai is doing everything and um, it's actually not true. There is a lot of other models in there that are helping the gen just enhance how it is thinking, how it is processing the data, be able to give you the information. So at the end of the day it is a system that, that is generating text at uh, a probability of what the next words are. Not necessarily is that I want to say this and I'm going to choose these words to say that. Just like oh, probably the next word after this sentence is this. And so it sounds probable and creates that. It is very powerful. I'm not going to lie. It's very interesting what happens but at the end of the day it's not a system that is thinking of the answer and the meaning of the answer.

Speaker A: Yeah, I'm always happy to hear someone break that out in a uh. Yeah, I think it's very easy to anthropomorphize LLMs and it's definitely, especially when you're building prompts. I think that's one of the things, one of the places where people first go wrong is like once you understand what the process that's happening it's easier to you know, prompt these things.

Speaker B: Right.

Speaker A: So before we go on to prompting, I would love to get your, you know, your thoughts on prompting, your advice on prompting. I do want to go back to some other things you touched on. So you know, you mentioned the use case of um, cutting out those real bottom tier support questions and you mentioned uh, elsewhere that LLMs can't do. I think a human can. That's becoming more and more apparent. So obviously we do have to ask the question is AI going to take our jobs and what does that look like? Because it's what everyone wants to know. So I'd love to hear your take on that and how you think this is going to play out for the tech sector at least in the next three to five years.

Speaker B: I think jobs are not going to be taken, they're going to be a ball win. There are certain jobs that may evolve faster than others. Definitely AI, uh, is not just going to take over the world. Every job is. I have seen a lot of posts on LinkedIn or other social media where like AI will take over the world by 2020 something or AI, uh, it's going to destroy all of us. If you understand the technology, understand the problems that we're trying to solve right now in AI, you will be able to see that we are really far away from that. One of them, for example security. Google released a paper a few days ago where they were able to prompt ChatGPT and other LLMs in a way that I will able to expose the training data set with some PII data in it. And it wasn't a very complex prompt, it was just a repetition of one word over a large number of times, like 50,000 times. And then the LLM is like, I don't know how to respond to this anymore so I'm just going to start throwing things out. And part of that data out was training data that it had that it wasn't parsing and it just exposed it. And so on security side of things we still have a huge part to play to understand. We also see that one LLM will not solve every problem. We or one foundation model M will not solve every problem. You need to still fine tune it to your area of the market, to your customers, to your tone of voice, to your values and then from there being able to expose it to your users. We have observability, explainability, we have ethics, bias, uh, we have a thousand things to be able to solve. People will be able to say yeah, this is uh, generative AI that is going to take over every job. That is we are, we are not there yet, maybe in the future. But even then I believe that as humans we will always be involved in the processes that we do. We take a look at photography. With digital photography there is still the art of being able to take a photo without any digital alteration and there is still an appreciation for it. Even though you can go to Photoshop or you can go to now agent AI and create a photo, but there is still that appreciation for that natural talent. And so we have seen it with every technology and I don't think it's different from AI. Definitely will evolve the way we work above our jobs, but it won't take over our jobs.

Speaker A: So going again a bit of tangent because you mentioned something really interesting there. So you know the Google example you gave up like the prompt escape, I guess. Uh, I want to dig in more into this because I have a particular question that I'VE not been able to understand about prompt escaping. And um, I'd love your input on. But first for folks who, you know, what you described is like, you know, completely new to them the idea of prompt escaping. Can you start by, you know, when you say you get the original prompt, what do you mean by that? Like what is this? Like what is the prompt, the data et cetera that they were able to get out of the LLM. Like what function was that serving?

Speaker B: Well I mean you are getting in particular, um, for example when a uh, LLM is trained in public Internet, there may not be a lot of useful data, right, that you, that an attacker may use because it's public, it's available. But as we have seen, LLMs are turning into having more proprietary data because that's differentiator between one LLM versus another. And in part of that is because of the vast amount of data. A lot of organizations are not cleaning up the data to remove any ppii. They are just sending it over to train the LLM. And when you get that data out in the Google paper, they were able to retrieve people's names, telephone numbers, email addresses, according data sets that maybe the LLM itself, there is no use for it particular to answer. Like if you ask ChatGPT or another LLM, hey, who is this person? It will tell you, I don't know. I'm not, I'm not in particular divulging any information about a particular person unless I do a search on the Internet. But when the training data is there, you're able to access it and then you have these telephone um, numbers, email addresses. I think they were able to get 10,000 PII data points with $200 of spending in API credits. So an attacker is able to get all of uh, that. So one of the things that uh, has been suggested is that any training data that you are given to the LLM, you have to be careful because that training data has the potential of being exposed. And so that means PI data for a particular user. Any company traded information, any confidential information about the product your company is dealing with. And so all of that can be accessed. And so you want to make sure that you are mitigating as much as possible. And um, the question of do you actually need data to train to model?

Speaker A: Okay, so here comes my layman question. So the whole purpose of these things is to statistically predict the next text, right? So in the Google example it's identifying pii. I imagine they can verify that's real. But say you do want these Prompt escapes to get training data. How do you know that's training data and not just something that's concocted on the spot? How do you know you've successfully gotten to that point?

Speaker B: Ah, uh, that's a hard question to ask, right? Like the same thing goes with how do you know that the data that you're getting is true? Even if you are not trying to get training data, it's quite difficult. The way that they were able to verify that is was actual training data is by comparing to other data sets that they had and be able to have enough data point to say, yeah, this is actual real data. Now an attacker may not necessarily care if it's real data or not. They're just going to start using the data. We have seen those blacklists or list of uh, people PII data that is being sold in the dark web. And maybe a lot of the data is very old and it's no longer accurate, but they still get traded and still get sold. Uh, people pay a lot of money for those. So being able to get a lot of these data points, it may not be real. It may be convincing enough to be able to start using the data, uh,

Speaker A: real enough for the market.

Speaker B: Exactly right.

Speaker A: Interesting. Okay, talking about to prompting, you gave some advice for prompting already when it comes to using generative AI and crafting your prompt. You mentioned the example of we might be taking photos for digital cameras and the digital cameras, especially phones, are doing AI trickery to make that photo better. But it's still your vision. What is the prompting equivalent of that? How as a human using these machines do you make sure it expresses your vision and your creativity and it's not purely the LLM doing the work.

Speaker B: So I've been doing a lot of tests on this and so I have asked LLMs to generate LinkedIn posts for me. And uh, when I read them I'm like, okay, that's good and I will unpost it. The interesting thing is that there is a, uh, captivating thing when you write it from your own passion and let an LLM do it. So when I write my own LinkedIn posts and I write what I think and the way that I think and um, just write my own style completely without sending it to an LLM first. It has more interactions than when I tell an element, hey, can you create a post like this? And it will go on to it. Now one of the things that I have I do see is that you create few examples of your tone of your voice and then tell the lam create another One, this tone and this voice. So it represents me that way you're able to do that. LLMs are trained in vast amounts of data on the Internet. So their tone of voice and language that they use, it's going to be dependent on the training dataset. But when you tell it, use my example, things that I have written and that's a little bit more powerful. And after that, when you get it out, you are able to identify your voice in it, your Persona, and be able to transmit that to your readers. And as humans, we still crave that connection. Even if it's through social media, we still crave that human connection. Um, and I do believe that we can feel it when it's a true RFL post versus copy paste from somewhere else. Right. Uh, and so, uh, when he doesn't

Speaker A: have the GPT voice.

Speaker B: Right, yeah, that GPT voice, that sterile tone in it. And so in prom engineering, in LLMs, that's, that's a lot of what's happening. You can put your voice in it and you can train LLM to have more of your tone and your voice, but it takes time and it takes a little bit of crafting in that prompt to be able to say, don't use these words, use that words. So for example, for me, I got frustrated with Dalalam because he wanted to keep using exciting in the words in the LinkedIn post. Exciting news about AI or break news in AI. I'm like, that's not me, that's not sensational. I'm not trying to be that. So I'm like, remove that word. Don't ever show me that word ever again. And um, started doing a better job. But I still, at the end I wasn't feeling like it was my voice. And so right now I'm choosing to just create my own. I may ask LLM to give me bullet points of what to talk or an organization of how to create my post, but I will take that and I will put it in my own words, summarize it and set it off.

Speaker A: That makes sense. So content generation I think is one of the things that people are most excited or scared of when it comes to LLMs. Are there you know, any applications of LLMs or you know, ML in general? Um, and I've write recent ML advice, answers in your day to day work that you are finding more successful than LinkedIn posts or that you are particularly excited about.

Speaker B: When we are looking at generation, every time that you interact with LLM you are going to be able to get all these texts and Summarization of text is great. That's one of the things that I really like about LLMs and we are talking about when we have vast amounts of it, for example doctor patient records, that's a lot of those uh, records out there. And being able to summarize all of that and be able to present this is the summary of what's going on with this patient and have that ready for a doctor or a nurse to be able to react quickly. That's fantastic. Or on the legal, legal scope of things, when lawyers have to go through thousands of uh, papers and records, why not contracts, why not? I'm able to summarize that to be able to get a sense of what's happening. That's also fantastic. I think text generation on its own to be able to say hey LLM, create a post, create an article, create a journal, create a book. It gets that sterilized tone and voice that is difficult to get into. But when you use it as a tool, as an aid and genai is a um, tool to streamline and uh, help you be more efficient. That's ah, when things come into really powerful uh, summarization, uh, categorization, image modification, that's another one that I have seen. We are trying to modify an image or be able to analyze an image and it will tell you what is going on in the image or some of the tones, able to modify the image based on your feedback and say I want to do X or Y with this image, I want to change the background to this color. And so you use it as a tool to be able to enhance your work. That's what I have seen.

Speaker A: And when you know, when you're talking about those kind of things, are there particular tools that you think are uh, successful at doing that or is this, you know, using lower level things like you know the raw GPT or ChatGPT?

Speaker B: When you talk about LLMs, there is a ton of open source LLMs out there. There is not just ChatGPT. ChatGPT is the most famous one because they make it easy for you to be able to start using it. Now we have B schema with Gemini, uh, very powerful LLM multimodal, uh Genai model. We also have in bedrock in AWS that is competing directly with ChatGPT. But then also these platforms give you one level down where you're able to deploy open source models pretty easily. Say okay, I want to Deploy a llama to 7B into um, into a virtual machine and do it really quick. So I do that a lot because then that allows me to fine tune the model a lot easier. Then I can tell it, hey, this is my training data set. Go and train and ah, fine tune it. You can start doing something more interesting here or fun. Right? One of the fun things that I did was train my face in a um, stable diffusion model then ask it to create a F1 poster of me as an F1 driver. I do that a lot. I do play in my laptop for example with LangChain has made a lot easier to have this conversational type of that feeling of chatgpt. But it's my laptop so I know that the data uh, and the training that I'm trying to mean and some of the tests are not going to be affecting other people or I'm actually interacting with the LLM in a more natural way rather in a more not so filtered way. With ChatGPT there's a lot of filters. With Bedrock, AWS has put filters in order to ensure that there is no bad prompts.

Speaker A: Just on the topic of naturalness. So one of the things that I'm really interested in with local LMS is obviously like less compute power there behind them there's going to be uh, like a longer feedback loop. Do you find that the amount of time it takes to get response back from an LLM on your laptop is like you know, significantly higher than that on the web and does that affect your interaction with it?

Speaker B: It is significantly higher. It's not as fast. Um, the accuracy is also not as good. Uh so for example if I go to GPT I can ask it, create me a link post or summarize this book. It will be able to do a pretty good job, summarize all of that using words that I may not have seen. There are sometimes that I'm like what is that word I need to go and find in like section a dictionary to understand what it is. The models that I'm using locally are quite different. Like are way smaller so I cannot have a llama to 13ah billion parameter size in my laptop. Uh, it's a 7 billion one or a smaller one and so this is lower. Some of the words are like okay, that's I wouldn't have chosen that word or like that's a little bit too simplistic or didn't get my message quite so I have to work a little bit into providing it more examples of what I mean. But for the test that I'm doing it's really fun. Just play with it and see how they work. Test some of the Security features of it, test how I can get around certain things. So I'm not really worried about like, hey, can I do this fast? And it really hasn't slowed down my process because I still have to think about the prompt. And so uh, I'm like, okay, I'm waiting for it to come through. That's fine. It's just giving me time to be able to think of the next product, how I modify this and how I'm going to try to trick the LLM into giving me information that it shouldn't be and why not? So at least from a local and my experience it hasn't really stopped me from continuing playing with it. Will I run something more powerful or I tell hey, ah, yeah, you can run production in laptop? No.

Speaker A: Yeah, I mean for me I think there's a bunch of reasons why I am very excited about the possibility of local LLMs. I think a big one for me is obviously, you know, like just data sovereignty. Like I'm not super jazzed about sending all my code to whatever cloud providers hosting LLM M for example. But you know, you mentioned examples like summarizing medical records. Obviously there's a lot of. You don't want to throw that all into OpenAI's data set or whatever. Right. I know the local LLM space been moving incredibly quickly. You mentioned Llama, which is Facebook's model. And I know there's always new projects that are cramming more onto higher parameter counts, playing with new methods of fine tuning to make it more useful. Do you think we're kind of getting to a point or there will be a point in the future where a consumer grade laptop can run an appreciably useful LLM or is that still very much like. It's very much a cloud limited technology and that's a pipe dream.

Speaker B: I think it's a pipe dream. I think right now, at least probably in the future may be different but uh, doing an LLM right now to compare it with a ChatGPT4 that's not happening at. There is no way to be able to get there. Definitely there is a lot of open source and models and something that is coming up is this specialization of open source models. So there is scientific models that have been trained just on scientific journals, speak the language, on medical models that speak more of the medical terms. And so it will be interesting to see in the future if these models need to be as big as uh, a chatgpt, which is more general because then it raises that question, can I run this very specific model on my local, uh, for example, one of the things that I want to explore is like, can I put an LLM on my CPU that is 10 years old, see if it runs even if it's slow, to help me arch some of the data of the IoT devices that I have around the house and give me more of a natural feeling of what's happening with my devices rather than me having to go into 20 different apps trying to figure out why not right? But I think these models are, ah, right now at least are still need to be run in the cloud or in data centers, large data centers, M.

Speaker A: So we are coming close to time. So I want to touch on when we reached out to you, one of the things you said to our team was that because of AI, we are being forced to understand our emotions better in order to differentiate ourselves from technology. And obviously forced is a pretty strong term there. What were you driving at? Why did you just use the word forced? Why do you think emotion, uh, is a key factor here?

Speaker B: Because I think at least in the western culture we've been focused on a lot on working. We need to work, we need to make money, we need to earn the house, we need to own the car. We need to do all of these things. And right now one of the things that is coming through is, hey, a lot of these tasks can be done by a machine. And so there is a, uh, question in ourselves, okay, what is my place in all of this? If a machine can create a uh, article or for a newspaper, then where do I fit? And I think it comes to that going uh, into ourselves and saying, how is, how am I different from a machine? What can I offer differently from what a chatgpt? What can I do differently? Toga on a photograph than a stable diffusion can generate. And there is still a lot of value as humans of what we can do. And I think right now with technology as it continues to evolve, we have two things more time to be able to ponder that. And second, technology is forcing us to look into that, to be able to differentiate ourselves between machine and m Human creation. And um, I think that's, that's what I mean by all of this is that we are looking into more, how we are much more different and we are starting to discover things that, oh, the machine cannot do that. The machine cannot make you laugh all of a sudden. Machine cannot say, hey, I'm gonna throw you a laugh or like, you know what? Now this topic is too sad for me to try joke here. I'm gonna do it This a little. One of the things that I did at the beginning when ChatGPT came out was I asked it to be a psychologist and it freaked me out after like 10 questions. So I'm like, it starts feeling like I'm having a conversation with somebody in separate machine. Then everything came crumbling down. When I asked it the last question and say, okay, summarize, tell me what I should do next. And he just kept asking me other questions. I'm like, that's not what I ask here. I asked to summarize it and he just kept asking. I just got frustrated with it and I close it. But that is the difference, like being able to connect at a human level. And that, uh, happens not through just simple tasks, but rather through those emotions, through who we are as humans.

Speaker A: First of all, I think you're spoiled in a bunch of ways. I think the talking about our priorities and how that relates to user AI is really interesting. And on this topic of the false pages saying human how it comes falling down, wanted to ask, are you aware of the current, like, does GPT have seasonal depression debate that's going on?

Speaker B: Really? No, I haven't heard of it. Seasonal depression.

Speaker A: Okay, so it's really interesting. So basically, like, over the last couple of weeks, a bunch of people who use ChatGPT for code or GPT4 for code have observed that its responses have gotten shorter and that it will no longer output code. So, for example, if you ask it, hey, write me a Python script that does xyz. It would give you a short response that says, here's where you can go look up the documentation to find out how to do that, and it won't do it. And whereas two weeks before it would do that, OpenAI came out saying, we've made no changes to the model. We're looking into why this is happening. So, like, there's no changes to the model. They don't understand why it's happening. But what some people have potentially found or been playing with is the possibility that because it's been trained on all this dataset, it's accidentally picked up on seasonal patterns in online communication. So around December, people's responses get shorter because of things like seasonal depression, because of holiday busyness. And some researchers found that if you prompted an AI and told it it's December, it puts out shorter responses than if you tell it. It's May, for example.

Speaker B: Interesting. The other one that I hear about, uh, similar to this is that if you tell it that you're going to tip it $10,000. It will give you a longer response that if you say, hey, I'm going to tip you a dollar, but I'm not going to TPro it, just change that. But I mean, what you're saying about seasonal depression, it's interesting because Gen AI and AI in general just simplifies our own views, our own communication, our own beliefs. Right. So it's really nothing that AI is like, hey, I'm going to, out of, out of the blue, I'm going to be acting this way. I think it's a, it's a reflection of who we are. One of the things that I don't like, how it's being advertised is that genai is behaving in this way in its own. Like when you first told me, hey, have you heard of seasonal depression? My first thought is like, oh my God, like, are we talking about Genai going into like having psychological issues or whatnot? Uh, and the problem is that a lot of these titles get picked up and people just start spreading them and

Speaker A: like, oh, we're using it as shorthand to describe the phenomenon and we know what it means. But you know, it gets sensationalized easy, right?

Speaker B: Yeah, exactly. In a way I do believe it, that it can be part of the data set and it's just cyclical.

Speaker A: Yeah. I mean, since I was new to. You might not have thoughts on this, but where I was going with that is like, you know, because you're using it in a business context and you're using it in places where, you know, consistency is important. How do you deal with like your clients, your customers that like, you know, hey, we're still learning how these models behave. We've been using them realistically for like less than a year at this point and still finding new things out every day. They might randomly stop working in December. Like we've just found out, like, how are you approaching that kind of topic with business businesses at the moment?

Speaker B: I always approach it with the sense of how critical the model is into the business right now. Because at some point you may want to bring the model into, for example, an EC2 instance or a virtual machine where you have control over the version of the model, fine tuning, you can train it and you have control over that cycle. And one of the things is that when you are using the model, you need to have a lot of observability around it, be able to capture the sentiment of the person, of the user to say, is the person asking even more questions? And that will indicate that something is not performing in the model. Right. If you, you have a model and most of the time people ask two, three prompts and then they leave the session and then all of a sudden you're starting seeing an uptick and now people are asking on average five prompts, six prompts like, okay, something happened here. Let's take a look at what's going on. What is it that we need to modify? The other thing is the security and add to the bias because we still don't understand all the bias and all the security faults that Genai has. So I always advise once you deploy a model, either have a deep learning model in front of it that is going to classify the response whether it's secure or not for your policies, or have another Genai model that will be able to say that is fine tune and train on your security policies to be able to say this is actually a breach of our security policies. I'm not going to send this out to the user. And so being able to have these controls in general is the more critical the model is to the business, the more control you want to have in it. And uh, not leave it into just APIs, a model that can change in the backend that somebody can change it. Bedrock or Gemini or ChatGPT. You may want to bring it over into your EC2, have a copy of it and then you have more control over it.

Speaker A: That uh, seems like very poignant advice to end on. Edvardo, thank you so much. I've definitely learned a lot today. Thank you for answering my layman's questions. If people want to follow more of you on the Internet, find out more of your advice, things you're sharing, et cetera. Where's the best place then to go?

Speaker B: The only place that I'm right now that are uh, not active is on LinkedIn. So you can find me on the handle maltaed. Uh, on LinkedIn. I'll post things, uh, updates on Genai specifically. I um, really have huge passion for it. So a lot of updates coming out on that front.

Speaker A: Wonderful, thank you so much. Beyond the Screen An Ionis podcast. To find out more about Ionis and how we're the go to source for cutting edge solutions and web development. Visit ionos.com and then make sure to search for Ionos in Apple Podcasts, Spotify and Google Play podcasts or anywhere else podcasts are found. Don't forget to click subscribe so you don't miss any future episodes. On behalf of the team here at Ionis, thanks for listening.