Designing for Human-AI Interaction | Product Builders | AI-Native #2 | PRODUCT

Speaker A: M

Speaker B: hi. Welcome to Product Builders AI Native, a community hosted by Boulder. This is a space for practitioners to share knowledge, exchange perspectives and discuss real experiences in building products the AI Native way. Delivery strategy, discovery and growth, all in just 25 minutes. I'm Anna, uh, your host today. I hope you enjoy this episode. One more find Product Builders AI Native on substack

Speaker C: and this conversation today. Conversation is about something I keep bumping into in my own work and I suspect you do too. We add AI to products and we call it copilot or an assistant or sometimes. And this is where it gets really interesting. Autopilot. And then we wonder why users don't quite behave the way we expected them to. Uh, so the question this episode is asking is what does it actually take to design human AI interaction? Well, not just to ship it, but to design it honestly. And I have exactly the right person to explore that with. Agata Jawosinska researches human data interaction and teaches human computer interaction at SWPS University in Poland. She is regional project leader for the Global index on responsible AI, tracking progress in responsible AI, um, across 138 countries right now. And she's completing her PhD in human computer Interaction at Newcastle University. Agata, really glad you are here.

Speaker A: Thank you very much for having me. Just to say completed Ph.D. so happily finished.

Speaker C: Okay, so it's finished. That's good. That's why your name, that's why we

Speaker A: can use the title.

Speaker C: Nice. Yeah, so, so great to know it. Um, okay, so let's start at the beginning. Uh, maybe not about the, the PhD itself, but I think a little bit about it as well. And I want to anchor it, um, in something concrete because I think it makes the question land, um, differently. In 2018, uh, a Tesla on autopilot drove for a 7:37, I suppose, minutes without the driver touching the wheel before the crash. And the driver had been watching a video and the car and this, let's say solution was called autopilot. In aviation, the word means something very specific, something pilots trained for extensively for a long time in their life in a consumer car, nobody had defined what it meant. We borrowed a word that carried meaning. And that's not just a naming failure. I think it's a design description failure. Um, we are building interactions we haven't fully described yet or named yet. How is AI different from other technologies in terms of being the object of design? And what do we know about designing for AI already and what is still unknown about.

Speaker A: Okay, so, um, let's maybe try to organize. I'll Try to organize my reply to that question into like two fields. Because first field that we can talk about is product designer, CEO of the company, thinking about the AI that they are adding or developing AI native products under their, under their brand. Or we can talk from the user perspective. And I uh, would argue that the problem with the name Autopilot doesn't necessarily comes from the design perspective, but rather from the marketing and branding and trying to play with the name in order to sell something. Because when we think about uh, from the designer and from the CEO perspective, what we are basically facing is the situation when we are dealing with two big scales, two big phenomenon. First one would be the AI's um, system capability which is borderless. The second one is the output complexity which again can be very wide. And those two axes would help us um, navigate the first scope of what we are planning to design. With the term Autopilot. We are already trying to um, aim for something which is very complex, which has huge capabilities and the human is not in control. This is like a top tier of, top tier, top quarter of the spectrum to which we can uh, dream of, to which we can uh, be ambitious to gain with AI. But there is much more. It's also the thing that when we are thinking about AI, when we are thinking about automotive technologies, we always think about this replacing of human with automation. There is so much that can happen in between. We are super happy with using AI as not needing to intervene, just letting it do it work. But we are also sometimes happy to have it work on a very great scale with very dense and intense and detailed human, um, human reach, human oversight over it. So what I would say makes AI difficult to design and to conceptualize. It is like those first two things that I said. It's like systems capacity, um, the enormous amount of users that we can think of. And it's also that the system is learning along the way. It's also that the system is developing based on what the user is inserting into the system. We can have one technology but then giving it to 10 users and the technology can learn. It's very difficult to prototype for these kind of situations. It's very difficult to come up with scenarios that can happen. Designers also face a problem with

Speaker B: maybe

Speaker A: not lack of toolkits, but current toolkits being quite uh, difficult to apply to this situation. When we talk about system capacity we are thinking on the scale of system is very well defined. System can um, automatically or not but conduct the activities that we have pre designed it to do. And we Define them. There's like a close set of activities that it can do. But on the scale we can go further down to the unknown number of capabilities because the system will be able to learn with the user. That would be like let's think about our first axis. Then the second would be the outputs complexity. So the variety not only of the modalities but also of the variety of outputs that um, system can perform with or also deliver uh, to the user. And it can be as simple as right or wrong answer to the question. That's very good. If someone can automate this decision for us, we will be very happy. But that's also something like Autopilot that will take care of the whole car, try to take care of the whole car and navigate with the number of uh, using the number of sources of information. I would work first with uh, these two scales and these two spectrums of output complexity and system capacity.

Speaker C: And of course above or behind, I don't know. Uh, sometimes it's changing. Uh, there is always a business and as you said it's some kind of marketing. We want to sell the idea, we want to um, explain a little bit what we are building or making. Uh, and even we have a great decisions. I see maybe it's too much but I see a huge problem with the naming for a moment. I think it's underestimated. Before any design decision there is a naming decision or during the designing there is naming decisions. And names carry cognitive contracts. Some contracts with people and autopilot implies you can disengage as a person. I can be not involved. Assistant implies it like it supports me, but I lead. Copilot implies shared responsibility but who has the control and when do you think that names we give to AI systems are themselves a form of design failure? And what would it look like to name them more like? I don't know honestly. Or better.

Speaker A: I think we still don't know how to name it. Uh, and uh, I would still say like if we think about interacting with the product, uh, I would still try to divide between name coming from the marketing and name of it coming from the design. Right. Because you can design a very good autonomous car and sell it as it is a car that still needs your activity without calling it autonomous car. The car on the autopilot. The design, the design part in which you enter the machine. You see the, I um, don't know, you see the wheel, you see the M seat next to you, you know how to operate this machine. And this is something that HCI has been Working with for a long time. Like we could also put this argument in that interacting with the machine or interacting with the automation is nothing new for the research perspective. We've done it, we know many ways of how to do it, we have many guidelines how to design an interaction with something that is autonomous. And the question is like, are we with the, with the proliferation of AI, what is required of us is like to accidentally and instantly imagine the, the scope of human action that is happening with the machine. And this is not something that we are adapted to. And we know like we had first, first uh, ChatGPT was like four years ago, three years ago and the technology is everywhere. This doesn't happen with, with any kind of car, with any kind of, I don't know, lamp, cell phone, mobile, nothing. And uh, the problem with um, with naming I think is that both designers, both users are still learning about what exactly is happening in this interaction between a user and the machine or any application like desktop application or mobile application. Uh, when we don't know we are using our well known resources like we go to this fairy tales about something magical or dreams. Yeah, or dreams like, or like, you know, these, these imaginations of something material suddenly becoming alive and we forget about the whole variety of different levels of interaction that can happen in between. And we don't stop to think about the elements and the scales that we can use when trying to describe this interaction. Uh, this is actually something that once we learn how to do it, we will be much quicker in understanding the core design problem from the perspective of the designer and from the user perspective, we will be much better in understanding what kind of uh, buttons or various types of interactions we can design for the user to elevate their experience of safety, of control, of understanding, of satisfaction, of using the product, of joy, of creativity, of um. Once we know better how this works from the perspective of the user, we are much better at coming up with uses of the technology that have not been fought for by the designers. And this is the beautiful part when it happens.

Speaker C: Beautiful but a little bit scary at the same time because as you said, this is design as it had been before, as it was, but the framing is already shifting in 2026. Today agentic AI systems talk to each other um, through protocols like MCP. And when we build the systems we are designing interactions where neither party is human. Does your research framework still apply and how evs.

Speaker A: Well, without the framework that I've introduced a little bit it would get even more dangerous I would say because the first of the Framework that I said about the system capacity output complexity is one. Second one that I spoke about is defining control. What we would ideally aim for is high output complexity, high system capacity and high human control. Not leaving it to the autonomous case like the autopilot, like right, uh, but we can deal with it somehow. Like car accidents show us that there is a limit to it. But to some extent we can deal with it even now without fully applying to these frameworks. But when we talk about agentic interaction, it's impossible to understand what is happening if the system doesn't talk to you. This is the moment when the side of the interaction with the user is super important. The first framework that I wanted to talk about is um, is Amerchi Salim, uh, Amershi. She is a Microsoft researcher and Microsoft actually develops 19 guidelines for human AI interaction. But don't stick to the name. Think about if the interaction, if the guideline says explain what the system does and explain how well it does, what it is designed to do. We want it from the agent, we want to know exactly, okay, I'm good at this job. I'm not the best designed to conduct any other task. We want the agent to define to us what it does when uh, uh, it meets a problem. We want the agent to translate to us. Okay, I'm, I don't know, limiting the number of the activities that I can conduct because the situation is this and this and that. And I will explain this to you. So from the level of the interaction uh, uh, with the agent M and with the interaction with the systems that are only agent based, the transparency becomes important. But I don't want to stick to the word transparency because it got so much confabulous and different types of implications that we can go into very, not necessarily concrete discussion. But if we stick to this idea of like 19 guidelines when we define the system, what the system does, how well does it do, um, explain why it behaved that way, explain to the user what will be the consequences if user continues to use the system in this way. These are very practical, very pragmatic information about what we need from this, that we need from the system. And in order to, like I said, safely, creatively, funny, peacefully enjoy the uh, interaction with the system.

Speaker C: I'm not fully convinced that I am not a little bit afraid. But of course, um, as you said we always needed some kind of frameworks for us as people as well. So I think that this is not so scary. Uh, it's about evolving with what we are building right now and so many People, uh, are researching and building these things for us. So definitely it's something really quick as almost everything right now. And um, to bring this somewhere m even more practical before we close, ah, you've mentioned you are open to collaborations and uh, the flow works. When you work with product teams, what does an ideal collaboration between a researcher like you and a tech company look like and what does it produce that neither side could make alone?

Speaker A: Okay, um, well, I think that this kind of collaboration happens best when everybody is aligned according to the design process where we are and either, uh, we are exploring, either we are prototyping and we understand what we bring to the table. So I trust the design team, I trust the research team. What me as an academic researcher can bring is absolutely the competence of organizing chaos and to being able to name what is happening in these unknown spaces. We have some time to think, we academics and we have some time to test our frameworks. So also I wanted to strongly underline that, you know, frameworks that I talked about are researched, are confirmed, uh, are like tested in many, many ways. These are not like out of my head. Um, but I think what happens also very good in the, in these kind of, in these kind of, um, settings when we are trying to collaborate is being able to define how can you help each other. Because without it, I can do the best I can, but if it doesn't help you, that's not needed. And I think the problem with collaborations is that we very often focus on only what I would like to have from it, but not how my work could actually support this specific project.

Speaker C: But you know, sometimes it's really difficult to, to say what do you need from other side? Do you have any, I don't know, common or most important or like the basics, uh, you offer as a researcher? Because right now with all the data, uh, we can, I will put it in brackets, uh, gather from the Internet. From my perspective, many product designers or product people think that they don't need any kind of person from outside to bring the knowledge. As I get it, this is not about the knowledge, it's about organizing a chaos, as you said. But what does it mean, the secret sauce of this kind of good, uh, collaboration?

Speaker A: I think it would be, rather than, like you said, you can be a specialist in your knowledge area, uh, in your competence. But like working with someone who can come and give you this analytical organizing of your specific knowledge is as simple as, like sometimes getting someone from outside to look at your space and just benefiting from it. This can also be your mother it doesn't have to be a researcher. Sometimes if you talk to someone from the outside, that's as simple as that. But I think in the context of design, um, specifically we are still learning. As much as we would like to think, we are all specialists, we are still learning about how to design the human AI interaction. There is so much that can benefit the grounded research. The work that is using some already researched frameworks, like a combination of both. In the time when we are doing something new, something for the first time, I think that's something we should all aim for.

Speaker B: Okay.

Speaker C: Yeah. I really believe that, um, any person from outside, especially when you build not only something new but something challenging, it means that it can be really small thing, but very different that you've done before. Uh, it's good to have someone, um, to ask proper questions. Sometimes I say that even our teams, our designers are people not designing as drawing, but designing as asking questions. The right ones.

Speaker A: Absolutely.

Speaker C: Yeah, I think it's about that, especially right now.

Speaker A: Knowing where to find a hole.

Speaker C: Yes.

Speaker A: Knowing also which question is the most important to solve first and to organize the order of the questions. What connects with what and what comes out as a result of what. That's absolutely good. Uh, mind work here.

Speaker C: Yes. Okay. Agatha, this has been, uh, exactly the kind of conversation I hoped, uh, it would be. And thank you very much for being here. Um, for everyone watching, we will have links to the frameworks Agata mentioned that we've discussed in this episode, uh, in the notes, uh, in the YouTube or in Spotify and Agata's contact details if you want to explore a collaboration as well. Uh, yeah. So thank you very much and see you in the next one.

Speaker A: Thank you very much. That was a really nice chat.

Speaker C: Thanks. Bye. Bye.

Speaker B: Thanks for listening to this episode. I invite you to join the Product Builders AI native community powered by Boulder. And if you would like to join feature future live sessions, follow us on social media. All the links are in the episode description.