HCAI 11 - Ubiquitous HCAI with Niklas Elmqvist

Speaker A: Welcome to a new episode of the Human Centered Artificial Intelligence podcast. And today we have the great honor of having Professor Niklas Elmkvist with us. Niklas, you're a professor of computer science. Would you like to tell us a bit more about yourself? Sure.

Speaker B: Thank you for having me. Um, my name is Niklas. As you heard, I'm a professor here at Oris University in Denmark. I have, um, just joined this department about a year ago. Before then I was a faculty member in the United States at two different places, University of Maryland and Purdue University. And before then I spent some time as a postdoc in France and also did my PhD, uh, in the beautiful city of Gothenburg, where I think this is being recorded in Sweden. So I'm Swedish originally. Northern Swede to be exact.

Speaker A: All right, awesome. Thank you. So, as you know, Niklas, in this podcast we're trying to figure out what human centered artificial intelligence is. And so far we've had 10 different perspectives on AI, from data science to governance to explainability. Your perspective is more from human computer interaction, right?

Speaker B: Yeah.

Speaker A: And I know that some of your work are things like working on multiple devices, ubiquitous analytics, visual analytics, and so on. What kind of relationship do you see between that HCI field and human centered AI in general?

Speaker B: Yeah, that's a good question.

Speaker A: I think.

Speaker B: Let me first say I'm a newcomer to human centered AI. I think that may be true for most people. It's a fairly new field and we're still trying to figure out what it means. But I approach this from, as you said, from technical HCI and from data visualization. So my home research area is data visualization. And I think in general, human centered AI is about this intersection of human concerns, human values and so on on the one hand, and the more traditional artificial intelligence, machine learning and so on on the other. So in that intersection, I think there's space at the table for people of all different disciplines and backgrounds. I represent the more, I suppose, technical, not you, uh, know, technical side of human computer interaction. And I can tell you about this through, through this, this episode. I think in the last year I've been, been giving some talks, invited talks on human centered AI and how it applies to data visualization. And what I tend to say there almost in the, in the beginning of those talks is that for me approaching artificial intelligence has been a little bit of a, uh, journey of discovery and to some degree facing my fears. Because we have this growing focus on artificial intelligence in all of computer science, in all of society as a whole. And it's come to the point where I realized I need to learn more about this field and see how it intersects with my own.

Speaker C: All right.

Speaker A: I find that the last few years where people from other fields are coming into this, this field and joining and merging and you know, adding their perspectives makes this just so much more interesting and fun to work with. So I'm really happy to see this type of things like, you know, adaptive visualizations and so on. Getting a voice in the AI field.

Speaker C: I mean, uh, I'm kind of getting curious here. How do you then in your talks or just for now, like, what's the relationship here between data visualization and AI?

Speaker B: Yeah, so that's precisely the question I've been grappling with over the last years. I saw, you know, this last year in particular, I've had this nagging fear, I think, of a field like mine, data visualization, where the human is so centric, it's part of this interaction loop and worrying is this field going to go away or are we going to be replaced by AI? Do we actually need a person analyzing data? Do we need these fancy visual interfaces to perceive data? If we can just, you know, short circuit that whole thing by some advanced model? And I think, you know, I'm not sure I have all the answers to that question. But what I have discovered to cut to the chase over this last year is that in many ways I have been utilizing AI in my research for at least, uh, a decade. That's kind of what I found after some of this investigation. And that maybe is part of it, is understanding that the field of AI is not just the most recent flavor of the month. It's not just generative AI, uh, it's not just large language models, it's not just deep neural networks, but it's an entire spectrum from basic statistical methods, machine learning methods, regression and so on, all the way up to the most advanced models. And if you take that picture, this becomes, I think the field of human centered AI becomes a much larger tent that is much more inviting to people from all kinds of backgrounds to feel that they have something they can contribute and can participate in this work.

Speaker A: Some of the, um, works you've been doing are on visual analytics and I feel that that's a very nice application of AI that we haven't really seen as an AI field in the past. But it's become so clear that it is an evident application of AI.

Speaker B: Mhm. Yeah. I mean, at the moment, in fact, one of my collaborators here at URUS and I are writing an Article for IEEE CGNA Computer Graphics and Applications. And we're calling it, slightly tongue in cheek, Automating the Path. Because the original research and development agenda for visual analytics was called Illuminating the Path. I actually have the book here, this One is from 2005 and you look at this book, it was written after a series of workshops. A bunch of, I think 30 or so people, mostly Americans came together and they had these Canadians too, I think, had these workshops and some training and ideation. And they wrote this in a very short amount of time and it sparked the research field of, uh, visual analytics. So I think it's interesting because, yeah, the book was written very much from a Department of Homeland Security in the US kind of perspective. So it's very US oriented. But since then the field of visual analytics has been established in the international scientific community and it has become an accepted part of all this. And almost to the point, I think, at least in my mind, visualization and visual analytics are almost synonymous. But if you look at this agenda, they do talk about this intersection between visual interfaces, human reasoning, cognition, perception and automatic algorithms, which could be cluster analysis, could be natural language processing. Certainly many of these are what we'd call AI or machine learning techniques, but certainly nothing of the more advanced that we see today. So the thesis that I'm trying to develop here is to say that visual analytics is precisely, I mean that was essentially what visual analytics had in the beginning, this idea of a meaningful human AI teaming initiative. And if 2005, 20 years ago almost was where that discipline was founded, we don't so much need a new discipline in 2024, 2025, that when this article will come out. Instead we want to adopt human centered AI as a way to maybe correct the course, a little bit of visual analytics so that we do take these advanced model, large language models, generative AI and so on into account.

Speaker C: Okay, so just to see if I understand here where you're coming from, so are you in any way kind of suggesting or hinting towards that one could get the idea, and this is not what you're advocating, but that someone could get the idea that you could replace visualization with AI, where like AI could replace whatever utility that would be for visualization.

Speaker B: So these are very good questions that we're grappling with. And I'll say already that a lot of the work that I did with my students ten years or even five years ago, for example, we've done some work on recommendation systems for computational notebooks like Python and creating suggestions to tell, uh, a person, how could you proceed with this analysis that you've done so far based on data mining? Lots and lots of jupyter notebooks on GitHub. That kind of work is almost obsolete by now because now you can just upload your data set into ChatGPT or Claude or some equivalent model and say, okay, can you help me analyze this? And it's going to do it. So I think it's clear that it would be naive to say, oh, we're still going to have humans to play the same role they did before, but at the same time. So that will certainly change. But at the same time I think there's clear value in thinking of AI, uh, not as a replacement, but as something that enhances our work. And in that regard I certainly subscribe to my former colleague, UMD colleague Dan Schneiderman's view of human centered AI. So I brought more props here. So I have his book on human centered AI and, and he of course was uh, a good mentor of mine when I was a faculty member at umd and he was the first director of the Human Center Human Computer Direction Lab that I also directed for a while. So I've had conversations about this and, and with him in the past and, and his focus, which I also have adopted myself, is that we should be redirecting these efforts to create tools, to create human centered AI tools, just having the same role like tools have had throughout all of human history. So these are not supposed to be, these are not supposed to be partners or you know, virtual humans or maybe not even assistants. But these should be just like the shovel or the excavator or the word processor. These are our tools where the humans still retain agency and control and we are not just just inserting some black box model in place of the human. So that I think is, that's my key own fundamental philosophy in this, in how do we use AI, uh, to create tools that support the human rather than replace the human.

Speaker A: So that book that you showed, we're quite familiar with it, we started a, ah, master's on human centered AI three years ago and we used that book as sort of ideation, um, and inspiration for the entire program. So I think every faculty member in our department has a copy of it nowadays. How would you say that the perception of AI as a tool in HCI has changed over the last few years?

Speaker B: That's, that's a good question. I mean, how do I respond to that? I, I think perhaps, you know, in hci, the fundamental view always has been an interactive framework where you have a human and a computer. And there is, and that's particularly familiar to those of us who do data visualization because we have interfaces that enable someone, a person to perform actions on the computer application. And then we have visual output, often visual representations, interactive visual representations that show the state of the model. And so I would, I'm not gonna, I, I, I honestly don't think that this view of AI has changed so much within hci. I mean even work on mixed initiative interaction from, from 20 years ago still talks about an interchange and uh, between these two. Even Ben Schneiderman's and patting Mace debate at IUI and kai where they talked about agents versus direct manipulation. I mean in that case, Ben was representing the HCI side and Patty was representing more of the AI side. And it was a bit of a clash, but they still came to this synthesis of an interact, the interactive loop where there's a human in the center, the human is part of the loop. So I think if I would redirect your question a little bit, I think what we have seen instead is that where AI research for the longest term time has been its own silo and has now grown massively in the last few years. And of course it's gone through summers and winters since the beginning, we're now starting to see AI researchers also recognizing that we're going to have to involve people like Fei. Fei Li had that New York Times op ed thing 2018 where she said we should not replace humans, we should enhance them. And Michael Jordan, I think Berkeley, same year, he said, oh, we need a discipline that looks at this intersection between humans and computers. Maybe not realizing that discipline already exists. But what I'm saying is that we're seeing that change. I mean, yeah, it's happening even within AI that they recognize that their viewpoint needs to change. And a lot of the incentive for human centered AI actually is coming from the AI side too. So I think coming back to your question, HCI has grappled with how do we fit AI into this, into this interactive loop. AI is coming to terms how to do it. And that's uh, what we're seeing now, I think.

Speaker C: Well, I'm curious because you said in the beginning that you came from sort of technical AGI point of view. Right. Do you have the same sort of approach to AI that you think of it from a technical side of AI or do you have more of a human side into AI and a technical side into AGI or.

Speaker B: Yeah, first of all, I can say I'm a consumer of AI. I am not so much an innovator or researcher in AI, so this has also been slightly something of a journey of discovery. Again understanding that intersection and where the more we know about the AI models the better we can use them. But I think in order for this to be an effective partnership, going to need more HCI people on that side of the camp that, that can, can uh, influence the model so they have the necessary hooks to provide insights into the models and not just treat them as black boxes where we don't really see what's going on. But in general data visualization is, you know, very data intensive. Of course there are different flavors of researchers in this field. There are those that had more of a uh, uh, psychologist, uh, cognitive approach or more on computer graphics, more on design. I, I call myself a full stack visualization researcher which means that I. One way to see it is that I, I'm a generalist, I work across all these different areas. Another way to say it is that I have a low attention span so I spread myself very thin. But yeah, I think all that we do in my group has a human focus, even if we have fairview. All my students are software engineers. They build stuff, they have a very technical focus. But especially now that I am recognizing this human centered AI field, I am redirecting a lot of my research towards that direction, which means it will have more of the humane or human concerns that you mentioned. So thinking explicitly about this is transparency, about communicating the models to understand provenance between the work that a human and an LLM did. If they're writing a text, for example, I have a student that's been working in this field for a couple of years where it's all about AI, uh, co writing and how do we make this maximally transparent, how do we show what you did and what the model did and so on. So I think it's a little bit of both. To answer your question, that it can't just be technical. That's the good thing about hc, AI.

Speaker A: This makes me think of a term you said earlier. You said human AI teaming. Right. So when you have an LLM as a co writer, I'm assuming this is an example of human AI teaming. Aside of this LLM co writer example, what other aspects of human AI teaming are there?

Speaker B: Uh, that's a good question. We've been trying to think specifically within data visualization and data science. What are the roles of models like this? And to go back to, to Dan Schneiderman's tool perspective, he talks about four different capabilities. He talks about amplifying people's abilities, existing abilities, like increasing someone's ability. He talks about augmenting their ability. So adding new capabilities is how I interpret that. He talks about empowering people to do things they couldn't do before or enhancing the work, so improving the quality. So I think that's a useful, those four capabilities. What is a human um, centered AI tool capable of doing? I think it's a useful model for how to think about how AI can be applied to different domains. So one way is for. You said about co writing, which is work, uh, that my student has done and even that work has some visualization component because you know, at the core I'm a vis person. So then what does that mean? It means and these four terms they do like amplify, augment and power enhance, they do kind of are kind of blurred. It's hard to say exactly which one is which. But you would improve the quality of your writing. You might reduce. So that would be enhancing, reducing spelling errors or being able to increase your outputs so that you can write faster than you would otherwise. That might be something about amplifying or even being able to translate something to a new language that I don't even know. I mean I'm in Denmark now. I'm Swedish, I don't really speak Danish. I speak some weird mix. But I've had many situations. Or I asked Claude to take uh, um, an email. I wrote in English and just translated to Danish. So that's another example I think of maybe empowering. So that's one way that lens of thinking of what are potential capabilities and then applying it to a domain. In the domain of visualization, at the moment, my students or my collaborator, uh, and I are trying to think what do these four capabilities mean when we instantiate them for data exploration in a visualization or when we're trying to create a report or when we're trying to extract knowledge. And so that becomes a little bit of a design space and uh, generated design space. You take these four capabilities and you cross them with the uh, different parts of visualization and sense making. And then you try to come up with what does that mean for each combination, what's already out there and what are the potential future capabilities where this could help.

Speaker C: So you mentioned black boxing and you mentioned how like one aspect, if I understand correctly of heai is to kind of sort of remove the black box, right? I guess you can see the black boxes in different ways. One way to kind of deal with it is sort of an uh, onion with Layers on layers on layers. And I guess you can't really go all the way into the core of it. Possibly. Or perhaps you can like uh, what sort of obscurity do you think is most important to kind of remove.

Speaker B: Yeah, I would say visualization in general is one of those few disciplines that I think really can help, not just can help AI research and provide insight into the models. Um, how does that mean though? I mean how does that work? I mean that's a different uh, topic. Like I said, I'm more of a consumer of these. I think it requires some specialized insight on how the models work because you could. It doesn't make sense to visualize a graph of a few billion parameters.

Speaker C: Right.

Speaker B: Which is what these look like in practice and seeing what gets activated. I mean there's a reason that you can't really generalize what the connectivity structure of a deep neural network looks like to understand the underlying principle. It, there's no generalizable knowledge that way. So I think you're right when you say you have to peel away the layers and maybe there's some layers you can look at, but maybe not the innermost ones where it's just down to nodes getting, you know, nodes getting activated. So for example, where it might be useful if you have an image model, resnet, let's say that classifies images and then visualizing the attention of um, where, which different parts of the image is generating different types of output, that certainly is fairly straightforward. It would enable someone to debug the model to see it's misclassifying a part of the image and that's part of the image. And maybe you can then understand why by just eyeballing the image and understanding it. But I think it's uh, I don't think there's a general approach to visualizing AI models that can be applied to uh, any of them. I think it needs to be custom made for each particular model.

Speaker C: So I mean, I guess there's so many different aspects of black boxes as well. And I think when we talk about it as we're more, or you see yourself more as a consumer of AI perhaps then it's not so much important how the model works. As you say, visualizing the parameters and the weights and stuff like that and activations that might not be important when you alert to this kind of permanence, who has been involved in the production of whatever outcome is if it's a image being generated or text being co written then being able to show that it wasn't just a human Writing, sitting, tapping with his little fingers on the keyboard. But it was actually like using a tool, using a large language model as a kind of aid in the production of the text. I mean, yeah, I guess it's another way of seeing the black box being opened up. Right?

Speaker B: Yeah. And I mean just embedding these models into this interactive framework where it's not just a pipeline. I ask a question, I get an answer and then I'm um, I should just satisfy myself with that answer and not do anything more. But having the familiar, now familiar Chat interface with ChatGPT, where it becomes not just one prompt and one output but you can build and you can start, uh, a conversation is obviously a form of transparency, a form of explainability because you can get, you get more, it becomes the process itself, the interaction itself becomes part of understanding and part of seeing insight into the model. And a lot of, at the moment, a lot of HCI work in this field are um, little more than thin layers on top of chat GPT. We're still in a bit of a gold rush where, where a lot of people are just grabbing ideas and they're managing to publish it. I think we're, we're getting past that phase to some degree but, but I still think there's value in turning or to bending the pipeline from a straight line into uh, a cycle where we do have feedback and there's a conversation between the human and the loop and making it more human centered in that way.

Speaker A: I ran across this term explanatory transparency somewhere, I can't remember where. So when you have explainable AI, you refer to local, uh, or global or model specific or model agnostic explanations and those are fairly technical. Right. Whereas you can have explainable AI with an LLM that sort of explains or justifies why something happened without necessarily going into the nitty gritty details of the model, making the explanation very much more sort of user centric or human centered to sort of get someone a feeling of understanding on an abstract level rather than like, oh, you know, that specific network triggered this and this. I mean, I don't think that that makes sense most of the time.

Speaker B: No, exactly, yeah. And I mean the curious thing is you can ask ChatGPT or Claude to tell you a little bit about how did you come up with this answer? And it will try to tell you and it's not. I mean you could ask yourself is that really true? Is that really what's happening? It's the same thing if you, if you're as a human computer interaction person you run a think aloud protocol and you ask someone tell me what you're thinking and they will tell you one thing and may not necessarily be what they're actually thinking, but still it does give you some, some reasoning power. Now you have such complex models, like a large language model.

Speaker C: Yeah.

Speaker A: Another line of work that you've been doing is multiple devices. Uh, so I saw this video uh, for your CACM paper where there was a laptop and then you added a cell phone and you know, it sort of augmented the view and then you added a tablet and it augmented the view again. And I'm um, thinking how much is AI an aspect of that and how. So how does the sort of, the sharing of content work?

Speaker B: Yeah, I mean that work on cross device visualization, I call it ubiquitous analytics, has been part of my research repertoire since I started as a faculty member in 2008 or so. And so my worldview of human, uh, centered AI and the work I've been doing is slowly coming together this year has been um, instrumental for that. I gave my first invited talk in January this year and I'm trying to see now how these things fit together. So my answer might be a little half baked, but I still think that the core focus of my uh, research on ubiquitous analytics is to understand how visualization, data analytics, in fact any interactive work that involves a computer is not just an interplay between a person and a device like you see through a screen, your monitor or your smartphone, but it's a system, especially for data visualization where it's an intensely cognitive ability that requires you to synthesize lots of sources of information and analyze data and you might use a piece of paper to write notes. You might talk to your colleague about the data you're seeing, you might work on it collaboratively, you might go to sources on the web. So the point is that in the work that I've been doing in ubiquitous analytics and especially for cross device settings, it is trying to recognize that the entire world around you needs to be instrumented so that you can support the person doing the sense making who's trying to understand the data. So that's been the focus of my work for the last 15 years. That's my Willem center here in Aurus on, on, and that's the CACM article as well on, on data analytics anywhere and everywhere. And now I'm trying to, to, to understand how that fits into human centered AI. Uh, there are AI, uh components in all the work we've done. You know, we're doing, we're doing Cluster analysis, we're doing natural language processing, we're doing regression analysis in many ways already across devices where you can bring in a smartphone, you can have a large display, you can have your computer and so on. You can have not just the mouse and keyboard, but also gestural interaction, time interaction and so on. But now I'm starting wondering, how do we incorporate gen AI and how do we incorporate large language models into this? For example, by having more advanced methods for organizing the visualizations around us, perhaps by having voice control instead of, you know, the typical interfaces that we might surround ourselves with by having some kind of data almost like an assistant. I mean that, that can, where you can author a visualization by speaking. You can ask for insights into a data set, maybe using the spoken commands, maybe using gestures, pointing and so on. So like I said, the answer is a little half baked. But I think this idea of um, um, ah, cognition as a distributed ability that involves not just your brain but also everything you surround yourself with, all your physical devices, all you, the other people around you is eminently applicable to human centered AI, to incorporating AI models so that we don't just instrument the physical world that surrounds the analyst, but we can also infuse it with AI, essentially.

Speaker C: So I'm trying to see if there is a way to sort of place the AI or where is AI here? Uh, a soft blanket over this whole thing, like all these things, the distributed cognition, you know, you have your padlocks or your screens and maybe other devices and stuff like that. Is AI? Should we think of AI as yet another device, another one of those? Or is AI something that's omnipresent? Like is there a way to place AI? Or perhaps there is not one AI, Right? Maybe there are many.

Speaker B: Yeah.

Speaker C: In what way are you thinking of it?

Speaker B: Yeah, how does that fit into my worldview? Uh, we have many examples of very local models that belong to certain devices that belong to certain contexts. And in that sense it is very localized. I mean, I'll give you an example. One of my PhD students over the last five years has been working on modeling human attention, um, for example, so that if you're designing a new visualization, you can run it through a, uh, virtual eye tracker instead of running a real eye tracker experiment. It will tell you what, based by training a deep learning network, it will tell you what this visualization, this is what people are going to look at. Is that what you expected? That's a very local model and it's useful for a few things. It's very highly Specialized and it can help certain settings. So that's one role of AI. And that's what I said, that's what I realized I've been doing in the last 10 years or so actually. Uh, these are examples of AI. They're not large language models, they're not the current trend, but still part of my repertoire. But now I'm seeing what you say this could be more omni present, something that's, that's more. Binds things together. When you have very capable large language models and foundation models where it can become essentially a kind of a glue. Like in the past we've always used the human in this interactive loop as the glue that binds things together. And that's why in general, the part of computing that has a user in the loop had some very early successes. Right? Because there's a human that interprets, they get the data in some form. Computing will help improve the displays and I can make better decisions. So the human is the blue. But now with these very capable models, it is becoming possible to have the AI also serve as a glue, a multifunction all around kind of medium. I think maybe that's the way to, to think of it. A medium where you can, you can ask it to do all kinds of things that are just nice like they, they bind things together and offload the human in many ways. And I'll, I'll see if I can come up with a good example. Well, that could be. We have an ongoing research project on creating augmented reality visualizations. It's an authoring system in augmented reality that runs on a, uh, you know, normal Quest 3 headset. It just uses WebXR, it's all web based, open source and so on. And the problem is when you're in xr, you're an immersive space. You normally don't have a keyboard, you normally only have your hand controllers, your hands. So authoring complex visualizations, often you see JavaScript or JSON or something. It's not something you do right because you'd have to sit down and normally you take off the headset because the video pass through is not good enough for you to read the screen. All kinds of problems. But now we've found that we can just hook up the OpenAI whisper model to do voice input and then get the voice input to send to uh, GPT, right? And we can say, here's a visualization I'm viewing at the moment, can you please add a filter on the X axis to show houses that are between 1 million and 5 million krona let's say. So that's the glue. I mean it is kind of magic, a magic medium. Because now the AI is, the GPT is the glue that I don't have to have a person go in and do these manual actions. We can offload those interpretation to actually change the JSON source code to affect that change. And so it's opening a lot of possibilities, the scope of which I don't think we really fully realize yet. And also we perhaps don't realize the limitations.

Speaker A: But I guess this somehow connects to how AI is becoming this tool. Right. So sort of, uh, using uh, an engine to power something, you don't need to know how the engine works, you just know that it can do this. Right. And this is sort of what you're, what you're describing. Like we're now using AI as this tool for stuff that um, that we didn't have before. And we don't necessarily need to know how it works. We just need to know that it will actually deliver what we want it to and then linking it to whether it's omnipresent or not. I think from this perspective, it has to be, in order to work this way, it has to be this glue. Otherwise it'll just be uh, a model like the models that we've had before.

Speaker B: Yeah, I agree. I think I had a conversation the other day where we're getting even to the point in AI where these large language models are a commodity and we buy them off the shelf, we don't have to be involved in them. And you know, the interesting thing is what we do with them, not, not the models themselves. I'm sure AI researchers will dispute what I'm saying. But, but from the perspective of, uh, how these models will be used to society, they have become a commodity, a utility almost that we can tap into. And the other thing you said, which I think is worth thinking, um, about is, and it relates to what I was saying, we have some models that are local, very specialized, and then we have this new thing that is almost kind of a medium, it's a glue that binds things together, is that it's worth thinking about whether we want to send all our data to OpenAI servers or to anthropic, um, servers and then trust that they don't use it for bad purposes or for training their own data. So all of a sudden I'm going to see my own business secrets show up in output five years or two, two years down the line. So. And I think that Apple, for example, is, is recognizing this with their new OS updates, where some of the you can say, I want to run my model locally, it's just going to be, it's not going to leave my computer very clear what is the scope of what I'm asking. And try not to erase the boundary between local and Internet or global models, which might mean you lose some of the capability because it's just going to run on your machine now. But, uh, on the other hand, you're not leaking any information.

Speaker C: Sometimes I think the trouble talking about AI is we often talk about it as one thing, whereas it's obviously many different things. Yeah, talking about microchips as one thing and, you know, it's used in many ways in a computer these days. And in this case in your example, I mean, you have like the whisper that's sort of the input modality, and then you have the chatgpt writing the code. That's sort of the output modality, creating the graph, whatever. But I think, uh, a missing part there is also the sort of what's new. And that is like the understanding. Right, the understanding, our intent or change. Change this. So I see this on the X axis. Like, you know, it's been a long, long winding road to figure out how to create a machine that can understand what you're referring to.

Speaker B: Right, yeah.

Speaker C: Uh, and now we sort of have that. And I think that sort of seems to be the new thing. And I guess that's, that's the part that can become this glue like we've been glued before. And now that the sort of understanding capability, uh, yes, comes to glue.

Speaker B: But I would also add to that, which is that as an HCI person, it is frustrating to see literally decades of, um, progress on graphical user interfaces, direct manipulation and local. You know, all these things go out the window and now we're back to a command line, or maybe not a command line, maybe even worse. It's a natural language interface, very versatile, but also not very precise. So I'm excited about work. You know, some people say I already talked about a gold rush in hci where people are building all kinds of thin layers on top of ChatGPT. And some people say it's just glorified prompt engineering. Sometimes I certainly, you know, see that part. But I also think that there are, there are interesting innovations to be made in that space where how can we take in some of these old ideas, like direct manipulation and like instrumental interaction, minimizing the indirection in the interface, um, being able to reify commands that I issue as maybe natural language, but then reuse and undo and redo and so on so that we can have some kind of golden, I don't know, combination of, um, these advanced models, plus all that we have learned in HCI over the last 50 years.

Speaker C: I don't think, uh, there's any better way to end this conversation. That is so good. And that is so, uh, right down my alley. I think that sometimes we feel like, you know, the pace of AI development is going super fast. For me, personally, I don't think AGI is moving fast enough. Together with AI, I think it's like, it's a shame. We just see these kind of AI, uh, sprinkling on top of existing stuff. There's so much we can do and there's so much we will do, and you are really alluring to that, which is really refreshing and really nice. So, Niklas, thank you so much for joining us in this conversation, and I hope this is not the last time.

Speaker B: No, absolutely. Thank you for inviting me. This has been great.