Educating the data and AI policymakers of the future

Speaker A: Today we are living through and witnessing in real time a revolution. It's a technology and data driven revolution that's affecting every aspect of our lives, from how we educate ourselves and our kids to how we access health care and how government decisions are informed that uh, then affect every aspect of our lives. So it's never been more important than right now that we understand how to think critically about these technologies and indeed the information and data that they use. We need to be skeptical about these things. Tech Bros in the deepest depths of Silicon Valley have a habit of moving fast and breaking things. And if we're not really careful, we may find that these technologies start to break things, break us a little bit more than we may have wanted. Today's guest on the Tech for Good podcast has a view on all of this. She is Dr. Eleanor Carme and she's a senior lecturer in data politics and data justice at City St. George's University in London. She's an author, an academic, a researcher, a journalist, uh, a dj, but she's also co director of a new master's degree program at the university that is aiming to create informed policymakers for the future. Eleanor, welcome to the Tech for Good podcast. We have an interesting and potentially, um, wide ranging and extremely long conversation to have, I think. Um, so we're going to try and uh, pare this down a bit to um, something that makes a little bit more sense, uh, a bit more focus. And what that focus can be is a new uh, master's degree program, uh, that you are co director of at C University London. And it's all about data and how we should manage it, uh, ethically, how, uh, we should think about it from an engineering perspective at the same time as thinking about it from a human perspective. That's my understanding. But before we get into all of that, maybe you could give us the headlines. You've got this incredibly rich and interesting academic and journalistic background. Um, tell us a little bit more about yourself and your position at City University. Uh,

Speaker B: so first of all, thank you so much for inviting me to this podcast. I um, really like the kind of the premise of the podcast and I think that's kind of the guiding, um, motto for me as well. You know, how can we use technology in a good way, a good way for society and not for, necessarily only for profit or for tech CEOs. So thank you so much for inviting me because I think it's so important to kind of highlight these issues. Um, so I'm a senior lecturer in uh, data politics and social Justice. And in the past kind of 10 to 15 years I've been looking at kind of the politics of what's happening behind our screens. So that includes all of the ad tech industry and how they make money, what actually funds the things that we use every day, the politics around that, different, uh, kind of dark patterns and how different these kind of services design things to make us use them in one way over another. And then I was really interested in kind of how does that then shape how people understand and engage with these technologies. And at the moment I'm looking at what can we actually do if we want to have a say in all of these kind of processes. Um, so I'm really driven in kind of uh, peeling off the kind of the taken for granted or kind of things that we take think of as common sense. So I also criticize a lot computer scientists and how they explain and define different kind of things. Um, and I think it's really important, kind of leaning into, into the, the new kind of uh, MSC that we have right now because, you know, we're being fed with a lot of news about different types of technology, data, AI, and it sometimes can be overwhelming. We feel, oh, this is something that only computer scientists can talk about. But what I'm trying to say is that, no, this is something that affects all of us and therefore we should all have a SE table and kind of talk about it. Even though if we don't know the exact programming skills, we still know how does that affect us? And we can ask different kind of challenging questions about where it's going and how it's conducted.

Speaker A: Yeah, so there's, it's obviously an enormous nebulous kind of topic. Our world is being transformed before our eyes by data and the use of data, especially when it, when it's used to power things like AI. Um, the agenda seems to be being driven from a technological perspective and a bit of a FOMO aspect to it, where you've got an awful lot of people spending an awful lot of time and money saying these are incredibly powerful tools, we have to use them right now. But nobody's really sitting there thinking, hang on, let's really understand what the, the underlying ethics of all of this and how do we trust all of it just because a tech bro somewhere says this is going to transform your healthcare service or your whatever, um, what's actually happening with data, who's in control of it and um, what data is being used and what data isn't being used, etc. And so that's why it seems to me that the course that um, you're now um, leading at City University is quite interesting because, uh, correct me if I'm wrong, it's City, uh, university itself, although it's actually called City St. George's now I believe, as of exactly.

Speaker B: Yeah. Just kind of merged with the health school. Yeah.

Speaker A: Right. Um, so, uh, there's a long history at City of computer science and engineering and things like that, but there's also a very long history in terms of journalism and social sciences and things like that. And this course is very specifically a collaboration between those two schools, I believe. Right. So there are modules where you're looking at engineering and computer science and things, but then there's also uh, the social sciences side of it and uh, it's two schools in one that students will benefit from. So what is the power that that brings to this and what makes that kind of unique?

Speaker B: Well, first of all, it's also important to mention that it's very much a university of profession. So we really want to prepare people to whichever job they decide to take. And we're really focused on kind of policy. Now policy can be in kind of national level, local level. A lot of companies have data and AI in their, you know, in their different services or products that they're doing. So for us I think what is really important is first of all to give people a firsthand experience of, of what does that mean. So we're collaborating with the ICO, which is the UK's kind of personal data regulator, and also the Demos, which is a digital rights NGOs and we're connecting students to different kind of organizations that have a different kind of uh, outlook on these things and they get a first hand experience of what does that actually mean? Which kind of decisions or kind of decision making process do we need to make when we're making decisions in our own organization. And I think one of the, of the kind of the benefits of this course is that it brings together people from computer science and sociology together to think about, you know, stuff that basically affects society. Right. And up until now I think most of these kind of things were taught separately. So you know, if you are doing AI, uh, oh, I'm just going to do computer science, or if you're doing something about society, you're going to study sociology and criminology. And what we're thinking and what we believe is that the people who will make decisions around policy, whichever policy, again nationally, locally or in different kind of organizations, need to have, need to be equipped with both kind of computer science, hat of, you know, being able to analyze the data, being able to kind of, um, design, uh, the different technology with kind of, um, ethical considerations, but also to have the different kind of theories and understanding of what does that mean and the politics of, um, as you said before, inclusion, exclusion in data, ah, how do categories affect different kind of things? What are the environmental aspects of all of this, which is obviously also very important. So we believe that we're designing something that's very unique and very special and we're kind of giving people the right tools to be the best policymakers in whichever company organization that they will be.

Speaker A: Yeah, absolutely. And it's hugely important because I think going back to what I was saying before about an agenda that's been largely driven by the people who've invested in these technologies and the people who fear that they're going to get left behind if they don't use them is a bit of a hole, it seems to me, where policymakers, and I suppose we could look at that from a government perspective as the most obvious example, um, are somewhat following behind in the dust trail of these people that are driving the technologies on. And obviously legislating and creating policy is always a somewhat agonizing and slow process, meaning it's always trying to catch up. Um, but these are considerations that presumably should be at the forefront of decision making when developing these technologies and choosing how and what data is being used in them. And I guess, therefore, ideally you're hoping that there's a cohort of people that emerge from City of London, um, having done this course, having thought about ethics at the same time as computer engineering, and can then spread themselves around the world again and actually start to influence conversations in that regard? Um, is that the kind of hope that you might have for students that, uh, exit the MSC and go off to forge careers? Is that the opportunities that you hope they'll be able to, um, find?

Speaker B: Yes, definitely. And I think another really important thing that we want to hopefully give our students is tools to think about. Do we actually need the technology? Many times, as you said before, CEOs are trying to push different types of technology when actually that technology doesn't necessarily benefit the company, it might cost actually more, or your consumers might not want to interact with that. So I think just to the basic stuff like, do we actually need this technology? What does this technology do for us? Can we design it in a way that fits our purposes and not the CEOs? And all of these kind of things are things that I think we're not being taught because we're being taught that the tech CEOs know what they're talking about and we need to take their words verbatim and you know, we just need to accept everything that they tell us. So what we're trying to teach our students that no, we need to challenge the tech CEOs and have different kind of negotiations with them and maybe refuse or maybe negotiate the different kind of things and ways we want these technology to be developed by. And I think that is so important as critical citizens in a democratic society because we need to have a better kind of, uh, negotiating position, uh, where we want to lead, uh, the technology and not that the technology would lead us, if that makes sense.

Speaker A: Yeah. And that is the way it's going. And I can think of one really good example of that. There's a risk here that I end up becoming something of a Luddite, but I don't mean to do that. Um, before we started recording, we were talking about, um, a radiology company that I'm familiar with, um, that they're developing AI driven solutions in radiology. And uh, if there's any radiologists listening to this, they'll understand this problem. And one of a radiologist that I spoke to as part of that recently said, hey, look, with the evolution of AI to help generate reports from, um, things like MRI scans and so on, huge volumes of which now exist because of increasing demands. He said, one of the problems we've got is that about 15 years ago, um, we had some new technology brought to us which was essentially speech recognition so that the radiologist can dictate reports onto a screen. And what happened then was that the people, the assistants that they used to have next to them who would do the transcription for the radiologist as they were talking to them. So a human being typing, um, went away and now we've just got machines. And that caused a massive problem for the last 15 years. You don't have a human being expert next to you who understands not just the structure of a report as you're writing it, but the, but the consequences of what's being said. The oncology that you're talking about, all the other, all the details that as a medical professional you kind of have in your head disappeared because new technology came along and transformed everything. And only now with AI are they bringing, um, the assistance back, if you see what I mean. So actually that was a good example of technology misapplied. It caused problems because it extracted expertise because technology was favored. And now we're Only actually looking at AI as getting us back to square one, if you like. And so maybe someone should have thought back then that simply being efficient isn't always going to deliver the outcomes that you really want. What it left, what it ended up with was 15 years of, uh, radiologists leaving the industry. And the ones that are left are now burnt out and exhausted because it's just them and a Dictaphone. Um, so that's, I think, a good example of how we need to be cynical perhaps about everything that's happening right now. And when we were having a conversation before, um, I think cynicism was a word. Cynicism or skepticism. M. I'm not sure which is worse or better. Um, but if we sort of say, look, one of the things that you might learn on this course perhaps is how to think more critically about technology, to be a bit more cynical about it, to not just believe the hype and to say, let's look at this through a human lens. Is that something that you think might be a benefit from the course?

Speaker B: Yes. And I think you gave such a great examples because. Example, because you point out into a few really important things that we emphasize, first of all that top down decisions of adoption of technology are never good because you don't really consult with the people who actually use these technologies. Right. So many times what we're seeing is that you have company heads that are making decisions because they want to cut costs. And I get it, I know that, you know, everybody's not in a great position, the market is not great, but many times when you try to cut costs, it actually will cost you more in the future. Because as you said, if, if more and more people are dropping out or if the actual reports are not as good, then actually you kind of miss the whole point. Right. And I think that this is so it points to two things. First of all, if you're trying to cut costs, you should really think what might, you know, the drawbacks may be. And also maybe you can actually involve the people who would use the technology in the decision making process and see how you can maybe have a, uh, middle ground kind of solution where you say, well, why don't we uh, have AI do some of the parts. But at the end of the day the final decision making will be by humans because they have a lot more expertise. And I think, you know, the, the kind of, the notion of uh, kind of outsourcing expertise to machines is really dangerous because again, we're humans, you can never substitute us. That's what I think Even though, you know, I'm a huge Terminator fan since I'm very young, I still don't think that machines can ever change a human decision making process. And so in order to avoid mistakes and you know, many times when you actually ask people, it actually the adoption of technologies that were not consulted with the people who actually use them increase the volume of work and increase the mistakes. And so I'm just saying, you know, like you say, we can be skeptical, but we can also offer other types of solutions where you can still cut costs but not fire people who have expertise and have the people who actually understand the working process consult with them before you adopt the technology and see, maybe we can actually adopt it in a way that fits us and helps us. Because, because again, I don't say, I'm not trying to say that technology is bad. I use technology every day. Right. Um, I mean this is a technology first of all, glasses. But um, there's so many types of technology. I think it's really exciting. But just because you're critical about the technology doesn't mean that you're a Luddite and that you're against technology. You would just want it to work for you and not against you. And so I think these, this is a really great example because it shows that cutting cost is not always going to cost you customer in the long run. If you really care about the quality of work, you have to consult with the people that it affects them. And that's by the way, not only the people who use it, for example in this case the radiologist, but maybe sometimes also the patients. Right. And especially in a country where still we have public health service and not um, privatized, which I hope it will stay that way. Uh, uh, we as a society should have a say in how it's applied. And you know, that people would have, you know, an ability to, you know, maybe question that. Maybe I don't want my data to be uploaded as well in various places and things like that.

Speaker A: So yeah, yeah, um, and the same thing. So excuse me, we've been talking about. I, you know, I just pulled an example out of the healthcare industry and you could apply the same lessons all over the healthcare industry, which is as soon as you peel back um, the layers of healthcare and where technology has an impact, you can find all sorts of opportunities and all sorts of problems and um, a horribly fragmented kind of landscape that is definitely still a challenge. But you can take the same things and apply them in any other area of uh, society where people are using Technology to make big decisions that affect people, whether that's people in the treasury making budget decisions or whether it's um, uh, local authorities making decisions about how to allocate scant resources towards things like adult social care and things like that, um, critically important decisions that are getting made around, um, limited resources that always ultimately end up being delivered by human beings. And yet those human beings who actually ultimately deliver those services are often the last people that get asked about whether the service is going to be a good use of technology or not, uh, or would actually help them or not. And it's fascinating to see all of this going on, uh, around us, but until now I've never been aware that there might be a university master's level degree course that you could take where you're actually trying to think about these things in combination. So you come out with an understanding of actual engineering and what, what it means and how you can process data and understand and use it, um, aligned with some, uh, a more humanities type approach to trying to understand what the consequences and how to think about that in a human centered way. And I think that sounds really fascinating. But, um, you said something interesting last time we had a conversation, which was that. And uh, I've seen you in a video saying this as well before. Look, let's start by understanding that data isn't neutral. It's not neutral. So that seems like a nice springboard for a conversation about a slightly wider conversation I think, which is, um, what do you mean by data isn't neutral? Presumably that means there's an actor behind it. There's someone. Data is sourced and used intentionally by someone for some purpose and we should never assume otherwise. Is that what you mean? And how can we use that phrase, data is not neutral as a way to think critically about how we're uh, using new technologies.

Speaker B: Well, some, you know, historically some people had more data about them than others. And that was usually more marginalized communities, whether it's the black community or prisoners or people that were monitored and kind of spied on, whether it's by the government or whether it's by, you know, other types of sources. So the problem starts when, you know, we have all of these kind of automated systems that are then relying on kind of historical data to predict the future. And so in that way that creates a really problematic way for us to kind of, uh, kind of recreate the same problems that we have in the past, but also ignoring other things. So we just, just before we started to record, we talked about how there's actually Very little data about women's health, especially about perimenopause and menopause. Um, and that also relates to, you know, other kind of intersectionality, so black women or other types of intersectionalities. And the problem with that is that if you need, for example, thinking about the health system, if you want to create different kind of predictions according to the past, then obviously you're just going to recreate what already exists. And that's not going to help us, uh, to kind of move forward as a society. Because if you're just going to recreate what happened in the past, relying on data that is already skewed because some data is more represented and some, some other data about other types of people or, or places or things is less represented, then that is a huge problem. We're also seeing that right now with the Trump administration, who is intentionally erasing different kind of historical data or data about the environment or about gender and sexuality. Now, this is not a new thing, Right. You know, previous kind of authoritarian regimes have also been really intentional to delete and erase specific data. And the reason for that is that they realize how it is for us to understand ourselves better as a society and have a kind of a more diverse understanding about different types of situations. Right. So data is never neutral in the sense that you always make decisions of what you want to include or exclude from a particular type of data. Right. So that's the kind of stuff that we're teaching our students. Right? What, what do you include in the different kind of things? So I'll give you an example. One of the things that wasn't calculated during the pandemic, Long Covid, we only got the data about who died. Right. But long Covid is actually a really important, uh, data because first of all, uh, it have, it tells us an indication of, you know, different types of the way that the virus affected people. It also has huge links to the fact that there is a big unemployment, uh, kind of numbers that are happening since the pandemic because a lot of people are not able to go back to work. Right. And those data then can influence different types of policies. Right. We don't have, you know, people kind of forgot. It's like, oh, we don't have Covid anymore. No, Covid still exists and it still affects people. I know people who never came back to work after Covid. Right. And unsurprisingly, uh, the most people who have, uh, kind of chronicled diseases are women. And so, you know what I mean, this kind of kind of inter overlaps of how, you know, women have been historically kind of ignored about the different kind of diseases that they have and especially if they're chronicle diseases around pain and different kind of things like that. And we're seeing that with COVID again. Why didn't we count the amount m of people who have long Covid and because we didn't count them. There is much less treatment for these people. And that means that if we'll have another pandemic, and I'm sure that we will in one way or another, then we're unprepared for that. We're only kind of treating people as if it's about life or death, but it's also about the quality of life and that affects the market as well. Because if you have so many people who are unemployed because of long Covid and you're not treating that, that is a problem.

Speaker A: Yeah. And obviously Covid and um, the lockdowns and everything else had psychological impacts as well. So people, we don't really understand what it is. But there's definitely been a uh, uh, longer tail effect of the upheaval that we all experienced around then that's affected um, the work economy if you like. So that people are working differently and they're not necessarily working as enthusiastically. They might be doing different jobs, they might be less inclined to work for all sorts of different reasons. But because we don't have the data, we don't really have a chance of understanding it. And the last thing that anyone needs to be doing is for ideological reasons deliberately deleting information. Because you can interpret information however you want. But it is slightly alarming to witness what's going on over in America right now, uh, where there seems to be be actual data sets being removed, um, being made inaccessible to the public, which is kind of nuts. It's like, well, let's, we can, we can make an issue go away that we believe is a political issue by simply deleting information. Well, that's, that's just knowledge or just deleting knowledge. That, that, that is sort of alarming. But you know, this isn't a political conversation and I don't want to sort of jump up and down on that, but it is slightly scary. But hopefully people that come out of degree courses like yours will be able to understand that there is data and there's information and then there's uh, the nature of it and how well it's collected and what's missing from those data sets, how to address that and then how you can take that into proper policy. Making kind of decision making processes that are, uh, actually treating the whole population with as much understanding as possible. And I guess that really comes down to the root of everything, doesn't it? Um, and I think I also want

Speaker B: to add, I also want to add, I was just interviewed, uh, I think last week by a journalist about the ONS numbers, about unemployment. And now there is an increased number of women who are unemployed. And I think again, first of all, I would disagree with you. I think all of this is very political. Um, everything around data and AI is political and it is being influenced by specific political ideologies. Um, but let's talk about why are there more unemployed women? Right? A lot of it has to do with something that feminists have always been saying, is that, you know, if you're a stay at home mom, it doesn't mean that you don't do work, right? But as a society we don't value work from home, right. When you go to pick up your kid, you don't get paid for that, right? Or when you clean the house or when you cook for your kids or when you take care of your kids. But if you will outsource it for you to go to work, you will still have to pay for it, right? So that work has value in our society, right? But if you're a woman and you decide to stay at home, because childcare is extremely expensive in the uk, unlike, um, a lot of European countries who have realized that actually if we want to help women, which are traditionally, uh, kind of taking care of the kind of the family, um, and the children and kind of all of this stuff that's related to kind of homework, um, then, you know, we should actually pay for childcare and then women would actually go to work and that would actually be good for our society. The UK unfortunately is a bit behind and I'm sure I don't need to tell you how they expensive child care is.

Speaker A: Right, yeah.

Speaker B: So, you know, so this is a society decision of how do we value different types of work and how do we count that? Do we count that as work? If we don't count that as work, then, oh, surprise, surprise. And there's a lot of women who are unemployed. Are these women really unemployed or are they doing work that is not valued in a specific society and therefore not counted? This is the kind of things that we're asking, right? What is counted? What is the value of counting specific work over other. And what does that say about the specific society that these kind of counting is being conducted?

Speaker A: Yeah, yeah, it's tricky. We are getting into really big, big topics now, aren't we? But it's a case of saying, well, um, and then what. What does value even mean? Right? So, um, obviously what we are is we are all sacrificial lambs on the altar of GDP growth. But, you know, what does that even. What does that even mean? Um, I don't think any of us, when we come to be lying on our, um, deathbeds at the end of the day, think about what was valuable in our lives and wonder, well, I wonder what my GDP contribution was. Is per capita GDP now slightly higher because I existed? Well, then I've succeeded. That's not how it works. And so that's an even bigger conversation, I guess.

Speaker B: Of course.

Speaker A: And it is very, very political. And I think depending on your political outlook, you can look at, um, that question and come up with very different answers. So, for example, there's definitely, uh, uh, a suggestion now among certain policymakers that unemployment or the unemployed, um, where we think actually there's all sorts of reasons and Covid might even be a really big one, is not really considered. What we're actually looking at is a much increased number of scroungers, um, of, you know, people who aren't working as hard as they should. And they should definitely be getting back into the office because that's the solution to a problem that we haven't even investigated. But for political reasons, we think it's easy to blame certain sections of people, uh, based on no information. But, yeah, we could, we could really. We could talk about this all day. I could talk about this all day. I absolutely know. But. But, um, we're sort of slightly out of time, so let's just end it by saying, look, if. If people really do want to talk to you personally about this a lot more, um, they should probably enroll on the course, um, uh, when it's, uh, appropriate for them. And how should they do that? Where should they go? And how can they find out more about, um, all the different modules that are available for them?

Speaker B: Well, first of all, they can email me, and I'm happy to have a chat with whoever wants to join the program. You can also check the program, uh, on our university website where you can actually see which modules we offer and kind of who's teaching. We have an amazing staff from, again, both sociology and criminology and the computer science department. So you can kind of check out who is lecturing and things like that. Um, and yeah, that is kind of the two main places.

Speaker A: Yeah. And I know just from stalking you on LinkedIn, um, that you've got this really interesting background that I think goes all the way back to broadcasting psychedelic trance on radio as a dj and you've gone through journalism and you've written a lot of stuff. And I would argue that, uh, anybody who's interested in the course should just go and check out your own profile page because it's extensive and kind of impressive.

Speaker B: Um, well, not everybody has patience for LinkedIn, so, you know, that's a specific, uh, skill. But yeah, I mean, I think everybody, you know, everybody brings their own wealth of experience. Each one of the lecturer is amazing and super experienced with working with private companies as well. So I think for us it's, you know, we're not kind of this ivory tower that we're just kind of talking about theory. We're actually, you know, we're working with companies and we have, uh, experience with working again with companies, but also we have collaborations with again, both ICO and demos and we will invite other kind of companies to collaborate with us. So the kind of experience that you'll get is quite massive in terms of hands on. What does it actually mean to create different kind of policies, which kind of consideration you need to take into account and all of these kinds of things, which I think is extremely valuable.

Speaker A: Right. And my understanding as well is that there is, um, there are specialisms that you can go into. There are kind of specific modules that you can take the take that are specialized either in the direction of computer science or in the direction of the sort of sociological side of things. I'm trying to use an umbrella term, but, um. Is that right just before we wrap this up, that people with a particular interest, maybe in one area or the other, can actually specialize there their, their masters, Is that right?

Speaker B: Yes. So first of all you have the core modules and then you have electives, and electives go mainly between the computer science and the sociology and criminology. But also there's a very few electives that are from the economics department and also international politics. So you kind of get a really wide range of topics that are really, I think, relevant for whichever kind of specialization that you want to go.

Speaker A: Brilliant. Well, it sounds really exciting to me and I, um. When, when did the course start?

Speaker B: Actually it's starting this September, so.

Speaker A: So it's really neat.

Speaker B: It's really fresh from the oven.

Speaker A: Right, okay. So people should jump in. I'm tempted to just abandon what I'm doing and join in as well, actually. I just know it.

Speaker B: You should you should definitely do you

Speaker A: do part time for middle aged people?

Speaker B: Yeah, there is part time, yeah.

Speaker A: Excellent. I might consider that. But uh, in the meantime we'll put links to uh, yourself and to other things on our various places that we publish this. Good, um, luck with everything. I hope you get, I hope you're massively over subscribed.

Speaker B: Thank you so much. I really appreciate it.

Speaker A: Yeah, and it'd be great to talk to you again at some point down the line um, once things are up and running. And uh, it's a fascinating and critically important topic, especially right now, um, with the political forces that seem to be driving us in an interesting new direction. So um, it would be great to recap at some point down the line if you're up for that.

Speaker B: Yeah, I would love to. Thank you very much.

Speaker A: Excellent. Thanks very much.