Navigating regulation: The EU AI Act and its impact

Everyone who works in any compliance management systems knows that there is no such thing as perfect quality management system of perfect information security management systems. It's about the process. Welcome to STARS Limited podcast series about AI regulations. My name is Andriy, I'm head of Health Tech Marketing and today I'm happy to introduce you to a very special guest of mine, the head of Quality and Regulatory consulting at star, Antonina Borlichenko. Hi Antonina. Thank you Andriy. So as Andriy said, my name is Antonina Porlachenko. As of now my role at STAR is head of Regulatory consulting. So me and my team, we are supporting our clients with regulatory strategies, quality management systems definitions and information security management system definitions. Our topic today should resonate with a lot of enterprises and businesses and startups because it will be about the AI based services and products, about their regulations and generally about the upcoming EU AI Regulation Act. So to have some context around this discussion, Tonya, tell me what is the EU AI act, why its launch is so important not only in the European Union but worldwide as well? Well, European AI act is the first full scale regulation of AI systems and it will definitely affect not only Europe but also the global market as such, because the act will be applicable to all the companies who deliver their AI services to European market basically so it will affect all global companies who are present in Europe and obviously all European companies as well. Does it mean that the AI systems will be classified to some extent or like what are the implications to this categorization or what's the consequences for those categories? Yes. So AI act same as many other regulations. It introduces risk based approach and what it brings. It categorizes AI systems into three major categories. The first one is prohibited devices. So those products bring unacceptable risk and European Union is not willing to have these kind of devices being deployed. So it's different kinds of social scoring systems, manipulation techniques using different patterns and exploiting vulnerabilities in particular social subgroups. So all of these kind of systems will be not allowed in European Union. The next category is high risk AI systems and here we speak about AI systems which might have a serious impact on individuals, environment and society. As such, I am sure we will speak more about these systems shortly. And and the third group is non high risk AI systems. So all the applications of AI which are not falling into the category of high risk ending up here. So different kind of chatbots, deep fakes or emotion recognition systems will be in this category. Is there any specific criteria that basically filters if the product goes into high risk category or in lower risk category. Yeah. So AI act has a definition of which products are classified as high risk AI systems. So generally the definition as such has two parts and in simple words it boils down to the following. So in case if your product is or AI system is a system itself or a component of a system which is already being regulated by European Union and by one of the legislations which I listed in Annex 2 of a Act, then your product will also be regulated by AI Act. So that is first part of the definition. And then additionally There is Annex 3 which lists additional criteria which basically define what system would be high risk. So here we speak about different biometric identification systems, any systems relating to managing of the critical infrastructure, different systems related to like being used in the educational setup or in the employment setup. So for example for managing profiling of employees, all of these kind of use cases and then essential services use cases. And it basically covers both private and public essential services, any systems for law enforcement, migration related systems, border control and any systems for being applied for ensuring justice and democratic processes. So as you see it's very broad and there is a specific source. So basically I think it's contained in the document that anybody can check all these criteria that you outlined. Right. So there is a specific link, people can go and check those. Yes. So it is all part of the AI Act. And the first part, the definition as such is quite broad and it basically refers to to Annex 2 of the AI app. And then there is Annex 3 which lists additional use cases which will be considered a high risk having these classifications. How does it impact the development and deployment and management of those systems eventually for companies. So for businesses who build these kind of AI products and services. Yeah. So the first obvious thing is that prohibited systems won't be able like won't be prohibited to be a place on the European market. In case of violation of these, the there will be high penalties. The second implication would be that for high risk AI systems, developers of these systems won't be able to place it on the market without prior conformity assessment procedure and CE marking. So for some industries it'll be not anything shocking. So for example, health tech, in health tech companies got used to the fact that they have to go through a conformity assessment prior to releasing their products to the market. And for some other industries it will be quite novel thing so might be a little bit shocking. And then for these non high risk systems you can say they will feel this AI act release the list. So they might decide to implement some of the aspects of the AI act, they will have to inform their users about the fact that they are dealing with AI technology, but other than that there is no much enforcement on them. And can you then briefly describe what a conformity assessment may look like for the companies? So is it a particular body that checks it within the company and how the whole process looks like for the company? Yes. So it will be fairly similar to how it is happening now for health tech products. So in order to get the CE marking on the medical device, it is a software medical device or any other companies have to go through a conformity assessment procedure conducted by a notified body and demonstrate that they followed all the required processes, show that they had PMS implemented and followed it, and then all the technical documentation of the product is available which is an evidence that the product was developed responsibly with following all the required processes. And it will be fairly the same for AI act. And in some cases, like for medstec, they will have to like this confirmatory assessment procedure for AI act will be part of the usual conformity assessment procedure. So when they place the medical device on the market, they will just have another set of questions from that body and they will still have the same declaration of conformity for the device. But now they will have to just include AI act requirements, reference to AI act in IT as well. If the product is within the prohibited classification or there is some kind of incompliance within the product, what are these fines that you described previously and what it pretty much can mean for the company itself? So AI act describes penalties for non compliance with the requirements it sets up. The first and highest amount of potential penalty is for placing a prohibited product on the market. So in this case it might be up to 30 million euros or 6% of the worldwide annual turnover of the organization. And then the same amount of penalties goes to those companies who do not follow the requirements of Article 10 of AI Act. And that Article 10 describes data governance requirements. So AI act has a lot of focus on proper data management. And all of us understand that quality of data which is being used for machine learning solutions, it is of high importance. And then a little bit lower penalty will be issued for non compliance with other applications for high risk AI systems. So in this case it will be up to 20 million euros or 4% of worldwide annual turnover of the organization. And then the lowest fine is up to 10 million euros or 2% of the turnover for providing incorrect, misleading or incomplete information to the notified body during the conformity assessment or Any other follow up reporting. And then it is worth to say that the amount of these fines and penalties will depend a lot on the nature, gravity and duration of the infringement and depending on all the consequences which happened due to that. And also depending on the size of the market share of a particular company who was involved in this non compliance with AIF requirements. Yeah, these sounds like a quite substantial penalties and they are quite big even for small and medium businesses. And I'm not mentioning even startups. What would you say is the best strategy then for startups to do today to avoid these kind of penalties and ensure that pretty much they are building an AI compliant product? Well, I would strongly recommend starting somewhere. So everyone who works in any compliance management systems knows that there is no such thing as perfect quality management system of perfect information security management systems. It's about the process. So did the same with AI management system. You have to define what is the most critical area of your organization and just start making sure that you describe the processes for that area. And with AI management systems potentially it would be a good idea to just as a first step, make the scope of that system only to the highest risk product in your portfolio and just play with it a little bit, set up processes and then extend it further. So my main call for all the organizations out there is to just start doing something and not wait because this process takes time and it's better to just start doing it. So can you say then that documenting having the AI management system also contributes to traceability, explainability, transparency of AI models and basic basically understanding how they work in any kind of setting. Yes. So quality management system has a number or AI management system has a number of different processes and controls which bring and contribute to the quality of the product in the very end. In those processes you can also define what is requirements and what is your approach to ensuring transparency, explainability, making sure that you assess risk of bias on every single step of the machine learning system development. So definitely management systems help you to be in control of what is happening. And when you're in control it obviously benefits the end result. Okay, if we talk about geographies, meaning the EU AI act obviously is more about the European Union. Does it influence somehow companies from abroad who are hosting their, who are building their products elsewhere rather than in European Union and what it means for them if they want to expand to the European Union from abroad? Well, it's the same as with other regulations. So if the company is outside, outside of the European Union but wants to place their product or services on the European market. They will just have to make sure that they follow the and conform to the requirements of the AI act and go through the conformity assessment prior to placing that product on the European market. That's one part. And another part is that European UN took this first step and soon AI act will be published. So I am pretty sure that other major markets and countries will start catching up. You talked also briefly about AI management systems and as I know there is already a standard out there that pretty much is the certification and the standard itself for these AI management systems. So I saw 42.0.0.1. Can you quickly go through like describe what it is, how it helps and is it something that companies should be aiming for acquiring and incorporating into their workflows? Good question. So yes, ISO 4201 was released just recently in the end of 2023 in December. It is very, you can say unique, a little bit standard because it's a management system which covers a little bit of other systems as well and integrates really nicely with other systems. So for example safety related management systems or information security management systems. And it is also industry agnostic. So basically it would benefit any company who develops or uses AI products. So therefore I think that it would be a really good idea to start with ISOFET 2.01 implementation as a first step building compliance towards European AI Act. And as of now it's worth noting that there is no accredited notified body to assess the conformity with ISO 22001, but notified bodies around the globe are working on that. So I would expect in a couple of months there will be notified bodies who will be able to run this conformity assessment procedure and issue certificates of compliance with ISO 4201 requirements. With all these requirements to manage the whole process of AI development, would you say that now there is a specific need to have a team or like at least one person who is dedicated to managing compliance within the AI development on the team or like outsourcing it elsewhere or just having someone who can oversee these operations? I think it depends a lot on the organization size, on the complexity of all the products and complexity of the organization's context, external and internal. But generally it doesn't matter if it will be internal person or external consultant, or if it will be a group of people working together. What matters is that organizations should start really making sure that someone is moving them towards compliance and straightening up those processes and implementing the missing pieces just to make sure that they are not behind. Obviously the whole compliance thing introduces the next level of things to consider, things to manage inside the product, inside the business. How would you say it impacts the timelines of AI product development and innovation itself? Everything about processes and requirements seems to be kind of boring and heavy and not needed. But in my opinion it's about trade offs. So on the one hand you can have speed of development and path to market. On the other hand, knowing the power of AI as a technology, you might bring up something really harmful to the market in the very high risk use case. So in my opinion, for all of us as users and a community, it's better that products are being developed responsibly at least. You know, obviously that process will not reduce risk to zero, but it will at least minimize it and somehow control it. So my opinion is that yes, it's not easy, it's not the easiest way, but as I already said many times that the easiest way is not always the right way. And usually the right way is, has its own traction. So I think it's just right thing to do and we have to do it. And we have to work, all of us like we as consultants can support companies, we can work with notified bodies and regulators to make sure that we collaborate and make this transition as smooth as possible. I totally agree. I'm fully with you on this topic and this argument that it's always about the trade off and eventually you're talking about quite sensitive decision making or producing data from the AI which should be regulated, which should be quality assured, which should be managed to some extent. And it also should be documented quite clearly so it's easy to follow how the system works itself. AI and machine learning as technology is really powerful multiplier of any bias in our human behavior. So if there is one human who is biased, it is bad, obviously. But then if we use the data generated by that person and we train an AI model on that, the deployment of that model and the model itself to process the similar cases will be way higher than one human being. So we will multiply all the biases and scale them to some potentially huge scale. So we have to just remember about this and make sure that we at least try to build machine learning and AI systems in a responsible way. I think it's a very good saying that this is a multiplicator of biases in this way. And I think it may be even worse than fake media because people start using it, people start believing it as it somehow provides some sources that might not be truthful. And again, I think it's a very valid Argument that these systems, this AI should be observed, should be trained, should be monitored and managed properly and also documented within the AI management systems. So I'm fully of you on that. And again coming back to the topic of trade off. Yes, I think it's the valid trade off to do to build those systems in the right way. What is the timeline of EU AI act right now, how it looks like and where is it going? So as of now the expectation is that they will review and update the final text of the AI act and then it should be published. And for the first there was 21 days to come in force and be officially published in the journal. And then in half a year this prohibition of some of the prohibited unacceptable risk AI systems will come in force. Then there is a 24 months timeline for the rest of the systems. There is still time, but organizations should start doing something about it. Then give us two, three immediate steps that companies that had none AI management before that they should do right now. What are these steps that they should take care of? I would start with implementation of AI management System compliant with ISO 4201. So I think it's a good first step and as already previously said, I would start with defining its scope really, really as narrow as possible. So for example, starting with one particular department in the organization or one particular product, the most high risk, for example one and then just working their way towards the full blown AI management system within the organization. And I know that many consulting companies including us, and are ready to support businesses with this compliance. So for example, in our case our team already prepared a set of policies, processes and templates and we are ready to help companies to build AI management System compliant with ISO 42001. In working with customers and clients, what is the most common pain point or like request that you hear from them regarding this topic. So regarding the AI management and where you see the most value of your team helping as well, AI management is just being developed now, so it's all pure innovation. And yes, so 4201 it was just released. So obviously companies struggle to interpret some of the requirements of the standard or some of the processes are not clear to them. So I think that I see already now a lot of interest from the business side in the different kind of trainings related to AI Regulations and ISOFET 2.01 as well as support with the implementation of the standard. And which industries are they coming from? What is the major parts of industry that you encounter different? I've spoken already to automotive representatives to health tech industry representatives, so different industries. All right, anything else you would like to say to the audience regarding the AI act as the some kind of advice or some kind of guidance overall in order to implement a compliance management system first from scratch? For a person who is not familiar with this kind of systems, it takes a lot of time to do it on a good level. So I think engaging at least for a short period of time with someone who is experienced and can help you to cut the corners and implement it faster, I think it's beneficial. So I would consider actually getting help from the people who are doing it as their main job. And generally if there is some kind of request for any challenge within the company for developing this AI product, they can pretty much contact you or contact your team and get at least some kind of initial guidance or initial thoughts on where they are going and how should they develop that. Right? We do not guide companies from a technical perspective how they should develop something. What we do, we interpret for them the regulators expectations and phrase those into human understandable language and help them to establish a process which is compliant with a particular standard or regulation. All right, I think that's pretty much it from my side. Thank you for tuning in to today's episode on navigating the new landscape of EU AI Act. It was a very good dive into the upcoming regulations, what to do and how startups and small and medium businesses should act right now in order to comply in the future. Thanks Tanya for joining. Thank you. It was really interesting discussion. Please be sure to subscribe to learn what to do to help AI businesses simplify their compliance journey at startup Global RegulatoryAI and learn more about technology insights from Health Stack and beyond from our Insights section. All the links to the resources will be available in the podcast description and please also know that we will be doing a dedicated webinar on ISO 42001 on 22 May. So please follow the announcement and we'll be happy to see you there. Keep ethical innovation going and don't break the rules as we don't want AI to rule the world. Until next time.