AI Act. A safety framework constructed around risk categories

On the 14th of June the Members of the E.U. Parliament voted with a large majority the AI Regulation.

The EU AI Act is one of the first major laws to regulate artificial intelligence, and as Adam Satariano observes for the New York Times “a potential model for policymakers around the world as they grapple with how to put guardrails on the rapidly developing technology.”

 

It’s worthwhile mentioning that the EU Parliament amendments are deviating from the European Commission original text in the sense that strict requirements meant for high-risk AI systems have been extended to other useful AI applications that pose very limited risks.

 

In fact, since the AI Act proposal happened, the technology has developed to the point of ChatGPT breaking the EU plan to regulate AI.

Therefore, discussions on systems trained on large data sets or large language models (LLM) have spurred amendments to the initial text with the EU Parliament wanting a clear distinction between general purpose AI (GPAI) and large language models, aiming at introducing a stricter regime for the latter.

 

What’s on the plate

• LLM to keep appropriate levels of performance, explainability, corrigibility (by human input), safety, and cybersecurity throughout their lifecycle.
• Employment of independent experts, documented analysis, and extensive testing for above purpose.
• LLMs to be subjected to obligations to implement a quality management system to provide the relevant documents up to 10 years after the system is
  launched.
• LLM systems to be registered on the EU database.
• Generative AI (GenAI) systems to be submitted to heightened transparency obligations and safeguards against generating content in breach of EU law.
• Disclosure obligations regarding the use of training data protected under copyright law.
• Disclosure obligations regarding the computing power required and the training time of the model.

Let’s be very clear. Behind the hype, what is the Regulation meaning for companies?

The AI Regulation is a product safety framework constructed around risk categories.

The law creates new conformity obligations, largely based on the technicality of high-risk AI systems themselves, but also on the technicality of certification and standardization mechanisms. 

 

The legislator provides that high-risk AI systems must as a minima follow conformity assessment procedures before they can be placed on the Union’s market in order to minimize the following risks: confidentiality and data analysis, precision, robustness, resilience, explicability, non-discrimination.

 

Manufacturers of an AI systems will have to manage large amounts of data, including personal data for which they must guarantee their integrity and confidentiality.

 

Data governance obligations are part of the law if high-risk AI systems use techniques involving learning models with data based on training, validation, and test datasets. Datasets must be reliable, which is why the law imposes that training, validation and testing data sets to be subjected to appropriate data governance and management practices, such as the relevant design choices, annotation, labelling, cleaning, enrichment and aggregation, a prior assessment of the availability, quantity and suitability of the data sets that are needed, examination in view of possible biases, the identification of any possible data gaps or shortcomings, and how those gaps and shortcomings can be addressed, transparency on the source of the datasets, their reach and key characteristics, how the data were obtained and selected, etc.

 

Furthermore, the legislator establishes rules for a graduated risk management to help technical operators meet their obligations.

 

A risk management system consisting of an iterative, continuous process conducted throughout the life cycle of a high-risk AI system must be established, implemented, documented, and maintained.

The management system consists mainly of obligations to identify and minimize foreseeable risks, to manage residual risks and tests. Risk minimization is based on the adoption of suitable measures as reflected notably in relevant harmonised standards or common specifications.

 

Residual risks are considered acceptable, provided that the high-risk AI system is used for its intended purpose or under conditions of reasonably foreseeable misuse.

 

The user must be informed of these residual risks, while risk minimization or mitigation measures must be taken.

 

High-risk AI systems must be tested, to determine the most appropriate risk management measures at any time during the development process and, in any case, before being placed on the market or put into service, based on previously defined metrics and probabilistic thresholds adapted to the purpose of the high-risk AI systems.

 

Which are the High-Risks AI Systems?

• Biometric identification and categorisation of natural persons
• AI used for management and operation of critical infrastructure.
• AI used for educational and vocational training
• AI systems used in employment for i.e., recruitment or promotions
• Some AI systems used for law enforcement
• AI systems used for asylum and border control managemen

MEPs expanded the classification of high-risk areas to include: