White Paper on Artificial Intelligence in the EU

The EU Framework on Artificial Intelligence

Artificial intelligence is a family of technologies that display intelligent behaviour by analysing their environment and taking actions, with some degree of autonomy, to achieve specific goals. AI-based systems can be purely software-based, acting in the virtual world – i.e., voice assistants, search engines, or face recognition systems – or AI can be embedded in hardware devices – i.e., advanced robots, autonomous cars, or drones. Many AI technologies require training data to improve their performance. Once they perform well, they can help improve and automate decision-making in their specific domain. In general, AI can optimise existing processes or enable brand-new activities, offering new opportunities and benefits for private and public services – including Law Enforcement – but also introducing serious risks. As first observed by the European Parliament in the Resolution of 16 February 2017 on Civil Law Rules on Robotics, the use of systems regulated by AI involves risks that are different from those linked to the human factor, inevitably posing ethical and legal problems [1]. With regards to privacy, it can be endangered by the unregulated use of facial recognition in public spaces. Furthermore, based on the design and type of data entered, AI systems could reproduce the existing discrimination in the offline world, making decisions influenced by ethnicity, gender, or age class. The so-called “deep fakes” – false but extremely realistic visual and audio contents, which are increasingly used in the field of information warfare – are also created through AI.

Still, the benefits brought by artificial intelligence are enormous. Faced with the rapid technological development determined by the growth of solutions based on artificial intelligence – the number of patent applications published in the last decade has increased by + 400% – and with an international context in which the main competitors of the European Union are heavily investing in this technology, the European Commission has adopted a series of initiatives aimed at regulating AI.

The fact that fragmentation of national actions with regard to AI applications are considered risk to the EU global competitiveness and intention to set standards were the main reasons that prompted the EC to launch the European Strategy on Artificial Intelligence in April 2018 [2]. The main assumption at the basis of the European strategy is that the EU “can lead the way in developing and using AI for good and for all, building on its values and its strengths”. These strengths inter alia include: world-class researchers, labs, and start-ups; the Digital Single Market; a wealth of industrial, research and public sector data which can be unlocked to feed AI systems [3]. Within its strategy, the European Commission then identified three distinct but complementary commitments: (a) increase investments to a level that corresponds to the economic weight of the European Union in the world; (b) leave no one behind – especially in the education field – and ensure a smooth transition to the era of artificial intelligence in the workplace; (c) ensure that new technologies reflect European values and principles. With respect to this last commitment, the EC made explicit reference to the General Data Protection Regulation (GDPR) of 2016 on data protection and privacy in the European space – which at the time was not yet in force [4] – and to Article 2 of the Treaty on European Union (TEU), which lists the founding values of the European political community: “respect for human dignity, freedom, democracy, equality, the rule of law and respect for human rights, including the rights of persons belonging to minorities” [5].

In the aforementioned Communication, the European Commission also announced the adoption of a series of initiatives on artificial intelligence, including the launch of the European AI Alliance, which is a multi-stakeholder forum that has rapidly attracted members of civil society, industry and the academic world, and the institution of a High-Level Expert Group on Artificial Intelligence (AI HLEG) [6]. The 52 experts of the AI HLEG were asked by the EC to develop a set of ethical guidelines, published in April 2019 under the name of Ethics Guidelines for Trustworthy AI [7], and to make policy and investment recommendations, which were presented in June 2019 in the Policy and Investment Recommendations for Trustworthy AI [8]. Overall, these two documents highlighted the need to join forces at a European level, in order to develop a human-centred approach to artificial intelligence as the main feature of “AI made in Europe”. This vision was reaffirmed by the EC itself in COM (2019)168 entitled “Building Trust in Human-Centric Artificial Intelligence” of April 2019 [9]. Finally, on July 2020 the AI HLEG presented its final Assessment List for Trustworthy Artificial Intelligence (ALTAI), identifying seven key requirements – human agency and oversight; technical robustness and safety; privacy and data governance; transparency; diversity, non-discrimination and fairness; environmental and societal well-being; accountability – to ensure that users benefit from AI without being exposed to unnecessary risks by indicating a set of concrete steps for self-assessment [10].

The European Strategy on Artificial Intelligence was followed by the White Paper on Artificial Intelligence of February 2020, accompanied by a Communication from the EC itself outlining the European Strategy for Data [11] [12]. In general, the document suggested establishing within the European space both an “ecosystem of excellence” in the development and diffusion of AI systems, and an “ecosystem of trust” based mainly on a human-centric approach to artificial intelligence. The White Paper was also accompanied by the “Report on the Safety and Liability Implications of Artificial Intelligence, the Internet of Things and Robotics”, concluding that the current product safety legislation contains a number of gaps that needed to be addressed, notably in the Directive 2006/42/EC – the so-called “Machinery Directive” [13] [14].

During the development of the EU framework on artificial intelligence, the European institutions have also given importance to the security aspect of AI systems. In December 2020, the European Union Agency for Cybersecurity (ENISA) presented a report called “Artificial Intelligence Cybersecurity Challenges”, warning that AI may open new avenues in manipulation and cyber-attack methods, as well as new privacy and data protection challenges for citizens, enterprises, and institutions [15].

In defining its approach to AI, the European Union has decided to play the role of pioneer in the sector, similar to what it did with the GDPR of 2016. With COM (2021)205 of 21 April 2021, the EC has in fact announced an ambitious regulatory project on AI, which is still under development [16]. On the same data, the European Commission proposed to the European Parliament and the Council of the EU a regulation on harmonised rules regarding AI applications – the so-called “Artificial Intelligence Act” – emphasising that its approach is shaped by European values and risk-based, ensuring both safety and fundamental rights protection [17]. Once approved, this regulation would represent the first legal framework in the world on the AI sector. As stated in the proposal: “By improving prediction, optimising operations and resource allocation, and personalising service delivery, the use of artificial intelligence can support socially and environmentally beneficial outcomes and provide key competitive advantages to companies and the European economy. Such action is especially needed in high-impact sectors, including climate change, environment and health, the public sector, finance, mobility, home affairs and agriculture. However, the same elements and techniques that power the socio-economic benefits of AI can also bring about new risks or negative consequences for individuals or the society” [17]. The EU has therefore decided to regulate these elements and lay the necessary legal bases so that artificial intelligence has rules and specific guidelines within the common European space.

The appropriate balance between fundamental rights protection and public security is indeed one of the main pillars of the proposal. The European Union wants to ensure that European citizens can benefit from safe, transparent, ethical, and impartial AI systems under human control, thus placing specific requirements for all European or foreign AI systems used in the EU territory. Specifically, it aims at addressing risks of specific uses of AI, categorising them into 4 different levels: “unacceptable risk”, “high risk”, “limited risk”, and “minimal risk”. In doing so, the AI regulation will make sure that Europeans can trust the artificial intelligence they are using. For instance, the “unacceptable risk” category includes AI applications in which algorithms track users’ behaviour to automatically assess what level of credit score to assign to individuals and companies – as widely used in China. Examples of elements classified as “high risk” are the following: AI systems that autonomously control critical infrastructures; AI applications that could endanger the life and health of citizens; CV sorting software for hiring procedures. All these systems will be carefully evaluated before being placed on the market, will be subject to minimum transparency obligations, and will be monitored throughout their life cycle. Anyway, the vast majority of artificial intelligence systems fall into the category of “minimal risk”, therefore potentially not subject to the new European legislation.

Particular attention must be paid to biometric surveillance. Artificial intelligence powers the use of biometric technologies, including facial recognition applications, which are used for verification, identification, and categorisation purposes by private or public actors. While facial recognition markets are poised to grow substantially in the coming years, the increasing use of facial recognition technologies (FRTs) has emerged as a salient issue in the worldwide public debate on biometric surveillance. While there are real benefits in using facial recognition systems for public safety and security, their pervasiveness and intrusiveness, as well as their susceptibility to error, give rise to a number of fundamental rights concerns with regard, for instance, to discrimination against certain segments of the population and violations of the right to data protection and privacy [18]. In October 2021, the European Parliament passed a non-binding resolution that prevents the use of real-time facial recognition systems in publicly accessible spaces for the purpose of law enforcement, along with the creation of private facial recognition databases. With this resolution, the EP recognized that the use of AI for mass surveillance and other unlawful interference, such as the profiling of citizens in order to rank them and restrict their freedom of movement, pose a serious threat to fundamental rights [19]. The non-binding resolution sends a strong signal on how the EP is likely to vote in upcoming negotiations on the Artificial Intelligence Act.

The legislative framework on artificial intelligence will have a huge impact worldwide, as it was for the GDPR of 2016, which has become an international standard in its sector since it came into effect in 2018. With this proposal, the EU wanted to strengthen its competitive position with respect to its main competitors – China and the United States of America – by anticipating them in the definition of a regulatory framework that could thus become the reference standard on the global scene. This political dimension was reaffirmed by the Coordinated Plan on Artificial Intelligence 2021 Review, which goes hand in hand with the proposal for the Artificial Intelligence Act [20]. The new plan builds on the collaboration established between the EC and Member States – plus Norway and Switzerland – during the 2018 Coordinated Plan on Artificial Intelligence, which was a joint commitment to maximising Europe’s potential to compete globally and an essential first step in defining actions and funding instruments for the uptake and development of AI across sectors. Moreover, it encouraged Member States to develop national strategies [21] [22]. The revised plan proposes around 70 actions for closer and more efficient cooperation between the EC and Member States on artificial intelligence between 2021 and 2027.

As already outlined in the White Paper on Artificial Intelligence of February 2020, the European Commission has thought about a series of tools to support the future legislation, in order to favour the birth of a public-private partnership on artificial intelligence, data and robotics to define, implement and invest in a joint strategic research and innovation program for Europe. These tools include the establishment of centres of excellence for AI, the birth of new digital innovation poles that act as one-stop shops to provide access to technical skills and experimentation – so that companies can “test before investing” – and the creation of a central European database of AI resources needed for the uses of private companies and the public sector. With funds provided by the Digital Europe (DIGITAL) and Horizon Europe (HE) programs, the European Commission intends to invest around one billion euros per year in AI and mobilize further investment from the private sector and Member States through their National Recovery and Resilience Plans (NRRPs) for a total of 20 billion a year [23].

Schematically, the European approach to artificial intelligence has four fundamental objectives: (a) establish the enabling conditions for the development and diffusion of AI; (b) build a strategic leadership in high impact sectors; (c) making the EU a place where AI can flourish; (d) ensure that AI technologies serve people. These objectives fall within the broader concept of a continent that sees in technological progress, while attentive to the environment and human society, not only one of the keys necessary for the post-pandemic restart, but above all an indispensable tool for an ever-greater integration between Member States in a single entity capable of relating equally to the great world powers.

On March 2022, the European Parliament’s Special Committee on Artificial Intelligence in a Digital Age (AIDA) adopted a report on artificial intelligence. On the one hand, it emphasised that the digital transition in the EU must be human-centric and compatible with the Charter of Fundamental Rights of the European Union. On the other hand, the report cautioned that the EU has fallen behind in the global race for technological leadership. This might result in a risk for standards that need to be developed elsewhere in the future, often by non-democratic actors [24]. The delay of the EU compared to its main competitors is the reason why the European Commission proposed the creation of the EU-US Trade and Technology Council (TTC), which was established in June 2021 to promote coordination between the two shores of the Atlantic Ocean on everything related to the technology sector – from regulation to taxation, passing through cybersecurity [25]. On May 2022, meeting at the second Ministerial Summit of the TTC in Paris, both parties discussed the implementation of common AI principles and agreed to develop a joint roadmap on evaluation and measurement tools for trustworthy AI and risk management [26]. However, the European approach places the European Union at the forefront of regulation in the field of artificial intelligence, as happened with the GDPR of 2016. In the end, given the European focus on the values underlying the rules, aimed at avoiding the systematic violation of privacy and individual freedoms as happens in the autocratic regimes, it seems that the EU and the US are destined to converge in this sector.

The legislative process relating to the proposed regulation is currently proceeding. The EP Committee on the Internal Market and Consumer Protection (IMCO) and the EP Committee on Civil Liberties, Justice and Home Affairs (LIBE) jointly released a draft report on the EC proposal in April 2022. The document includes proposed amendments to the original text proposed by the European Commission. The most significant changes proposed in the draft report include the ban on using artificial intelligence to implement predictive policing practices, the obligation to register AI-based technologies and greater alignment with the GDPR [27].

AI for Law Enforcement Applications

Referring to AI systems used by law enforcement agencies (LEAs), an important document has been elaborated by EUROPOL through the launch of the Accountability Principles for Artificial Intelligence – the so-called AP4AI project – in February 2022 [28]. This multidisciplinary project is led by EUROPOL and the Centre of Excellence in Terrorism, Resilience, Intelligence, and Organized Crime Research of Sheffield Hallam University (CENTRIC), and represents apractical toolkit to support AI accountability within the internal security domain. The project is specifically designed for security and justice practitioners and is aimed at preventing misuse of AI by internal security practitioners and safeguarding accountability. The document states the legislative lack in terms of accountable use of artificial intelligence within the internal security domain and addresses the challenge of creating a comprehensive global framework for the accountability of Policing, Security and Justice [28]. The AP4AI should be seen as a “living document” for the further creation of an AI Accountability Agreement (AAA) [28].

Accountability is considered by the AP4AI as the core value for AI deployments within internal security domains [28]. Accountability is defined as “the acknowledgement of an organisation’s responsibility to act in accordance with the legitimate expectations of stakeholders and the acceptance of the consequences” [28]. Accountability should be taken also as a basis for creators in order to develop AI coherently with the legal use they are allowed to. AP4AI is innovative in aiming at creating a comprehensive legal framework that does not refer only to LEAs but to all the stakeholders (i.e. industry, non-governmental organisations, researchers, citizens) who take part or are affected by AI. Hence, while there is widespread knowledge of risk assessment within the internal security sector, there is scarce awareness of how the risk can be mitigated in practice and who are the actors involved. The report briefly focuses on EU efforts and then refers to other countries’ legislation (that of the US in particular) to take it as a model approach for further legislation.

AP4AI consists of the introduction of 12 principles that together define requirements for achieving accountability in the use of AI, namely legality, universality, transparency, pluralism, independence, commitment to robust evidence, enforceability and redress, compellability, explainability, constructiveness, conduct and learning organisation. Here, the 12 principles will not be revised in detail, but it will be provided with an overview of the main concepts to implement an accountable use of artificial intelligence. The document argues the necessity to encompass national approaches and provide enforcement mechanisms applicable to the entire AI system and associated actors. Within the scope of the document, covering the entire AI system means ensuring accountability in all the areas of the AI lifecycle – from design and development to concrete application in various contexts – and to all the stakeholders involved in artificial intelligence. In this regard, a multi-level collaboration within civil society, public and private organisations is necessary.

Given the speed of development of AI, the document recognises the exigence of having a regulatory assurance body that identifies the risks and can give advice to stakeholders and the government.

Human-Centric AI

The main vision characterising the EU approach to artificial intelligence is the creation of human-centric AI, which ensures it works for people and protects the fundamental rights of European citizens. The EC proposal for the Artificial Intelligence Act states that AI systems must always be under human control. However, no mention is made of the training of personnel responsible for supervising these systems, except that it has to be adequate for the task [17]. This lack of attention to this practical aspect has concerned the whole process of elaboration of the EU framework on AI.

The Ethics Guidelines for Trustworthy AI, published by the High-Level Expert Group on Artificial Intelligence in April 2019, promoted a set of 7 key requirements that AI systems should meet in order to be deemed trustworthy, including “human agency and oversight”. According to this key requirement, AI systems should empower human beings, allowing them to make informed decisions and fostering their fundamental rights. At the same time, proper oversight mechanisms need to be ensured, which can be achieved through human-in-the-loop, human-on-the-loop, and human-in-command approaches [7]. With COM(2021)205 of 21 April 2021, the European Commission has accepted the content of the aforementioned document, but it has not taken steps to regulate the training issue. Article 14 on “Human Oversight” states that “high-risk” AI systems should be designed and developed in such a way that natural persons can effectively oversee their functioning [17]. For this purpose, appropriate human oversight measures should be identified by the provider of the system before its placing on the market or putting into service. Where appropriate, such measures should guarantee that the system is subject to in-built operational constraints that cannot be overridden by the system itself and is responsive to the human operator, and that the natural persons to whom human oversight has been assigned have the necessary competence, training and authority to carry out that fundamental role [17].

In general, the European Commission’s proposal does not go beyond the recognition of the need for the training of the personnel responsible for controlling AI systems to be adequate for their supervision. Furthermore, it is not established whether this training should be regulated at European level or left to the competence of Member States.

As a comparative example, the US National Artificial Intelligence Initiative (NAAI) – which became law in January 2021 – focuses on training an AI-ready workforce. The US is investing in current and future generations of American workers through apprenticeships, skills programs, and education in science, technology, engineering, and mathematics (STEM), with an emphasis on information technology, to ensure that American workers are able to take full advantage of the opportunities of AI [29]. The lack of any legislative provision in this regard is particularly serious considering that AI systems may be responsible for the management of sensitive sectors and infrastructures within the EU territory. In recent years, the European Union has launched a set of initiatives aimed at developing knowledge of AI systems, but they have always been conceived as a support to the European digital transition rather than training the personnel who has to supervise the artificial intelligence implemented in critical areas or defence systems.

To start with, open to businesses, organisations and public administrations from all over the continent, the Digital Europe Programme (DIGITAL) is actually investing in learning and training opportunities – i.e., specialised masters and education programmes in key capacity areas – that will create new AI experts within the European Union [30]. Moreover, the Digital Education Action Plan (2021-2027) is a renewed European policy initiative to support the sustainable and effective adaptation of the education and training systems of EU Member States to the digital age.  In order to enhance digital competences for the digital transformation of Europe, this policy aims to update the European digital skills framework to include AI and data skills [31].

The NATO Approach

One of the consequences of the lack of a European legislative provision on the training of personnel in charge of supervising the AI ​​systems used is that in the defence sector it will continue to be carried out within the context of the Atlantic Alliance.

Over the last few years, NATO has paid particular attention to the so-called “emerging and disruptive technologies” (EDTs), endorsing a Coherent Implementation Strategy on EDTs in February 2021 [32]. Their importance for deterrence, defence and capability development was also recognised by the report entitled “NATO 2030: United for A New Era”, which was commissioned by the Secretary General Jens Stoltenberg and published in November 2020 [33]. In particular, the Atlantic Alliance is developing specific plans for each of the following technological areas: (a) data and computing; (b) artificial intelligence; (c) autonomy; (d) quantum-enabled technologies; (e) biotechnology and human enhancements; (f) hypersonic technologies; (g) space; (h) novel materials and manufacturing; and (i) energy and propulsion [34] [35]. Of all these dual-use technologies, artificial intelligence is known to be the most pervasive, especially when combined with others like big data, autonomy, or biotechnology. Due to its cross-cutting nature, AI will pose a broad set of international security challenges, affecting both traditional military capabilities and the realm of hybrid threats. This the reason why NATO has prioritized AI, identifying it as critical for its operations and a key enabler for modernisation and cooperation in the Atlantic Alliance.

At the Meeting of NATO Ministers of Defence held in Brussels in October 2021, the Allied Defence Ministers formally launched the NATO Artificial Intelligence Strategy [36]. Only a summary of the document has been made public. The strategy is meant to provide a common policy basis to support the adoption of AI systems among Member States in order to achieve NATO’s three core tasks – collective defence, crisis management, and cooperative security. In particular, in accordance with international law and values of the Atlantic Alliance, the document established six basic principles of safe and responsible use of artificial intelligence in the field of defence: (a) lawfulness, (b) responsibility and accountability, (c) explainability and traceability, (d) reliability, (e) governability, and (f) bias mitigation [37]. All AI systems developed by NATO and its partners will have to comply with these principles, which are quite similar to the Ethical Principles for Artificial Intelligence adopted by the US Department of Defense in February 2020 – but with a plan to verify that the principles are followed [38]. By adopting a comparative approach, the EC’s proposal for the Artificial Intelligence Act seems to be more restrictive for high-risk applications of AI, although its impact on defence will be indirect, as it do does not apply to the military domain. The Artificial Intelligence in Defence Action Plan – finalized by the European Defence Agency at the end of 2020 – shares more similarities with the NATO Artificial Intelligence Strategy, as it focuses on identifying modes and means for EU Member States to collaborate in the development of AI for their militaries [39].

While it emphasizes collaboration with private technology companies, academics and start-ups, the new strategy needs further refinement as AI would help NATO’s military and civilian personnel interlink devices on different platforms, perform rigorous data analytics, and quicken response time in response to conventional or hybrid attacks. In this sense, the 2022 Strategic Concept will play a central role. The Strategic Concept is one of NATO’s most important documents, as it informs military alliance’s planning, resource allocation, and programming based on changes in the threat environment. The last version of the document has not been updated since 2010 [40]. But, as established at the 2021 NATO Summit in Brussels, the Atlantic Alliance will adopt its new Strategic Concept at the 2022 NATO Summit in Madrid, which will be held in 2022 [41].

By setting NATO’s strategic direction for the next decade and beyond, the 2022 Strategic Concept should highlight the essential role of EDTs in collective defence, ensuring that the military alliance will continue to adapt to a changing world. However, the document should focus less on the emergence of new technologies and more on how NATO’s military and civilian personnel use them – i.e., human training on artificial intelligence and other EDTs. In order to build greater digital capacity within the Atlantic Alliance, NATO institutions are aware of the importance of providing education, training and instruction to both military and civilian personnel in various areas consistent with the objectives and priorities identified by NATO’s new security policies. During the 2021 NATO Summit, the Heads of State and Government of the thirty member countries decided to support internal cooperation and technological development through the creation of two new structures: the Defence Innovation Accelerator for the North Atlantic (DIANA) and the NATO Innovation Fund. Both bodies aim to consolidate the technological advantage within the Atlantic Alliance, considered precisely one of the foundations on which NATO’s ability to dissuade and defend itself from potential external threats is based [42]. In this perspective, the 2022 Strategic Concept should establish deeper cooperation between NATO and the private sector, academia and non-governmental organisations, in order to provide new tools, strategies and practices to improve the knowledge, expertise and capability of personnel in the supervision of AI systems.

Conclusions

In the context of the global race for artificial intelligence, the European Union aims to strengthen its competitive position with respect to its main competitors – the United States and China – by anticipating them in the definition of a comprehensive regulatory framework on AI that could become a global standard. Indeed, acting as a tech regulator, the European Commission believes that the Artificial Intelligence Act will become an international point of reference for similar legislation, thanks to its balanced approach between fundamental rights protection and public security. Structured around a risk-based approach, the proposed regulation introduces tighter obligations in proportion to the potential impact of AI applications.

AI systems are efficient tools at the disposal of security practitioners and citizens but it is necessary to safeguard accountability and avoid misuse that can endanger national security and the respect for human rights. In this direction, the AP4AI Project has been established by Europol in February 2022.

The NATO approach was described to explore possible parallelisms between defence and civil security with regard to Artificial Intelligence preparedness of human operators. However, related documents are focused on the emergence of new technologies rather than on how military and civilian personnel use them.

The EC proposal for the Artificial Intelligence Act of April 2021 does not address the issue of human training on AI. The proposal simply states that it has to be adequate for the task, without establishing minimum technical requirements or setting up specific training structures [17]. Therefore, this fundamental aspect will not be regulated in the European framework – meaning, the training of personnel responsible for supervising AI systems will be informally delegated to the structures and initiatives of the Member States.

The concerns about this rather significant gap should be expressed here, especially since the future EU framework on artificial intelligence aims to become an international standard and advises the need to promote new tools, strategies and practices to improve the knowledge, expertise and capability of personnel in the supervision of AI systems used for Security, Intelligence and Law Enforcement. From a long-term perspective, it is crucial that the EU – in the search for strategic autonomy – continues to allocate public resources for the development of an “AI made in Europe” and favours the creation of a European environment that stimulates private investment.

Disclaimer

This work was supported by the NOTIONES (iNteracting netwOrk of inTelligence and securIty practitiOners with iNdustry and acadEmia actorS) project, that has received funding from the European Unions’ Horizon 2020 research and innovation programme under grant agreement No. 101021853.

References