What Practices Are Prohibited by the EU AI Act?

Social Scoring and Manipulative AI

The EU AI Act identifies specific artificial intelligence (AI) practices that it prohibits due to conflicts with European values, ethics, and fundamental rights. Among these are the use of AI for social scoring by governments and manipulative AI systems. Article 5 of the Act describes these bans to ensure AI is developed, implemented, and used in a responsible and ethical manner.

Social Scoring by Governments

Social scoring involves assessing or categorizing individuals based on their social behavior, personal traits, or other data collected over time. The EU AI Act specifically bans such systems when used by public authorities or governments. This ban addresses the risks such practices pose to fundamental rights, including privacy, equality, and freedom from discrimination.

Key Points About the Social Scoring Ban

• Unjustified Treatment: The Act prohibits AI systems that unfairly or harmfully treat individuals or groups in situations unrelated to the original purpose of the data collected. For instance, using data from a workplace to influence someone’s access to housing or education is not allowed.
• Disproportionate Consequences: Social scoring systems that impose penalties that are excessive compared to the behavior being evaluated, such as denying access to public services, are also banned.

Ethical Concerns

Social scoring systems, like those implemented in some non-EU countries, raise concerns about mass surveillance, breaches of privacy, and increased social inequality. The EU bans these systems to protect individual rights and promote fairness across society.

Manipulative AI Techniques

The EU AI Act also prohibits AI systems that manipulate people’s behavior without their awareness. These techniques take advantage of vulnerabilities such as age, disability, or financial situations to influence decisions or actions in a harmful way.

Subliminal Manipulation

AI systems that use subliminal techniques to influence individuals without their conscious awareness are banned. These methods work by embedding subtle, undetectable cues in media or advertisements to influence actions, such as purchasing decisions or voting. The EU Act prohibits these systems when they result in significant harm to individuals or groups.

Exploitation of Vulnerabilities

AI technologies that exploit the vulnerabilities of certain populations, such as children, older adults, or those facing economic challenges, are also banned. For example, ads targeting children with manipulative content promoting unhealthy products fall under this category.

Real-World Examples

Manipulative AI raises both ethical and real-world concerns. For example, the Cambridge Analytica case demonstrated how AI-driven psychological profiling was used to influence voter behavior. The EU AI Act prohibits such practices to safeguard individuals’ ability to make autonomous decisions.

The bans on social scoring and manipulative AI in the EU AI Act highlight the EU’s focus on creating safe, ethical, and human-centered AI systems. These prohibitions are in place to protect individuals from harm, uphold fundamental rights, and prevent the misuse of advanced technologies. By addressing these issues, the EU establishes itself as a global leader in responsible AI regulation.

Real-Time Biometric Identification and Subliminal AI

The EU AI Act sets strict limitations on artificial intelligence technologies that could violate basic rights or compromise ethical principles. Among these regulations are those addressing real-time remote biometric identification and subliminal AI methods, as both present serious ethical, security, and privacy risks.

Real-Time Biometric Identification

Real-time remote biometric identification systems use unique biometric data—like facial features, fingerprints, or iris patterns—to identify individuals, often in public spaces. These technologies are a key focus of the EU AI Act due to the risks they pose to privacy, personal freedom, and autonomy.

General Prohibition and Exceptions

• Primary Ban: The Act generally bans the use of real-time biometric identification in publicly accessible areas. This restriction is designed to prevent mass surveillance and to stop individuals from being monitored or identified without their consent.
• Law Enforcement Exceptions: Limited exceptions apply under strict conditions, allowing law enforcement to use these systems only when absolutely necessary. Permitted scenarios include:
  • Finding victims of abduction, human trafficking, or sexual exploitation.
  • Preventing immediate threats to life or public safety, such as terrorist attacks.
  • Identifying suspects of severe crimes, provided the offense carries a custodial sentence of at least four years in the Member State concerned.
• Safeguards: Even in these rare cases, authorities must ensure the system is used in a proportionate and targeted way, focusing only on specific individuals identified beforehand. The deployment must also comply with the data protection rules outlined in the General Data Protection Regulation (GDPR).

Ethical and Legal Concerns

Real-time biometric technologies create risks of mass surveillance, misuse by authorities, and erosion of privacy. The EU AI Act’s restrictions aim to balance public safety with individual rights, ensuring the technology cannot be misused for oppressive or discriminatory purposes.

Subliminal AI

Subliminal AI systems influence people’s decisions or behavior without their conscious awareness. These systems work by embedding subtle and undetectable cues in media or interactions, which can alter decisions in ways people wouldn’t normally choose.

Explicit Prohibition

• Manipulative Techniques: Subliminal AI methods that distort individuals’ decisions or behavior in harmful ways are completely banned. These systems bypass people’s cognitive defenses, compromising their autonomy and ability to make informed choices.
• Examples of Subliminal Manipulation: Such AI techniques can appear in contexts like:
  • Steering consumer behavior with hidden messages in advertisements.
  • Shaping voter decisions by embedding biased signals in political campaigns.
  • Pressuring individuals into financial choices under the pretense of personalized recommendations.

Exploitation of Vulnerabilities

The Act also bans AI systems that exploit vulnerabilities related to age, disability, or socio-economic status if they cause harmful distortions in behavior. For example:

• Advertising unhealthy products directly to children.
• Convincing elderly individuals to purchase unnecessary or harmful services.

Ethical Implications

Subliminal AI conflicts with ethical principles of transparency and autonomy. Its potential misuse in political, commercial, and social settings underscores the need for strong regulation. The EU AI Act’s ban on these systems reflects a commitment to protecting individuals from hidden manipulations and ensuring that AI respects human dignity.

The restrictions on real-time biometric identification and subliminal AI show the EU’s focus on ethical and human-centered AI. By regulating these high-risk technologies, the EU AI Act seeks to protect fundamental rights, build trust in AI, and set a global standard for responsible AI development. These regulations ensure AI systems are used in ways that align with societal values and prevent harm or inequality.