17th Apr 2024


How far will we — and the EU — let AI go?

  • Consumers, despite seeing benefits in the rollout of AI, had low trust in the use of AI systems respecting their personal data and were concerned it could manipulate their decisions (Photo: Jonathan Kemper)
Listen to article

In the last few weeks, the world has been gripped by a remarkable artificial intelligence tool. ChatGPT can generate all types of texts such as essays, media articles, and even poems. It is both fun and fascinating. And yet ChatGPT is also raising difficult questions, with its potential for disinformation giving legislators a headache.

Brussels is currently at a crossroads for dealing with the enormously important questions about the role we see for AI in our society and, importantly, where we need to draw red lines.

Read and decide

Join EUobserver today

Get the EU news that really matters

Instant access to all articles — and 20 years of archives. 14-day free trial.

... or subscribe as a group

  • Consumers also generally don't trust authorities exerting effective control over AI systems (Photo: Jonathan Kemper)

It has been almost two years since the European Commission published draft legislation to regulate AI systems, and discussions in the EU institutions are now reaching the critical stage. The European Parliament will finally set out in the next few weeks what it thinks the AI Act should do.

MEPs have to get it right and protect people from harmful uses of AI systems which could significantly impact our lives as citizens and consumers, and our society as a whole. So far, neither the commission nor national governments have done enough. MEPs must think carefully about how we want to use AI technology in our society, and how we don't.

Consumers in the hands of AI?

That is because AI systems will soon be doing much more than writing fun or elegant texts the way ChatGPT does.

In the rental accommodation market, Airbnb has already patented AI software that can supposedly predict a person's traits and their actions based on data it holds on them, for example from social media.

Such an AI system would determine how much a consumer pays, or whether they would even be able to access particular accommodation.

It begs the question of how the consumer could ever know if they were being discriminated against and what criteria were being used to reach this decision. In particular, we should be asking if we want to be subjected to social scoring by businesses at all.

AI is increasingly being used across the insurance industry to calculate how much a premium for car or home insurance will cost, or whether the customer should be offered a policy at all.

The vast datasets that these systems will be able to rely on and analyse at ever faster speeds are likely to lead to more and more personalised prices without us being able to know which factors are taken into account for this personalisation, or to contest the AI's decision.

AI could have a devastating impact on some people's personal finances if the data is incorrect or there are negative biases in the algorithm's reasoning.

In 2021, Frances Haugen revealed how Facebook's algorithms were causing physical and psychological harm to teenagers by forming addictions and habits. TikTok is being investigated in the US for similar concerns.

These algorithms need greater public scrutiny, and public authorities must reassert control over them if a company doesn't take remedial action.

The solution: regulate where harm can likely occur.

Our own survey from 2020 showed that consumers, despite seeing benefits in the rollout of AI, also had low trust in the use of AI systems respecting their personal data and were concerned it could manipulate their decisions. Consumers also generally don't trust authorities exerting effective control over AI systems.

Over the next few weeks, the European Parliament must step up its ambition and push for effective consumer protection in the AI age.

First, parliament should ban AI systems which carry an unacceptable risk of harm for consumers.

Social scoring by companies, where consumers are commodified as much as the product or service, surely has no place in a society where we value an individual's right to privacy, autonomy and dignity.

Facial recognition by businesses in publicly accessible spaces, where our faces and every move are detected and scrutinised, should also be forbidden. The commercial value of companies knowing what we do and where we go might be high, but we should be free to go where we want without a company's cameras and sensors following us. Respect for the fundamental rights of privacy and autonomy must prevail over so-called innovative business ideas.

Secondly, the EU parliament needs to make sure that the scope of high-risk AI systems under the law is broader.

Systems which can cause consumers harm should be included in that category so that they have to meet specific obligations. Content recommender systems, home assistants, smart meters, all retail insurance which uses AI systems, and any AI likely to be used by children, need to be considered high risk.

Thirdly, all other types of AI systems must respect certain broad principles such as fairness, transparency and accountability which are integral to our society. ChatGPT should be no exception. The EU's AI Act must be a flexible, future-proof, piece of regulation that can also address risks as they arise.

Finally, a technology of this complexity and reach cannot be rolled out without giving strong rights to the people who will be affected by it. Such rights must include a right to object to a decision by an AI system and to receive an explanation, but also to seek redress from the company in case its AI system has caused harm. Importantly, it has to be possible for consumers to go to court as a group to seek collective redress.

The ball is now in the camp of the European Parliament. This new AI law is a great opportunity for the EU to foster and lead on socially valuable innovation. It should ban technology which can cause serious harm. It should put consumers, citizens, and democracies at the front and centre in the AI age. ChatGPT isn't the only AI application coming our way.

Author bio

Ursula Pachl is deputy director general of the European Consumer Organisation (BEUC).


The views expressed in this opinion piece are the author's, not those of EUobserver.


The challenge of artificial intelligence

The fast-growing impact of artificial intelligence will be the biggest challenge for business and consumers in Europe's single market of tomorrow.

Eight EU states miss artificial intelligence deadline

Pan-European strategy "encouraged" member states to publish national artificial intelligence strategies by mid-2019. Germany, France and the UK have already done so - others are lagging behind.

EU lawmakers 'hold breath' on eve of AI vote

European lawmakers regulating the risks of artificial intelligence are likely to face resistance from EU states ahead of negotiations later this year and possibly even among their peers ahead of a plenary vote.


Okay, alright, AI might be useful after all

Large Language Models could give the powers trained data-journalists wield, to regular boring journalists like me — who don't know how to use Python. And that makes me tremendously excited, to be honest.


What do we actually mean by EU 'competitiveness'?

Enrico Letta and Mario Draghi are coming up with reports on the EU's single market and competitiveness — but although 'competitiveness' has become a buzzword, there's no consensus on a definition for what it actually means.

Private fears of fairtrade activist for EU election campaign

I am not sleeping well, tossing and turning at night because I am obsessed about the EU election campaign, worried by geopolitical tensions, a far-right next parliament, and a backlash against the Green Deal, writes Sophie Aujean of Fairtrade International.

Latest News

  1. EU leaders mull ways to arrest bloc's economic decline
  2. Police ordered to end far-right 'Nat-Con' Brussels conference
  3. How Hungary's teachers are taking on Viktor Orban
  4. What do we actually mean by EU 'competitiveness'?
  5. New EU envoy Markus Pieper quits before taking up post
  6. EU puts Sudan war and famine-risk back in spotlight
  7. EU to blacklist Israeli settlers, after new sanctions on Hamas
  8. Private fears of fairtrade activist for EU election campaign

Stakeholders' Highlights

  1. Nordic Council of MinistersJoin the Nordic Food Systems Takeover at COP28
  2. Nordic Council of MinistersHow women and men are affected differently by climate policy
  3. Nordic Council of MinistersArtist Jessie Kleemann at Nordic pavilion during UN climate summit COP28
  4. Nordic Council of MinistersCOP28: Gathering Nordic and global experts to put food and health on the agenda
  5. Friedrich Naumann FoundationPoems of Liberty – Call for Submission “Human Rights in Inhume War”: 250€ honorary fee for selected poems
  6. World BankWorld Bank report: How to create a future where the rewards of technology benefit all levels of society?

Stakeholders' Highlights

  1. Georgia Ministry of Foreign AffairsThis autumn Europalia arts festival is all about GEORGIA!
  2. UNOPSFostering health system resilience in fragile and conflict-affected countries
  3. European Citizen's InitiativeThe European Commission launches the ‘ImagineEU’ competition for secondary school students in the EU.
  4. Nordic Council of MinistersThe Nordic Region is stepping up its efforts to reduce food waste
  5. UNOPSUNOPS begins works under EU-funded project to repair schools in Ukraine
  6. Georgia Ministry of Foreign AffairsGeorgia effectively prevents sanctions evasion against Russia – confirm EU, UK, USA

Join EUobserver

EU news that matters

Join us