Tuesday

26th Sep 2023

Magazine

Worse than 'deep fakes' — disinfo's new and more-powerful apps

  • Berlin's mayor unknowingly holding a Zoom meeting with a fake AI-generated Kyiv mayor "Vitali Klitschko" in June (Photo: Twitter)
Listen to article

On 24 June 2022, Berlin's mayor Franziska Giffey had a "completely normal [video] conversation" with Kyiv's mayor Vitali Klitschko.

Or so she thought.

Read and decide

Join EUobserver today

Become an expert on Europe

Get instant access to all articles — and 20 years of archives. 14-day free trial.

... or subscribe as a group

  • Volodomyr Zelensky telling Ukranian troops to surrender — or not. Another 'deep fake' (Photo: Twitter)

She started to become suspicious when the supposed mayor asked her to support for a gay pride parade in the middle of war-torn Kyiv.

It was not Klitschko, it turned out, but an impostor. Giffey's office later said the person was probably using deepfake technology to trick Berlin's mayor (though the tech behind it has remained unclear).

A year or two ago, few were familiar with deepfakes; today most people are. Its popularity is in large part due to its prominence on popular apps, such as face-swaps or AI-powered lip-syncing tech on TikTok.

Once merely a tool for entertainment, disinformation actors have become begun to leverage them. This year, 2022, alone saw multiple similar high-profile stunts, from those that were relatively speaking less harmful — such as the scam on JK Rowling — to potentially dangerous ones, like the deepfake imitating Ukrainian president Vladimir Zelensky instructing his citizens to lay down their arms.

But what's scarier is that deepfakes are themselves rapidly becoming an 'old-fashioned' way to create fake video content.

The new kid on the block this year is fully-synthetic media. Unlike deepfakes, which are partially synthetic and graft the image of one person's face onto the body of another's in an existing video, fully synthetic media can be created seemingly out of thin air.

This year saw the rise of text-to-image software that does exactly that.

It's not actual magic, but the technology behind the generators is hardly less mystifying. The models powering text-to-image software rely on machine learning and vast artificial neural networks that mimic your brain's natural neural networks and their ability to learn and recognise patterns. The models are trained to recognise millions of images paired and their text descriptions.

The user need only enter a simple text prompt and — hey presto! — out comes the picture. The most popular programs are Stable Diffusion and DALL-E — and both are now free of cost and available open access.

This points to troubling potential: these tools are a dream for a disinformation actor who need only to be able to imagine the 'evidence' they need to support their narrative, and then create it.

These technologies are already starting to penetrate social media and images are only the beginning.

Just recently in September, Meta released 'Make-A-Video' that enables users to create "brief, high-quality video clips" from a text prompt. Experts warn that synthetic video is even more troubling than synthetic images, given that today's social media landscape already favours fast and clipped videos, over text or images.

Entertainment aside, the penetration of synthetic media onto an app like TikTok is particularly troubling. TikTok is centered on user-generated content, encouraging people to take existing media, add their own edits, and re-upload — an operating model not too different from deepfake creation.

Recent research by the Associated Press has shown that one-in-five videos on TikTok are misinformation and that young people increasingly use the app as a search engine on important issues like Covid-19, climate change, or Russia's invasion of Ukraine.

It is also significantly harder to audit than other apps like Twitter.

In short, the TikTok app is a perfect incubator for such new tactics, which then commonly spread across the web through cross-platform sharing.

Most disinformation is still created using commonplace tactics like video and sound-editing software. Altering videos by splicing, changing the speed, replacing the audio, or simply taking the video out of context, disinformation actors can already easily sow discourse.

Seeing is still believing

Yet, the danger of text-to-image is already real and present. One does not have to expend too much creative energy to imagine the not-too-distant future when untraceable synthetic media appears en masse on our phones and laptops. As trust in institutions and reliable media is already tenuous, the potential impact on our democracies is terrifying to contemplate.

The sheer density of news today is a compounding part of the problem. Each of us only has a finite capacity to consume news — let alone fact-check it. We know that debunking is a slow and ineffective solution. For many of us, seeing is still believing.

We need to provide an easy and widespread solution to empower users to identify and understand false images or videos immediately. Solutions that do not empower users — and journalists — to identify fake news faster, easier, and more independently will always be a step behind.

Currently, the most promising solutions focus on provenance: technology which embeds media with a signature or invisible watermark at the point of creation, as proposed by Adobe's Content Authenticity Initiative. It is a promising but complex solution that requires collaboration across multiple industries. Policymakers, especially in Europe, should give it more attention.

We live in a fast-paced world and disinformation moves faster than our current solutions. It's time we catch up.

This article first appeared in EUobserver's magazine, Digital EU: the Good, the Bad — and the Ugly, which you can now read in full online.

Author bio

Heather Dannyelle Thompson is manager of Digital Democracy at Democracy Reporting International, the Berlin-based NGO watchdog on elections and democracy.

Column

'Deepfakes' - a political problem already hitting the EU

Last month, the Foreign Affairs Committee of the Dutch Parliament had an online call with Russian dissident Alexi Navalny's chief of staff. Or so the parliamentarians thought. It turned out that they may have been talking to a deepfake.

Digital EU: the Good, the Bad — and the Ugly

The European Union has impressive digital ambitions and an equally-impressive array of initiatives, proposals, directives and regulations, all designed to make the bloc 'fit for the digital age'. But what do they all mean — and will they all work?

Vestager: 'Technology must not steal our time'

Given the rapid pace of digitalisation, the EU has rushed to set and regulate digital standards. Many new initiatives are led by Margrethe Vestager, EU commissioner for competition, who says the focus must be on making technology work for people.

Latest News

  1. EU trade chief in Beijing warns China of only 'two paths' forward
  2. Why should taxpayers pay for private fishing fleets in third countries?
  3. Women at risk from shoddy EU laws on domestic workers
  4. EU poised to agree on weakened emission rules
  5. China trade tension and migration deal This WEEK
  6. Europe's energy strategy: A tale of competing priorities
  7. Why Greek state workers are protesting new labour law
  8. Gloves off, as Polish ruling party fights for power

Stakeholders' Highlights

  1. International Medical Devices Regulators Forum (IMDRF)Join regulators, industry & healthcare experts at the 24th IMDRF session, September 25-26, Berlin. Register by 20 Sept to join in person or online.
  2. UNOPSUNOPS begins works under EU-funded project to repair schools in Ukraine
  3. Georgia Ministry of Foreign AffairsGeorgia effectively prevents sanctions evasion against Russia – confirm EU, UK, USA
  4. International Medical Devices Regulators Forum (IMDRF)Join regulators & industry experts at the 24th IMDRF session- Berlin September 25-26. Register early for discounted hotel rates
  5. Nordic Council of MinistersGlobal interest in the new Nordic Nutrition Recommendations – here are the speakers for the launch
  6. Nordic Council of Ministers20 June: Launch of the new Nordic Nutrition Recommendations

Stakeholders' Highlights

  1. International Sustainable Finance CentreJoin CEE Sustainable Finance Summit, 15 – 19 May 2023, high-level event for finance & business
  2. ICLEISeven actionable measures to make food procurement in Europe more sustainable
  3. World BankWorld Bank Report Highlights Role of Human Development for a Successful Green Transition in Europe
  4. Nordic Council of MinistersNordic summit to step up the fight against food loss and waste
  5. Nordic Council of MinistersThink-tank: Strengthen co-operation around tech giants’ influence in the Nordics
  6. EFBWWEFBWW calls for the EC to stop exploitation in subcontracting chains

Join EUobserver

Support quality EU news

Join us