4th Feb 2023


Worse than 'deep fakes' — disinfo's new and more-powerful apps

  • Berlin's mayor unknowingly holding a Zoom meeting with a fake AI-generated Kyiv mayor "Vitali Klitschko" in June (Photo: Twitter)
Listen to article

On 24 June 2022, Berlin's mayor Franziska Giffey had a "completely normal [video] conversation" with Kyiv's mayor Vitali Klitschko.

Or so she thought.

Read and decide

Join EUobserver today

Become an expert on Europe

Get instant access to all articles — and 20 years of archives. 14-day free trial.

... or subscribe as a group

  • Volodomyr Zelensky telling Ukranian troops to surrender — or not. Another 'deep fake' (Photo: Twitter)

She started to become suspicious when the supposed mayor asked her to support for a gay pride parade in the middle of war-torn Kyiv.

It was not Klitschko, it turned out, but an impostor. Giffey's office later said the person was probably using deepfake technology to trick Berlin's mayor (though the tech behind it has remained unclear).

A year or two ago, few were familiar with deepfakes; today most people are. Its popularity is in large part due to its prominence on popular apps, such as face-swaps or AI-powered lip-syncing tech on TikTok.

Once merely a tool for entertainment, disinformation actors have become begun to leverage them. This year, 2022, alone saw multiple similar high-profile stunts, from those that were relatively speaking less harmful — such as the scam on JK Rowling — to potentially dangerous ones, like the deepfake imitating Ukrainian president Vladimir Zelensky instructing his citizens to lay down their arms.

But what's scarier is that deepfakes are themselves rapidly becoming an 'old-fashioned' way to create fake video content.

The new kid on the block this year is fully-synthetic media. Unlike deepfakes, which are partially synthetic and graft the image of one person's face onto the body of another's in an existing video, fully synthetic media can be created seemingly out of thin air.

This year saw the rise of text-to-image software that does exactly that.

It's not actual magic, but the technology behind the generators is hardly less mystifying. The models powering text-to-image software rely on machine learning and vast artificial neural networks that mimic your brain's natural neural networks and their ability to learn and recognise patterns. The models are trained to recognise millions of images paired and their text descriptions.

The user need only enter a simple text prompt and — hey presto! — out comes the picture. The most popular programs are Stable Diffusion and DALL-E — and both are now free of cost and available open access.

This points to troubling potential: these tools are a dream for a disinformation actor who need only to be able to imagine the 'evidence' they need to support their narrative, and then create it.

These technologies are already starting to penetrate social media and images are only the beginning.

Just recently in September, Meta released 'Make-A-Video' that enables users to create "brief, high-quality video clips" from a text prompt. Experts warn that synthetic video is even more troubling than synthetic images, given that today's social media landscape already favours fast and clipped videos, over text or images.

Entertainment aside, the penetration of synthetic media onto an app like TikTok is particularly troubling. TikTok is centered on user-generated content, encouraging people to take existing media, add their own edits, and re-upload — an operating model not too different from deepfake creation.

Recent research by the Associated Press has shown that one-in-five videos on TikTok are misinformation and that young people increasingly use the app as a search engine on important issues like Covid-19, climate change, or Russia's invasion of Ukraine.

It is also significantly harder to audit than other apps like Twitter.

In short, the TikTok app is a perfect incubator for such new tactics, which then commonly spread across the web through cross-platform sharing.

Most disinformation is still created using commonplace tactics like video and sound-editing software. Altering videos by splicing, changing the speed, replacing the audio, or simply taking the video out of context, disinformation actors can already easily sow discourse.

Seeing is still believing

Yet, the danger of text-to-image is already real and present. One does not have to expend too much creative energy to imagine the not-too-distant future when untraceable synthetic media appears en masse on our phones and laptops. As trust in institutions and reliable media is already tenuous, the potential impact on our democracies is terrifying to contemplate.

The sheer density of news today is a compounding part of the problem. Each of us only has a finite capacity to consume news — let alone fact-check it. We know that debunking is a slow and ineffective solution. For many of us, seeing is still believing.

We need to provide an easy and widespread solution to empower users to identify and understand false images or videos immediately. Solutions that do not empower users — and journalists — to identify fake news faster, easier, and more independently will always be a step behind.

Currently, the most promising solutions focus on provenance: technology which embeds media with a signature or invisible watermark at the point of creation, as proposed by Adobe's Content Authenticity Initiative. It is a promising but complex solution that requires collaboration across multiple industries. Policymakers, especially in Europe, should give it more attention.

We live in a fast-paced world and disinformation moves faster than our current solutions. It's time we catch up.

This article first appeared in EUobserver's magazine, Digital EU: the Good, the Bad — and the Ugly, which you can now read in full online.

Author bio

Heather Dannyelle Thompson is manager of Digital Democracy at Democracy Reporting International, the Berlin-based NGO watchdog on elections and democracy.


'Deepfakes' - a political problem already hitting the EU

Last month, the Foreign Affairs Committee of the Dutch Parliament had an online call with Russian dissident Alexi Navalny's chief of staff. Or so the parliamentarians thought. It turned out that they may have been talking to a deepfake.

To lead in cyberspace, the EU needs to avoid digital tribalism

To avoid digital tribalism the EU needs a strategy to better engage with the Global South, including the emerging digital powers such as Brazil, Egypt, Ghana, India, Indonesia, Jamaica, Kenya, Mexico, Singapore, South Africa, and Senegal.

Digital EU: the Good, the Bad — and the Ugly

The European Union has impressive digital ambitions and an equally-impressive array of initiatives, proposals, directives and regulations, all designed to make the bloc 'fit for the digital age'. But what do they all mean — and will they all work?

Vestager: 'Technology must not steal our time'

Given the rapid pace of digitalisation, the EU has rushed to set and regulate digital standards. Many new initiatives are led by Margrethe Vestager, EU commissioner for competition, who says the focus must be on making technology work for people.

Latest News

  1. Greece faces possible court over 'prison-like' EU-funded migration centres
  2. How the centre-right can take on hard-right and win big in 2024
  3. Top EU officials show Ukraine solidarity on risky trip
  4. MEPs launch anonymous drop-box for shady lobbying secrets
  5. Hawkish ECB rate-rise 'puts energy transition at risk'
  6. MEPs push for greater powers for workers' councils
  7. How Pavel won big as new Czech president — and why it matters
  8. French official to take on Islamophobia in EU

Stakeholders' Highlights

  1. Party of the European LeftJOB ALERT - Seeking a Communications Manager (FT) for our Brussels office!
  2. European Parliamentary Forum for Sexual & Reproductive Rights (EPF)Launch of the EPF Contraception Policy Atlas Europe 2023. 8th February. Register now.
  3. Europan Patent OfficeHydrogen patents for a clean energy future: A global trend analysis of innovation along hydrogen value chains
  4. Forum EuropeConnecting the World from the Skies calls for global cooperation in NTN rollout
  5. EFBWWCouncil issues disappointing position ignoring the threats posed by asbestos
  6. Nordic Council of MinistersLarge Nordic youth delegation at COP15 biodiversity summit in Montreal

Stakeholders' Highlights

  1. Nordic Council of MinistersCOP27: Food systems transformation for climate action
  2. Nordic Council of MinistersThe Nordic Region and the African Union urge the COP27 to talk about gender equality
  3. Friedrich Naumann Foundation European DialogueGender x Geopolitics: Shaping an Inclusive Foreign Security Policy for Europe
  4. Obama FoundationThe Obama Foundation Opens Applications for its Leaders Program in Europe
  5. EFBWW – EFBH – FETBBA lot more needs to be done to better protect construction workers from asbestos
  6. European Committee of the RegionsRe-Watch EURegions Week 2022

Join EUobserver

Support quality EU news

Join us