What Are Deepfakes & Why Are They Dangerous to Businesses?

Since the advent of Gen AI, the world has worried about the ways in which it might be misused. The technology has come a long way since the days of OpenAI's DALL-E image generator, which allowed users to create renderings of anything they could imagine from a simple text prompt.

The launch of Sora, Veo 3 and other recently unveiled video generating models has shown just how far AI's capabilities have come in the past couple of years. Many people are now concerned that AI is becoming too indistinguishable from reality, with sophisticated fraud and coercion campaigns now a much more powerful prospect, especially for non-digital natives.

Deepfakes are the thing that is concerning industry insiders the most.

Deepfakes are a form of AI-generated content that generate convincing imitations of real people speaking or appearing in videos and they have evolved from mere curiosities into formidable threats to businesses, politicians and citizens alike.

The destabilising potential of the technology became very apparent on the campaign trail before the UK's 2024 general election, when audio of now Prime Minister Sir Keir Starmer was circulated across social media, with the recording seeming to show him expressing his dislike for the city of Liverpool.

Subsequently, the Labour Party insisted that this was the work of a sophisticated deepfake.

Political figures including Joe Biden and former presidential candidate Dean Phillips have also been impersonated through AI-generated audio, raising concerns about electoral integrity and democratic discourse.

What makes the technology difficult to prevent is that it analyses extensive sequences of existing footage or audio to learn speech patterns, facial expressions and mannerisms.

Once trained, these systems can make individuals appear to say or do things they never did. What once required resources akin to those of Hollywood studios now operates on consumer hardware, capable of producing credible fakes from just minutes of source material.

As deepfakes become more sophisticated with AI, businesses are left wondering how to protect themselves. Fraudsters are weaponising AI to create fake videos and audio recordings that can fool even experienced executives.

Banks have suffered heavy financial losses due to voice clones that bypass current security systems. Additionally, corporate executives have found themselves impersonated in video meetings authorising fraudulent transactions, raising pressing concerns about how enterprises can safeguard themselves from such threats.

The challenge of defending against deepfakes

The swift maturation of deepfake technology has outpaced existing defences, compelling businesses to contend with an entirely new fraud category.

This is already manifesting on a concerning scale. For example, the former CEO of the world’s largest advertising group, WPP, was targeted by fraudsters who created a WhatsApp account using his photograph, later deploying voice cloning in a Microsoft Teams meeting.

The attackers impersonated Mark Read and another senior executive to extract money and personal details from someone within the company’s network.

“Fortunately, the attackers were not successful,” Mark notes in an internal staff email. He further cautioned, “We all need to be vigilant to the techniques that go beyond emails to take advantage of virtual meetings, AI and deepfakes.”

The danger of voice cloning

Entities within the financial sector have increasingly become victims of voice clones that accurately replicate authorised users. This technology reproduces speech patterns, accents and vocal nuances to such a degree that it can deceive both automated systems and human operators.

The now-defunct start-up Ozy serves as a cautionary tale of corporate deepfake fraud. An executive there pleaded guilty to fraud and identity theft after allegedly using voice-faking software to pose as a YouTube executive, attempting to deceive Goldman Sachs into investing US$40 million in 2021.

Given that voice calls and video interactions are key channels for critical financial decisions, this makes them prime targets for fraudsters. Current tools can generate eerily realistic voice imitations from just minutes of audio.

Public figures face heightened risks because their voices are easily accessible due to speeches, interviews, and media appearances. Corporate executives also frequently join this category as businesses underscore thought leadership and public engagement.

The ethical concerns

Deepfake misuse extends far beyond financial fraud. Creating fake intimate images without consent has become weaponised against women in public life. Former UK Conservative Cabinet Minister Penny Mordaunt, who served as an MP for 14 years, suffered from this form of exploitation when her face appeared in AI-generated pornographic content alongside other female politicians.

"The people behind this... don’t realise the consequences in the real world when they do something like that... It plays across into people taking actual real-world actions against ourselves,” she says during an interview with BBC Newsnight.

Victims report feeling violated and helpless, especially when fake content proliferates across social media. Even school principals have been targeted by fake audio recordings. A Baltimore principal faced suspension over fabricated audio recordings containing offensive remarks, which were eventually exposed as deepfakes created by a colleague.

Corporate defence strategies

Against deepfake threats, companies must adopt multi-layered defence strategies. Technology alone cannot resolve the problem since deepfake capabilities continually evolve.

Authentication protocols are among the initial defences companies should establish, particularly for high-stakes communications involving sensitive or financial information. Verification procedures could encompass call back measures, secondary confirmation channels, or in-person verification for crucial decisions.

Employee education is an equally vital component of deepfake defence. Training programmes should educate staff on recognising subtle audio quality issues, atypical speech patterns, or requests deviating from standard procedures.

Mark from WPP highlights specific warning signs in employee communications, cautioning staff about requests for passport information, money transfers and references to secret transactions unknown to other employees.

“Just because the account has my photo doesn’t mean it’s me,” he emphasises, showing that visual verification alone is insufficient for authentication.

The response of regulators

Governments worldwide are challenged with crafting fitting regulatory responses to the rise of deepfake technology. The UK proposes legislation to criminalise creating or distributing sexually explicit deepfakes, acknowledging the rise in non-consensual intimate imagery. However, accountability on social media platforms remains contentious.

These platforms face mounting pressure to implement detection and removal systems, but the volume of content and the sophistication of deepfakes present enormous hurdles.

Penny champions stronger age verification measures on online platforms, believing that technology leaders possess the capability to devise effective solutions.

“Elon Musk is taking the human race to Mars. I’m sure he can figure out age verification,” she states, referring to the owner of the social media platform X.

The escalating sophistication in cyberattacks on senior leaders, as highlighted in Mark’s email, indicates that regulatory and corporate solutions must evolve in tandem with emerging threats.