Financial scammers are enjoying a productivity boost thanks to advances in artificial intelligence (AI). Generative AI, which can draft highly personalised phishing emails, clone voices and create realistic impersonations, is proving especially useful. It allows fraudsters to amplify their attacks in both volume and sophistication.
The financial sector is among the most exposed. A recent study by IBM and Red Hat, two technology firms, found that it was the second-most targeted industry in 2024, after manufacturing, accounting for 23% of reported incidents, up from 18% a year earlier. Deloitte, a consultancy, forecasts that in America alone losses from AI-powered scams could rise to $40bn by 2027, up from $12.3bn in 2023.
And the threat is spreading. Besides scamming the unsuspecting public, cyber-criminals, often backed by organised crime networks, have successfully defrauded bank managers and finance staff at multinational companies, showing just how effective their campaigns have become.
Digital deceit
The main reason AI has helped scammers so much is because it lets them achieve personalisation at scale. Models can “not just create and send phishing emails in vast numbers, but can also collect background information [to personalise those emails],” explains Fred Heiding, an AI-security researcher at Harvard Kennedy School.
Criminals can mount thousands of attacks at once, yet tailor them to mimic real people or trusted institutions. Among the most alarming developments is the rise of deepfakes, which are created through deep-learning algorithms that convincingly replicate voices, faces and even video interactions. Deepfake-based identity-verification “face swap” attempts surged by 704% in 2023, according to iProov, a biometric company.
Aided by recent AI models, voice-cloning scams have gone far beyond the unconvincing mimicry they used to be. In the “Hi Mum” or “Hi Dad” con, conducted over WhatsApp or SMS, victims receive pleas for money from what appears to be a relative or a friend who claims to have lost access to their phone or bank account. Increasingly, these messages are accompanied by convincing voice notes that exploit familial and amicable trust.
Video-based deepfakes—again, made increasingly more believable with advanced AI—are often used to defraud vulnerable citizens in romance scams. In one case in Britain, a suitor who used doctored videos, including a fake marriage proposal, managed to withdraw £17,000 from the pension of an unsuspecting 77-year-old woman. Businesses are vulnerable too. In 2024, scammers used an AI-generated video of a company meeting to impersonate a multinational firm’s chief financial officer and thus dupe an employee into transferring them $25.6m. The total losses from scams affecting Americans alone could be as high as $50bn a year, estimates Erin West, a former prosecutor who led some of America’s first attempts to try online scammers.
In phishing campaigns, fraudsters use natural-language models and machine learning to create emails, text messages and even voice calls that are linguistically polished and emotionally manipulative. AI can mimic tone, syntax and brand language, making fraudulent communications increasingly difficult to detect. Some attacks even deploy chatbots programmed to mimic human conversation, reinforcing the illusion of legitimacy. The rise of generative-AI tools has facilitated an estimated increase of 1,265% in such scams since the launch of ChatGPT in late 2022.
AI is also strengthening the execution of distributed denial of service (DDoS) assaults and password cracking. DDoS attacks, whereby scammers flood a website with traffic to bring down its services, increased by 55% year-on-year in 2024, with finance the most targeted sector. AI makes such attacks more adaptive, mimicking legitimate traffic and identifying vulnerable points in systems to maximise disruption and minimise detection.
Passwords are becoming more crackable too. AI can check billions of passwords per second and generate more plausible entries, by learning from data like patterns and distributions spotted in password leaks. And platforms are emerging to industrialise cyber-crime through tools such as WormGPT, FraudGPT and DarkBARD, which offer everything from malicious code generation to inbox flooding and credit-card fraud, designed specifically for criminal enterprises.
Fighting back
But AI is also helping in cyber-defence.The same techniques that enable criminals to mimic trusted voices can also be used by banks and payment platforms to spot and outwit scammers. Facial and voice recognition algorithms, for example, increasingly power protective fraud-detection systems, transaction monitoring and authentication.
Self-healing cyber-security systems are another AI-powered innovation that can help counter scams, similar to how a human immune system would. Automated “patching” tools can respond within minutes to detected threats, such as DDoS attacks or malware bots. “The good thing here is that once a vulnerability is established, it’s easier to protect against it,” notes Mr Heiding. Until now, it was common for defenders to know about vulnerabilities in their systems but not have enough developers to patch them. AI-assisted patching mitigates this and there are active efforts to make such self-healing cyber-security systems more common. For example, in 2023, the American government launched the AI Cyber Challenge, a programme to spur their development. “When these tools are released into the world to patch vulnerabilities, we raise the bar for what attackers need to do,” says Mr Heiding.
But improvements to online security and governance could also restrain cyber-criminals. For example, these groups often rely on domain names that sound similar to the trusted brands they seek to impersonate, a process called “cyber-squatting”. Experts believe there should be limits, or flags, to prevent the bulk-buying of such sites.
Some of the issues are broader. “It’s not reasonable that it's so easy and cheap to buy a ton of domains,” argues Mr Heiding. “[In the field of cyber-security] we talk about ‘secure by design’, which is an initiative that shifts responsibility from the users to the producers of the technology. But we haven’t seen enough of this [in the fight against] phishing. This is something domain providers need to be much better at.”
Customers must take steps to protect themselves too, similar to being mindful of their digital footprint. “It’s not as easy as just removing information; people often want to ‘market’ themselves, but that information can also be used against them,” Mr Heiding points out.
Digital identity wallets could be one technological fix. Backed by the World Economic Forum and the European Union, these use biometric verification and privacy-preserving technologies to reduce identity theft and deepfake risk, such as detecting user liveness and protecting data-sharing processes, helping institutions verify individuals while offering a more secure and user-controlled identity framework. Keeping pace with a productive enemy will require greater vigilance among consumers, enterprises and intermediaries and an embrace of AI’s full powers.