The Future of Cybersecurity is Fighting AI with AI

How can businesses realise the promise of generative AI and defend against the new threats it creates?

If generative AI takes off like other computing innovations, we’re at the beginning of a major shift in the ways people use technology. “Hockey stick” growth has just begun, as the early adopters in every organisation explore ways to be more efficient and apply new generative AI tools to workflows.

Generative AI growth by sector

Percentage of companies that have used AI to produce goods and services by sector

Unfortunately, the bad guys are far ahead of the curve. While enterprises see opportunity in generative AI for new methods of interacting with customers and eliminating the tedium associated with building code or writing reports, the threat actor sees generative AI as a low-cost way to supercharge existing attack strategies and exploit new vulnerabilities at scale.

“The adversaries are moving faster, weaponising vulnerabilities by reverse engineering the patch,” says Michelle Abraham, research director, security and trust, IDC. “Exploits are being developed more quickly.”

Take phishing. Clumsy emails purporting to be from your bank or your boss, with misspelled subject lines and dodgy links, have polluted inboxes for decades. Countermeasures included stricter filtering of inbound emails and admonitions by IT departments not to click on suspicious links.

Deepen Desai, CSO of cloud security platform provider Zscaler, warns of a new generation of phishing exploits boosted with generative AI capabilities. Phishing-as-a-service not only allows attackers to target thousands of companies simultaneously, but, when armed with generative AI, can also be instantly tailored to every employee at each one of those organisations.

“The template can be directed to pretend to be the manager or IT security administrator and write an email to your organisation's employees advising them of some security best practices,” Mr Desai explains. “Generative AI is able to quickly add in customisation to a phishing email template, and it's able to do it in different languages and with different personas. Threat actors can use those phishing kits to launch large-scale, highly targeted attacks, which are heavily customised, and all in an automated fashion.”

How threat actors can leverage AI across the ransomware attack chain

Zscaler’s ThreatLabz analysed 18bn transactions across its Zero Trust Exchange between April 2023 and January 2024, revealing key trends in how enterprises are adapting to the rapidly shifting AI landscape.

The findings are presented in the ThreatLabz 2024 AI Security Report and reveal that AI and machine learning (ML) tool usage skyrocketed by almost 600% in that period, rising from 521m AI/ML-driven transactions in April 2023 to 3.1bn in January 2024. Meanwhile, enterprises are blocking 18.5% of all AI/ML transactions—a 577% increase in blocked transactions over nine months.

The report makes clear the growing concerns around AI data security and companies’ reluctance to establish AI policies. But although some enterprises are exercising extreme caution and using blunt tools, others are not monitoring generative AI usage at all.

“There are still a lot of organisations that are not confident in their visibility of everything that's in their environments,” IDC’s Ms Abraham explains. “It could be developers spinning up a cloud instance and not putting the right configuration around it, or SaaS [software-as-a-service] applications that are being used unofficially.”

Prompt engineering is another concern. Users can use creative prompts to nudge generative AI models to provide more useful output, such as a more descriptive summary or a shorter block of code. However, there is also a real risk of systems being tricked into yielding IP that they shouldn’t, or delivering different outcomes than the business wants, such as oversized refunds or trip cancellation policies that don’t exist.

Employees may also be unaware that copying and pasting data into a publicly facing generative AI website may expose sensitive IP.

“If your employee copies finance information or your code and asks ChatGPT to explain it, inadvertently, that information is now part of the public ChatGPT model,” Mr Desai warns. “Your information is leaked—it's not reversible.”

A Zero Trust Approach to New Threats

Legacy security frameworks are not able to counter the growing generative AI threat. Perimeter defence may have worked when applications ran on a server in a utility closet or a corporate data centre, and employees were granted access through local permissions or virtual private networks (VPNs). That model falls apart with post-pandemic working-from-home policies, SaaS applications, ubiquitous mobility, and slick generative AI apps promising time-saving shortcuts and dazzling output.

“Say the environment is flat, and it’s using legacy architecture and controls. There is limited protection for machine learning environments involving large language models. The environment is prone to lateral propagation attack risks, it's prone to exploitation, it's prone to remote access, and it's prone to data exfiltration as well.”

Deepen Desai, Zscaler

High-risk scenario

Furthermore, the traditional approach to shutting down breaches—responding to alerts of an attack, manually combing through event logs to determine the cause, and helping users and management to triage the resulting damage—simply does not scale with AI-enabled attacks.

The solution? Fighting AI with AI. Specifically, combining the well-established Zero Trust architecture with a data fabric that contextualises millions of data points in real time, leading to proactive threat mitigation.

“Zscaler’s approach has been to create user-to-app segmentation and make sure that the user is never on the network, but is only connected to the resource that they need access to,” says Syam Nair, CTO of Zscaler. “This is the power of the Zero Trust Exchange.”

An effective Zero Trust infrastructure will limit the blast radius of an attack. If a single asset is tied to a single identity—for example, a laptop used by a marketing executive in Berlin—it is extremely difficult for a threat actor to move laterally through the organisation if that device is somehow compromised.

Context is key. For instance, is the marketing executive connected to a corporate network, or is he using a Wi-Fi hotspot at a café? What sanctioned apps are running on the device? Do any have unpatched vulnerabilities? Has someone else on his team recently been the target of a breach attempt? Are there anomalies in network traffic indicating unsanctioned apps running in the background?

“We collect about 500trn signals from 400bn transactions that go through our system daily,” Mr Nair says. “Instead of asking customers to manage the logs, we want to provide them with actionable outcomes based on the logs that we have connected with the other data that they already have.”

AI can streamline the response at several stages. It can help with data classification and applying data loss prevention (DLP) policies. It can provide recommendations to security professionals, and in certain situations take proactive steps, such as isolating compromised assets or blocking access to a critical resource.

“The power of generative AI is not sentence completion,” Mr Nair explains. “It's going to be in the context of being able to translate signals it sees into intent that’s articulated in a simple, readable way. And to understand that intent—to create, investigate, or take whatever action needs to be taken, and make it seamless.”

Apathy is not an Option

Corporations, governments and other organisations are still grappling with how to balance the promise of generative AI with its inherent risk. Although the pace at which attackers are leveraging AI to weaponise any available weak point gives legitimate cause for concern, there is real hope that AI-enabled security tools can help to counter—and ultimately, predict and prevent—the threat.

The banking sector, with a rich trove of personally identifiable information (PII) and access to financial transactions, is high on the target list for new exploits, including those augmented by generative AI. Darin Hurd, CISO of mortgage provider Guaranteed Rate, says that effectively leveraging AI for cybersecurity defence is not optional—it’s a requirement.

“AI enables our program to proactively block emerging threats, identify malicious anomalies, streamline and automate operations, and enhance our teams’ ability to respond to cyber-attacks,” Mr Hurd says. “Strategic partnerships with firms like Zscaler make these AI capabilities a reality.”