
Why is AI so persuasive—even when it’s wrong?
London Business School’s Keyvan Vakili explains why generative AI’s persuasiveness may be one of its biggest risks.
By Florence Wilkinson, journalist and filmmaker
Artificial intelligence has moved from specialist debate to everyday conversation. But as AI becomes more familiar, how well do we really understand what it is, and where its biggest risks lie?
According to London Business School's (LBS) Keyvan Vakili, associate professor of strategy and entrepreneurship, most people misunderstand the nature of AI. Rather than a single, all-knowing technology, it is a broad set of systems that learn from data in different ways: by recognising labelled examples, identifying patterns or improving through trial and error. A spam filter, for example, learns from emails already marked “spam” or “not spam”; other systems can group similar data points without being told what to look for, or improve by testing which actions lead to better outcomes.
Generating content is not verifying facts
That distinction matters because much of today’s debate is focused on one particularly visible type of AI: large language models (LLMs). These include tools such as ChatGPT, Gemini, Claude and DeepSeek. Because they’ve become the most familiar face of AI, many people now use “AI” as shorthand for LLMs.
That assumption can be misleading. Not every system is designed to generate language, and not every language-generating system should be treated as a source of knowledge. LLMs are trained on vast quantities of text to predict the next word in a sequence. Their strength is producing fluent, plausible content—not determining whether that content is true.
That distinction matters for organisations. When they treat LLMs as neutral sources of knowledge, they risk mistaking a well-written answer for a reliable one. In a business context, that can shape more than a draft email or presentation. It can influence research summaries, client recommendations, internal strategy documents and decisions that require judgement as well as speed.
LLMs are trained on vast quantities of text to predict the next word in a sequence. Their strength is producing fluent, plausible content—not determining whether that content is true.
The paradox of persuasion
"I expect AI to be capable of superhuman persuasion well before it is superhuman at general intelligence, which may lead to some very strange outcomes." Sam Altman, CEO of OpenAI on Twitter (now X), 25th October 2023
Keyvan has been working on the risks associated with LLMs. "A lot of people believe that these tools are becoming more reliable with every new generation," he explains, "but that's not always the case. These models are not designed primarily to be accurate. They are generators—there's no engine for accuracy in them."
LLMs are trained using internet data and are only as accurate as that content. "And what's the main objective of people who produce content on the internet?" Keyvan continues; "it's not to be accurate, it's to persuade—to convince people to do something, click on something or consume something."
The irony is that internet data has taught LLMs the language of persuasion, but not the discipline of verification. They can make mistakes fluently—and wrap them up in jargon to sound convincing. That is what makes their errors so difficult to manage: they don’t always look like mistakes. They can arrive neatly structured, confidently argued and wrapped in the language of expertise.
Keyvan tested this in a recent working paper with his colleague, Professor Bryan Stroube. They asked participants a simple maths question, then presented some with arguments by ChatGPT in favour of the correct answer, and others with arguments in favour of the wrong one. They "managed to convince more than 50 percent of the people who were initially correct to change their answer," Keyvan says. The risk, then, is not simply that AI gets things wrong. It is that it can make the wrong answer sound reasonable enough to override a person’s own judgement. For organisations, that raises a more troubling possibility: AI may not simply assist decision-making, but quietly distort it.
The risk is not simply that AI gets things wrong. It is that it can make the wrong answer sound reasonable enough to override a person’s own judgement.
The risk of invisible adoption
Keyvan's research highlights a critical paradox in how generative AI tools are being adopted. The individuals most likely to use LLMs are often those with the least experience, or lower confidence in their own abilities. They may benefit most from the productivity boost, but they may also be less able to spot when an output is flawed.
Compounding the risk, many behave like "secret cyborgs": they rely heavily on AI, while concealing their use of it from colleagues. This makes AI's influence harder to monitor and manage within organisations, and increases the risk that inaccurate outputs are absorbed into work without being challenged. Without visibility over where and how these tools are being used, leaders may struggle to know whether AI is improving work, weakening it or introducing risks no one has formally assessed.
Generative AI may be changing how work gets done, but it does not remove the need for human judgement. The more convincing AI becomes, the more carefully it must be questioned. The organisations that benefit most will be those that treat it not as an authority, but as a tool whose outputs still need to be tested.

To learn more about Think at London Business School, visit: