Will AI improve healthcare outcomes or amplify existing system failures?

Hospitals are rushing to deploy artificial intelligence. But without disciplined systems, good governance and human-factors design, it will merely automate today’s failures. The real risk is not that medical algorithms will be wrong—it is that they will be right in ways that could make unsafe systems even more dangerous.

Every year, millions of patients are injured or killed not by their disease, but by the systems meant to treat them.^[1],[2] Harm persists because healthcare remains organised like a cottage industry: reliant on heroic clinicians, fragile processes and hospital boards with limited visibility of what happens on the clinical ground. Technology alone will not fix this.

Artificial intelligence, introduced into low-reliability systems such as healthcare may merely automate unreliability. In patient-care terms, reliability means how often a clinical process delivers what it is supposed to deliver, for the right person, at the right time, under routine conditions. At this level, one in five care steps fails.^[3], ^[4] The evidence shows that in most hospitals clinical processes operate at roughly 80% reliability (based on our assessment of this study).^[5]In complex care pathways involving dozens of steps, failure becomes statistically inevitable.^[6]In contrast, high-reliability organisations—airlines, nuclear power plants, aircraft-carriers—take a different approach. Reliability in safety-critical industries like aviation often approaches 99.999% (the “five nines”) for certain critical processes, meaning errors occur only in one out of 100,000 events.^[7],[8] They design work so that errors become extremely rare, aligning processes, culture, threat management, event reporting, accountability, standardisation and measurement into a single operating system.^[9] Healthcare has yet to do this at scale. AI makes that omission impossible to ignore.

These types of organisations are preoccupied with failure, assuming that things will go wrong, so they design systems to detect weak signals early.^[10] They resist the comforting fiction that complex systems behave in simple ways. They defer to expertise rather than hierarchy. They stay sensitive to operations, knowing what is really happening and listening to those on the front line. Above all, they are committed to resilience: the ability to adapt when the unexpected occurs. In contrast, many healthcare organisations often treat quality primarily as a clinical issue rather than a core governance responsibility. As a result, critical quality data frequently arrive months after the fact, limiting timely decision-making, and unacceptable variability in care delivery is often tolerated rather than systematically addressed. AI is introduced as a tool, not as a rethinking of how dynamic risk is managed. The result is that AI can quietly make fragile, understaffed systems already operating close to their limits more brittle and prone to failure.

Human‑factors science explains how people interact with technologies, tasks and systems that improve safety, performance and well-being. Patient-safety research has demonstrated that harm is seldom caused by individual incompetence alone.^[11] Most healthcare systems fail because they are poorly conceived—how information and work flow, how decisions are made and whether teams and patients can speak up.^[12]

Systems‑engineering work shows that hospitals are complex adaptive systems.^[13] When a new technology is introduced, everything else shifts: workload, timing, communication and the location of risk. Research has shown how poorly designed tools increase interruptions, cognitive load and co-ordination failures. Clinicians do not fail because they do not care. They fail because systems make it hard to do the right thing.^[14]

In the operating theatre, thoughtfully applied AI feedback exposes the root causes of errors and near-misses, redirecting focus from individual blame to the systemic factors that influence surgical performance.^[15] The most effective surgical teams advance by gaining clear, unbiased insight into their own work—free from fear or judgment. When data is used for learning rather than punishment, skill improves. Used badly, it becomes surveillance.

Most healthcare AI today is built to answer narrow technical questions: is this cancer? Will this patient deteriorate? Was that surgical step done correctly? These tools can help.^[16] But the metrics they optimise often reflect engagement and efficiency more than diagnostic accuracy or patient safety. ^[17]The bigger danger is not that AI will be wrong, but that it will be right in a way that undermines teamwork and trust. Opaque recommendations encourage automation bias. Clinicians stop questioning and deskill. Hierarchies harden. Situational awareness narrows. When the model errs, the system is less able to recover.

High-reliability organisations do the opposite. They design systems that make it easy to challenge, escalate and learn. Technology supports collective sense‑making, not blind obedience. AI should reduce cognitive load, improve co-ordination and make it easier to speak up. If it does not, it is making care less safe, not more.

AI is not a panacea. Its value lies in being seamlessly integrated into clinical systems rather than functioning as an isolated tool. Rigorous simulation testing is essential, and success should be measured by its impact on teamwork, workload, communication and staff wellness—not solely on accuracy for patient care. Human involvement in AI-guided decision-making remains critical, supported by robust governance frameworks. Healthcare must adopt a similar approach to the aviation sector by investing in human factors and systems engineering. The true potential of AI is in enhancing system reliability and safety; without this, existing inefficiencies risk being digitised at greater speed, potentially exposing patients to dangers.^[18] AI can make surgical errors measurable and visible—but whether that leads to safer care or just more scrutiny depends entirely on how healthcare systems choose to use that information. Patient safety must be viewed as the foundational architecture, with thoughtful human design ultimately determining success.

Views or opinions expressed are those of the author and any individuals cited, and do not necessarily reflect those of Economist Enterprise or any other member of The Economist Group.