More than the ghost in the machine: human agency in agentic AI deployment

The emergence of agentic AI is rewriting the procurement handbook, and thus the skills needed by procurement professionals. As digital agents take over high-volume tactical work, the human role is shifting from task execution to designing – and continuing to manage – the entire architecture within which these tasks take place.

The challenge becomes greater as agents proliferate within individual businesses. Firms generally begin their AI journey by deploying a handful of agents in specific areas. But this is very different from managing dozens or hundreds across areas such as sourcing, contracts, risk and logistics, which is the direction of travel. While the focus in the first instance will be on designing and deploying agents, and creating the guardrails and systems they work within, the next step will be to ensure that the humans in charge remain up to the evolving job. 

Who’s in charge?

The question of who ultimately owns agent decisions, and what oversight is needed when agents operate across functions, sits at the heart of this evolution. And this should always be who, not what. As AI becomes more deeply embedded in business processes, however, it is not always clear where that responsibility lies.  

“It cannot be on the functional person who is using these agents,” warns Khushboo Jajoo, vice president, global delivery at GEP. “That's where businesses need proper guidelines on how these ecosystems work, and where the right intervention points should be.”

“The answer is straightforward: the process owner remains accountable,” says Joni Kuusiniemi, general manager, indirect procurement at Wärtsilä, a Finnish manufacturer in the marine and energy markets. However, because of the complexity of these interactions, this must be disseminated across business, technology and data owners. The process owner owns the outcome, technology teams own the platform and data owners are responsible for the quality of the information used by the agent. “If any of those elements fail, agent performance will suffer,” he explains. 

Teams therefore need behavioural guardrails, continuous validation of outputs and monitoring for drift or bias. External regulations and internal policies will also require audit trails, security controls and clear processes for updating agents as requirements change.

“A solution must clearly identify who is the business owner for the outcome, who is the technical owner for this AI agent, and who will be responsible for performance and reliability,” says Ms Jajoo. 

“Human validation is critical,” Mr Kuusiniemi adds. “AI should improve productivity and decision quality, but it should never become a black box.”

Changing profiles

All of this is changing the skills procurement teams need. As agents absorb administrative and analytical work, the human workforce will need a greater focus on data fluency, systems thinking and AI orchestration skills.

“I don't think procurement suddenly becomes a data science function, but the profile is definitely evolving,” says Mr Kuusiniemi. “The mindset needs to shift from ‘AI gave me an answer’ to ‘AI gave me a recommendation that I am responsible for validating.’ ”

“You cannot just be a functional expert and leave technology to someone else,” agrees Ms Jajoo. Procurement professionals need a broader understanding of how and why processes are automated, and of the strengths and limitations of the technology they are using.

“Just having one capability is not enough. We want more techno-functional experts, rather than just technology or functional knowledge,” she explains.

Ms Jajoo draws on her own professional development to illustrate the point: “Seventy per cent of what I do is still people business. Thirty per cent is now technology, so I've not completely shifted into technology. But I think my own journey and what I’ve seen in the industry has been that the boundaries are blurring.”

Like employees… sort of

This requires a command of end-to-end process by the human workforce, rather than just knowing how to write AI prompts for one corner of the operation. Keeping agents safe and effective is not like running office software. “Agents are more similar to employees, where you will need ongoing training, coaching, performance reviews and interventions when something is not going well,” explains Ms Jajoo.  

Probably the biggest difference between previous digital automation and agentic AI is that the latter makes autonomous decisions. Judgment calls are hardwired in. To the extent that agents are making decisions and influencing outcomes, they need to be managed accordingly.

“Treat AI as another small team within your procurement team – someone like a high-performing team member, which will need clear objectives, defined responsibility and continuous coaching,” says Ms Jajoo. 

Operationally, that is a very different approach from how firms have traditionally managed technology. Companies need clear triggers for human intervention and, just as importantly, people with the training and judgment to know when, why and how to intervene.

Human augmentation: from skills to strategy

The firms that succeed in agentic AI deployment will not be those that replace humans with bots. AI augments the human role. But it also changes it. The procurement team’s role becomes less about executing processes and more about defining goals, constraints and outcomes.

“In the traditional world, everything was rules-based,” explains Ms Jajoo. “There’s a code; it runs as you have written it. But AI agents are fundamentally different because you need to apply context. There will be human interpretation needed before certain action is taken or triggered.” 

Recognising and retaining core skills

The change is not simply technical. The capabilities that agents cannot easily replicate include strategic negotiation, relationship building, ethical judgment and stakeholder management. In short, those interpersonal skills that make us truly human. 

Such business-critical decisions must remain with procurement professionals. “I would be cautious about fully delegating decisions that carry significant commercial, financial, legal, regulatory or reputational consequences,” warns Mr Kuusiniemi. 

Key areas that should remain human decisions – supported by AI, rather than delegated to it – include supplier selection, major sourcing awards, contract exceptions and risk acceptance decisions, he says. 

“These are exactly the areas where procurement professionals continue to add the most value,” Mr Kuusiniemi explains. “AI can analyse data, identify patterns and support decision-making, but it does not replace trust, influence, judgment or relationship management.”

The future of procurement is therefore not about replacing human intelligence with artificial intelligence. It is about building a workforce in which the two operate together, while humans retain responsibility for the outcomes that matter most.