Governing the use of autonomous AI agents is no longer a job humans can do alone and now requires the intelligence of AI itself, says Volker Rath, Field CTO, Cloudflare.
Organisations are increasingly adopting agentic AI, moving beyond simple automation to deploy intelligent systems capable of complex orchestration.
Unlike traditional automation, which follows rigid, rule-based processes, agentic automation integrates advanced AI technologies to create a dynamic workforce.
This shift enables systems to handle intricate workflows and perform critical tasks – such as fraud detection, customer support and forecasting – with unprecedented autonomy. The momentum is undeniable: McKinsey’s 2026 state of AI report finds that 40% of large organisations are already significantly scaling up their use of AI agents.
As these agents take on more responsibilities, businesses must confront a new reality: governing them is no longer a job humans can do alone, it now requires the intelligence of AI itself.
The common afterthoughts in a gold rush
As organisations rush to integrate emerging AI technologies, striking parallels are emerging between current scaling efforts and the early days of cloud adoption. While the primary focus remains on upscaling capabilities, critical enterprise safeguards – such as lifecycle management, continuous assurance, automated compliance and cost control – are at risk of being treated as afterthoughts.
While CIOs and CTOs move full steam ahead, CFOs, CISOs, CROs and audit teams try to keep pace with the fast-changing technology and risk landscape.
In the cloud era, retrofitting these safeguards after the fact led to ballooning costs and unmitigated operational risks. Because AI adoption is accelerating far faster than cloud ever did, applying those hard-earned lessons is imperative; building mature, proactive enterprise governance capabilities today is essential to staying within risk appetite and truly realising the business outcomes AI promises.
A thought experiment that shapes the mindset
Good decision-making requires a mindset that looks forward – one that sets future goals and needs and imagines a North Star capable of steering the organisation towards real business outcomes.
Here’s a thought experiment that can help define that North Star for enterprise-scale AI governance capabilities:
Imagine your organisation runs 10,000 AI agents in the future. All of them act on behalf of someone else and some of these people may have already left the organisation. Some of the use cases may already be outdated or obsolete.
As AI inference doesn’t come cheap, your company asks you to reduce cost by 20%. The question is: how do you best select which agents to shut down? What aspects should you consider that lead to the best possible outcomes without disrupting the business? What key metrics would you like to consider for each of these aspects and what data do you need from each agent to calculate these metrics on a continuous basis?
You’ll likely need a standardised decision framework that can be used by an automated AI governance layer that has access to key metrics across quality, customer satisfaction, cost, importance of the use case, dependability of other agents and use cases, security, compliance and other risk and success factors.
How do you measure quality in an automated fashion? You may also want to consolidate AI agents that do similar work, but do you have enough metadata on each agent to even make that comparison?
You may be wondering: should agents proactively send all required data to a central AI registry, to enable automated AI governance at scale? Maybe AI agents should only be allowed to run in production when they satisfy all governance needs by proactively sending data to the AI registry.
This registry could also store additional metadata such as owner, cost center, assigned budgets and other agent-specific context and issue a license to operate when agent registration is successful.
AI governing AI
When we dive deeper into this thought process, it becomes clear that independently operating AI agents require a continuously operating governance layer that enables visibility and control at any scale.
Tracking thousands of agents and their metrics in real time requires more than simple automation, it demands an AI-powered one. Questions such as ‘What agents have a similar purpose?’ can only be answered when metadata is available and when many free-text fields can be analysed for complex questions like this.
Organisations should consider the entire AI agent lifecycle early on to avoid afterthoughts that cause unmanaged risks and costs later.
Non-critical internal AI agents may not need to be managed within such a tight framework, but when AI agents perform business functions critical to the organisation’s success, we should govern them the way we govern people.
We already collect sufficient information throughout the year to conduct an annual review. We have insights into people’s work, the quality they provide and whether they are reliable, trustworthy and meet behavioural expectations. We also have documented processes for discussions about promotions and pay raises. Managing AI agents demands the same discipline we apply to managing human resources.
AI governance is not just a tool
Building an AI governance system takes new technology, but at its core, is a framework built backward from future needs. The future is approaching faster than most organisations are ready for.
