Page 15 - The Annual AI Governance Report 2025 Steering the Future of AI

The Annual AI Governance Report 2025: Steering the Future of AI

Multi-agent security. When agents interact, new attack surfaces appear, such as secret collusion
that distorts markets, error cascades that spread misinformation and self-replicating 'agent
worms' like Morris II, which can infect entire application networks. Surveys show that current
12
defences, such as sandboxing (running code in a restricted environment to limit what it can access
or do), cross-examination (one AI agent to test another model’s responses for safety, accuracy,
or signs of manipulation) and cooperative 'AutoDefense' agents (agents working together
to detect and block harmful prompts), can reduce the success of jailbreaks, but they remain
basic. Future frameworks will require standards for isolation, authenticated communication,
13
and incident response across distributed agent ecosystems. This will entail moving beyond the
robustness of a single model to achieve system-level resilience.

1.3 Infrastructure for Agent Deployment

Critical infrastructure is being developed to support the deployment, monitoring, and control
of AI agents at scale.

Agent-infrastructure framework. Think of a future “agent-net” layered on top of today’s internet:
shared rails that let autonomous software act while giving humans levers to supervise it. Chan
et al. (2024) propose three core functions for this infrastructure: (i) attribution: attaching a
persistent identifier and “agent card” to every action, (ii) interaction shaping: real-time
14
monitors and permission systems that can pause or roll back risky behaviour, and (iii) harm
15
remedy: tamper-evident logs that regulators or courts can inspect after an incident to trace
responsibility. This moves governance toward prevention—from punishing bad outcomes to
16
designing environments that make good conduct the default. 17

Economic Infrastructure. E-commerce was designed for human fingertips—password boxes,
CAPTCHAs, and card numbers—meaning agents still struggle with the basics: proving identity,
discovering services, and processing payments. Researchers discussing agentic finance suggest
that, without verifiable credentials and transparent loss-allocation rules, merchants have nothing
to base their trust around when it comes to code they have never encountered before. Fintechs
are rushing to retrofit the infrastructure: Stripe's open-source Agent Toolkit enables an LLM to
generate one-time virtual cards or initiate bank transfers with a single command. Meanwhile,
18
Visa has announced pilots that connect autonomous shopping agents directly to its global
network, indicating that a fully agent-driven checkout process is now a priority for the industry.
19
Still missing, policy analysts note, are interoperable identity proofs (e.g., verifiable credentials)
and liability frameworks that allocate losses when an agent misfires. 20

12 Cohen, S., Bitton, R., & Nassi, B. (2024, March 5). Here Comes The AI Worm: Unleashing Zero-click Worms
that Target GenAI-Powered Applications. arXiv.org.
13 Deng, Z., Guo, Y., Han, C., Ma, W., Xiong, J., Wen, S., & Xiang, Y. (2025). AI Agents Under Threat: A survey
of key security challenges and future pathways. ACM Computing Surveys.
14 Chan, A., Ezell, C., Kaufmann, M., Wei, K., Hammond, L., Bradley, H., Bluemke, E., Rajkumar, N., Krueger,
D., Kolt, N., Heim, L., & Anderljung, M. (2024). Visibility into AI Agents. 2022 ACM Conference on Fairness,
Accountability, and Transparency, 958–973. Page 963.
15 Chan, A., Ezell, C., Kaufmann, M., Wei, K., Hammond, L., Bradley, H., Bluemke, E., Rajkumar, N., Krueger,
D., Kolt, N., Heim, L., & Anderljung, M. (2024). Visibility into AI Agents. 2022 ACM Conference on Fairness,
Accountability, and Transparency, 958–973. Page 961.
16 ibid.
17 See also: Kraprayoon, J. (2025, April 17). AI Agent Governance: A Field Guide. Institute for AI Policy and
Strategy.
18 Add Stripe to your agentic workflows. (n.d.). Stripe Documentation.
19 Visa wants to give artificial intelligence “agents” your credit card | AP News. (2025, April 30). AP News.
20 Birch, D. G. (2025, May 24). Agentic commerce does not work without agent identities. Forbes.

6