Page 41 - The Annual AI Governance Report 2025 Steering the Future of AI

The Annual AI Governance Report 2025: Steering the Future of AI

infrastructure. Local governments and research networks in Kenya , Nigeria , and India
186
184
185
are leveraging open weights to develop domain-specific tools in healthcare and agriculture.
At the same time, some actors critique how “open” often still means reliance on infrastructure
(e.g., cloud, datasets) dominated by a handful of companies. 187

Future Trajectories and Governance Questions. As more governments and multilateral
organisations consider standards for model openness , key questions remain: who decides
188
which models can be open? How can transparency and safety be balanced without excluding
less-resourced actors? Proposals such as ‘responsible open release’ and ‘tiered access’ are
gaining traction but risk entrenching asymmetric control. Meanwhile, a parallel governance
189
ecosystem—of open science initiatives, public compute efforts, and South-South cooperation—
may offer more pluralistic models for AI development.

6.5 AGI, Existential Risk, and Social Resilience

Artificial General Intelligence (AGI): AGI is usually defined as an AI system that can match or
exceed humans across the full range of cognitive tasks, but opinions on when—or even whether—
it will arrive diverge sharply. On the optimistic side, Google DeepMind CEO Demis Hassabis
190
told TIME that with a few big breakthroughs beyond today’s large-language models, AGI could
be five-to-ten years away , while Google co-founder Sergey Brin floated a “before 2030” target
191
at Google I/O. Large-scale surveys paint a slower trajectory: the 2023 AI Impacts survey of
192
nearly 3,000 ML researchers gives only a 25% chance of “high-level machine intelligence” by
the early 2030s and 50% by 2047, a result echoed in a 2025 meta-review of expert forecasts.
193
Meanwhile, prominent sceptics argue that scaling current LLMs is not a royal road to AGI.
Apple’s study “The Illusion of Thinking” empirically shows frontier reasoning models falter as
problem complexity rises, challenging claims that GPT-4-style architectures already contain the
“sparks” of general intelligence. 194

Theoretical Foundations and Definitional Debates: Existential risk (also referred to as x-risks)
in AI refers to the potential for AI, and AGI in particular, to cause severe damage on a global
scale to human well-being. This could range from the irreversible loss of human control to the

184 Farmonaut. Smart farming solutions boost Kenyan agriculture productivity. Retrieved June 19, 2025.
185 Yakubu, M., Yakubu, U., Yakubu, H., & Mayun, F. A. (2025, February 13). The effective use of artificial intelligence
in improving agricultural productivity in Nigeria. Journal of Basics and Applied Sciences Research, 2(4).
186 Lakhani, A. L., L., Kathiria, R. K., & Vadher, A. L. (2024). Government initiatives for artificial intelligence in
agriculture. Just Agriculture, 112.
187 Van Der Vlist, F., Helmond, A., & Ferrari, F. (2024). Big AI: Cloud infrastructure dependence and the
industrialisation of artificial intelligence. Big Data & Society, 11(1).
188 White, M., Haddad, I., Osborne, C., Liu, X. Y., Abdelmonsef, A., Varghese, S., & Hors, A. L. (2024, March 20).
The Model Openness Framework: Promoting completeness and openness for reproducibility, transparency,
and usability in artificial intelligence. arXiv.org.
189 Seger, E., Dreksler, N., Moulange, R., Dardaman, E., Schuett, J., Wei, K., Winter, C., Arnold, M., Héigeartaigh,
S. Ó., Korinek, A., Anderljung, M., Bucknall, B., Chan, A., Stafford, E., Koessler, L., Ovadya, A., Garfinkel, B.,
Bluemke, E., Aird, M., Gupta, A. (2023, September 29). Open-Sourcing Highly Capable Foundation Models:
An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives. arXiv.org.
190 Morris, M. R., Sohl-Dickstein, J., Fiedel, N., Warkentin, T., Dafoe, A., Faust, A., Farabet, C., & Legg, S. (2023,
November 4). Levels of AGI for operationalizing progress on the path to AGI. arXiv.org.
191 Perrigo, B. (2025, April 15). Demis Hassabis Is Preparing for AI’s Endgame. Time. Retrieved June 19, 2025.
192 Crowley, K. (2025, May 21). Google leaders see AGI arriving around 2030. Axios. Retrieved June 19, 2025.
193 Todd, B. (2025a, April 10). Shrinking AGI timelines: a review of expert forecasts. 80,000 Hours.
194 Shojaee, P., Mirzadeh, I., Alizadeh, K., Horton, M., Bengio, S., & Farajtabar, M. (2025, June). The Illusion
of Thinking: Understanding the strengths and limitations of reasoning models via the lens of problem
complexity (Apple Machine Learning Research Paper). Retrieved June 19, 2025

32