Page 47 - AI Standards for Global Impact: From Governance to Action

AI Standards for Global Impact: From Governance to Action

step in the process of creating an AI system, from data-set collection through to deployment
(and beyond). A key aspect of this process involves developing methodologies for constructing
and training AI systems that are safe or ensuring their alignment with specific contexts and
scenarios. Part 2: Thematic AI

Berkeley University presented the challenges posed by AI security risks and the research
on trustworthiness and risk assessment of different Large Language Models (LLMs), which
identified different ways today’s LLMs can be attacked. It was noted that, with the enhanced
capabilities brought about by Agentic AI systems, there is a need for collaboration among
scientific communities to share lessons learned and best practices.

The Netherlands Organisation for Applied Scientific Research (TNO) has been conducting
research on AI security, working closely with various Dutch government bodies and NATO
partners. As AI becomes increasingly integrated into critical systems, ensuring its security is not
just a technical challenge but imperative for national safety. Despite the growing importance
of this field, they observed a significant gap: the lack of openly accessible tools to assess and
enhance the security of AI systems. TNO is working on the development of an AI Security
Assessment Framework tailored to the needs of the Dutch government. This framework aims
to provide a structured approach to evaluating the security and trustworthiness of AI systems.
Their findings underscore a clear call to action: the AI community must collaborate to create
and share more open, practical tools that support the secure deployment of AI.

Atlas Computing presented the research concept of Flexible Hardware Enabled Guarantees
(FlexHEG). As AI advances, so does the potential for catastrophic risks resulting from accidents,
misuse or loss of control over dangerous capabilities. For example, severe misuse in domains
such as disinformation or cyber attacks seems plausible within the next few years. As such,
governance of AI technology — whether by national governments, industry self-governance,
intergovernmental agreements, or all three — is a crucial capacity for humanity to develop, and
quickly.

Hardware-enabled governance has emerged as a promising pathway to help mitigate such
risks and increase product trustworthiness by implementing safety measures directly onto
high-performance computing chips. These hardware-enabled mechanisms could be added
to powerful AI accelerators used in datacentres.

However, it is not yet clear which compliance rules will be most appropriate in the future.
Therefore, these hardware-enabled governance mechanisms could be considered for the flexible
updating of compliance rules through a multilateral, cryptographically secure input channel,
without needing to retool the hardware.Concepts like FlexHEG could enable multilateral control
over AI technology, thus making it possible for a range of stakeholders to agree on a variety of
potential rules, from safety rules to robust benefit-sharing agreements. Mutually agreed-upon
rules could be set and updated through a multilateral and cryptographically secure mechanism
to guarantee that only agreed-upon rules are applied.

6�2 International collaboration on AI testing

The session discussed the gaps in testing AI systems, evaluation of trust in AI systems, and
opportunities for international collaboration to ensure the effective design, development, and
deployment of AI systems that integrate considerations for AI trust. There was general agreement
among participants on the need for better collaboration internationally on methodologies and

35