Page 38 - AI Governance Day - From Principles to Implementation

AI Governance Day - From Principles to Implementation

• How do we uphold human rights and ethical considerations in AI development such as
transparency, accountability, privacy and others for all individuals globally?
• How do we have the global majority take part in shaping the future of AI technologies
and AI governance?

4.5.3 Open vs closed sourcing of (generative) AI models

The debate surrounding the open- vs closed-sourcing of increasingly capable AI models
highlights the tension between the benefits of transparency, external oversight and rapid
innovation against the potential risks of misuses and unintended consequences. "Open-
sourcing AI models" for our purposes means that the AI model architecture and the associated
weights are freely and publicly available to anyone to use or modify. Closed, or proprietary,
refers to AI systems, particularly foundation models, where the underlying algorithms, model
architecture, datasets, and training methodologies are kept confidential by the entity that
developed them.

Arguments in favor of open sourcing of (generative) AI models:

• Promotes innovation: By making the models public, a broader range of developers can
contribute to and enhance the technology.
• Increases transparency: Open-sourcing allows for community auditing of the models,
which can lead to improvements in model safety and ethics. More eyes spot more bugs.
• Fosters collaboration: Open-source AI systems encourage collaboration among
researchers and developers, leading to faster innovation and improvement of technology.
• Increases access: Open-source AI systems are more accessible to a wider range of users,
including those from resource-constrained environments, promoting inclusivity and
access to AI technology. This can reduce the knowledge and resource gap between
large corporations and smaller entities, counteracting the centralization of power in AI
companies.
• Prevents vendor lock-in: Users of closed-source AI systems may become dependent on a
single vendor, making it difficult and costly to switch to alternative solutions in the future.
• Reduces monopolistic practices: Provides opportunities for smaller entities to participate
in and benefit from advanced AI without the prohibitive costs of developing proprietary
models.

Arguments against open sourcing of (generative) AI models:

• Security risks: There is an increased risk of misuse as more actors can access powerful AI
tools, potentially leading to harmful applications.
• Quality control: It may be challenging to maintain high standards of quality and reliability
when control over modifications is decentralized.
• Irreversibility: If a model has been released with a flaw that would allow grave misuse, or
inherent safety risks of the model, there is no straightforward way to prevent someone
from continuing to use the model or to ensure that users install patches to fix the model.
• Regulatory challenges: Open-source models could complicate efforts to enforce
compliance with ethical standards and legal regulations due to their widespread and
uncontrolled distribution.

Potential Discussion Questions

• How has open-sourcing of AI models benefited startups, academic institutions, and
developers in resource-constrained environments?
• Are we seeing the advantages/disadvantages from already open- or closed-sourced
models?

28