Page 23 - U4SSC Guiding principles for artificial intelligence in cities

• Scoping the impact assessments (cities can opt to conduct impact assessments for certain
potentially high impact AI systems rather than all of them, due to costs and efforts incurred
during the impact assessment).

3.2.9 Enabling human autonomy

It has been observed in practice that AI systems have penetrated into various applications in cities
and that this trend is projected to continue in the foreseeable future.

In this context, it is important to make sure the owners and operators of AI systems in cities preserve
and maintain the human autonomy to support automated and human decision-making.

Humans should determine:

– how AI systems should be designed;
– what their capabilities and constraints are; and

– when and how humans would intervene in AI systems during their functioning.

Hence, this principle allows cities to enable complete human autonomy over AI systems.

Implementation Considerations: Cities can adopt various mechanisms to enable human autonomy
over their AI systems. These mechanisms include:

• Setting AI systems design guidelines and specifications to include human oversight and “human
in the loop” provisions;

• Establishing governance models that support deployment of AI systems;
• Determining the level of autonomy of AI systems (e.g., fully autonomous, semi-autonomous);

• Specifying clearly and explicitly when and how humans will intervene during AI systems’
operations (e.g., pre-specified points during the tasks, in case pre-determined conditions occur,
exceptions);

• Determining whether AI systems should be deployed or not based on sufficient testing and
confidence; and
• Maintaining the right and the capability to switch off AI systems.

Data and algorithm aspects of AI systems: AI systems rely on algorithms operating on datasets. It is
of the utmost importance to consider algorithms and data distinctly for each of the above principles.
Compliance of one with respect to a principle does not necessarily warrant the compliance of the
other. Hence, data and algorithms should be analysed separately and assessed for compliance
with the principles that cities adopt.

13