Page 45 - FIGI - Big data, machine learning, consumer protection and privacy

establishing the reliability of inferences, particu- er how, with different input attributes, they might
larly those with high social importance, risk and obtain different decisions from the automated
legal effect, and in relation to protected groups. In decision-making system. In circumstances where
addition, standards could be developed for test- these are considered to be viable, standards could
ing inferences before and after deployment. Such be developed for providing post-decision coun-
standards may require different approaches to terfactual explanations.
different types of services. 7� Developing best practices in processes for allow-
6� Developing standards for explanations of auto- ing consumers to obtain human intervention, as
mated decisions, including asserting the relevance well as for identifying the appropriate degree of
of data used to inferences drawn by the system, human intervention that maintains the integrity
the relevance of such inferences for the type of and value of the model, while also offering the
automated decision, and the accuracy and statis- consumer a meaningful opportunity to be heard
tical reliability of the data and methods used. This by a human being.
could involve encouraging developers of scoring 8� Developing principles of international best prac-
models to share with consumers (and if required, tice and harmonization of accountability mecha-
regulators) the key attributes used in a model, and nisms, including procedures for contesting auto-
their relative weighting, and ensuring that docu- mated decisions, standards for establishing prima
mentation and audit trails are provided in case of facie harm, and ultimately frameworks for assess-
legal process. Developing standards for explana- ing liability for design and operation of artificial
tions could also include examining the potential intelligence and machine learning models.
for using counterfactuals to inform the consum-

Big data, machine learning, consumer protection and privacy 43