Page 9 - FIGI - Big data, machine learning, consumer protection and privacy

countries, where bias is unintentional, it may never- are coming under increasing scrutiny, and laws pro-
theless be unlawful if it has “disparate impact,” which viding consumers direct rights are being introduced.
arises where the outcomes from a selection process Conventional requirements to provide notice of
are widely different for a protected class of persons. the intended purpose of using a consumer’s per-
A key question is to what degree firms should sonal data when the purpose may as yet be unclear,
bear the responsibility and cost of identifying poten- or obtaining consent for something the consumer
tial bias and discrimination within their data algo- largely cannot understand, are under strain. Risks
rithms. Firms relying on big data and machine learn- from inaccuracy of data inputs, or bias and discrim-
ing might employ tools (and under some laws be inatory treatment in machine learning decisions
responsible) to ensure that their data will not amplify also raise difficult questions about how to ensure
historical bias, and to use data to identify discrimina- that consumers are not unfairly treated. The diffi-
tion. Ethical frameworks and “best practices” may be culty of ensuring transparency over decisions gen-
needed to ensure that outcomes will be monitored erated by algorithms, or of showing what harm has
and evaluated, and algorithms adjusted. been caused by artificial intelligence techniques that
The vast amounts of data held by and transferred would not have otherwise been caused, also pose
among big data players creates risks of data secu- challenges for consumer protection and data privacy
rity breach, and thus risk to consumer privacy. Per- law and regulation.
sonal privacy may be protected in varying degrees The challenges arising for the treatment of big
by using privacy enhancing technologies (PETs). A data and machine learning under legal and regulato-
market is growing in services for de-identification, ry frameworks for data protection and privacy sug-
pseudonymization and anonymization. Differential gest that the development of robust self-regulatory
privacy is also increasingly being employed. Regu- and ethical regimes in the artificial intelligence and
lation may need to ensure that privacy enhancing financial services community may be particularly
technologies are continuously integrated into big important. Facing legal and regulatory uncertainty,
data and machine learning data processing. This may businesses may introduce risk management systems,
require establishing incentives in legislation that cre- employ privacy by design and develop ethics.
ate liability for data breaches, essentially placing the There are various areas for further exploration and
economic burden not on the consumer by obtaining development of standards and procedures, including
their consent but on the organizations collecting, in relation to acceptable inferential analytics, reliabil-
using and sharing the data. ity of inferences, ethical standards for artificial intel-
Big data and machine learning are made possible ligence, provision of post-decision counterfactuals,
by intermediaries, such as third-party data brokers documentation of written policies, privacy principles
who trade in personal data. Transfer of personal data for design, explanations of automated decisions,
creates risk of breach and identity theft, intrusive access to human intervention, and other accountabil-
marketing and other privacy violations. Data brokers ity mechanisms.

Big data, machine learning, consumer protection and privacy 7