Page 13 - FIGI - Big data, machine learning, consumer protection and privacy

(referred to as Convention 108), as recently amended CompuCredit had been reducing consumers’ credit
by the Amending Protocol to the Convention for the limits based on a model that reduced their scores
Protection of Individuals with regard to Automatic where they engaged in certain transactions, such as
Processing of Personal Data. 18 visiting pawn shops, personal counselling and pool
Three core tenets of data protection and priva- halls. 21
cy law are purpose specification, data minimization, The treatment of data available on individuals,
and the treatment of data of “protected” or “special” and in particular the process of profiling them and
categories of groups (such as racial, gender, reli- drawing inferences about them, is thus central to
gious and other groups). These tenets come under the provision of such financial services. Consequent-
strain when the specific purpose of collecting and ly, achieving fairness, accuracy and transparency in
processing data may only become understood as financial services must take into account what and
the machines themselves learn from high volumes how personal data is being collected, being used,
of observed and performance data, producing more and being shared with third parties. 22
accurate analysis. Personal data can also serve as a These challenges are made more complex by the
proxy for membership of a protected group. variety of regulatory frameworks applying to differ-
These new technologies also present risks, some ent types of digital financial service providers, some
even say tendencies, of bias in decision-making, dis- of which are regulated as banks, and others of which
crimination and invasion of privacy. Analytics may are barely regulated at all. Even when they provide
19
be used to draw inferences (and in some cases make similar services, different restrictions may apply to
predictions) about a person’s race, gender, sexual ori- the data they may collect and use, and different rem-
entation, relationships, political views, health (includ- edies may be available for consumers.
ing specific disease), mental state, personal interests, The challenges arising for the treatment of big
creditworthiness and other attributes. Discrimination data and machine learning under legal and regulato-
may be embedded in the data processing, effec- ry frameworks for data protection and privacy sug-
tively leading to results that would be prohibited by gest that the development of robust self-regulatory
gender or race discrimination laws if decisions were and ethical regimes in the artificial intelligence and
carried out through human (as opposed to machine) financial services community may be particularly
processes. important.
These risks are particularly relevant to financial ser- This paper provides background for policy makers,
vices. Unlike many consumer products and services, regulators, digital financial service providers, inves-
offers and pricing of financial services depend on the tors and other organizations concerning the need
profile of the individual consumer. The decision to for solutions and standards on protecting consumer
offer a loan, and at what interest rate, the decision data privacy in the context of big data and machine
to issue a credit card, and with what credit limit, and learning. These issues are still emerging as the tech-
the decision to offer different types of insurance, all nologies, use cases and adoption rapidly increase. As
depend on assessing the risk the individual presents. a result, while the issues are increasingly understood,
Thus, like the decision to employ or not to employ there are few areas in which there is widespread con-
someone, many financial services have an important sensus on definitive best practices. Approaches will
personal dimension. depend on how policy makers, legislators, regulators
20
This can enable services to be better tailored to and market participants weigh up trade-offs and
the individual’s risk profile, and thus facilitates access synergies among policy objectives such as experi-
to financial services that might otherwise not have mentation and innovation, economic productivity,
been offered. However, at the same time, the indi- trust in services, and consumer protection.
vidual may be unaware of the data relied on to draw This paper explores various views, citing organi-
inferences or the reason for a decision not to extend zations’, academics’, and thinkers’ suggestions on
services to them, and may lack a way to dispute the commonly adopted approaches to protecting con-
data, inferences and decision. sumer data privacy and the associated laws and
Access to data about individuals enables such regulations. The purpose of this paper is to highlight
decisions to be based increasingly on individual these ideas and not to take a position. It seeks to
behaviour, but with potential invasion of privacy. In support those who must wrestle with these mat-
2008, the US Federal Trade Commission intervened ters at a policy, legislative and regulatory level in
to stop unfair practices by CompuCredit, which mar- the coming years. Rather than recommending best
keted credit cards to people with subprime credit. practices, this paper therefore focuses on identifying

Big data, machine learning, consumer protection and privacy 11