Page 87 - ITU KALEIDOSCOPE, ATLANTA 2019

TOWARDS INTERNATIONAL STANDARDS FOR THE EVALUATION OF
ARTIFICIAL INTELLIGENCE FOR HEALTH

1
Markus A. Wenzel , Thomas Wiegand 1,2
1 Fraunhofer Heinrich Hertz Institute, Berlin, Germany
2 Technische Universität Berlin, Berlin, Germany

ABSTRACT and journal branches to digital health with a focus on ML and
AI [cf. 5, 6, 7], and the English National Health Service has
Healthcare can benefit considerably from advanced created several guiding documents on AI in healthcare [8, 9,
information processing technologies, in particular from 10].
machine learning (ML) and artificial intelligence (AI).
However, the health domain only hesitantly adopts these The capacity of ML and AI for applications in healthcare is
powerful but complex innovations so far, because any founded upon the increasing availability of digital health
technical fault can affect people’s health, privacy, and data, which can be used to train and to apply advanced
consequently their entire lives. In this paper, we substantiate models, on increasingly powerful computing infrastructure,
that international standards are required for thoroughly potentially accessible from around the globe via the Internet.
validating AI solutions for health, by benchmarking their Available data include, e.g. radiology or microscope images,
performance. These standards might ultimately create well- free text, sensor time series, lab measurements, and other
founded trust in those AI solutions that have provided information stored in electronic health records. These data
conclusive evidence to be accurate, effective and reliable. enable ML and AI algorithms to learn to perform a wide
We give reasons that standardized benchmarking of AI range of recognition, early-detection, classification,
solutions for health is a necessary complement of established prediction, image-segmentation and image-reconstruction
assessment procedures. In particular, we demonstrate that it tasks that only humans had been able to perform previously
is beneficial to tackle this topic on a global scale and and that can now be automated very fast and at large scale
summarize the achievements of the first year of the with computers.
ITU/WHO focus group on “AI for Health” that has tasked
itself to work towards creating these evaluation standards. Medical image analysis is a large field of application for ML
methods [11, 12, 13]. For instance, ML models can recognize
Keywords – Artificial intelligence, benchmarking, lung nodules from radiology images [14], detect malaria
evidence, ITU-T, machine learning, medicine, from microscope images of blood samples [15], or classify
standards, validation, World Health Organization skin cancer based on dermoscopy images [16]. The models
can process electronic health records in order to categorize
1. HEALTH AI patients, make clinical decisions, predict patient trajectories
[17] or forecast the outcome [18], e.g. in the intensive care
Advanced information technologies from machine learning unit [19, 20]. From the electrocardiogram, algorithms can
(ML) and artificial intelligence (AI) have been attracting detect myocardial infarction and heart arrhythmia [21, 22].
much interest in the health domain lately. The recent global In addition, AI methods not rooted in ML can contribute to
strategy on digital health of the World Health Organization healthcare: Knowledge-based expert systems can ask
(WHO) mentions AI as technology that has “the potential to systematic questions about symptoms and advise on how to
enhance health significantly” [1]. Regulatory bodies, in proceed, e.g. recommend visiting a hospital immediately
particular the U. S. Food and Drug Administration, are [23]. Access via the Internet is a considerable driver for
beginning to permit AI-based medical devices [2, 3], and health AI technology because it makes novel types of health
start the discussion on the regulation of continuously delivery possible, as well as innovative business models, in
learning ML/AI-based medical software [4]. Renowned particular via mobile applications that can give expert advice
medical and scientific journals are dedicating special issues to virtually everyone, everywhere, e.g. based on text input,
camera or even cellphone-based, deep-learning-enhanced
microscopy [24].

978-92-61-28401-5/CFP1968P-ART @ ITU 2019 – 67 – Kaleidoscope