Background information and scope of the WG-CE
The aim of the WHO/ITU Focus Group on AI for Health is to develop a framework for standardized benchmarking of AI-systems in healthcare by creating (a) guidline documentation on how to evaluate AI for health from various perspectives and (b) an online platform and associated tools for the benchmarking of AI for health. The existing evidence-based frameworks to evaluate medical innovations and prove that they are safe, effective, and comparable to current standards of care in health have their limitations when applied to AI-based systems. The scope of the working group on clinical evaluation (WG-CE) is to build a community of collaboration with other groups and stakeholders to consider how AI models for health should be evaluated. The WG-CE will produce guidance documentation that outlines current best practices for the evaluation of AI for health and that can be used by researchers, clinicians, patients, developers, and policymakers.
Purpose of the workshop
The purpose of this first WG-CE workshop is to bring interested participants and experts from related fields together to facilitate (a) an exchange about the different perspectives that have to be considered with evaluation of AI for health and (b) to define concrete issues that have to be addressed. Furthermore, the workshop offers the opportunity to learn more about FG-AI4H and the people that are involved in this initiative.
Audience
This workshop is open to all and addresses everyone interested in clinical evaluation of AI for health, including members of the FG-AI4H and interested participants that work or are specialized in the interface of AI and health from research institutes, clinical practice, government agencies, developers, academia, funders, and other interested stakeholders/parties.
Activities
The original plan was to organize a physical workshop on clinical evaluation of AI for health, but given the SARS-CoV2 pandemic, the workshop will now be held virtually on 14 October 2020, 2-6 pm CEST. Most of the activities and work of FG-AI4H is already accomplished by virtual meetings and online collaboration because FG-AI4H members are located all around the globe.
The following topics and activities will be addressed during the CE-workshop:
- Presentations of experts stemming from different fields to share their expertise and thoughts on evaluation of AI for health in the context of their specialty followed by a panel discussion
-
Define main topics of clinical evaluation of AI for health that are relevant for future guideline documentation, based on the main output document of the WG-CE deliverable No. 7.4 'Clinical Evaluation of AI for health' as a starting point
-
Assignments (voluntarily) to contribute to one of the main topics and organization in subgroups
Organizational details/Participation
1. The workshop is open to all and will be held online via an ITU-zoom account
(installation of 'Zoom' on your computer is required)
2. The Zoom registration is available here. Once you have registered you will receive an email with your link to access the Workshop.
Structure of the Workshop
The agenda can be found in the programme that will be available soon.
Background on ITU/WHO FG-AI4H
Health technologies require careful evaluation under consideration of both technical and health-related aspects, prior to wider usage. Several factors complicate this evaluation and, thus, the deployment of artificial intelligence (AI) solutions in the health context. Therefore, the International Telecommunication Union (ITU) and the World Health Organization (WHO) have initiated an international standard-seeking effort to address these challenges by creating a joint Focus Group on Artificial Intelligence for Health (FG-AI4H). The ITU and the WHO are two specialized agencies of the United Nations authorized for creating global standards in the fields of information technology and health, respectively. The mandate of the ITU/WHO focus group is to undertake crucial, exploratory steps towards evaluation standards that are applicable on a global scale. FG-AI4H has begun working towards establishing a rigorous evaluation process for AI4H solutions, under the supervision of ITU and WHO, with a global community of experts from health, machine learning, AI, both from academia and industry, and regulation.
Detailed information on the work FG-AI4H is doing can be found on the website, a commentary in The Lancet and the following white paper. Furthermore, all documentation can be accessed via the online collaboration system (a free ITU account is needed; see instructions for help).
