• Documentation
  
• ​Mailing list: fgai4h@lists.itu.int
  
• TG-Symptom mailing list: fgai4htgsymptom@lists.itu.int​
  
• Join the mailing list (ITU account needed):
  Experts | Step-by​-step
  
• Collaboration site​ (​ITU account needed - instructions​)
  
• Templates: Documents​ | Liaison Statements​
  

AI-based symptom assessment is one of the most promising applications in the field of AI4H. The World Health Organization estimates the shortage of Global Health workers to increase from 7.2 million in 2013 to 12.9 million by 2035 . The 2017 Global Monitoring Report by the WHO and the World Bank reported that half of the world’s population lacks access to basic essential health services. The growing shortage of health workers is likely to further limit access to proper health care, reduce doctor time, and worsen patient journeys to a correct diagnosis and proper treatment.

At the same time, patient-centered apps for AI-based symptom assessment have become widely available. However, the lack of consistent standardization makes it difficult for organisations like the WHO, governments, and other key players to adopt symptom assessment systems as part of their solutions to address global health challenges.

The implementation of a standardised benchmarking for this class of applications as part of the WHO/ITU’s AI for Health Focus Group will therefore be an important step towards addressing this issue. The safe and transparent application of AI technology will help improve access to healthcare for many people. It will enable earlier diagnosis of conditions, more efficient care-navigation through the health system, and ultimately better health as it is currently pursued by WHO’s sustainable development goal number 3 (SDG 3).

To create a standardised benchmarking, the topic group will consider all relevant aspects. In addition to the aforementioned general objectives, this will include more topic specific questions like:

• Which output features should be benchmarked (triage, next-steps advice, differential diagnosis, explanations, question-flow, etc.)?
• Which scores and metrics should be used to assess the different dimensions of “AI-based symptom assessment” performance within the different fields of application?
• Which terminologies and ontologies (SNOMED, ICD, etc.) should be used for representing labeled test cases as well as for the communication with the AI-systems?
• How to set up benchmarking in a way that protects the intellectual property of the AI contributors, the rights of the data providers, and is still transparent?
• What should the first minimal viable benchmarking look like and how can this be refined over time to eventually cover all relevant aspects of symptom-based decision support?