Summary - F.748.74 (06/2026) - Requirements and framework for evaluation of multimodal foundation models

Multimodal foundation models have achieved significant progress in integrating and understanding diverse modalities such as text, images, audio and video. The need for comprehensive and reliable evaluation becomes increasingly critical. Effective assessment supports model trustworthiness, facilitates root-cause analysis of failures, and ensures consistent performance across diverse application scenarios.
Recommendation ITU-T F.748.74 establishes a comprehensive framework for evaluating multimodal foundation models, covering test scenarios, datasets, tools, workflows and model capability requirements, which is applicable to providing references for institutions engaged in the research, development and application of multimodal foundation models.