AAP Recommendation

F.748.74: Requirements and framework for evaluation of multimodal foundation models

Study Group
21

Study Period
2025-2028

Consent Date
2026-05-01

Approval Date
2026-06-13

Provisional Name
F.RFE-MM

Input used for Consent
SG21-TD199R1/PLEN (2025-10)

Status
A

IPR
Site

Multimodal foundation models have achieved significant progress in integrating and understanding diverse modalities such as text, images, audio, and video. The need for comprehensive and reliable evaluation becomes increasingly critical. Effective assessment supports model trustworthiness, facilitates root-cause analysis of failures, and ensures consistent performance across diverse application scenarios. This Recommendation establishes a comprehensive framework for evaluating multimodal foundation models, covering test scenarios, datasets, tools, workflows and model capability requirements, which is applicable to providing references for institutions engaged in the research, development and application of multimodal foundation models.

AAP Current Status
Step # Action
Start / End
Status Announcement Related documents Comments / Resolution logs