|
Work item:
|
F.748.74 (ex F.RFE-MM)
|
|
Subject/title:
|
Requirements and framework for evaluation of multimodal foundation models
|
|
Status:
|
Approved on 2026-06-13
|
|
Approval process:
|
AAP
|
|
Type of work item:
|
Recommendation
|
|
Version:
|
New
|
|
Equivalent number:
|
-
|
|
Timing:
|
-
|
|
Liaison:
|
IEEE 3303
|
|
Supporting members:
|
Shenzhen Tencent Computer System Co., Ltd., Beijing Baidu Netcom Science and Technology Co., Ltd., Alibaba (China) Network Technology Co., Ltd, China Mobile Communications Group Co.,Ltd, Huawei Technologies Co., Ltd
|
|
Summary:
|
Multimodal foundation models have achieved significant progress in integrating and understanding diverse modalities such as text, images, audio, and video. The need for comprehensive and reliable evaluation becomes increasingly critical. Effective assessment supports model trustworthiness, facilitates root-cause analysis of failures, and ensures consistent performance across diverse application scenarios. This Recommendation establishes a comprehensive framework for evaluating multimodal foundation models, covering test scenarios, datasets, tools, workflows and model capability requirements, which is applicable to providing references for institutions engaged in the research, development and application of multimodal foundation models.
|
|
Comment:
|
-
|
|
Reference(s):
|
|
|
Historic references:
|
|
Contact(s):
|
|
| ITU-T A.5 justification(s): |
|
|
|
|
First registration in the WP:
2025-02-18 17:31:32
|
|
Last update:
2026-05-05 15:28:58
|
|
