|
Work item:
|
F.EML
|
|
Subject/title:
|
Evaluation Method for Adaptability of AI Systems to Foundation Models
|
|
Status:
|
Under study
|
|
Approval process:
|
AAP
|
|
Type of work item:
|
Recommendation
|
|
Version:
|
New
|
|
Equivalent number:
|
-
|
|
Timing:
|
2027-08 (Medium priority)
|
|
Liaison:
|
ITU-T SG13
|
|
Supporting members:
|
Institute of Computing Technology Chinese Academy of Sciences (ICTCAS),State Grid Corporation of China,China Telecommunications Corporation.
|
|
Summary:
|
AI systems built for foundation models serve multiple users across diverse applications. These systems face complex challenges: for instance, model accuracy may fall short of functional requirements, or users must balance trade-offs among latency, throughput, and request concurrency to meet performance demands. Traditional evaluation method often focuses narrowly on simple model validation or on isolated performance metrics, lacking a comprehensive and scenario-driven assessment approach.
This Recommendation specifies an evaluation method for assessing AI systems' adaptability to foundation models, covering both functional support and operational performance. It incorporates representative multimedia model workloads and scenario-driven performance configurations to reflect real-world usage. The method aims to provide objective assessments, thereby facilitating efficient deployment and operation of foundation models within multimedia applications.
This Recommendation supports system selection, capability enhancement, and the development of high-performance AI applications in the multimedia industry.
|
|
Comment:
|
-
|
|
Reference(s):
|
|
|
Historic references:
|
|
Contact(s):
|
|
| ITU-T A.5 justification(s): |
|
|
|
|
First registration in the WP:
2025-09-09 11:15:22
|
|
Last update:
2026-09-04 13:53:10
|
