|
Work item:
|
F.748.91 (ex F.LLMO)
|
|
Subject/title:
|
Requirements and evaluation metrics for large language model operators
|
|
Status:
|
Approved on 2026-08-29
|
|
Approval process:
|
AAP
|
|
Type of work item:
|
Recommendation
|
|
Version:
|
New
|
|
Equivalent number:
|
-
|
|
Timing:
|
-
|
|
Liaison:
|
-
|
|
Supporting members:
|
Institute of Computing Technology Chinese Academy of Sciences (ICTCAS), China Telecommunications Corporation , State Grid Corporation of China
|
|
Summary:
|
The effective processing ability of large language models (LLMs) for text information is supported by operators, which are the fundamental building blocks to implement the model. This Recommendation provides an overview and framework of large language model operators, and specifies requirements and evaluation metrics for the operators commonly used in training and inference scenarios of LLM, to ensure that LLMs efficiently process multimedia information in different hardware environments. Specifically, requirements include Adaptation Requirements, Function Requirements, Development Requirements, Service Requirements, and API Requirements. Evaluation metrics include computation error, computation efficiency, resource efficiency, robustness, and portability.
|
|
Comment:
|
-
|
|
Reference(s):
|
|
|
Historic references:
|
|
Contact(s):
|
|
| ITU-T A.5 justification(s): |
|
|
|
|
First registration in the WP:
2025-02-24 15:29:30
|
|
Last update:
2026-07-22 16:56:21
|
