|
Work item:
|
Q.4081 (ex Q.MMAI)
|
|
Subject/title:
|
Methods and metrics for monitoring Machine Learning/Artificial Intelligence in future networks including IMT-2020
|
|
Status:
|
Consented on 2025-11-26 [Issued from previous study period]
|
|
Approval process:
|
AAP
|
|
Type of work item:
|
Recommendation
|
|
Version:
|
New
|
|
Equivalent number:
|
-
|
|
Timing:
|
Q4-2025 (Medium priority)
|
|
Liaison:
|
ITU-T SG13, ISO/IEC SC42, IEEE
|
|
Supporting members:
|
China Telecom, China Unicom, Zhejiang Lab, Xi'an Jiaotong University
|
|
Summary:
|
Monitoring ML/AI refers to the continuous, real-time tracking and observing an ML/AI system's in production environments. Monitoring ML/AI evaluates the performance of ML/AI model to determine whether it operates effectively. When the ML/AI model experiences some performance degradation, appropriate maintenance measures should be taken to restore performance.
ML/AI models are trained based on historical data and assumptions about the operational environment. However, the environment is dynamic. These dynamics can lead to model degradation — a decline in predictive accuracy or decision quality over time — caused by phenomena such as data drift, concept drift. Therefore, the environment and running state of ML/AI model should be monitored in order to determine whether the model should be updated or not.
To overcome various issues that resulted in performance degradation, a set of parameters and events should be defined and monitored. As a result, choosing appropriate monitoring strategies based on the specific use case, data characteristics, and business requirements is a critical step in ensuring the long-term reliability and effectiveness of ML/AI systems.
This Recommendation will give a guide and reference of monitoring ML/AI methods and metrics.
|
|
Comment:
|
-
|
|
Reference(s):
|
|
|
Historic references:
|
|
Contact(s):
|
|
| ITU-T A.5 justification(s): |
|
|
|
|
First registration in the WP:
2023-06-01 12:40:24
|
|
Last update:
2025-12-12 16:03:49
|
