|
Work item:
|
H.FMLS-QMEM
|
|
Subject/title:
|
Quality Metrics and Evaluation Methods for Federated Machine Learning Services
|
|
Status:
|
[Carried to next study period]
|
|
Approval process:
|
AAP
|
|
Type of work item:
|
Recommendation
|
|
Version:
|
New
|
|
Equivalent number:
|
-
|
|
Timing:
|
-
|
|
Liaison:
|
SG12, SG13, SG17, IEEE
|
|
Supporting members:
|
China Telecom, MIIT (China)
|
|
Summary:
|
Federated Machine Learning (FML) is an emerging distributed framework that creates a calculating ecosystem with privacy-aware for multiple data providers to enable collaborative machine learning (ML) and model building across decentralized datasets.
Federated machine learning services refers to the federated machine learning computing process realized by multiple participants through information exchange in order to meet the goals of the initiator.
Nowadays, the development and application of federated machine learning and privacy computing are facing critical challenges, for example how to fairly and scientifically price the data of the participants in the federated machine learning system.
In order to motivate data owners with high-value data to join federated machine learning, it's necessary to guide a fair and scientific data contribution evaluation method by measuring the contribution of participants, data value, data quality indicators. Secondly, this standard is instructive and is an executable evaluation method covering FML-related services, models, algorithms and others. On the other hand, more and more organizations begin to use FML technologies in daily production, it is necessary to form assessment methods for FML services, evaluate the contribution of FML services, and therefore further promote the large-scale application of FML technologies.
This draft recommendation provides quality metrics and evaluation methods for federated machine learning services, along with a set of complete and operable quality evaluation standards and last give some specific quality evaluation methods for FML typical scenarios.
|
|
Comment:
|
-
|
|
Reference(s):
|
|
|
Historic references:
|
|
Contact(s):
|
|
| ITU-T A.5 justification(s): |
|
|
|
|
First registration in the WP:
2023-08-22 10:54:56
|
|
Last update:
2024-09-20 10:20:58
|
|
