Committed to connecting the world

PP-18 coneference

Executive Summary, SG12, May 2018

​​​​Executive summary: Meeting of ITU-T Study Group 12 (Performance, Quality of Service, Quality of Experience), Geneva, 1-10 May 2018

Note: This is not an official record and is subject to correction or modification

Overview

ITU-T Study Group 12 held its third meeting of the study period in Geneva, 1-10 May 2018. The meeting was attended by 128 participants from 43 countries. 13 delegates used the remote participation facilities.

The meeting received 74 Contributions​ and handled 183 TDs​. Delegates reviewed 43 incoming Liaison statements and adopted 18 outgoing Liaison statements.

Nine Recommendations were consented (4 new, 2 revised, 3 Amendments), and three non-normative texts were agreed (1 Amendment, 1 Implementers’ Guide, 1 Technical Report) by the meeting (see below for details).

A webinar (announcement | recording | slides) was held on 10 May 2018 following the closing plenary, summarizing key meeting results. Photos of the SG12 opening plenary can be found here​.

Major accomplishments

Completed work

ITU-T Rec. No. Question Reference New/Rev. Title AAP/Last Call
P.5705/12 TD472R2NewArtificial noise-fields in laboratory conditions AAP​-35
P.501 Amd.​16/12 TD532R1Rev.Test signals for use in telephonometry AAP-35
P.8087/12 TD529NewSubjective evaluation of speech quality with a crowdsourcing approach AAP-35
P.8097/12 TD512NewSubjective evaluation methods for gaming quality AAP-35
E.802 Amd.212/12 TD450R1Rev.Framework and methodologies for the determination and application of QoS parameters  AAP-35
E.84012/12 TD438R3NewStatistical framework for end to end network-performance benchmark scoring and ranking  AAP-35
G.107013/12 TD541R2Rev.Opinion model for video-telephony applications  AAP-35
Y.154317/12 TD543R1Rev.Measurements in IP networks for inter-domain performance assessment  AAP-35
Y.1546 Amd.117/12 TD542R1Rev.Hand-over performance among multiple access networks  AAP-35

The meeting consented new ITU-T Recommendations on
(1) ​ Artificial noise-fields in laboratory conditions (will be known as ITU-T P.570);
(2) ​ Subjective evaluation of speech quality with a crowdsourcing approach (ITU-T P.808);
(3) ​ Subjective evaluation methods for gaming quality (ITU-T P.809);
(4) ​ Statistical framework for end to end network-performance benchmark scoring and ranking (ITU-T E.840).

The meeting consented revisions and amendments to existing ITU-T Recommendations on
(5) ​ Test signals for use in telephonometry (ITU-T P.501), Amendment 1 on AM-FM test signal for super-wideband and fullband applications;
(6) ​ Framework and methodologies for the determination and application of QoS parameters (ITU-T E.802), Amendment 2 on updates and additional information on the degree of variability function in support of E.802;
(7) ​ Opinion model for video-telephony applications (ITU-T G.1070);
(8) ​ Measurements in IP networks for inter-domain performance assessment (ITU-T Y.1543); and
(9) ​ Hand-over performance among multiple access networks (ITU-T Y.1546), Amendment 1 on the IP-based service availability function.

 Doc# Question Reference Title
PSTR-CROW7/12 TD528Technical Report on subjective evaluation of quality of media with a crowdsourcing approach
P.863 Imp9/12 TD507R1 Implementers’ Guide for Recommendation ITU-T P.863 (03/18) on Perceptual objective listening quality assessment
​J.343 Amd.1​19/12TD571​​Hybrid perceptual bitstream models for objective video quality measurements

SG12 agreed on three non-normative texts:​
(1) ​ Hybrid perceptual bitstream models for objective video quality measurements (ITU-T J.343), Amendment 1, new Appendix containing information about test vectors for the J.343 family of Recommendations;
(2) ​ Implementers’ Guide for Recommendation ITU-T P.863 (03/18) on Perceptual objective listening quality assessment;
(3) ​ Technical Report on subjective evaluation of quality of media with a crowdsourcing approach.

The meeting also endorsed the way forward developed by the ad-hoc group on E.CEMI (Customer experience management index for popular services in operators’ network to score service quality that customers experience in terms of key network performance parameters), a text consented at the previous meeting. SG12 agreed to convert existing E.CEMI into a new work item, E.CEMIF (Framework), and to consequently not to approve the existing text of E.CEMI as an ITU-T Recommendation.

New work items

SG12 established new work items on
(1) ​ Application guide for the concept of QoE (G.CQoE);
(2) ​ Super-wideband E-model (G.107.2);
(3) ​ Computational model used as a QoE/QoS monitor to assess one-to-one videotelephony services (G.CMVTQS);
(4) ​ Crowdsourcing embedded solutions for measuring end-to-end QoS in fixed broadband networks (E.CrowdESFB);
(5) ​ QoE factors for new services in 5G network (G.QoE-5G);
(6) ​ QoE factors of augmented reality (AR) (G.QoE-AR);
and initiated work on two new technical reports on (1) the considerations on the use of algorithms based on machine learning and artificial intelligence for QoS and QoE purposes, and (2) on the concept of broadband access.

WTSA matters

The update of SG12’s operational plan for implementation of WTSA Resolution 95 (ITU-T initiatives to raise awareness on best practices and policies related to service quality) was approved. The meeting reviewed the preliminary results of the questionnaire on service quality regulatory frameworks circulated after the last meeting via TSB Circular 62 and agreed on follow-up activities.

A comprehensive overview of the results is scheduled for presentation at an ITU Workshop on Telecommunications Service Quality Regulatory Frameworks and Experience Driven Networking, to take place in Geneva, 26 November 2018, a day before opening the next SG12 meeting.

In addition to advancing its work related to WTSA Resolution 95, SG12 liaised with other ITU-T Study Groups on topics including alternative calling procedures (WTSA Resolution 29), SG12 work related to software-defined networking (WTSA Resolution 77), and the impacts of counterfeit and non-compliant devices on the QoS/QoE of mobile networks (WTSA Resolution 96).

Other activities

Work is advancing on the ITU-T Software Tool Library (Recommendation ITU-T G.191) on GitHub​, with inputs from 3GPP and the wider community. A revision of ITU-T G.191 is likely to be considered at the next SG12 meeting.

A lab was set up in the basement of the ITU Montbrillant building to advance the practical work on P.NATS phase 2: Parametric bitstream-based and pixel-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport for HEVC and UHD.

Technical tutorials were given by on the topics of (1) extended KPI for train-related end-to-end quality metrics for train-related use cases, and (2) assessing the impact of environmental factors on perceived speech quality using smartphone sensors. ITU/TSB introduced delegates to the new Rapporteur Group Meeting (RGM) Document Management System.

Approximately 20 people from 13 countries attended the SG12 newcomers’ session.

SG12 in the news

Results by group directly reporting to Plenary

Q1/12

Q2/12

SG12 Regional Group on QoS for the Africa Region (SG12RG-AFR)

Quality of Service Development Group (QSDG)

Results by working party

Working Party 1/12

Working Party 2/12

Working Party 3/12

Future meetings and workshops

Meetings of SG12

Meetings of Rapporteur groups

(Please see​here for updated information)​

Meeting of Quality of Service Development Group (QSDG)

SG12 Workshops