​Executive summary: Meeting of ITU-T Study Group 12 (Performance, Quality of Service, Quality of Experience), Geneva, 26 Nov – 5 Dec 2019

Note: This is not an official record and is subject to correction or modification

Overview

ITU-T Study Group 12 held its sixth meeting of the study period in Geneva, 26 November - 5 December 2019.

The meeting was attended by ​137 participants from 51 countries. 24 delegates exclusively used the remote participation facilities.

The meeting received 52 Contributions​ and handled 162 TDs. Delegates reviewed 42 incoming Liaison statements and adopted 12 outgoing Liaison statements.

Two Recommendations were approved (1 AAP, 1 TAP) with no opposition, and 14 Recommendations were consented (11 new, 1 revised, 1 Amendment, 1 Corrigendum) by the meeting (see below for details).

The meeting also agreed on one new Technical Report and one Implementers’ Guide.

A webinar (Announcement | Recording | Slides) was held on 5 December 2019​ following the closing plenary, summarizing key meeting results.​​

Major accomplishments

Recommendations approved during closing plenary (TAP, Resolution 1)

See TSB Circular 217

Recommendations approved during closing plenary (AAP, Recommendation ITU-T A.8)

See TSB Circular 218

Recommendations consented during closing plenary (AAP, Recommendation ITU-T A.8)

See last call in AAP-72

Texts agreed during closing plenary

New work items

SG12 establi​shed work items for new Recommendations on the following topics: 

1. Subjective evaluation of gaming qual​ity with a crowdsourcing approach (P.CROWDG);
2. Taxonomy of telemeeting systems from a QoE/QoS perspective (G.TeleMeTax);
3. Opinion Model for Mobile Online Gaming applications (G.OMMOG);
4. Video quality assessment of streaming services over reliable transport for resolutions up to 4K with access to transport information (P.1204.1); and
5. Video quality assessment of streaming services over reliable transport for resolutions up to 4K with access to video frame information (P.1204.2).

WTSA matters

Implementation of WTSA Resolution 95 (ITU-T initiatives to raise awareness on best practices and policies related to service quality) is progressing.

The web page containing information on a questionnaire released by SG12 on the status of national quality regulatory frameworks, as well as country case studies and SG12 activities to implement the Resolution was updated, see https://www.itu.int/en/ITU-T/studygroups/2017-2020/12/Pages/resolution95.aspx.

In response to WTSA-16 Resolution 95, SG12 approved new Recommendation ITU-T E.805 – Strategies to establish quality regulatory frameworks (ex E.RQUAL). Work on Res.95-related draft Recommendations progressed, in particular in Question 12/12.

Approved in October, ITU-T G.1033, a new ITU standard introduces QoS and QoE aspects of digital financial services. The document is accompanied by a new specification (consented in December 2019) describing how to test the QoE of digital financial services. Both standards are catered to meet the requirements established by WTSA-16 Resolution 89 on the use of ICTs to bridge the financial inclusion gap.

The meeting also saw the continuation of the preparations for WTSA-2020. The initial SG12 proposal for Res.2 and Questions texts was adopted for submission to TSAG.

Other activities

Approximately 15 delegates from 12 countries attended the SG12 newcomers’ session.

Two tutorials were held during the meeting, discussing

• An open-source framework for writing & publishing standardization documents; and
• Personalized network-based voice enhancement.

SG12 in the news

• ITU News: Webinar: ITU standards to assess the quality of video streaming, cloud gaming and fintech

Results by group directly reporting to Plenary

• Adopted initial proposals concerning SG12 for ITU World Telecommunication Standardization Assembly (WTSA-20, Hyderabad, November 2020)
• Discussed the use of machine learning techniques in the context of developing quality assessment models in ITU-T SG12
• Progressed work on an application guide on the concept of Quality of Experience

SG12 Regional Group on QoS for the Africa Region (SG12RG-AFR)

• Reviewed its action plan;
• Coordinated African contributions to SG12;
• Adopted liaison to TSAG on regional groups.

Quality of Service Development Group (QSDG)

• Provided briefings on recent workshop in Singapore;
• Reviewed and updated its work programme for 2020.

Results by working party

Working Party 1/12

• Produced the following texts for consent in plenary:
• New Recommendation ITU-T P.918: Dimension-based Subjective Quality Evaluation for Video Content;
• New Recommendation ITU-T P.1150: In-Car Communication Audio Specification.
• Established work items P.CROWDG and G.TeleMeTax.

Working Party 2/12

• Produced the following texts for consent in plenary:
• New Recommendation ITU-T E.475: Guidelines for Intelligent Network Analytics and Diagnostics;
• New Corrigendum 1 to Recommendation ITU-T G.107.1: Wideband E-model - Corrigendum 1;
• New Recommendation ITU-T P.565: Framework for creation and performance testing of machine learning based models for the assessment of transmission network impact on speech quality for mobile packet-switched voice services;
• New Recommendation ITU-T P.1204: Video quality assessment of streaming services over reliable transport for resolutions up to 4K;
• New Recommendation ITU-T P.1204.3: Video quality assessment of streaming services over reliable transport for resolutions up to 4K with access to full bitstream information;
• New Recommendation ITU-T P.1204.4: Video quality assessment of streaming services over reliable transport for resolutions up to 4K with access to full and reduced reference pixel information;
• New Recommendation ITU-T P.1204.5: Video quality assessment of streaming services over reliable transport for resolutions up to 4K with access to transport and received pixel information;
• Revised Recommendation ITU-T P.1401: Methods, metrics and procedures for statistical evaluation, qualification and comparison of objective quality prediction models.
• Produced the following texts for agreement in plenary:
• New Implementers’ Guide to Recommendation ITU-T P.863;
• New Technical Report: No-reference pixel-based video quality estimation algorithm
• Established work items P.1204.1 and P.1204.2.

Working Party 3/12

• Produced the following texts for approval in plenary:
• New Recommendation ITU-T E.805: Strategies to establish quality regulatory frameworks;
• Revised Recommendation ITU-T Y.1540: Internet protocol data communication service – IP packet transfer and availability performance parameters.
• Produced the following texts for consent in plenary:
• New Recommendation ITU-T G.1034: QoE metrics for mobile telephony communication during rail travel;
• New Recommendation ITU-T G.1072: Opinion Model Predicting Gaming QoE for Cloud Gaming Services;
• New Recommendation ITU-T P.1502: Methodology for QoE testing of digital financial services;
• New Amendment 1 to Recommendation ITU-T Y.1540: Internet protocol data communication service - IP packet transfer and availability performance parameters - Amendment 1
• Established work item G.OMMOG​.

Future meetings and workshops

Meetings of Study Group 12

• Geneva, 15-24 April 2020

Meetings of Rapporteur groups

Please see ​​here for​ updated information​. Additional conference calls of Questions will be announced on the respective mailing lists.​

• TBC: Qs 13, 14 and 17/12: Germany or Sweden, February 2020
• Q19/12: Seattle, Washington, USA, in conjunction with VQEG and IRG-AVQA, 9-13 March 2020

Meetings of SG12 Regional for QoS for the Africa Region (SG12RG-AFR)

• ​N’Djamena, Chad, 2-5 March 2020 (jointly with workshop)

Meetings of Quality of Service Development Group (QSDG)

• Details TBC: Amman, Jordan, 3Q 2020 (jointly with workshop)

SG12 Workshops​

• N’Djamena, Chad, 2-5 March 2020 (jointly with SG12RG-AFR)
• Details TBC: Amman, Jordan, 3Q 2020 (jointly with QSDG)

Meetings of Intersector Rapporteur Group Audiovisual Quality Assessment​ (IRG AVQA)

• Seattle, Washington, USA, in conjunction with VQEG and Q19/12, 9-13 March 2020