Executive summary: Meeting of ITU-T Study Group 12 (Performance, Quality of Service, Quality of Experience), Geneva, 19-28 September 2017
Note: This is not an official record and is subject to correction or modification
Overview
ITU-T Study Group 12 held its second meeting of the study period in Geneva, 19-28 September 2017. The meeting was attended by 119 participants from 43 countries (see
list of participants). 14 delegates used the remote participation facilities.
The meeting received 81
Contributions and handled 217
TDs. Delegates reviewed 60
incoming Liaison statements and adopted 23
outgoing Liaison statements.
Ten Recommendations were consented (3 new, 6 revised, 1 Corrigendum), and two Supplements were agreed (1 new, 1 revised) (see below for details).
A webinar (announcement |
recording |
slides) was held on 28 September 2017 following the closing plenary, summarizing key meeting results. Photos of the SG12 opening plenary can be found here.
Major accomplishments
Completed work
ITU-T Rec. No. |
Question |
Reference |
New/Rev. |
Title |
AAP/Last Call |
E.831 | 12/12 |
TD254R3 | New | Customer experience management index for popular services in operators' network to score service quality that customer experience in terms of key network performance parameters |
AAP-21 |
G.1032 | 13/12 |
TD327R1 | New | Influence Factors on Gaming Quality of Experience |
AAP-21 |
P.804 | 7/12 |
TD308R1 | New | Subjective Diagnostic Test Method for Conversational Speech Quality Analysis |
AAP-21 |
P.862.2 Cor.1 | 9/12 |
TD350R2 | New | Wideband extension to Recommendation P.862 for the assessment of wideband telephone networks and speech codecs - Corrigendum 1 |
AAP-21 |
P.1203 | 14/12 |
TD355 | Rev. | Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport |
AAP-21 |
P.1203.1 | 14/12 |
TD352 | Rev. | Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport - video quality estimation module |
AAP-21 |
P.1203.2 | 14/12 |
TD353 | Rev. | Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport - audio quality estimation module |
AAP-21 |
P.1203.3 | 14/12 |
TD354 | Rev. | Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport - quality integration module |
AAP-21 |
P.1301 | 10/12 |
TD347 | Rev. | Subjective quality evaluation of audio and audiovisual multiparty telemeetings |
AAP-21 |
P.10/G.100 |
2/12 |
TD202R3 | Rev. | Vocabulary for performance, quality of service and quality of experience | AAP-22 |
The meeting consented
new ITU-T Recommendations on
(1) Subjective diagnostic test method for conversational speech quality analysis (will be known as ITU-T P.804);
(2) Customer experience management index for popular services in operators' network to score service quality that customers experience in terms of key network performance parameters (ITU-T E.831);
(3) Influence factors on gaming quality of experience (ITU-T G.1032).
E.831 was initially introduced to SG12's
Quality of Service Development Group (QSDG) before being elevated to Study Group level and includes an example of the use of the index in an operator's network.
The meeting consented
revisions of existing ITU-T Recommendations on
(4) Vocabulary for performance and quality of service (ITU-T P.10 / G.100);
(5) Subjective quality evaluation of audio and audiovisual multiparty telemeetings (ITU-T P.1301);
(6) Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport (ITU-T P.1203);
(7) Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport - Video quality estimation module (ITU-T P.1203.1);
(8) Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport - Audio quality estimation module (ITU-T P.1203.2);
(9) Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport - Quality integration module (ITU-T P.1203.3).
The revisions to the Recommendations in the P.1203 family fix ambiguities in descriptions that prevent operators from correctly implementing the respective Recommendations, and increase readability.
SG12 also consented a
Corrigendum to an existing ITU-T Recommendation on
(10) Wideband extension to Recommendation P.862 for the assessment of wideband telephone networks and speech codecs (ITU-T P.862.2).
The Corrigendum addresses a systematic under-prediction of P.862.2 caused by the incorrect transformation of the sound pressure level.
Doc# |
Question |
Reference |
Title |
G.1020 Suppl.61 | 11/12 |
TD239R2 | Supplement on IP aware QoS management |
P.Suppl. 26 | 10/12 |
TD318 | Scenarios for the subjective evaluation of audio and audiovisual multiparty telemeetings quality |
SG12 agreed on two
non-normative texts:
(1) Revised Supplement 26 to ITU-T P series of Recommendations: Scenarios for the subjective quality evaluation of audio and audiovisual multiparty telemeetings;
(2) New Supplement to ITU-T G series of Recommendations: IP aware QoS management.
New work items
SG12 established new work items on
- Voice service quality monitoring and troubleshooting framework for intrusive parametric voice QoE prediction (P.VSQMTF);
- Considerations for realizing virtual measurement systems (Y.cvms);
- The effect of SIM-boxing on QoS and QoE (E.QSIMbox);
- Amendment 2 to Recommendation ITU-T E.802 (E.802Amd2);
- QoS management model for bridging the gaps between QoS and QoE (E.QoSMgtMod);
- Amendment 1 to Recommendation ITU-T Y.1545.1 (Y.1545.1Amd1);
- Assessment of the LTE circuit switched fall back - impact on QoE and QoS (G.CSFB);
- Measurement scenarios, advanced measurement systems and sampling methodologies to monitor the QoS in mobile networks (E.MTSM);
- QoE-based network capacity planning (G.NCP);
- Objective model for assessing impact of initial loading delay on user experience (P.QUIT);
- Subjective test methodology for assessing impact of initial loading delay on user experience (P.QUITS); and
- QoS and QoE aspects of digital financial services (G.DFS).
Work items E.802Amd2, E.QoSMgtMod, G.CSFB, E.MTSM and Y.1545.1 are related to the mandate given to SG12 by WTSA Resolution 95. E.QSIMbox is related to WTSA Resolution 29, and G.DFS related to WTSA Resolution 89.
WTSA matters
SG12's operational plan for implementation of
WTSA Resolution 95 (ITU-T initiatives to raise awareness on best practices and policies related to service quality) was approved, as updated in
TD380. The meeting requested TSB to disseminate a questionnaire on service quality regulatory frameworks contained in
TD347R1 via TSB Circular. SG12 approved a liaison to ITU-T Study Groups 2 and 3 on matters related to alternative calling procedures (WTSA Resolution 29) and SIM-boxing.
Other activities
SG12 released the
ITU-T Software Tool Library (Recommendation ITU-T G.191) on GitHub, enabling the wider community to use, improve and contribute to the code.
A computer lab was set up in the basement of the ITU Montbrillant building to advance the practical work on P.NATS phase 2: Parametric bitstream-based and pixel-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport for HEVC and UHD.
SG12 approved working methods enabling it to collaborate with ETSI TC STQ WG MOBILE in selected areas of interest, e.g., update and maintenance of the ETSI TS 102 250 series and Recommendation ITU-T E.804; QoS and QoE aspects of digital financial services.
ITU/BDT colleagues joined SG12 on two occasions to introduce the QoS Regulation Manual and the QoS Training Programme, both developed with the support of SG12 delegates.
Nineteen delegates from developing countries participated in a bridging the standardization gap hands-on training session.
SG12 in the news
Results by group directly reporting to Plenary
Q1/12
- Coordination on WTSA Resolution 95 (Hammamet, 2016)
- Approved working methods to collaborate with ETSI TC STQ WG Mobile
- Approved eight outgoing liaison statements
Q2/12
- Launch of STL on GitHub
- Produced the following Recommendations for consent in plenary:
- Revised Recommendation ITU-T P.10 / G.100: Vocabulary for performance and quality of service
- Adopted one outgoing liaison statement
SG12 Regional Group on QoS for the Africa Region (SG12RG-AFR)
- Reviewed its action plan
- Coordinated African contributions to SG12
Quality of Service Development Group (QSDG)
- Reviewed and updated its work programme for 2017-18
Results by working party
Working Party 1/12
- Produced the following Recommendations for consent in plenary:
- New Recommendation ITU-T P.804: Subjective diagnostic test method for conversational speech quality analysis;
- Revised Recommendation ITU-T P.1301: Subjective quality evaluation of audio and audiovisual multiparty telemeetings.
- Produced the following non-normative texts for agreement in plenary:
- Revised Supplement 26 to ITU-T P series of Recommendations: Scenarios for the subjective quality evaluation of audio and audiovisual multiparty telemeetings
- Adopted one outgoing liaison statement
Working Party 2/12
- Produced the following Recommendations for consent in plenary:
- Corrigendum 1 to Recommendation ITU-T P.862.2: Wideband extension to Recommendation P.862 for the assessment of wideband telephone networks and speech codecs;
- Revised Recommendation ITU-T P.1203: Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport;
- Revised Recommendation ITU-T P.1203.1: Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport - Video quality estimation module;
- Revised Recommendation ITU-T P.1203.2: Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport - Audio quality estimation module;
- Revised Recommendation ITU-T P.1203.3: Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport - Quality integration module.
- Established new work item P.VSQMTF
- Adopted five outgoing liaison statements
Working Party 3/12
- Produced the following Recommendations for consent in plenary:
- New Recommendation ITU-T E.831: Customer experience management index for popular services in operators' network to score service quality that customers experience in terms of key network performance parameters;
- New Recommendation ITU-T G.1032: Influence factors on gaming quality of experience.
- Produced the following non-normative texts for agreement in plenary:
- New Supplement to ITU-T G series of Recommendations: IP aware QoS management
- Established new work items Y.cvms, E.QSIMBox, E.802Amd2, E.QoSMgtMod, Y.1545.1Amd1, G.CSFB, E.MTSM, G.NCP, P.QUIT, P.QUITS, G.DFS
- Adopted eight outgoing liaison statements
- Adopted a Questionnaire on service quality regulatory frameworks
Future meetings and workshops
Meetings of SG12
- 1-10 May 2018, Geneva, Switzerland
- 27 November - 6 December 2018, Geneva, Switzerland
Meetings of Working parties under SG12
- WP2/12: 15 February 2018, Geneva, Switzerland (ITU)
Meetings of Rapporteur groups
(Please see
here for updated information)
- Q4/12: 23-24 January 2018, Novi, Michigan, United States (BlackBerry)
- Q9/12: 14-15 February 2018, Geneva, Switzerland (ITU)
- Q12/12: TBC 12-16 March 2018, Dakar, Senegal (jointly with SG12RG-AFR)
- Qs 13 and 14/12: 27-29 November 2017, Krakow, Poland (AGH University) (jointly with VQEG)
- Qs 13 and 14/12: 26-28 February 2018, TBC: Stockholm/Berlin (Ericsson/T-Labs)
- Q17/12: TBC 27-28 February 2018, TBC: Stockholm/Berlin (Ericsson/T-Labs) (with Qs 13 and 14/12)
Meeting of SG12 Regional Group on QoS for the Africa Region (SG12RG-AFR)
- TBC: 12-16 March 2018, Dakar, Senegal, with workshop and Q12/12 rapporteur group meeting
Meeting of Quality of Service Development Group (QSDG)
- TBC: September 2018, hosted by Türk Telekom
SG12 Workshops