Committed to connecting the world

gsr-19

Question 14/12

Question 14/12 – Development of parametric models and tools for multimedia quality assessment
(Continuation of Question 14/12)

Motivation

A major challenge for emerging IP-based networks is to provide adequate Quality of Experience (QoE) and Quality of Service (QoS) for new multimedia services and applications such as IPTV, internet media including over-the-top (OTT) video, and 3D video. In the previous study period, the new recommendations P.NAMS and P.NBAMS have successfully been developed to meet the challenge of assessing the quality for video transmitted over UDP in a scalable manner, proposing different no-reference, trace analysis based audio, video, and audiovisual quality models.

In this Question, the application of P.NAMS and P.NBAMS for different in-service deployment scenarios needs to be studied further. The synchronization between audio and video is considered as an addition to the P.NAMS and P.NBAMS recommendations. TCP based and OTT video services are very popular, and methods like P.NAMS and P.NBAMS will be extended to include TCP and OTT video, considering both adaptive and non-adaptive streaming. Based on the attention for 3D video service offerings, extensions of the P.NAMS and P.NBAMS work towards 3D video and audiovisual quality monitoring are a natural evolution of the work in this Question.

 P.NAMS and P.NBAMS are applicable to short input sequences up to 24 seconds. The extension to longer sequences that better reflect ecologically valid media sessions of up to minutes or hours durations, are further work items addressed by this Question. Multimedia service quality is a combination of a number of aspects; for example, the quality of experience depends on the content selection procedure, the start-up time of a video and the video quality itself. A new topic to study is how to combine these quality-related aspects into an integral quality index for general service monitoring.

The following major Recommendations, in force at the time of approval of this Question, fall under its responsibility:

 P.1200-series

Question

Study items to be considered include, but are not limited to:

  • What further aspects of a continued characterization of P.NAMS and P.NBAMS models should be considered?
  • How do P.NAMS and P.NBAMS need to be maintained, and what further application guidance towards, for example, network-centric monitoring solutions needs to be provided?
  • What are relevant subjective test methodologies, and which respective new standards need to be developed (possibly in cooperation with other standardization bodies)?
  • How is consistency with development of hybrid models by VQEG ensured?
    What relationship exists between the subjective responses of users at the terminals and the objective measurements made from the point at which the non-intrusive assessment system is connected?
  • How can different modes of operation be handled for the P.NAMS and P.NBAMS models?
  • How can P.NBAMS be extended to audiovisual quality predictions?
  • How can audiovisual synchronization be reflected in parametric models such as P.NAMS or P.NBAMS?
  • How should P.NAMS and P.NBAMS be extended for TCP-based over-the-top services?
  • How should TCP- and HTTP-based video related quality monitoring models be designed?
  • How should short-term measurements be temporally pooled for longer-term, per session multimedia quality monitoring?
  • How can different quality-related aspects of a multimedia service be combined to an integral quality index for general service monitoring? An example is that of an IPTV session with content selection, viewing, zapping, EPG usage, etc.
  • How should existing models be modified or new parametric and bitstream models be implemented for 3D video quality monitoring applications?
  • Considerations on how to help measure and mitigate climate change

Tasks

Tasks include, but are not limited to:

  • Maintenance of Recommendation P.NAMS Parametric non-intrusive assessment of audiovisual media streaming quality
  • Maintenance of Recommendation P.NBAMS Parametric non-intrusive bitstream assessment of video streaming quality
  • New Recommendation(s) on guidance for the use of P.NAMS and P.NBAMS in operational contexts (collaboration with Question R/12).
  •  Extensions of P.NBAMS towards bitstream-based audiovisual quality evaluation.
  • New Recommendation on non-intrusive assessment of TCP-based multimedia streaming quality (P.NATS).
  • New Recommendation for including audiovisual synchronization in P.NAMS and/or P.NBAMS type models (P.AVSQ – audiovisual synchronization quality).
  • New Recommendation for temporal pooling of multimedia quality for longer sequences (P.MMSTP – multi-media session temporal pooling).
  • New Recommendation on non-intrusive evaluation model of 3D audiovisual quality based on IP protocol information (P.3DNAMS)
  • New Recommendation on non-intrusive evaluation model of 3D audiovisual quality based on IP protocol and bit-stream information (P.3DNBAMS).
  • New Recommendation on integral index of quality for general service monitoring (P.INQX)

An up-to-date status of work under this Question is contained in the SG 12 work programme http://www.itu.int/ITU-T/workprog/wp_search.aspx?q=14/12

Relationships

Recommendations

  • P.564, G.1000-series, J series recommendations on video quality
Questions
  • 13/12, 17/12
Study Groups
  • ITU-T SG 9, SG 13, SG 16
Standardization bodies
  • ETSI (STQ, TISPAN), IETF (IPPM, AVT), ATIS (IIF); VQEG, HGI, DSL Forum
​​​