| (Continuation of Question 7/12) Motivation
The work of this Question will be primarily concerned with improved methods of assessing the subjective impact of time-varying impairments and with the design of laboratory testing of speech/noisy-speech/music/mixed content, considering the evolution of enhanced telecommunication technologies. These methods and tools will apply to narrow-band, wideband, superwideband and fullband audio telephony.
A Handbook on "Subjective testing practical procedures" was developed during the previous study period for speech and noisy-speech, music and mixed content signals, and will be maintained, adding new procedures whenever they become available and widely used in listening testing laboratories during standard exercises (e.g. for the development of a new coding algorithm within an SDO regular course of action).
As done so far, considering the need for standard subjective testing methodologies will continue to exist for the effective assessment of the transmission performance of new digital systems, like speech/music coders (telephony-band and higher quality bandwidths), or other devices and equipment designed for carrying voice and audiovisual signals, the Question should be mandated to continue to provide the necessary support to produce test/processing plans to execute appropriate subjective tests.
Input could also be provided due to the relevant work in other standards organizations, like ISO/MPEG or fora/consortia like 3GPP and 3GPP2.
The following major Recommendations/Supplements/Handbooks, in force at the time of approval of this Question, fall under its responsibility:
P.85, P.800, P.805, P.810, P.830, P.835, P.840, P.851, P.880, P Suppl. 24, P Suppl. 25, Handbook STP.
Question
Study items to be considered include, but are not limited to:
What new Recommendations need to be developed to evaluate news speech/noisy-speech/music and mixed content quality requirements? What enhancements to existing Recommendations need to be defined to improve the evaluation of degradations due to codec interoperability aspects, e.g. narrowband-to-wideband-to-superwideband-to-fullband codecs? What methods are required for evaluating the subjective performance of active signal processing devices (particularly noise cancellation, generic sound activity detectors / voice activity detectors, comfort noise generation algorithms/devices)? What new or revised subjective assessment methods are required for evaluating the effects of time-varying impairments (such as packet loss), and what guidance can be provided for the appropriate provision of sample/noise material for varying degrees of nonstationarity? What enhancements to existing Recommendations need to be defined to improve the subjective evaluation of speech-based or multimodal interactive services? What new or revised subjective test methodologies are required for evaluating web browsing and browser based applications? What new or revised subjective test methods are required for evaluating synthesized speech services and/or services based on spoken dialogue systems? What modifications to existing or new Recommendations need to be developed to assess new speech/music/mixed content digital coding systems, e.g. narrowband/wideband/audio speech and/or music and/or mixed content codecs operated over fixed and/or mobile networks (including Internet Multimedia Services)? What new test plans are needed to evaluate (subjectively) the above-mentioned new requirements for speech/noise/music quality improvement in end-to-end communications over fixed and/or mobile networks? What guidance can be provided for collection and post-screening of subjective test results, and global analysis of results from internationally coordinated exercises in general? What guidance can be provided for collection and evaluation of cultural/language/ nationality dependence of subjective quality? Which Questions within SG 12, and other standardization activities within ITU, require support for subjective testing?
Tasks Tasks include, but are not limited to:
Maintenance and enhancement of Recommendations in the P-series with regards to subjective testing methods and of the Handbook on Subjective Testing Procedures. Draft new and revised Recommendations P.ASPD, P.GSAD, P.MULTI, P.MUS, P.SUSE, P.CLN, P.STMWeb and maintenance /enhancements to the Handbook on Subjective Testing Procedures can be produced in the 2013–2016 Study Period.
An up-to-date status of work under this Question is contained in the SG 12 work programme http://www.itu.int/ITU-T/workprog/wp_search.aspx?q=7/12
Relationships
Recommendations
Questions
Study Groups
Standardization bodies
ISO-MPEG, 3GPP, 3GPP2, IETF, ETSI
|
