Question 7/12 - Methods, tools and test plans for the subjective assessment of speech and audio quality
(Continuation of Question 7/12)
Motivation
The work of this Question will be primarily concerned with improved methods of assessing the subjective impact of time-varying impairments and with the design of laboratory testing of speech/audio/music processing enhanced technologies. These methods and tools apply for both conventional (narrowband), wideband and audio telephony. A Handbook of subjective testing practical procedures was started to be developed during the previous Study Period, limited to speech and noisy-speech testing, and will be complemented with the procedures for testing of audio/music signals.
Considering the need for standard subjective testing methodologies will continue to exist for the effective assessment of the transmission performance of new digital systems, like speech/music coders (telephony-band and higher quality), or other devices and equipment designed for carrying voice and audiovisual signals, the Question should be mandated to provide the necessary support to produce test/processing plans to execute appropriate subjective tests.
Input could also be provided due to the relevant work in other standards organisations, like ISO/MPEG or fora/consortia like 3GPP and 3GPP2.
The following major Recommendations/Supplements/Handbooks, in force at the time of approval of this Question, fall under its responsibility:
P.85, P.800, P.810, P.830, P.835, P.840, P.851, P.880.
Question
Study items to be considered include, but are not limited to:
- What new Recommendations need to be developed to evaluate new speech/music quality requirements?
- What enhancements to existing Recommendations need to be defined to improve the evaluation of degradations due to codec interoperability aspects, e.g. narrowband-to-wideband
codecs?
- What enhancements to existing Recommendations need to be defined to improve the evaluation of encoded speech + environmental noise, or both kinds of signals (separately)?
- What modifications to existing or new Recommendations need to be developed to assess new speech/music digital coding systems, e.g. narrowband/wideband/audio speech and/or music codecs operated over fixed and/or mobile networks (including Internet Multimedia Services)?
- What methods are required for evaluating the subjective performance of active signal processing devices (especially noise cancellation, voice activity detectors, comfort noise generation algorithms/devices)?
- What new or revised subjective assessment methods are required for evaluating the effects of time-varying impairments such as packet loss, and what guidance can be provided for the appropriate provision of sample material for varying degrees of
nonstationarity?
- What new test plans are needed to evaluate (subjectively) the above mentioned new requirements for speech/noise/music quality improvement in end-to-end communications over fixed and/or mobile networks?
- What guidance can be provided for collection and post-screening of subjective test results?
- Which Questions within SG 12, and other standardization activities within ITU, require support for subjective testing?
Tasks
Tasks include, but are not limited to:
- Maintenance and enhancement of Recommendations in the P-series with regards to subjective testing methods and of the Handbook on Subjective Testing Procedures.
Draft new and revised Recommendations P.840, P.835, P.851, P.CONV, P.ASPD, P.VOAD, P.MUS and a Handbook on Subjective Testing Procedures can be produced in the 2005–2008 Study Period:
Relationships
Recommendations: P-series, G.700 Series
Questions: 6/12, 8/12, 9/12, 12/12
Study Groups: ITU-T SG 9, SG 15, SG 16 and ITU-R SG 8
Standardisation bodies: ISO-MPEG, 3GPP, 3GPP2, IETF, ETSI
» List of Questions «
|
