1 Scope
2 References
3 Definitions
3.1 Terms defined elsewhere
3.2 Terms defined in this Recommendation
4 Abbreviations and acronyms
5 Conventions
6 General recommendations concerning the subjective quality evaluation of
multiparty telemeetings
7 Multiparty specific aspects in subjective quality evaluation
8 Guidance to suitable test methods
8.1 Test method decision criteria
8.2 Flow charts to be used when selecting test methods
Annex A – Set-up of a multiparty telemeeting assessment test
A.1 Assessment of conversational quality – Conversation tests
A.2 Assessment of non-interactive quality – Non-interactive tests
Annex B – Assessment of telemeetings with text-based communication and
graphical information means (e.g., web conferencing)
Annex C – Assessment of video-only telemeetings
Annex D – Effect of transmission delays on telemeeting quality
D.1 Background
D.2 Existing test task recommendations
D.3 Recommended test tasks
D.4 Set-up of a delay test
D.5 Test subjects
D.6 Training session
D.7 Instructions
D.8 Test questions
D.9 Objective measurements
D.10 Effects of delay
Annex E – Assessment of spatial audio and 3D video reproduction of multiparty
telemeetings
E.1 General remarks
E.2 Recommendations concerning spatial audio
E.3 Remarks concerning 3D video
Annex F – Assessment of asymmetric multiparty telemeetings
F.1 Problem statement
F.2 Translation of technical degradations into possible perceptual
impairments
F.3 Influencing factors
F.4 Experiment design
F.5 Rating scales
F.6 Test stimuli
F.7 Data analysis
Annex G – Assessment of multiparty telemeetings with non-stationary quality
Annex H – Assessment of multiparty telemeetings using multi-dimensional scaling
methods
Appendix I – Influential factors
Appendix II – Overview of multiparty non-interactive test stimuli and
conversation test tasks
II.1 Non-interactive audio-only stimuli:
II.2 Non-interactive video-only stimuli
II.3 Non-interactive audiovisual stimuli
II.4 Audio-only conversation tasks
II.5 Audiovisual conversation tasks
Appendix III – Examples of multiparty conversation test tasks (audio-only and
audiovisual): Free conversation
Appendix IV – Examples of multiparty conversation test tasks (audio-only):
Three-party conversation test scenarios 3CTs
IV.1 Introduction
IV.2 Test scenario development
IV.3 Scenario validation
IV.4 Cultural aspects
Appendix V – Examples of multiparty conversation test tasks (audiovisual):
Audiovisual multi-point tasks for three parties (Survival task, Leavitt task,
Brainstorming task)
V.1 Overview and most suitable task
V.2 Leavitt task
V.3 Brainstorming task
V.4 Survival task
Appendix VI – Additional proposals for multiparty conversation test tasks
(audiovisual): Extended survival tasks scenarios and celebrity name guessing
task
VI.1 Overview and background
VI.2 Modification of survival scenarios from Appendix V
VI.3 Three additional survival scenarios
VI.4 Celebrity name guessing task
Appendix VII – Additional proposals for multiparty conversation test tasks
(audiovisual): Formal and informal multiparty video conferences
Page
Appendix VIII – Overview of documents describing suitable test methods
VIII.1 Baseline test methods on which the current Recommendation is
based
VIII.2 Multiparty specific recommendations to adapt the baseline test
methods
VIII.3 Further test methods, which are referred to in the annexes of
this Recommendation
Bibliography