1
Scope
2
References
3
Definitions
3.1
Terms defined elsewhere
3.2
Terms defined in this Recommendation
4
Abbreviations and acronyms
5
Conventions
6
General Recommendations concerning the subjective quality evaluation of
multiparty telemeetings
7
Multiparty specific aspects in subjective quality evaluation
8
Guidance to suitable test methods
8.1
Test method decision criteria
8.2
Flow charts to be used when selecting test methods
Annex A – Set-up of a multiparty telemeeting assessment test
A.1
Assessment of conversational quality – Conversation tests
A.2
Assessment of non-interactive quality – Non-interactive tests
Annex B – Assessment of telemeetings with text-based communication and
graphical information means (e.g., web conferencing)
Annex C – Assessment of video-only telemeetings
Annex D – Effect of transmission delays on telemeeting quality
D.1
Background
D.2
Existing test task Recommendations
D.3
Recommended test tasks
D.4
Set-up of a delay test
D.5
Test subjects
D.6
Training session
D.7 Instructions
D.8 Test
questions
D.9
Objective measurements
D.10
Effects of delay
Annex E – Assessment of 3D audio and 3D video reproduction of
multiparty telemeetings
Annex F – Assessment of asymmetric multiparty telemeetings
F.1
Overview
F.2
Interactions of different group sizes and different communication modes
F.3
Remarks concerning the experiment design
F.4
Remarks concerning scales
F.5
Remarks concerning data analysis
Annex G – Assessment of multiparty telemeetings with non-stationary
quality
Annex H – Assessment of multiparty telemeetings using multi-dimensional
scaling methods
Appendix I – Influential factors
Appendix II – Overview of multiparty non-interactive test stimuli and
conversation test tasks
II.1
Non-interactive audio-only stimuli
II.2
Non-interactive video-only stimuli
II.3
Non-interactive audiovisual stimuli
II.4
Audio-only conversation tasks
II.5
Audiovisual conversation tasks
Appendix III – Examples of multiparty conversation test tasks
(audio-only and audiovisual): Free conversation
Appendix IV – Examples of multiparty conversation test tasks
(audio-only): Three-party conversation test scenarios (3CTs)
IV.1
Introduction
IV.2
Test scenario development
IV.3
Scenario validation
IV.4
Cultural aspects
Appendix V – Examples of multiparty conversation test tasks
(audiovisual): Audiovisual multi-point tasks for three parties (Survival task,
Leavitt task, Brainstorming task)
V.1
Overview and most suitable task
V.2
Leavitt task
V.3
Brainstorming task
V.4
Survival task
Appendix VI – Additional proposals for multiparty conversation test
tasks (audiovisual): Formal and informal multiparty video conferences
Appendix VII Overview of documents describing suitable test methods
Bibliography