Table of Contents

 1     Scope            
 2     References   
 3     Definitions   
        3.1     Terms defined elsewhere
        3.2     Terms defined in this Recommendation     
 4     Abbreviations and acronyms  
 5     Conventions
 6     General Recommendations concerning the subjective quality evaluation of multiparty telemeetings    
 7     Multiparty specific aspects in subjective quality evaluation             
 8     Guidance to suitable test methods          
        8.1     Test method decision criteria         
        8.2     Flow charts to be used when selecting test methods 
Annex A – Set-up of a multiparty telemeeting assessment test    
        A.1     Assessment of conversational quality – Conversation tests   
        A.2     Assessment of non-interactive quality – Non-interactive tests              
Annex B – Assessment of telemeetings with text-based communication  and graphical information means  (e.g., web conferencing)    
Annex C – Assessment of video-only telemeetings    
Annex D – Effect of transmission delays on telemeeting quality    
        D.1     Background     
        D.2     Existing test task Recommendations          
        D.3     Recommended test tasks              
        D.4     Set-up of a delay test     
        D.5     Test subjects    
        D.6     Training session               
        D.7     Instructions      
        D.8     Test questions  
        D.9     Objective measurements               
       D.10     Effects of delay             
Annex E – Assessment of 3D audio and 3D video reproduction of  multiparty telemeetings    
Annex F – Assessment of asymmetric multiparty telemeetings    
        F.1     Overview           
        F.2     Interactions of different group sizes and different communication modes         
        F.3     Remarks concerning the experiment design               
        F.4     Remarks concerning scales            
        F.5     Remarks concerning data analysis              
Annex G – Assessment of multiparty telemeetings with non-stationary quality    
Annex H – Assessment of multiparty telemeetings using multi-dimensional scaling methods    
Appendix I – Influential factors    
Appendix II – Overview of multiparty non-interactive test stimuli and conversation test tasks    
       II.1     Non-interactive audio-only stimuli               
       II.2     Non-interactive video-only stimuli               
       II.3     Non-interactive audiovisual stimuli             
       II.4     Audio-only conversation tasks     
       II.5     Audiovisual conversation tasks    
Appendix III – Examples of multiparty conversation test tasks (audio-only and audiovisual): Free conversation    
Appendix IV – Examples of multiparty conversation test tasks (audio-only): Three-party conversation test scenarios (3CTs)    
       IV.1     Introduction    
       IV.2     Test scenario development           
       IV.3     Scenario validation        
       IV.4     Cultural aspects              
Appendix V – Examples of multiparty conversation test tasks (audiovisual): Audiovisual multi-point tasks for three parties (Survival task,  Leavitt task, Brainstorming task)    
        V.1     Overview and most suitable task 
        V.2     Leavitt task      
        V.3     Brainstorming task         
        V.4     Survival task    
Appendix VI – Additional proposals for multiparty conversation test tasks (audiovisual):  Formal and informal multiparty video conferences    
Appendix VII  Overview of documents describing suitable test methods