Annex 3 to AVC-443R

CCITT SGXV	TD 7
Working Party XV/1	January 1993
Experts Group for ATM Video Coding
(Rapporteur's Group on Part of Q.3/XV)

INTERNATIONAL ORGANISATION FOR STANDARDISATION
ORGANISATION INTERNATIONALE DE NORMALISATION
ISO/IEC JTC1/SC29/WG11
CODING OF MOVING PICTURES AND ASSOCIATED AUDIO

ISO/IEC JTC1/SC29/WG11
MPEG93/214
January 1993

	Source:	CCITT SGXV Experts Group on ATM Video Coding
		(Rapporteur's Group on Part of Q.3/XV)
	Title:	Error resilience framework for informative text and further experimentation.
	Status:	Proposal

The CCITT SGXV Experts Group on ATM Video Coding (Rapporteur's Group on Part of Q.3/XV) proposes the following means to progress activities leading to informative text in the MPEG2/H.26X standard. Section 1 below provides a framework for inclusion in the current WD at the Rome joint meeting. It includes several “holes” to be filled through contributions required by the next meeting. Section 2 provides some additional information on the status of ongoing studies of error resilience methods using tools that are not yet part of the agreed syntax, and therefore require further experimentation and decisions before they can be transferred to the WD. Summary tables, to assist with identification of appropriate error resilience techniques and any overlap between techniques, are also provided. Inputs are requested to update/complete these tables.
Note that text describing cell loss/error resilience techniques should include:
•	Assumptions about network capabilities or requirements, such as notification of cell losses from an AAL, or the availability of prioritised transmission, must be indicated clearly.
•	The encoder/decoder configurations, or profiles, that offer the tools that will be utilised to provide the error resilience must be identified.
•	An indication of the CLR, burst lengths or other error statistics that can be protected against using the technique.
1. Text for WD
The following is offered for inclusion as informative text in the WD, providing preliminary information and a basis for further expansion on specific error resilience schemes.
Error Resilience
The coded video bitstream generated by this Recommendation | International Standard may be carried by different transport systems including packet systems. The Asynchronous Transfer Mode (ATM) of B-ISDN is an example of a packet system. ATM uses relatively short, fixed length packets, called cells, consisting of a 5 byte header containing routing information, and a user payload of 48 bytes. The nature of errors on ATM is such that some cells may be lost, and the user payload of some cells may contain bit errors. At the receive terminal, the network interface forwards received cells plus indications of lost cells. Indication of cells containing bit errors may also be available, depending on AAL (ATM Adaptation Layer) functionality.
As an indication of the impact of cell loss in an ATM environment, the following table summarises the average interval between cell losses for a range of CLR and service bit rates. Note, however, that this summary ignores cell loss bursts and other shorter term temporal statistics.
	Average interval time of error	
CLR	   5 Mb/s   	   10 Mb/s  	   50 Mb/s  	  100 Mb/s   	
   10-2  	7.52*10-3  s	3.76*10-3  s	7.52*10-4  s	3.76*10-4  s 	
   10-3  	7.52*10-2  s	3.76*10-2  s	7.52*10-3  s	3.76*10-3  s 	
   10-4  	7.52*10-1  s	3.76*10-1  s	7.52*10-2  s	3.76*10-2  s 	
   10-5  	7.52       s	3.76       s	7.52*10-1  s	3.76*10-1  s 	
   10-6  	1.25       s	37.6       s	7.52       s	3.76       s 	
   10-7  	12.5       m	6.27       m	1.25       m	37.6       s 	
   10-8  	2.09       h	1.04       h	12.5       m	6.27       m 	

[TEXT REQUIRED TO INTRODUCE OTHER ERROR/LOSS CONDITIONS]
The following techniques of minimising the impact of lost cells and other error/loss effects are provided for reference, and indicate example methods of using the various tools available in this Recommendation | International Standard to provide good performance in the presence of those errors. Note that, while the discussion is centred around cell loss resilience, the techniques described may be equally applicable in the cases of packets of other sizes (e.g. LANs or certain storage media) or video data with uncorrected errors of different characteristics.
The error resilience techniques are summarised in three categories, covering the restriction of the influence of a loss or error in both space (within a picture) and time (from picture to picture), and the methods of concealing the error once it has occurred.
1.	Concealment
Concealment techniques attempt to hide the effect of losses/errors once they have occurred. Some concealment methods can be implemented using any encoded bitstream, while others are reliant on the encoder to structure the data or provide additional information to enable enhanced performance.
1.1	Concealment as a decoder-only function.
Independent of whether any attempt has been made at the encoder to make the generated video bitstream error resilient, a decoder always has the option of providing concealment of the errors by estimating the lost data from spatio-temporally adjacent data. Techniques may range from simply not updating a frame store in the event of lost macroblock data (i.e. concealment through estimation from the previous frame) to more sophisticated concealment methods that utilise motion compensation for the macroblock prediction.
[Reference system text required.]
1.2.	Encoder-assisted concealment.
Layered Coding
The components produced by the coding process can be placed in a hierarchy of importance according to the effect of loss on the reconstructed image. By coding and multiplexing components of similar importance into independent bitstreams, and treating each bitstream with due importance, superior error concealment performance may be possible. The independent bitstreams may be treated differently at one or more of the following locations: 
•	coder - different channel coding might be used
•	channel - the channel may be able to offer paths with different cell loss probabilities
•	decoder - error concealment could be performed differently within each bitstream
[Reference system based on the scalability and compatibility enhancements for error concealment.]
2.	Spatial localisation.
Spatial localisation encompasses those methods aimed at minimising the extent to which errors propagate within a picture, by providing early resynchronisation of the elements in the bitstream that are coded differentially.
[Reference text required on:	Small slices
	Adaptive slice sizes]
3.	Temporal Localisation
Temporal localisation encompasses those methods aimed at minimising the extent to which errors propagate from picture to picture in the temporal sequence, by providing early resynchronisation of the elements in the bitstream that are coded differentially.
[Reference text required on:	Intra pictures
	Intra slices]
4. Summary table
The following table summarises the above error resilience techniques, indicating their applicability and range of CLR for which they provide effective resilience.

Category	Technique	Profile/Applicability	Range of CLR	
Concealment (decoder only)	Simple (previous picture)	All		
	Motion compensated	All		
Concealment(Encoder assisted)	Layered coding	Freq. scalability,
Spatial scalability		
Spatial Localisation	Small Slices	All		
	Adaptive slice sizes			
Temporal Localisation	Intra pictures			
	Intra slices			

2. Further techniques for study
Other techniques, outside those described in Section 1, are under study for cell loss/error resilience. They are summarised below, using the same categorisation as in the proposed text for the WD, with a summary of the open issues requiring further investigation.
1.	Concealment
1.1	Concealment as a decoder-only function.

1.2.	Encoder-assisted concealment.
Use of I-frame or I-MB motion vectors for assistance with motion compensated concealment.
Open issues:	Under what circumstances does this method have advantages over others (that do not require syntax changes)
	Should the Intra MVs be provided for all I-macroblocks, or only those in I pictures?
2.	Spatial Localisation.

3.	Temporal Localisation
Leaky Prediction:
The CCITT-EG recommends one more round of testing to compare INTRA insertion (INTRA picture, slice, column) with leaky prediction for purposes of error resilience.
Several benefits of leaky prediction were discussed, including...
1. Leaky prediction provides fault tolerance with no special error detection or correction at the decoder.  Compared with simple replacement of errored MBs with previous frame values, leaky prediction provides significantly better picture quality.
2. After a random access or channel change, a pleasing picture can often be seen more quickly with leaky prediction than with INTRA insertion.
3. In low delay applications that require small buffers, better picture quality can often be obtained with leaky prediction than with INTRA pictures.
4. IDCT mismatch and frequency scalable drift are alleviated with leaky prediction.
5. Coding efficiency of leaky prediction and INTRA pictures with N=12 or 15 were comparable.
Several solutions to the problem of noisy background were presented, including...
1. Adaptive quantization with particular attention to reducing the step size in areas of low detail and motion.
2. Adaptive quantization of the DC component with low step size in areas of low detail and motion.
3. Leaky prediction of AC components only, with INTRA insertion of DC components.  This solution also has the advantage of extremely quick channel hopping.
Core Experiment F.3 will be completed with the independent verification and comparison of these three solutions.

4. Expanded comparison table

Category	Technique	Profile/Applicability	Range of CLR	
Concealment (decoder only)	Simple (previous picture)	All		
	Motion compensated	All		
Concealment(Encoder assisted)	Layered coding	Freq. scalability,
Spatial scalability		
Spatial Localisation	Small Slices	All		
	Adaptive slice sizes			
Temporal Localisation	Intra pictures			
	Intra slices			
	Leaky prediction