ITU - Telecommunications Standardization Sector
STUDY GROUP 16
Video Coding Experts Group (Question 15)
_________________
Sixth Meeting: Seoul, Korea, 3-6 November, 1998	Document  Q15-F-06
Filename: q15f06.doc
Generated: 31 Oct ’98	

Question:	Q.15/SG16	
Source:	Stephan Wenger
TU Berlin / TELES AG
Franklinstr. 28-29
D-10587 Berlin
Germany	
Tel:
Fax:
Email:	
+1-415-297-2063
+1-604-736-7853
stewe@cs.tu-berlin.de	
Title:	Report of the Ad Hoc Committee on Consideration of Simulation Conditions and Evaluations for Error Resilience Testing	
Purpose:	Information	
_____________________________
Summary
A lot of work was performed since Whistler, and especially after the Geneva study group meeting, with respect to error resilient coding.  Q.15 experts (especially Gary Sullivan) designed and implemented a H.223/M simulation software (Q15-F-16), after it became clear in Geneva that Q.11 experts were unable to provide that in a timely fashion.  Based on this simulator, and the test model and common condition documents decided in Whistler (Q15-E-50 and Q15-E-51), at least one group performed all the discussed simulations, which the author hopes will become available to other interested parties.  The author therefore believes that very good progress was made since Whistler with respect to mobile.

There was no substantial activity on error resilience related issues for packet networks, especially the Internet.  The author believes that, with respect to standardization work, this topic can considered to be almost completely covered, after RFC 2429 (H.263+ payload spec) is now published.  There is, however, some very interesting activity going on in the IETF on a FEC mechanism for packet based networks.

H.223/M simulation software
A very simple simulator of an H.223/M channel was implemented and made available (Q15-F-16).  This software allows for an uni-directional simulation of an H.223 annex B channel, allocating bandwidth for G.723.1 audio, video and H.223 overhead.  For video, a simplified AL3 mechanism is implemented, that does not perform any re-transmission.  The simulator expects as input and output a packet format similar to the one described in Q15-E-50.  The packization and de-packetization scheme is not part of the simulation software, but very easy to implement (sample source code is available from the author of this document).
There was also some discussion regarding the use of the MPEG-4 error simulator.  This simulator applies both bit errors and packet losses to a coded (MPEG-4) bitstream.  The error patterns are generated by a built-in error generator and not by an external error pattern source.  Based on reflector discussions it was decided not to use this simulator (although it is readily available), because it does not reflect the ITU-T system environment (especially H.223).  Additionally, there was a comment on the characteristics of the bit-error pattern generator of this simulator, which seems to show an error characteristic that is much more friendly to video coding at the same BER and average burst length rates, but less realistic than the error patterns we decided to use.
Error patterns
There was some discussion on the reflector regarding the DECT error patterns (Raleigh patterns).  It seems to be that the first 80 bytes of all 32 DECT patterns available show a quite serious burst of errors.  Discussions on the reflector lead to the conclusion that this is not a natural channel behavior, but might be related to some feature of the simulation software that generated those patterns.  Consensus was reached on the reflector to skip those first 80 bytes for the first iteration of use of the patterns.  

After several queries, we were finally pointed to the WCDMA error patterns, which currently reside in document Q11-F-05 (but will hopefully be moved to a more obvious place in the near future).  These patterns can be used ‘as is’, without any skipping of the first couple of bytes.  Their error characteristics allow for acceptable picture quality.

It was observed, although not discussed on the reflector, that the 4 dB Raleigh error pattern (mandated by Q15-E-50) is more than challenging for the use with coded video.  We should probably discuss to remove this pattern, or use the 6 dB pattern instead.  It was also observed, that the error patterns with similar SNR, but different Doppler speeds (and thus slightly different burst length) yield almost the same performance.  In order to minimize the effort of groups proposing error resilient syntax, we probably want to remove all error patterns except one for a given BER or SNR value.  A proposal for this can be found in Q15-F-38.
Mobile anchors
In the documents Q15-E-50 and Q15-E-51 a couple of sequence/error pattern/bitrate permutations and a description of the coding process were defined, that together form a description of what by the knowledge of the group is the best possible known way to use the H.263+ syntax in an error prone environment.  These “anchors” were planned to be prepared by UBC.  This has not happened by now.  UCLA/Samsung, however, have performed to generate those anchors as part of their experiment on data partitioning.  The obtained results are reasonably close to UBCs preliminary results, with the exception of a few sequences, where the UBC results are significantly better.

It should be noted, however, that UCLA did not use any RD optimization in their coding, as mandated by Q15-E-51.  Since they also did not use any RD optimization for their own RVLC scheme, the author believes that the results are still sufficiently meaningful.  In order to give groups with substantial experience in error resilience coding, but without an own RD optimized coder, a chance to pass an evaluation, we might want to relax our anchor document in this respect (and also with respect to the optional modes mandated), or generate a alternative ‘low complexity’ model.  

As part of the research work, UBC continued to evaluate a more sophisticated mechanism for choosing the intra update frequency (see Q15-F-37 and Q15-F-38).  Based on this results, UBC proposes a new mechanism for the anchor generation and promises to generate those anchors in a more timely fashion.
Packet networks / Internet activities
As already mentioned above, no significant progress was made on error resilience for Internet-based video transmission.  It should however be mentioned here, that during the Chicago IETF promising results were reported, and significant progress was made, in the standardization of a forward error correction scheme for the Internet.  This mechanism (draft-ietf-avt-fec-03.txt) is close to get RFC status.  A substantial amount of people in the IETF/avt voiced the opinion that this mechanism is preferable over error-resilient coding, because it work well with per-recorded data, and inflexible (hardware-) coder implementations that do not support any scalable error resilience support.  Based on a quick look at the data the authors provided in two not yet published journal papers, the author feels that there data and argumentation is compelling.