CCITT SGXV                                  					      Document AVC-399R
Working Party XV/1                        					      November 6, 1992
Experts Group for ATM Video Coding
(Rapporteur's Group on part of Q3/XV)

SOURCE:	CHAIRMAN OF THE EXPERTS GROUP FOR ATM VIDEO CODING
TITLE    :	REPORT OF THE NINTH MEETING IN IPSWICH AND LONDON 
	(27 October - 6 November 1992) - PART 2 
Purpose  :	Report
-----------------

Pert 2    Joint Sessions with MPEG

1. Introduction

Joint sessions with ISO/IEC JTC1/SC29/WG11 (MPEG) were held in London at the kind invitation of BSI (British Standards Institution) during 2-6 November 1992.  The Experts Group appreciated the support and hospitality of the hosting organization.

A list of documents considered during the joint sessions is attached to this  report as Annex 1.

2. REQUIREMENTS sub-group (K. McCann, S. Okubo)

2.1 General

The Requirements sub-group met every day during the November MPEG meeting, including joint meetings with the Systems and Implementation sub-groups.  Sakae Okubo chaired the meeting on Monday and Tuesday, whilst Ken McCann acted as chairman from Wednesday to Friday.

The major tasks were the finalisation of the integrated "Information on Requirements" document, the production of a first draft of the normative "Guide for Systems" and the definition of a means of producing sub-sets of the MPEG-2 toolkit.

2.2 Documentation

The following contributions were reviewed:

       Documents on Requirements Listing

92/674	Okubo	Report of Ad-hoc group for editing requirements document 
92/674a	Okubo          	Information on Requirements
92/740	CCIR SRG-UR	CCIR Comments on "Information on Requirements for 
		MPEG-2 Video"
92/791	CCIR 11/4	Liaison Statements from TG11/4 to Requirement sub-group 
		of MPEG
92/682	Erdem et al 	10 bit/pixel source material option for MPEG-2 
92/528	Futa	Requirements for the generic coding method of moving 
		pictures from the perspective of cable television distribution 
		in North America
92/714	Stevens        	Contribution to MPEG Requirements and MPEG Video in 
		the definition of MPEG-2 standard application profiles
92/725	SMPTE	SMPTE Header/descriptor Task force: Final Report 

      Documents on Profiles and the Core Profile

92/792	SC18/WG3       	Document Application Profile Proforma and Notation 
		Amendment (extract from N1779)
92/477 	Reader    	Contribution on MPEG 2 Video Requirements
92/476	USNB           	Proposal for Reorganised Syntax for MPEG 2 Video 
92/635	DIN            	Interoperability, definition of 'maximum core' and extensions 
92/569	NHK          	Consideration on HDTV/CDTV Compatibility
92/563	Sugiyama	Proposal for the Common Core
92/759	Lippman et al  	ATV Profile

2.3 Completion of the Requirements Information document

The participation of Dr Nishizawa and Mr Long of the CCIR Special  Rapporteurs Group on User Requirements contributed to a better  understanding of broadcaster's requirements and was greatly welcomed.  The document on Information on Requirements for MPEG-2, MPEG92/674a, was revised in the light of comments from the CCIR SRG-UR, and other inputs. 

It was noted that the CCIR are working on establishing a relationship  between subjective quality, an objective measure of picture  criticality and the probability of the occurrence of critical  sequences.  The offer from CCIR to collaborate in the MPEG subjective  test and evaluation programme was welcomed.

The suggestion of accepting 10 bits/pixel input signals in Document 92/682 was sympathetically received, but this was noted to be a new  requirement not contained in the normative video requirements  document.  Concern was expressed that adding this as an additional  requirement would adversely affect the timescales of the video work.

2.4 Guide for Systems

A first draft of the normative "Guide for Systems" was agreed.  The  Guide will be frozen at the next MPEG meeting after receiving feedback  for the Systems sub-group.

2.5 Definition of Profiles and Levels

2.5.1 ODA International Standardized Profile experience

Mr. Price, Convenor of SC18/WG3, kindly related the experience of SC18 with the development of International Standardized Profiles (ISPs) for ODA.  The ISPs are developed by informal workshops but are then formally registered by JTC1 ballot.  Three ISPs have been registered to date. 

2.5.2 Use of Profiles and Level by MPEG

It was agreed that:

  1)	The concept of International Standardised Profiles (ISPs) should be used as a guideline for the definition of sub-sets of the MPEG-2 parameter values and syntax.
  2)	Profiles could be developed by MPEG or by other bodies, such as CCITT or CCIR.  Plenary should be asked to consider whether it would be appropriate to ask ISO/IEC JTC1 to act as a registration authority. 
  3)	Profiles may contain different levels, which are distinguished only by allowed ranges of parameter values.  The number of levels should be kept as small as practical and, wherever practical, levels should be defined so that higher level conforming decoders can decode lower level bitstreams.
  4)	Wherever practical, the meaning of levels should be harmonised between different profiles. 
  5)	Full compliance to the same level of the same profile by two codecs should be a sufficient guarantee of their ability to successfully decode each other's bitstreams. 
  6)	MPEG profiles should be truly international in scope; national variants should not exist.  Some equipment designed for regional subsets of a profile might be used in practice (eg supporting of only 525 lines or only 625 lines) but such equipment could not be claimed to be fully compliant with the profile.
  7)	The number of profiles should be kept as small as practical.  In particular, the development of profiles which are very similar, but not quite identical, should be discouraged.

An example of profiles with harmonised levels is shown below:

      high  --------------------------------------------------
           |     ------------------------------------------   |
           |    |.............................Level 3......|  |
           |    |---------------------                     |  |
   MPEG-2  |    |.......Level 2.......|.......Level 2......|  |
  Parameter|    |                     |                    |  |
   Values  |    |.......Level 1.......|.......Level 1......|  |
           |    |                     |                    |  |
           |    |      Profile A      |      Profile B     |  |
           |     ------------------------------------------   |
      low   --------------------------------------------------
           <--                 MPEG-2 Syntax                -->

2.6 Core Profile

	2.6.1 Guidelines for developing the Core Profile

It was agreed that MPEG should develop a particular profile called the Core, using the following guidelines: 

  1)	The core profile should be suitable for use by the largest possible number of major MPEG-2 applications in terms of both functionality requirements and cost constraints.
  2)	The core profile should facilitate bitstream interchange between applications. 
  3)	The core profile should not contain redundant algorithmic techniques.  A single method of providing a particular functional element should be used wherever practical.

	2.6.2 Key Functionalities of the Core Profile

The joint meeting with the Implementation sub-group provided some useful indications of the likely implementation costs of important potential features of the core.  The precise definition of the Core Profile will require further work by the Requirements sub-group in consultation with the Implementation, Video, Audio and Systems sub-groups.  Consensus has been reached on the following skeleton:

The Core will support functionalities of:

     - Random Access (not necessarily at every frame)
     - Editability (not necessarily at every frame)
     - Robustness to bit errors and cell losses
     - Video windowing (e.g. 4:3 from 16:9 pictures)
     - Low delay (by simply turning off B frames)

The Core will not support functionalities of:

     - Trick modes (other than those automatically supported by the basic syntax)
     - Low delay (other than simply turning off B-frames)
     - Scalability
     - Compatibility

	2.6.3 Core coding techniques

The Core will include:

     - I, P and B frames
     - 4:2:0 resolution video
     - 8 bit pixels
     - No more prediction modes than strictly necessary (e.g. Frame and Field/Frame only) 
     - Non 8x8 DCT only if necessary (and only on a MB by MB basis)
     - No more scan modes than strictly necessary (e.g. zig-zag scan only) 

The Core coding techniques will exclude:

     - 4:2:2 or 4:4:4 resolution video
     - 10 bit (or other non 8 bit) pixels
     - Prediction modes whose necessity has not been demonstrated

	2.6.4 Core Performance Level

     - 720x480x29.97Hz and 720x576x25Hz
     - Bitrate up to 15 Mbit/s
     - Picture Quality close to Rec 601 source
     - Negligible degradation after up to 3 cascaded encoders/decoders

2.7 Recommendations of the Requirements Sub-Group

The Requirements Group recommends:

  1)	That the concept of International Standardised Profiles should be used as a guideline for the definition of sub-sets of the MPEG-2 parameter values and syntax.  Profiles are defined by functionality and coding techniques.  Within profiles, the concept of levels is introduced.  Levels are defined by performance constraints.

  2)	That WG11 should consider whether ISO/IEC JTC1 should act as a registration authority for such International Standardised Profiles, since they could be developed by other bodies (such as CCIR or CCITT).  

  3)	That MPEG should develop a profile called "Core" which is suitable for use by the largest number of major applications in terms of both functionality requirements and cost constraints and is intended for bitstream interchange.

  4)	That WG11 endorse the consensus view of the Requirements sub-group on the skeleton of the Core Profile given in the Requirements meeting report.  The precise definition will require further work by the Requirements sub-group in consultation with the Implementation, Video, Audio and Systems sub-groups.

  5)	The establishment of an ad-hoc group to further develop the draft Core Profile and consider profiles.  MPEG members are encouraged to join the ad-hoc group and to provide information on intended applications to assist in the profile development. 

  6)	That the MPEG syntax should not permit redundant algorithmic techniques. 

  7)	That the Video, Audio and Systems sub-groups should study the integrated "Information on Requirements" document.

  8)	That the draft Guide for Systems should be studied by the Systems sub-group with the intention of issuing a frozen set of normative systems requirements during the next MPEG meeting.

  9)	MPEG members are encouraged to provide information on the systems requirements of practical applications of the MPEG-2 standard.

  10)	That in order to meet the goal of a technically frozen draft by the Sydney meeting, an MPEG meeting is held in January 1993.  The Requirements sub-group further recommends that at least one joint Video-Requirements meeting is held during the January MPEG meeting. 

  11)	That the Convenor conveys to CCIR the appreciation of the Requirements Group for the participation of two members of CCIR Special Rapporteurs Group on User Requirements in the London MPEG meeting. 

3. TESTS sub-group (S. Okubo)

This sub-group decided to create procedures for and planning of subjective tests field trial for the purpose of showing the quality level of MPEG-2 standard to relevant organization including CCIR, EBU and SMPTE.

The following new test sequences were approved in addition to the current MPEG-2 official test sequences;

     - Composite version of Mobile and Calendar in 625/50 and 525/60
     - Stress materials proposed by General Instrument

Members are requested to provide HDTV test sequences for the purpose of MPEG-2 quality testing.

4. VIDEO sub-group (G. Bjoentegaard)

The video group met throughout the whole week.  The work was done in plenary, in joint sessions with SYSTEMS and IMPLEMENTATION and in subgroups.  The work was even done in "sub-sub" groups.  This way of spreading out the work made it difficult to have an overview of what was going on at all times.  

I followed the work in the video plenary as well as the subgroups on prediction and low delay.  The report on the work in the other groups is based on the subgroup reports back to the video plenary.  Special emphasis is tried given to areas of special interest for CCITT.

4.1 Joint meeting with SYSTEMS

The main issue was the impact of two - layer coding on systems.

4.2 Joint meeting with IMPLEMENTATION

The implementation group gave their reaction to the impact of various coding aspects.  There was basically no strong objection to the coding items proposed from the VIDEO group.

4.3 Prediction

The intelligent prediction modes (SVMC and DUAL+) showed significant coding gain, especially for M=1 where the gain with these prediction modes was typically 1 dB.  It was decided to continue tests for further simplification of intelligent prediction.  As a result of this, SVMC and DUAL+ will be merged to one mode for the next core experiment.

Tests had been performed on prediction on 8*8 block bases.  The results so far showed little improvement.  Further experiments will take place up to the next meeting.

4.4 Low delay

The work concerning regular update is concluded.  The recommendation is to use regular slice update.  An additional option for picture skipping is included in the picture header.  The coding performance for low delay depends basically on good prediction modes.  This is taken care of in the prediction group.  

Since the requirements concerning low delay seems to be fulfilled, it was not considered necessary to continue this work.

4.5 ATM - cell loss

The work in this group is basically based on the inputs from the CCITT experts group.  The group considered the use of leaky prediction as a method for cell loss recovery.

4.6 Compatibility/spatial scalability

The work is in progress, but many items still remain unsolved.  The main work items in this group are:

     - Compatibility with MPEG1 as well as H.261.
     - Fine tuning of spatio-temporal weights for optimum performance.
     - "Interlace-interlace" compatibility.
     - SNR scalability.

4.7 Frequency scalability.

Much work still remains before standardization.  Some of the present work items in the group are:

     - Reduction of drift problems in lower layers.
     - Single loop/multi loop approach.

4.8 Quantization.

This group covers several items related to the coding procedure.

     - An extra VLC for coefficient coding in INTRA frames have been adopted.
     - Alternative scanning of transform coefficients have been considered.
     - Increased precision of DC coefficients was considered.  The group recommended the use 
       of 9/10/11 bits for DC.
     - Alternatives to 8*8 DCT were considered.  8*1 DCT and NTC-coding were two 
        alternatives.
     - An alternative exponential table for MQUANT was considered.

4.9 Ad hoc groups.

The following ad hoc groups were formed to perform experiments up to the next meeting.

	Technical report (MPEG1)		A. Koster.
	Comments on syntax matters	J. Yonemitsu.
	Prediction				H. Watanabe.
	Quantization and entropy coding	N. Wells.
	Scalability				E. Viscito.
	Compatibility				A. Puri.
	ATM - cell loss			M. Biggar.
	Comments to working draft		T. Kogure.

5. SYSTEM sub-group (B.G. Haskell, T. Tanaka)

The systems group approved the Compliance section for MPEG1 and continued discussion of the various requirements for broadcast applications.  Participation was much larger in London than in Angra, mainly due to the broadcast interests.

They did not address ATM networks in particular, but instead focused the general broadcast problem.  The following model has been made;

A program consists of multiple audiovisual elementary bit streams having a common timing clock, while a program group consists of multiple programs each of which has an independent timing clock.  An adhoc group for "multiplexing for multiple programs" has been established to make further progress.

This sub-group also considered Directory for the program mux (MPEG92/672) and System ID extension.

The current Working Draft table is as follows;

Systems Working Draft Table of Contents

Contents
Forward
Introduction  -- Part I -- Systems
   Program Multiplex
   	Pack layer, Packet layer, System Target Decoder, Directory
   Super Multiplex
	Relationship to Program Multiplex, Function, Structure
   Master Multiplex
	Relationship to Super Multiplex, Function, Structure
   Stream Identification and Description Tables
   Conditional Access and Encryption
   Error Resilience
	Support for Channel Coding, Support for Residual Error Resilience
General Normative Elements
   Scope
   References
Technical Normative Elements
   Definitions
   Symbols and Definitions
   Method of Describing Bit Stream Syntax
   requirements
1-Annex A (Informative) Program Multiplex and Coding Layer
1-Annex B (Informative) Super Multiplex and Coding Layer
1-Annex C (Informative) Master Multiplex and Coding Layer
1-Annex D (Informative) Conditional Access
1-Annex E (Informative) Support for Error Resilience
1-Annex F (Informative) Support for Graceful Degradation
1-Annex G (Informative) Glossary

6. IMPLEMENTATION STUDIES sub-group (G. Morrison)

6.1  The Quantisation Sub group at Tarrytown had asked for implementation views on several items.

	6.1.1  VQ.  Documents 92/708 and 92/737 reported on implementation aspects of the proposal made at Tarrytown.  The vector search is very significant for encoders, approaching that of motion vector determination complexity.  The look up tables at coder and decoder are significant.  We made a recommendation stating these.

	6.1.2  Alternative VLC for intra coefficients, quant <8.

We agreed that this incurred little complexity.

	6.1.3 Range extension of coefficients.

We had been asked to comment on 10/11/12 bit coefficients.  We were willing to accept any or all of these.  If switchable, the indication should be at the picture header level.

	6.1.4  MQUANT extension

There were two proposals.  One was based on the step size being the sum of up to three consecutive powers of two while the other was a look up table with an approximately square law.  We expressed a slight preference for the former because it was easier in some implementations and just the same in the others.

	6.1.5  Zig-zag and vertical scan

We considered this to incur very small costs in decoders.  For encoders, an a priori decision method had been proposed with adequate performance and the implementation burden of it was acceptable.

6.1.6  Non 8x8 DCT

We had insufficient information to assess the implications of using an 8H by 1V DCT.  We requested fuller details.

6.1.7  Non transform coding

We assessed two proposals for non transform coding.  Both use a form of DPCM.

One method has the same prediction value for all pels in a block.  The differences are quantised with the existing quantiser and coded in the normal transform mode fashion but with a choice of two additional scan paths.

The other method uses DPCM from the previous pixel scanned along the zig-zag path and a square law quantisation of the prediction errors.  This scheme was seen to have two overheads.  Firstly, the order of quantisation and block to zig-zag scan is reversed from the transform case.  This is extra complication especially for encoders and decoders with dedicated hardware sub-units.  Secondly, the non-linear quantiser, being in the DPCM feedback loop in the coder must be high speed.

Accordingly we were more favourably inclined towards the former method but reserved final judgment on whether either should be included.  We requested that the Video group find a complete solution to those cases where transform coding does not work well.

6.2  Feedback from Video Group since Angra dos Reis

The Implementation Studies had requested the Video group to perform some experiments.  Results were reported in a joint session of Video and Implementation in the afternoon of 4 November.

	6.2.1  The effect of limiting the number of coefficient bits per macroblock to 256 or 512 were reported.  At higher bit rates defects were clearly visible.  These might be mitigated by more intelligent encoders which did more than just truncate excess coefficients.  However, the Implementation group had asked for experiments with dumb encoding to assess what could be achieved without adding significantly to encoders.

The original motivation behind the request for the experiments was realised to be part of a more general concern reported below.

	6.2.2  Macroblock address, skipped macroblocks, coded macroblock pattern and first 
	          coefficient trick.

Simulation results were reported that deletion of these had almost no effect on SNR and on the pictures when viewed.  However, it was pointed out that the use of other pictures and lower bit rates would refute this.  The Implementation group agreed that since the possible simplification of MPEG-2 only decoders was very small and that designs for chips able to decode both MPEG-1 and MPEG-2 would be made a little more complex we would accept the retention of these items in the syntax.

6.3  Issues arising at the London meeting

	6.3.1  Motion Vector for intra blocks

ATM cell loss error concealment can be improved if motion vectors for intra blocks are made included in the coded bit stream.  We concluded that:

There is trivial overhead on decoder implementations which choose not to implement this concealment technique.  The impact on decoders which do use it depends on the decoder implementation but is considered modest.

For encoders the overhead can be trivial if they are permitted to always insert zero vectors.  The next simplest is to insert the vector which may have been computed prior to the intra/inter decision.  Insertion of the vector for the macroblock below is more difficult.  However, this is not subject to standardisation.

	6.3.2  10 bit pels

There was discussion on permitting 10 bit pels which might be useful for some very high quality applications.  Our agreed view was that the extension to 10 bit pels has serious implications.  All signal handling parts of the decoder and most of them in encoder need changes. (Motion vector determination at 8 bits might be adequate.)  The major impacts are on the DCT/IDCT accuracy and the non-availability of 10 bit wide memories.

We agreed that it would not be commercially sensible to offer only 10 bit resolution MPEG chips.  Thus 8 and 10 bit MPEG-2 would be different markets.  Concerns were expressed that the market for 10 bits might be so small that it would only be addressed by costly professional offerings.

The concept was raised of a 10 bit MPEG-2 bitstream being decodable by an 8 bit MPEG-2 decoder.  This is essentially SNR scalability.

	6.3.3  Frequency scanning and MUVLC

This had been looked at previously at the Angra dos Reis meeting.  Extended discussion at the London meeting revealed that the encoder might incur less of a memory penalty than the original proposers had thought but we concluded that a corresponding technique at the decoder was not possible.  The temporary memory is one slice of coefficients.  For a slice size set to one row of macroblocks across 601 images, the storage is of the order of 200 kbit which is too much for on-chip.  Putting this off-chip incurs a very significant bus bandwidth penalty as every 12 bit coefficient is stored and fetched.  This is roughly equivalent to another three prediction sources.  We noted that using smaller slices reduces the memory size proportionally but leaves the bandwidth unchanged.  We were of the opinion that the coding efficiency would deteriorate markedly if the slice size were reduced sufficiently to allow on-chip storage.

(The encoder simplification arises from having small caches which hold the numbers of zero coefficients.  Non-zero coefficients are encoded as early as possible.)

We recommended that the video group make other proposals which we would then assess.

	6.3.4  We discussed the overheads of scalable and compatible algorithms.  We presented initial findings in a short session with the Requirements group.  We had a short presentation on the transform domain scalability methods (single and multiple loop).  We began to construct a table listing pertinent points of the transform and spatial split approaches.

	6.3.5  It was pointed out that although some prediction modes such as dual field have a smaller storage requirement than modes using B frames, the memory bandwidth is unchanged.  Device speed limitations may therefore mean that the required physical amount of memory is the same.

6.4  Restrictions on bit streams

We were informed of a difficulty that can arise in decoders when a very large number of MBA stuffing words are clumped together.  Further discussion indicated that this is a specific case of a more general difficulty. When the bitstream contains extended periods of large macroblocks (many bits) the VLC decoding time for them can become a significant fraction of one picture time.  Even though the following macroblocks may contain few bits (to satisfy the VBV) they may nevertheless require a large number of computations or memory accesses or both.  To complete these before the end of the frame slot may require very high clock rates and peak memory bandwidths.

The difficulty was seen to be architecture dependent.  Some of the participants had foreseen the matter and chosen their architectures to overcome it.  However it was accepted that there may be an (unquantified) cost penalty to do this.

The issue is that of designing a decoder which can accommodate all possible MPEG bit streams which could be generated by valid pictures and syntax or designing a decoder which works on all or most of the bit streams which are likely to be generated in practice by real encoders from real pictures.

The company which had highlighted the specific MBA stuffing issue had also brought a proposal to limit the bit stream.  This might place some burden on encoders but was thought (by the Implementation Studies participants) to have little or no effect on picture quality on real material.

We decided not to accept this hastily but agreed that the general problem identified has far reaching consequences.  Therefore it was felt that an ad-hoc should be established to consider the matter to sufficient depth before reaching a decision.

6.5  At the request of the MPEG Convenor we discussed what rules if any should be applied to hardware demonstrations (not simulation results) given at MPEG meetings.  A summary was presented at the closing plenary.

6.6  Need for January meeting

In the light of the need to close the restricted bit stream issue and the need to report on overall implementation aspects of the complete WD/TM it was agreed that the group should meet in January if the work was to remain in step with other MPEG groups towards the schedule.  The anticipated duration required is 3 to 4 days.

END