CCITT SGXV                                  					       Document AVC-444R
Working Party XV/1                        					       January 29, 1993
Experts Group for ATM Video Coding
(Rapporteur's Group on Part of Q.3/XV)

SOURCE:	CHAIRMAN OF THE EXPERTS GROUP FOR ATM VIDEO CODING
TITLE    :	REPORT OF THE TENTH MEETING IN TORINO AND ROMA
	(21-29 January 1993) - PART III
Purpose  :	Report

-----------------

Part III  Joint Sessions with MPEG

Contents

1. 	Introduction
2. 	REQUIREMENTS sub-group
3. 	VIDEO sub-group
4. 	IMPLEMENTATION STUDIES sub-group
5. 	SYSTEMS sub-group

1. Introduction

The joint sessions with MPEG were held at the Ministero P.T. Building in Roma during 25-29 January 1993 at the kind invitation of the Ministry of Posts and Telecommunications.  The Experts Group appreciated the support and hospitality of the hosting organization.

A list of documents considered during the joint sessions is attached to this  report as Annex 1.

2. REQUIREMENTS sub-group (S. Okubo)

The REQUIREMENTS sub-group met every day during the week (January 25-29) under chairmanship of Sakae Okubo.  In addition to our own sessions, we had joint sessions with other sub-groups as follows;

-	with VIDEO and IMPLEMENTATION, 16:30 - 17:15 on Wednesday to solve outstanding issues related to the Main Profile definition 
-	with SYSTEMS, 9:00 - 10:20 on Thursday to finalize the "Guide for Systems Work" document.

2.1 Information on requirements

	2.1.1 Relevant documents

047	NTV	Requirements for applying extended definition progressive scanning images	
100	Nasse	Proposal for MPEG-2 verification tests	
128	EBU G5	Replies from members of Subgroup G5 to MPEG documents	
133	CCIR TG11/4	Report of the first meeting of TG11/4	
179	Koster	Proposal for and discussion on the syntax to exploit the correlation between similar video signals over a pair of video codecs	
216	CCIR	Draft new Recommendation: Reference model for data broadcasting	
226	SMPTE	Contribution to MPEG on SMPTE File Format for Digital Picture Exchange	

	2.1.2 Agreements

-	To include progressive/interlaced indication in Picture Layer (MPEG93/047,179).

-	To include 720x480x1x59.94 format as another example of progressive format in the requirements listing document (MPEG93/047).

-	To recognize the necessity of a mechanism to indicate relationship between two correlated video signals in the video and/or system layer (MPEG93/179).  Reflection on the syntax is left to the decision of other sub-groups.

-	To consider the SMPTE document (MPEG93/226) in the study of presentation related indications which facilitate bitstream interchange across applications.

-	To return a liaison response to the EBU comments (MPEG93/128) including general statement describing current status of our study and Integrated Requirements Listing revised in London (MPEG92/674a).

	2.1.3 Items needing guidance of WG11

-	There has been recognized necessity of including source related indications such as colorimetry, color matrix in the syntax to make consistency with SMPTE "Digital Picture Exchange" (MPEG93/226).  We need a guideline on what application dependent indications are to be in the video syntax.  

-	At the plenary on Wednesday, Dr. Gill provided a proposal for our further study that VIDEO and AUDIO subgroups define their own extensible structures and parameters to accommodate presentation related parameters.  Considerations were that we want to put them in video and audio bitstreams, not in user data, and that we can not define all of them now.

2.2 Definition of Profiles

	2.2.1 Relevant documents

N0262	Convenor	Report of London meeting	
045	JNB	Comments for Rome meeting	
046	JNB	Profile proposal for MPEG-2 profiles	
059 	Reader	Report of ad-hoc group on draft core profile and additional profiles	
071	Fautier	Extended DC precision in the MPEG-2 core profile	
082	CCITT/J	Core profile based on communication applications consideration	
097	Reader	Classification of MPEG-2 algorithm techniques into profiles	
117	Kogure	Consideration on core profile	
141	Fautier	Proposal for CORE buffer size information	
154	ETSI NA5/VCM	Clarification and suggestion on profiles	
156	Yamada	A consideration for core profile	
168	Sugiyama	Proposal of limitation in core profile	
196	CCIR SG11	Progress report of the activities of SRG-UR1	
215	DIN	Comments on the resolutions of the 20th WG11 meeting	
237	REQ group	Definition of Profiles and the Main Profile	
238	Lippman	Suggestion for a revised set of profiles	

	2.2.2 Agreements

-	To adopt the descriptive text (MPEG93/237, contained in this report as Annex 2) generated during this meeting for definition of profile and Main Profile based on the London agreements.

-	To adopt the specifications for Main Profile at Main Level in terms of syntactic elements and their associated values if relevant, which were worked out during this meeting (MPEG93/237).  It should be noted that some items still need consultation with VIDEO and IMPLEMENTATION sub-groups.

-	To use a new name in substitution of "Core" because it sometimes convey wrong messages to the outside society, giving impression that there would be no profiles smaller than this.  After some discussion, we reached a conclusion to use "Main Profile" and "Main Level", leaving "Main Level" subject to further discussion.

-	To seek advice of CCIR whether coding format of 480 lines, instead of 486 lines, for 525 line television signals are acceptable for purpose of distribution to end users with our explanation of why 480 lines are considered appropriate.

	2.2.3 Discussion on profiles/levels other than Main

-	One view was that hierarchically structured three profiles (onion ring structure) may be sufficient as discussed in MPEG93/238.  

-	Another view was that it may practically be required to structure some profile(s) other than Main Profile in a non-hierarchical manner (crossing onion rings).

-	There was some discussion that a codec product may implement specifications outside Profile X but inside Profile Y (>X) and what conformance this product can claim.  A solution seems to be to claim decoder conformance as Profile X and encoder conformance as Profile Y.  

-	There was also a question whether a software decoder which may discard B pictures data and not display them can claim conformance with the standard.  There was another question to what extent regional variants of the international standard are allowed.

	2.2.4 Items to be further discussed

-	Levels other than Main Level in Main Profile.
-	Profiles other than Main Profile.
-	Whether 4:2:2 be in other level of Main Profile or in other Profile.
-	Profile and Level indicator

2.3 Structure of standard

	2.3.1 Relevant documents

057	Kogure	Report of ad-hoc group to process comments on the WD	
091	WD Ed. gr.	MPEG-2 Working Draft (Video part)	
092	Kogure	Presenting structures of profiles in WD	
195	JTC1/SC21	Proposal for a new work item: SQL multimedia and application packages (SQL/MM)	

	2.3.2 Discussion

Due to the urgent work for the Main Profile, this topic could not be handled during this week.  Inputs are requested toward the Sydney meeting.

2.4 Verification

	2.4.1 Relevant documents

100	Nasse	Proposal for MPEG-2 verification tests	

	2.4.2 Discussion

This verification process is intended to make sure all the requirements are supported by the draft standard.

-	In MPEG93/100, necessity of clarifying specifications of the profiles has been pointed out.  Interest in scalable coding, such as HDTV/CTV compatibility, has also been expressed.

-	What quality is obtained at particular bit rates is one of the most concerned characteristics for customers of the standard.

-	Due to lack of time at this meeting, this topic will be discussed further in Sydney focusing on methodologies to verify items addressing functionalities such as scalability, in addition to picture quality.

2.5 Guide for Systems work

	2.5.1 Relevant documents

92/770	REQ group	Guide for Systems Work, London Revision	
239	REQ group	Guide for Systems Work, Rome Revision	

	2.5.2 Agreements

-	To adopt the revised text as in MPEG93/239.

2.6 Very low bit rate audiovisual coding

	2.6.1 Relevant documents

213	CCITT WPXV/1	Terms of reference for Special Rapporteur for very low bitrate visual telephony	

	2.6.2 Discussion

We had a short free discussion on what would be the first activities, possible work methods and possibility of generic coding at very low bit rates.  It was a common feeling that we need technical breakthroughs to make this new standard.  As to the possibility of "generic coding", two solutions were mentioned; downloadable algorithm and heterogeneous standards.

	2.6.3 Action

-	To recommend starting activities at the Sydney meeting by reviewing the state of the arts with demonstration and having a brain storming session.

3. VIDEO sub-group (G. Bjøntegaard)

Unlike previous meetings, the video group this time stayed together as one group.  The group dealt with some 130 documents!  The bulk of the documents reported results from core experiments.  A major part of the video work was therefore to review these results and reach conclusions.  The video group also had joint meetings with requirements and implementation.  In addition there was a joint meeting with CMTT/2 on digital broadcast matters.

Joint meeting with CMTT/2.

CMTT/2 is dealing with digital TV coding.  The group wants to cooperate with MPEG on video coding matters.  The group presented its present work items.  It was decided to keep the contact and that members of CMTT/2 could cooperate with MPEG on the ad. hoc. group level.

Joint meeting with requirements and implementation.

The status of the different subgroups were presented.  There is at the moment a good co-operation between the subgroups.  This is helpful to the convergence process.  Video had put many questions concerning complexity to the implementation group.  The responses are reflected in the decisions on different coding items reported below.

Short reports on the main outcomes concerning different items are given below.

	Prediction.

Many documents reported results from the core experiments L.13, L.14 and L.15.  The main conclusions were:

- 	Special prediction mode (in particular Dual+) is accepted due to its performance for M=1.
- 	8x8 and 16x8 frame prediction was not accepted.  Further tests will be made until next meeting.
- 	Reverse order prediction was not accepted.  (Further tests will be made until next meeting?)

	Quantization.

-	Use of 8x1 DCT and NTC did not show good enough results to overcome problems with noise around edges.  (Further tests will be made until next meeting?)
- 	Use of adaptive VLCs for coefficient coding showed promising results. Further tests will be made until next meeting.

	Error resilience.

- 	Bitstream partitioning was accepted as a useful way of using different priorities to different parts of the bitstream.  This was accepted.
- 	Leaky prediction was not accepted because there are other ways (I slices, I pictures) to solve the same problem.  (Further tests will be made until next meeting?)
- 	Motion vectors for I pictures will be tested further until the next meeting.
- 	Structured packing was not accepted.

	Scalability.

- 	In spatial scalability progress was made in defining filters for Interlace -Interlace conversion.
- 	Work is done towards single loop solutions in frequency scalability.
- 	Effort is being made to define a combined syntax for both spatial and frequency scalability.

Good progress was made in the video group concerning convergence on coding issues.  It seems realistic that the remaining issues needed for the "March freezing" will be solved by the next meeting.

4. IMPLEMENTATION STUDIES sub-group (G. Morrison)

4.1 Sessions

The sessions were sole, joint with Video and Requirements and joint with Video

4.2 Participants

The following people took part for the vast majority of the sole sessions.  There were a few other participants for short times during the sole sessions.

Geoff Morrison	BT Labs (chair)
Adrian Wise	Pioneer
Richard Bramley	Thomson
Mark Veltman	Sony
Hironori Yamauchi	NTT
Chadd Fogg	Cascade Design Automation
Sam Narashiman	Singapore Microelectronics Institute
Barth Canfield	Thomson
Fathy Yassan	Thomson
Marco Gandini	CSELT
Thierry Fautier	Philips LEP

4.3 Topics discussed

The topics discussed were all related to the video algorithm.  The majority were raised by the Video Subgroup.  The remainder were prompted by the Requirements Subgroup, MPEG documents to the meeting or by the participants.

MBA stuffing

This had been the topic of an ad-hoc group established at the London meeting but a decision had not been reached.  After reviewing the inputs to the ad-hoc the options were to recommend:

1.  no change
2.  removal of MBA stuffing
3.  the introduction of a per frame limit
4.  the introduction of a per macroblock limit.

It was evident that there was no single choice which was optimum for all encoders and decoders.  However, after ascertaining that zero bit stuffing before start codes constituted a viable alternative for encoders, we agreed to recommend the removal of the MBA stuffing codeword from the MPEG-2 syntax.

Universal VLC

We had studied the MUVLC at the Brazil and London meetings and recommended that it not be incorporated into MPEG-2.  Mr. Descamps presented document MPEG 93/12 in an attempt to change our minds.  The ensuing discussion confirmed our earlier views that this VLC method has serious disadvantages (number of gates and memory size/bandwidth) compared to that already in the MPEG-2 Test Model.

Prediction modes

We noted that the Dual Prime prediction method has the highest memory bandwidth, being essentially equivalent to reading for B pictures plus a write back (which is not required for B pictures).  However, we were aware that the method gives significant coding performance benefit in applications which are delay sensitive and thus did not wish to reject it.

Our initial conclusions were that reverse order prediction increased delay, complicated memory management, had an implementation dependent effect on memory size and we did not support it.  After making these known in a joint session we were requested to revisit the issue by Dimitris Anastassiou.  Further examination revealed that the extra delay and extra memory are only required when M=1.  Ironically it also is in this case that the performance gains come.

Document MPEG 93/33 showed the increases in memory bandwidth incurred by 8 by 8 prediction modes.  We recommended that 8 by 8 and 16 by 8 frame modes not be included in the main profile.

Adaptive coefficient VLCs

There was some initial concern that switching between VLC sets based on the previous coefficient might have latency implications in some implementations giving timing problems if switching frequently between short codewords.  However, switching would be taking place because the codeword used was longer than it could be.  Thus frequent switching and short codes would not be encountered together.

Of more concern was the possible increase in table size but this was thought to be still acceptable provided they could be in ROM.  It was pointed out that start code emulation should be checked for any new VLC proposals.

Spatial-temporal weighting

This refers to the prediction in the upper layer(s) of the spatial scalability/compatibility algorithm.  We agreed that the simple mixing of the upsampled lower layer and the motion compensated current layer is trivial from the computational requirements.

We reported to the Video Subgroup that they might wish to consider reverting to a switch between (rather than a mix of) the two sources to reduce implementation costs.  The savings would be two-fold.  First, only either the lower or current layer would need to be extracted from memory thus reducing memory bandwidth.  Secondly, the operations of half pel interpolation on the current layer and upsampling of the lower layer are very similar, so that if they were not both required for a macroblock, hardware could be shared between these two functions.

Intra motion vectors

We had no reason to change our recommendation at the London meeting concerning the provision of motion vectors for intra macroblocks to aid error concealment at decoders.

4:2:0/4:2:2/4:4:4 compatibility

We had sympathy with the proposal in document MPEG 93/29 to provide compatibility between the colour formats by spatial scalability.  The reasons were that it retains an 8 by 8 IDCT in all cases and has very little impact on a 4:2:0 decoder (which is perceived as being the dominant type, at least initially).  We did have some concern that a more general descriptor was required to cover a wider range of scalability possibilities (not just colour differences) than considered in the above contribution.

We were not in favour of an alternative proposal, effectively a 2 band subband approach which required a 4:2:0 decoder to parse 4:2:2 encoded video.

We considered a proposal to have a flag indicating intra or inter field vertical downsampling of colour components, not to be an issue of any importance to us.

Non-transform coding

We did not support the scheme in document MPEG 93/181 as it introduced complex flow control and although the computations are easy the process is register intensive requiring on-chip cache.  In reply to a suggestion that the DCT transpose memory could be used, it was agreed that the latter is highly optimised and not suitable.

Support for fast forward and fast reverse

Document MPEG 93/31 (Revision 22 Jan 93) proposed a broadcast profile which enables random access and fast forward/reverse play mode with storage media.  Pointers are inserted in slice headers to intra coded macroblocks.  We did not favour this for two reasons.  Firstly, the technique complicates encoders because the pointer value cannot be known at slice header construction time and must therefore be filled in later.  Secondly, equivalent functionality can be achieved on any bitstream by adding a small (and potentially low cost) amount of MPEG decoding to the storage media player.

Data partitioning and frequency scalability

We were joined by Eric Viscito, Gunter Schamel and Thomas Sikora to consider implementation aspects of frequency scalability.  They outlined two schemes:

1.	Data partitioning, in which coded transform coefficients were separated into two or more sets, the break point being signalled in the slice header.
2.	Coefficient refinement.  Some coefficients appear twice being first quantised with a coarser quantiser and then the error quantised more finely.

A potential difficulty with layered schemes was pointed out by some VLSI implementers.  The assumption that the variable length decoding of two streams was no more demanding that the variable length decoding of one of the same total bit rate was challenged.  The issue relates to the on-chip caches required between the external coded data buffer and the variable length decoder to handle the very high peak rates which can arise.  The sum of the peak rates of multiple streams may exceed the peak rate of a single one, thus increasing the cache sizes.  Because the coded data buffer is in the same physical memory as the frame buffers the caches for these must be larger too.  Thus there is likely to be a significant impact on the design of the total memory interface.  This is already one of the most challenging areas in MPEG-2 implementations.

Because of the above we recommended that if video data partitioning becomes a part of MPEG-2, main profile decoders be permitted to decode only the first partition and that the partitions be multiplexed at a system level.  The latter relieves a decoder from parsing the entire video data to extract just some of it.

We noted that the coefficient refinement algorithm incurs the above plus increased inverse quantisation processing.

Flexibility of picture size

We agreed with the MPEG-1 concept that flexibility of picture size up to some maximum caused minimal and acceptable additional decoder complexity.  (This comment applies only to the decoding.  Handling the display of flexible picture sizes is another matter.)

Leaky prediction

We were not so concerned about the multiplication (which could be a shift and subtract) to introduce the leak as the fixes for the limit cycle problem.  As this did not seem to have been resolved by the Video subgroup we deferred further consideration.

Coded data buffer size

We recommended that the Video Subgroup investigate video buffer sizes and provide figures with evidence that they are the minimum acceptable.

Maximum bit rate

A figure of 15 Mbit/s was thought reasonable on processing speed grounds but the buffer size may also be linked to the coded bit rate.  There is a desire to keep memory size down, especially for main profile decoders.  Therefore we reserved our position until the Video Subgroup has studied buffer sizes.

Coded Block Pattern

Document 93/189 made a proposal to extend the Code Block Pattern to cover 4:2:2 and 4:4:4.  While this was acceptable to us in itself, the overall approach does not give the downward compatibility of MPEG 93/29.

Document MPEG 93/11 pointed out the inconsistency between MPEG block numbering (for 4:2:2 and 4:4:4) and JPEG's MDU and urged that MPEG change.  This prompted other participants to defend the MPEG numbering with other valid implementation arguments.  We decided not to request a change.

Motion vector range

We recommended that the range of vertical motion vectors in the main profile be restricted to about +/-128 pels (with half pel resolution).  No restriction in the horizontal direction is necessary as there is no implementation advantage.

Motion vector coding

We agreed to recommend changes to the coding of motion vectors

-	use of intra DC VLC
-	removal of 'f' codes
-	removal of half pel flags
-	addition at sequence layer of indications of maximum horizontal and vertical motion vectors

Variable bit rate operation

We did not perceive variable bit rate operation to introduce any additional processing difficulties provided the peak bit rate is consistent with the constant bit rate case.  However, we were not certain that decoder synchronisation, buffer management, etc were adequately investigated and specified by the Video and System Subgroups to support variable bit rate.

5. SYSTEMS sub-group (B.G. Haskell, C.J. McGrath)