Annex 4 to AVC-578R

INTERNATIONAL TELECOMMUNICATION UNION	Temporary Document 42 + 77(15/1)
TELECOMMUNICATION STANDARDIZATION SECTOR	Revised
STUDY PERIOD 1993-1996

Geneva, 7-17 September 1993
Questions: 2/15, 3/15

SOURCE:	RAPPORTEUR FOR Q.2/15
TITLE :	FIFTH PROGRESS REPORT OF THE EXPERTS GROUP FOR ATM VIDEO CODING
----------------

1. General

We met four times since we had presented the fourth progress report (Annex 4 to COM XV-R 105-E) at the previous Working Party XV/1 meeting in November 1992;

Meeting	Sole Sessions	Joint sessions with MPEG	
10th meeting in Italy	21-22 January 1993 in Torino at the kind invitation of CSELT	25-29 January 1993 in Rome at the kind invitation of the Ministry of Posts and Telecommunications	
11th meeting in Australia	5-7 April in Melbourne at the kind invitation of Telecom Australia	29 March - 2 April 1993 in Sydney at the kind invitation of Standards Australia	
12th meeting in USA	7-9 July 1993 in Boston at the kind invitation of PictureTel	12-16 July 1993 in New York at the kind invitation of Columbia University	
13th meeting in Belgium	2-3 September 1993 in Brussels at the kind invitation of BELGACOM	6-10 September 1993 in Brussels at the kind invitation of Commission of the European Communities, DGXIIIB - Advanced Communication	

Our meetings have been synchronized with those of ISO/IEC JTC1/SC29/WG11 (MPEG) since May 1991.  We have held joint session with MPEG as well as ITU-T sole sessions.  Lists of participants of the four meetings appear in Annex to this report (omitted here).

This document reports major achievements toward defining Recommendation H.26X for video coding in the ATM environments and its surrounding network aspects, and particular items for the consideration of Study Group 15.

2. Overall work plan

Recommendation H.26X is intended to provide a generic video coding standard which serves various services such as communicative, distributive, retrieval and messaging services in ATM environments.  SG15 decided that this Recommendation should be developed as common text standard H.26X|MPEG-2 between ITU-T and ISO/IEC.

The current work is proceeding as planned; technical specifications were frozen in July 1993 toward final approval of the Recommendation early 1995.  See Figure 1 below for more details.

3. Source coding

3.1 Test Model development

Test Model is a reference video coding scheme against which proposed improvements are tested.  If test results are promising, those improvements are incorporated into the next generation Test Model for further comparison.

Figure 1  Approval process of Recommendation H.26X|MPEG-2 standard

Since the first Test Model (TM1) had been defined in March 1992, it evolved into TM5 in April 1993.  This TM document includes definition of the coding algorithm as well as necessary experiments .

3.2 Basic coding algorithm

The basic coding algorithm is a hybrid one consisting of motion compensated interframe prediction to remove temporal redundancy and DCT to remove spatial redundancy.  Motion compensated interframe prediction includes both of forward and backward predictions.  This algorithm has been proven effective for wide range of picture formats and bit rates.

3.3 Improvements of coding efficiency

Particular efforts have been made to improve coding efficiency for interlaced signals which are common in the current television broadcasting.  The following is a list of such techniques adopted in the standard;

-	field/frame adaptive motion compensated prediction,
-	field/frame adaptive DCT,
-	special prediction (dual-prime).

In the quantization process, a new non-linear quantizing step size assignment has been defined to allow finer quantization control for high quality coding.  A new VLC table has been adopted for intra pictures, and a new DCT coefficient scanning method has also been adopted to cope with field pictures.

3.4 Functionalities

In order to make the generic coding standard applicable to as many applications as possible, the following functionalities have been incorporated;

	1) Multiple formats

The coding algorithm can handle interlaced or progressive formats of various sizes.

	2) Low delay operational mode

A cyclic refreshing method called "intra-slice" has been devised to reduce buffering delay.  This technique refreshes one or more strips of macroblocks at a regular interval.  Picture skipping is also introduced to cope with large pictures containing a number of bits due to e.g. a scene change.  Special prediction (dual-prime) is effective for the low delay operation.

	3) Random access/channel hopping

Fast access can be implemented by selecting refresh parameters.

	4) Scalability

Resolution, SNR and temporal scales are supported by spatio-temporal prediction and re-quantization.  The resolution scalability technique uses the enlarged low resolution picture as a possible prediction mode.  Hence if the decoder uses only the base layer bitstream, small size pictures (quarter size in the Example of Figure 2) are reproduced while the two bitstreams are used together, full size pictures are obtained.  If the base layer uses existing coding standards and formats, then this spatial scalability techniques provides backward capability if multiple bitstreams are supported by the system.

Figure 2  Spatial resolution scalability

SNR scalability provides two different versions of the same spatial resolution picture with different level of quantizing noises.  This may facilitate graceful degradation when transmission channels become vulnerable to errors.  The temporal scalability is useful for migration from interlaced television to progressive television.  Prediction errors from the interlaced signal to the progressive signals are transmitted as an enhancement signal to the encoded interlace signal.

	5) Cell loss resilience

The developed coding standard can support modest level of cell loss resilience by selection of appropriate cyclic refresh parameters and slice sizes in addition to concealment techniques which replace damages areas with motion compensated surrounding blocks.  Motion vectors for intra macroblocks can optionally be sent for this purpose.

If we need stronger cell loss resilience for CLR higher than 10-3 (Note), we need some layered coding.  Above mentioned SNR scalability is a tool to achieve this property.  Another tool is data partitioning which split DCT coefficients into high and low priority data.  Low priority data are much less sensitive to cell loss than high priority data.

Note - It should be noted that this figure is nothing more than a very general guide.  Layered coding may be the most appropriate choice in some circumstances at lower CLRs, and the error resilience performance of any system depends on the tools used, application, choice of parameters, detailed error conditions and the picture materials.

3.5 Profiles and Levels

H.26X is intended to be generic in the sense that it can serve a wide range of applications, bit rates, resolutions, qualities and services.  Applications should cover, among other things, digital storage media, television broadcasting and communications.  In the course of creating this standard, various requirements from typical applications have been considered, necessary algorithmic elements have been developed, and they have been integrated into a single syntax so that this standard will facilitate the bitstream interchange among different applications.

Considering practicality of implementing the full specifications of this standard at early stages, however, a limited number of subsets are also stipulated by means of "profile" and "level".  

Table 1  Profile/Level structure of H.26X|MPEG-2 Video
(Upper bounds of parameter values)

	Pels/line	1920	1920	1920	
High	Lines/frame	1152	1152	1152	
(up to 60 Mbit/s)	Frames/sec	60	60	60	
Note 4	Pels/sec	62.7 million	62.7 million	62.7 million	
	Pels/line	1440	1440	1440	
High-1440	Lines/frame	1152	1152	1152	
(up to 60 Mbit/s)	Frames/sec	60	60	60	
Note 4	Pels/sec	47.0 million	47.0 million	47.0 million	
	Pels/line	720	720	720	
Main	Lines/frame	576	576	576	
(up to 15 Mbit/s)	Frames/sec	30	30	30	
Note 4	Pels/sec	10.4 million	10.4 million	11.06 million
(Note 2)	
	Pels/line	352	352		
Low	Lines/frame	288	288	Not decided	
(up to 4 Mbit/s?)	Frames/sec	30	30		
	Pels/sec	2.53 Million	2.53 Million		

Level
(Note 1)	Simple
4:2:0
single layer
(Note 5)	Main
4:2:0
single layer	Next
4:2:2
scalable	
	Profile	

Note 1 -	Level for the Next profile indicates the upper layer of the two resolution scales.
Note 2 -	720x512x30 has been considered to accommodate 483 active lines of 525/60 TV.  The extension of the upper bound of the pel rate to cover half SCIF is an open issue.
Note 3 -	Multiples of 16 up to the upper bound are supported in number of pels per line and number of lines per frame.
Note 4 -	Data rates for other than MP@ML are to be determined .
Note 5 -	Main minus B-pictures
A “profile” is a defined sub-set of the entire bit stream syntax that is defined by this standard.  Within the bounds imposed by the syntax of a given profile it is still possible to require a very large variation in the performance of encoders and decoders depending upon the values taken by parameters in the bit stream.  

In order to deal with this “levels” are defined within each profile.  A level is a defined set of constraints imposed on parameters in the bit stream.  These constraints may be simple limits 

on numbers.  Alternatively they may also take the form of constraints on arithmetic combinations of the parameters.

For each defined subset (a profile at a certain level), conformance testing will also be defined.

The current structure of Profiles and Levels are shown in Table 1 with upper bounds of parameter values.

3.6 VBR

Study on the VBR coding has been continued to investigate its performance under Usage Parameter Control.  One is to experiment on the relationship between leaky bucket counter size and encoder buffer delay and its relevance to cell interleaving method for cell loss correction.  Another is to experiment on the utilization of periodic coding structure (I/B/P pictures) to obtain low delay in the sliding window environments.

SG13 indicated in its liaison response that UPC time constant in the range of seconds or more is not likely implemented in the network.  This UPC time constant affects the VBR effectiveness for video coding; if it is not sufficiently long compared to the picture period, we may not be able to fully exploit the advantages of VBR.

3.7 Working Draft for H.26X|MPEG-2

As the outcome of the New York meeting, the third Working Draft for H.26X|MPEG-2 was produced as in a separate document TD 39 (15/1).

4. System aspects

4.1 Audiovisual system configuration

According to the practice of N-ISDN audiovisual standardization, Recommendations in Figure 3 are assumed.  Dotted partitions between multimedia multiplex and AAL etc. are intended to represent the open nature of the current discussion.

Figure 3  Audiovisual communication terminal and system configuration

4.2 Cell loss correction in AAL Type 1

SG13's liaison response on support of H.320 terminals (which are cell loss critical) in B-ISDN indicated necessity of cell loss correction at bit rates not lower than 384 kbit/s and that requirements from CMTT/3 on high quality sound transmission indicate the same solution for cell loss.  Another factor is that we may need some bit error correction mechanism as well in video signal transport through ATM networks.

An AAL Type 1 solution which uses Forward Error Correction and cell interleaving has been studied both in this group and SG13.  Low delay characteristics have been sought for conversational applications.  As an example, use of Reed Solomon (94,88) and diagonal scanning will give processing delay of 15 ms at 384 kbit/s.

4.3 Multimedia multiplex and video support AAL

In the system model of Figure 3, we are now going to have video coding standard H.26X while ATM layer specifications have already been fixed.  To connect these two layers, we need multimedia multiplex and AAL which should have the following functions;

•	transfer of variable length data units.  Interval between data units may be fixed or variable.
•	multiplexing and synchronization of elementary streams
•	capability to synchronize source and receiver clocks 
•	error detection and/or correction capability

In the H.32X terminal these functions are to be distributed between;

•	H.22X - multimedia multiplexing and synchronization (possible common use of MPEG-2 Systems)
•	I.363 - the ATM Adaptation Layer

As to the multimedia multiplex and synchronization, MPEG-2 "system coding" provides a packet and time-stamp based method.  Each elementary bitstream is segmented into Packetized Elementary Stream (PES), and then respective packets are multiplexed.  MPEG-2 Systems consist of the following syntax;

•	"Program Stream" which is a multiplex of variable length PES packets and is similar to MPEG-1 Systems in functionality.  This is designed for use in error free environments.

•	"Transport Stream" which consists of 188 byte fixed length packets and has functionality of multiple program multiplexing (typically for broadcasting applications, each program having independent clock reference).  This is designed for use in error prone environments.

In the interest of service integration on B-ISDN, it is desirable that multimedia multiplexing is also common among various applications.  WP1/15 already expressed this view at its meeting in November 1992 and MPEG welcomed this policy.

It should be noted, however, that the above mentioned Transport Stream has close relevance to AAL/ATM specifications in functionality.  We are now finding an appropriate combination of part of MPEG-2 Systems, H.22X, and audiovisual support AAL (AAL Type 2 in particular).  The following is considered as alternatives;

1) PES packet + modified AAL Type 4 SAR
Figure 4  PES packet + modified AAL Type 4 SAR sublayer.

2) PES packet + modified Transport Stream packet (H.32X transport packet)+ AAL Type 5 CPCS/SAR

Figure 5  PES packet + modified Transport Stream packet + AAL type 5 CPCS/SAR

3) PES packet + AAL Type 5 CPCS/SAR

4) PES packet + AAL Type 1

4.4 Delivery of MPEG-2 Transport Stream packets

Given that it is required to deliver a Transport Stream across a number of systems, one part of which may be ATM/B-ISDN.  An issue is what is the best way to adapt Transport Stream (TS) packets to the ATM/B-ISDN part of the link.

For the B-ISDN connection, 

•	some error detection/correction capability is required,
•	it is believed that SAR/CS mechanisms common to other AALs should be used if possible. 

----------------------------

AAL	ATM Adaptation Layer	MID	Message Identification
BOM	Beginning of Message	PDU	Protocol Data Unit
COM	Continuation of Message	PES	Packetized Elementary Stream
CPCS	Common Part of Convergence Sublayer	SAR	Segmentation And Reassembly sublayer
CRC	Cyclic Redundancy Check	SN	Sequence Number
EOM	End of Message	ST	Segment Type
LI	Length Indicator

Consideration is given to the use of AAL Type 1 and AAL new Type mechanisms.  The latter example is illustrated in Figure 6.

Figure 6  Transport Stream packet and new AAL.

4.5 Independent clock sources and their recovery at the decoding end

There can be several independent clock sources in the audiovisual terminal.  They should be identified and the recovery method for each source should be clarified.  This is yet to be worked out for the H.32X terminal.

4. 6 Joint meeting with SG13 experts on AAL

A joint meeting with SG13 experts on AAL Type 1 and Type 2 for audiovisual communications has been suggested by Mr. Yamazaki, Rapporteur for AAL Type 1/2 in SG13.  We supported this idea, awaiting detailed arrangements to be settled between the two groups.  The topics will be cell loss correcting AAL Type 1, video support AAL, and TS packet delivery AAL.

5. Video mixing for continuous presence multipoint systems

This discussion is in response to the request of the Tokyo Rapporteur's meeting that the Experts Group be referred to the video aspects of the video mixing bridge proposal.  Our discussion results and some suggestions are summarized in a separate document TD 43 (15/1).

6. Audiovisual communication terminals as extension of H.320

In addition to audiovisual terminal Recommendation H.32X fully exploiting B-ISDN features, we have recognized the following two type of terminals which need early standardization;

•	H.32Y: Adaptation of H.320 terminals to B-ISDN environments
•	H.32Z: Adaptation of H.320 terminals to LAN environments

It is noted that functions of H.32Y should also be supported by H.32X, since H.32X terminals are required to interwork with H.320 terminals which are accommodated in N-ISDN.  This is similar to that H.320 terminals are required to interwork with analog telephones.

Necessity of H.32Z standardization was first discussed at the Rapporteur's meeting in Tokyo last May.  There are now several commercial products or announcements for desktop videoconferencing and videophone which are designed for LANs and WS/PC environments.  A stack of protocols for this purpose is an area of ITU-T SG15 responsibility.  SG15 should consider to provide Recommendations necessary to allow interworking between independently designed desktop videoconferencing and videophone systems over LANs.

At the subsequent meeting of the Experts Group in July 1993, there was a relevant input document discussing necessary functionalities of multimedia multiplexing, and several organizations expressed prompt interests in standardization of this area.  Furthermore a UK contribution at the September meeting proposed that this should be the prime focus of SG15.  We concluded that this Experts Group should draft necessary Recommendations and that time frame should be set for this work, seeking advice of WP1/15.

7. Interaction with other groups

In addition to the joint meeting sessions with MPEG in the area of common interests, the Experts Group had interaction with SG13 through correspondence.

from the Experts Group to Rapporteur for AAL Type 1/2 of SG13

July 1993 meeting sent the following;

•	Cell loss correction method for AAL Type 1 to support H.320 terminals in B-ISDN

from Rapporteurs of SG13 to the Experts Group

January 1993 meeting of SGXVIII (now ITU-T SG13) sent the following;

•	Update of the IVS Baseline Document
•	Request of information on maximum allowable AAL delay for supporting H.320 terminals in B-ISDN by introducing a forward error correction in AAL Type 1.
•	Traffic parameters for average cell rates, particularly measuring time constants of seconds or more
•	Interworking issues when using CLP bit and resource allocation options

June 1993 meeting of SG13 sent the following;

•	AAL Type 1 timing recovery issue
•	An error correction method for delay sensitive signal transport of AAL Type 1
•	IVS Baseline document
•	Updated release time table for B-ISDN standardization

8. Intellectual property

A contribution from FRG reminded the participants of the ITU-T "code of practice" in TSB Circular 6, particularly the disclosure of established or pending patents related to technical proposals.  The same contribution also proposed that the Experts Group should adopt the waiving option for Recommendation H.26X, but we felt that it should first be advised to and consulted with patent experts in each participating organization.

9. Work plan

We confirmed that the Tokyo Rapporteur's meeting had supported to set a target of early 1995 for "minimum system" Rec. to deter from prior divergence; also to show "committee draft" on the plan (May 1994?) to focus attention on preparation of first solid draft.  Here "minimum system" is meant to be a minimum total audiovisual communication system (H.32X) consisting of video/audio coding, multimedia multiplex, communication procedures, etc.

Chairman reminded the members of hardware verification trials which should take place in the final stage of standardization work to secure our Recommendations.  Since video coding/decoding specifications will be verified mostly through the bitstream exchange currently ongoing, our task is to test the above mentioned minimum system between two or more independently designed equipment.  The date could be sometime during 1995.  Interested organizations are requested to express willingness of participation and to contribute to materializing the plan.

10. Future activities (ITU-T sole sessions / Joint sessions with MPEG)

Meeting	Date	Sole sessions	Joint sessions with MPEG	
14th	October - November 1993	October 27-29 in Daejeon
(See AVC-535)	November 1-5 in Seoul	
15th	March 1994	in France (?)	in France	

11. Specific items requiring the consideration of Study Group 15

11.1 Intellectual property processing for H.26X

We owe to MPEG Convenor (Dr. L. Chiariglione) for his efforts to collect patent statements related to H.26X|MPEG-2 at this stage.  More than 30 organizations submitted statements indicating "non-discriminatory licensing on reasonable and fair terms".  These are addressed to MPEG Convenor.

The ITU-T should collect patent statements addressed to Director of ITU-TSB from those organizations by the time when SG15 initiates the approval procedure.  Though Chairman of the Experts Group advised the participants of the necessity at the September joint sessions with MPEG, Study Group 15 is requested to take appropriate actions to secure this statements collection.

11.2 Alignment between ITU-T Recommendations and ISO/IEC standards

The MPEG-2 standards are published in four Parts;

Systems - 	specifies the system coding layer of the Specification.  It defines a multiplexed structure for combining audio and video data and means of representing the timing information needed to replay synchronized sequences in real-time. 

Video - 	specifies the coded representation of video data and the decoding process required to reconstruct pictures. 

Audio - 	specifies the coded representation of audio data for multichannel (five plus one)

Conformance - 	specifies the procedures for determining the characteristics of coded bit streams and for testing compliance with the requirements stated in Systems, Video and Audio.

WP1/15 decided at its November 1992 meeting that the Video Part should be of common text with Recommendation H.26X.  How to handle the Systems, Audio and Conformance Parts should be decided.  MPEG is going to publish all of these in common text format even if ITU-T decides to take another approach.  For the consideration of Study Group 15, Working Drafts for Systems and Audio are also input as separate documents TD 37 (15/1) and TD 38 (15/1).

It is noted that the current Systems include DSM CC (Digital Storage Media Control Commands).  This is a set of commands between DSMs and the user to perform such operations as set up connection, selection of non-multiplexed bitstreams, playback, storage, edit, information request.  In our terms, DSM CC is a set of C&Is.

11.3 Time schedule

It is quite obvious that the video coding cannot stand alone; we need audio coding (G.XXX), multimedia multiplexing (H.22X), communication procedures (H.24X), call control (AV42X), whole terminal (H.32X) etc. as illustrated in Figure 1 to make interoperable communication systems.  WP1/15 made a small step at the November 1992 meeting by declaring that they will be "1994+" Recommendations in the "IVS Baseline Document" edited by SG13.  

One serious difficulty is that B-ISDN is not so visible as N-ISDN, but the other way to look at the situation is that we are in a good position to do the work before incompatible equipment appears in the market and to reflect audiovisual requirements to the network standardization.

As the SG15 meeting schedule is given for 1993-1995, we may be able to produce a WP work plan more precisely; what Recommendations we intend to make and at what time.  We need good target dates for Recommendations including H.32Y and H.32Z described in §6 above.

11.4 Terms of reference for the Experts Group

Currently the Experts Group is charged with mostly pure video coding in its terms of reference.  Considering 

•	that the generic coding Recommendation H.26X is going to become complete on schedule,
•	that the application Recommendation should specify how to use the generic standard,
•	that interworking is not possible without the total system and terminal specifications,

the role of the Experts Group should be reviewed in the light of the WP work plan for this study period and onward.

END