ITU-T TELECOMMUNICATION STANDARDIZATION SECTOR	     Temporary Document   -E (WP3)

STUDY GROUP 16								Original: English
___________________________

Geneva, 26 January-6 February 1998

Questions  :	15/16

SOURCE  :	Rapporteur (Gary Sullivan) and Associate Rapporteur (Keiichi Hibi) for Q.15/16

TITLE :	DRAFT call for proposalS for H.26L video coding

___________
The ITU-T SG16/Q15 ’Advanced Video Coding’ group has been very successful in the development of the H.263 very low bitrate video coding standard and subsequent revisions. The revised Recommendation H.263, known informally as H.263+, which extends Recommendation H.263 by adding more functionalities as well as incorporating coding efficiency improvements, is planned to be approved by the SG16 meeting in January 1998.
However, Q.15/16 recognizes that there is still a potential need for a new video coding algorithm capable of delivering much improved video quality and covering functionalities not supported by existing standards(e.g. H.261, H.262, H.263/the revision of H.263) or standards under development (e.g. MPEG-4).  Accordingly, Q.15/16 has initiated a new generation video coding standardization activity, currently named H.26L, which targets these areas.
Therefore, ITU-T Q.15/16 solicits technical proposals for the new generation H.26L video coding standard.
To achieve this goal, a list of requirements for H.26L has been defined to promote development of this new generation video coding algorithm, such that it should be superior in its performance to, and address areas not covered by, other video coding standards.  These requirements are presented in Annex C.
The enhanced functionality areas which should be supported by H.26L are currently identified as follows:
•	very low bitrate real-time profiles with both enhanced coding efficiency and low end-to-end delay
•	low complexity encoder/decoder permitting useful software only implementations
•	enhanced error robustness, including the needs of 3rd generation mobile networks
•	adaptable rate control mechanisms, for enhanced temporal and/or spatial resolution depending on application
•	applicability to various kind of source materials, such as fast motion, multiple scene changes, pans and tilts etc.
Although the H.26L standard should cover various types of application, such as both real time and non-real time applications, a generic single video codec solution applicable to a wide range of application is being sought. 
Possible applications of H.26L include:
•	real-time conversational services (e.g. video phone)
•	audiovisual communication over mobile networks
•	Internet video applications
•	video transmission for real-time sign language and lip-reading communication
•	video storage and retrieval services (e.g. video on demand)
•	video store and forward services (e.g. video mail)
•	multi-point communication over heterogeneous network environments
The detailed requirements for H.26L video coding are documented in the H.26L Requirements Document provided in Annex C. Although the final H.26L standard should meet all the requirements described in the requirements document, each technical proposal for H.26L need not fulfill all the requirements.
It is expected that activities in the ‘Advanced Video Coding’ group relevant to H.26L will integrate the proposals received into a single ‘super algorithm’.  This will be achieved by defining a ‘Test Model Long Term’ as the basis of the standard, with collaborative improvements being made by incorporation of innovative aspects of proposals received.  This type of effort can fully utilize the benefit of members’ technical cooperation and expertise, and has been used with great success in the development of the previous ITU-T standards.
At the Q.15/16 Rapporteurs group meeting in November 1998, all proposals received in response to this call will be technically evaluated. The first version of the Test Model of H.26L will then be defined.
Proposals for H.26L will be evaluated under the common test conditions described in Annex A to this document.  The detailed evaluation method will be defined later.  Proposers shall provide enough information concerning their proposed technology for evaluation by the Q.15/16 experts, e.g. a full technical description, a video demonstration of decoded sequences, etc. It is stressed that the evaluation of delay characteristics has very high importance to achieve the low delay features desired for H.26L. Proposers shall provide adequate information for evaluating delay and are strongly encouraged to provide information illustrating delay evaluation results based on the delay model defined in Annex B to this document.
The time schedule of H.26L development is given below.  Since the meeting dates of SG16 in 1999 and 2000 shown below are tentative, the time schedule may be slightly modified according to the SG16 meeting schedule actually adopted by ITU-TSB.  Two Rapporteurs group meetings are expected between SG meetings.
(1)	Call for proposal issued, January 1998 (SG16)
(2)	Initiate evaluation of the proposed technologies, November 1998
	A decision will be made at this time whether the proposed technologies have the potential to meet the requirements after the collaborative effort for improvement by the group.  If not, the schedule will be delayed.
(3)	The first Test Model of H.26L (TML1?) defined, April 1999
	The project steps into the collaboration phase towards Determination.
	Note: future dates may be subject to change based on an analysis of the suitability of technology proposals.
(4)	Determination, October 2001 (SG16)
	The result of collaborative improvement will be evaluated again at this point.  If it is decided that the requirements are not met, Determination will be delayed to a later date.
(5)	Decision, July 2002 (SG16)
It should be noted that ITU-T will always be open for new proposals submitted after the deadline specified in this call for proposals.  Such proposals will be evaluated under the collaborative phase of work and/or considered as candidate technologies for the work items in the next study period.
Questions for clarification or discussions regarding this project should be raised on the Q15 email reflector (itu-adv-video@listserv.iterated.com).  Subscription or “unsubscription” messages should be sent to itu-adv-video-request@listserv.iterated.com.  Further information may be found at the group ftp site: ftp://standard.pictel.com/video-site.

Annex A - Simulation Conditions for H26L Proposal.
_____________________________
1.	Introduction
This document contains a set of suggested common conditions for H.263ver3 and/or H.26L experiments. Those submitting proposals for H.263ver3 and/or H.26L are strongly encouraged to provide material using the conditions specified in this document to ease comparison so making their contribution more valuable.  These are considered as a minimum recommended set of sequences, bit-rates, frame rates, and spatial resolutions that each submission demonstration should contain. Those making a submission are free to provide additional demonstration material as appropriate. It is also expected that all results are derived from decoded bit-streams.
Suggested common conditions described in this document are subject to revision through discussion be made in Q.15 Rapporteur group, to make better evaluation of proposals against requirements for H.26L. The revised set of common conditions for H.26L experiments will be noted in the meeting report of any meeting at which such a revision is made, and is likely to also be discussed on the group email reflector.
2.	Common Conditions for INTER-Coding Experiments
The tests are performed using objective comparisons employing measures such as PSNR and bit-rate as well as subjective comparisons employing D-1 tape demonstrations. 
To provide a common starting point, all sequences should be downsampled to CIF or QCIF (as appropriate) by using the filter program developed for MPEG-4.  This will ensure that all submissions begin with the same source material.
The exact conditions described in this document shall be used when encoding anchor material.  The anchor shall consist of H.263ver2 with annexes: D (unrestricted MVs),  F (advanced prediction), and J (Deblocking filter) enabled.  All other annexes should be disabled. The bit-streams, PSNR values and bit-rates for the anchor will be up-loaded 3 weeks prior to the Rapporteurs Tampere meeting (April 21 to 24, 1998) to the ftp site: standard.pictel.com. These will be produced by the Test Model software that is provided by Peter List from Deutsche Telekom. Those making a submission are strongly encouraged to down-load these sequences and to decode them to provide the anchor for both objective and subjective comparisons.  For this purpose a decoder will be provided on the same ftp site.
For INTER-coding experiments the INTRA macroblock quantizer for the first frame is set QINTRA = 16 (using the H.263 coder in baseline mode). The INTER quantizer value is held constant throughout the entire sequence. The frame rate is kept constant over the complete sequence. These settings have been chosen so as to minimize the impacts of the various parameters involved.
In case the core experiment involves PB or B-frame coding, the corresponding PB- or B-macroblock quantizer is lowered by 2, i.e., QB = QPB = QINTER – 2.
2.1	Objective Comparisons 
For objective comparisons, parameterized rate distortion curves shall be plotted with the corresponding tables given in the documents as well. These curves show PSNR versus bit-rate measured for the complete bit-stream. PSNR is measured as the arithmetic mean of the PSNR values of each frame except the first INTRA frame. Bit-rate is measured as the average of all bits per frame except for the first INTRA frame multiplied by 30/frameskip (kbit/s). The plots shall have 0.5 dB grid lines on the PSNR axis.
A first set of rate distortion curves is generated by varying the INTER-macroblock quantizer over values as follows:
QINTER	4, 5, 7, 10, 15, 25	4, 5, 7, 10, 15, 25	
Frame Skip	2	1	
Resolution	QCIF	CIF	
Sequences	Mother and Daughter
Hall Objects
Container Ship
Silent Voice
News
Foreman	Foreman
Paris
Silent Voice	

A second set of rate distortion curves shall be presented by varying the frame skip using the following conditions:
QINTER	10	
Frame Skip	0, 1, 2, 3	
Resolution	QCIF	
Sequences	Mother and Daughter1
Silent Voice	

2.2	Subjective Comparisons 
The demonstration material should be presented on a D1 tape as a CIF image displayed side-by-side with a CIF image generated by the Test Model.  Images coded at QCIF resolution should be up-sampled to CIF (using the MPEG-4 up-sampling filter) for display.  These two images should be centered in a CCIR 601 size display.
The subjective tests shall be performed based on the following specifications:
Bit-rate (kbps)	About 10	About 243	About 1123	
Spatial Resolution	QCIF	QCIF	CIF	
Frame Skip	3	2	1	
Sequences	Mother and Daughter1
Hall Objects
Container Ship	Silent Voice
News
Foreman	Foreman
Paris2
Silent Voice	

In addition to this mandatory demonstration mode, those submitting demonstration material are encouraged also to present that material on alternative display technologies representative of those likely to be used in commercial products (e.g. on desktop and/or laptop computer systems).
2.3	Additional Aspects
Additional sequences, bitrates, and/or frame rates are also welcome.  Skipped frames are acceptable (so long as the sequences are realistically displayed and statistics are provides) and variable frame rate demonstration sequences are welcome.
A paper describing the tape demonstration is expected.  This paper should also identify any special features of the demonstration material such as frame skipping, degree of jitter, degree of delay, frame rate variability, etc.
For each sequence, the submitter should provide plots of the PSNR per coded frame obtained from the proposed algorithm and from the anchor.  Optionally, a submitter may provide plots of the PSNR per frame obtained when the coded sequence is padded out to the final frame rate (for a discussion of frame padding see LBC-97-016).
For each sequence the submitter should provide information on the number of bits used to code each frame.  If a plot of bits per frame is provided, the submitter should produce two plots, one which includes the bits spent on the first frame and another where the first frame bits are not included.  This is required since the large number of bits used to code the first frame tend to mask the variation in bits used to code the predictive frames.
In the context of the first frame proposers should note that at the December 1997 Rapporteurs meeting in Eibsee the Q15 experts group agreed that the bit allocation to the first frame should be limited to no more than the channel bitrate integrated over one second.  This is intended, particularly for the case of short test sequences, to prevent the effect of a very high quality initial frame unduly biasing the quality measures for the remainder of the sequence.

Annex B - H.26L Evaluation Delay Model User Guide

Contents List
1. Introduction	
1.1 Purpose of the Document	
1.2 Applicability of the Model	
1.3 Nature of the model	
1.4 Use of the model	
2. Definition of Terms	
2.1 Notation	
2.2 Special Terms	
3. The Model	
3.1 Underlying Assumptions and Constraints	
3.1.1 Assumptions	
3.1.2 Constraints	
3.1.2.1 Bit Allocation to the First Frame	
3.1.2.2 Initial Delay Value	
3.2 Components of Delay	
3.2.1 Encoder Delay    DE	
3.2.1.1 Encoder Processing Delay  DEP(m)	
3.2.1.2 Encoder-Wait-For-Data DEW(m)	
3.2.1.3 Total Encoder Delay DE	
3.2.2 Channel Delay DC	
3.2.2.1 Discussion	
3.2.2.2 Scheme 1	
3.2.2.3 Scheme 2	
3.2.3 Decoder Delay DD	
3.2.3.1 Discussion	
3.2.3.2 Decoder-Wait-For-Data DDW	
3.2.3.3 Decoder-Wait-For-Display DDD	
3.2.4 Total Delay	
4. Computational Operations — The General Case	
4.1 Discussion	
4.2 Log of Initial Data	
4.3 The Computational Process	
4.3.1 Initialization of variables for the first frame	
4.3.2 Pass 1 — Encoder Delay Computation	
4.3.3 Pass 2 — Channel Delay Computation	
4.3.4 Pass 3 — Decoder Delay Computation	
4.3.4.1	
5. Computational Operations — The Minimum Delay Case	
6. Data Presentation	

3.	Introduction
3.1	Purpose of the Document
This document describes, and sets out the means of use of, the Delay Model adopted by Q15 Coding Experts for generation of frame-on-frame delay values for proposed algorithms when used in hypothetical implementation contexts.  The model thus provides an opportunity for both proposers and evaluators to judge the delay characteristics of codecs when applied to test sequences under conditions comparable to production implementations.
3.2	Applicability of the Model
Use of this model is not mandatory for proposers of algorithms in response to the call for proposals.  However use of the model is strongly encouraged, especially for proposals/demonstrations that address the low delay scenarios in the call.  Proposers electing not to use this model should be prepared to justify claims of delay performance by other means if called upon to do so during algorithm evaluation.  For algorithms selected for more detailed examination following preliminary evaluation, use of the model may be made mandatory.
3.3	Nature of the model
The model consists of a number of additive terms each arising from a specific aspect of codec design.  Where such terms are dependent upon other terms the dependency is made explicit and integrated versions presented to allow computation.
Some parameters must be specified by the proposer since they involve specialized knowledge of the proposed codec.  Such parameters are identified and their characteristics specified.
3.4	Use of the model
The mode of use of the model is presented in Section 6 on page 1.  A technical justification is presented in Section 5, commencing on page 1.
4.	Definition of Terms
4.1	Notation
Term	Interpretation	
	A factor in the range   0    1   which enables the computation of  (q.v.)	
	A symbol denoting the adopted value of DDD — see Section 5.2.3.3	
	A constant power value used in computing one of the two alternative bit generation profiles contained in the model.  See Section 5.2.2.2	
S(i)	the time of commencement of processing of frame i 	
B(i)	The number of bits in the channel input buffer at the time of arrival of the final bit in the encoding of frame i.  The time taken for this bit to be clocked out of the channel input buffer and into the channel output buffer, through an assumed zero dwell time channel, is the component of total delay introduced by the channel.	
C(i)	the number of bits in the buffer at S(i)	
DEP(m)	Encoder-Processing-Delay.  The delay arising from the processing associated with transformation of the bits representing the version of frame m as acquired from the source sequence to the encoded version presented to the channel for transmission.	
DEW(m)	Encoder-Wait-For-Data.  The delay arising from a need to wait for the acquisition and processing of a frame that occurs later in the source stream than does the current frame m before encoding of m can take place.  This can occur where forward reference is included in the encoding of m.  Where such reference does not occur then DEW(m) is zero ; negative values are forbidden.	
k	A proportionality constant used in computing one of the two alternative encoder bit generation profiles contained in the model.  See Section 5.2.2.2	
m	The current frame.  More generally a bold, lower-case character denotes a corresponding frame.	
M{p}	A function that adopts the value of its parameter p when p is positive and adopts the value zero otherwise.  A compact way of denoting the function MAX{p,0}	
N(m)	The total number of bits allocated by the encoder to frame m.	
N(q)	The number of bits in the channel output buffer at time q for one of the two alternative encoder bit generation profiles contained in the model — see Section 5.2.2.2	
n(t)	The number of bits allocated to the encoding of the current frame by time t following the commencement of encoding of that frame; the integral of n’(t) q.v.  This function is used in computing the consequences of one of the two alternative encoder bit generation profiles contained in the model.  See Section 5.2.2.2	
n’(t)	The rate of bit generation by the encoder at time t since the commencement of encoding the current frame; the differential of n(t) q.v. This function is one of the two alternative bit generation profiles contained in the model.  See Section 5.2.2.2	
Q	The time when, during the encoding of a frame under ‘Scheme 1’, the rate of channel input buffer loading [by bit delivery from the encoder] equals the rate of emptying [by bit removal by the channel for transmission] — see Section 5.2.2.2	
R	The bitrate of the channel.  A constant.	
S(m)	The duration of the period during which the encoding of frame m would take place for reasonable assumptions regarding the implementation technology for a product based on the proposal.  This must be specified by the proposer on a frame by frame basis since it will be heavily dependent on the codec structure and the assumed implementation technology.  S(m) does not refer to the duration of the encoding period for the simulation used to generate data in support of the proposal.	
t	The time since commencement of encoding of the current frame [as used in the computation of Channel Delay — see Section 5.2.2]	
TD(i)	The time of completion of decode of frame i taking account of accumulated delay due to all causes up to and including this time	
Tout(i)	The time of presentation of frame i to the display	
tS(i)	The time of occurrence of frame i within the source sequence.  Since transfer of an acquired frame from the frame grabber to the encoder is assumed to be instantaneous, this term also denotes the time of acquisition of the frame.	

4.2	Special Terms
Term	Definition	
Channel Delay
DC(m)	The delay arising from the need to transmit pre-loaded (into the channel input buffer) bits before transmitting the final bit of the encoding of frame m 	
Decoder Delay
DD	The delay arising from a need to redistribute decoded frames in time relative to their times of availability to the display — see Section 5.2.3	
Decoder-Wait-For-Data
DDW	The delay arising from a need for the decoder to wait for the compressed representation of a frame which arrives later in the encoded data-stream than does the representation of the current frame.	
Decoder-Wait-For-Display
DDD	The delay arising from a need to temporally distribute the decoded versions of frames which may be available at the decoder, following decompression, out of source order or inappropriately separated in time.	
Encoder Delay
DE	The delay arising from the act of encoding a frame and of waiting for additional frames upon which that encoding depends — see Section 5.2.1	
Encoder-Processing-Delay
DEP(m)	The delay arising from the processing associated with transformation of the bits representing the version of frame m as acquired from the source sequence to the encoded version presented to the channel for transmission.	
Encoder-Wait-For-Data
DEW(m)	The delay arising from a need to wait for the acquisition and processing of a frame that occurs later in the source stream than does the current frame m before encoding of m can take place.  This can occur where forward reference is included in the encoding of m.  Where such reference does not occur then DEW(m) is zero ; negative values are forbidden.	
Forward Reference	for the purposes of this document ‘forward reference’ is taken to mean reference in the encoding of the current frame to one or more frames that occur later in the source sequence	

5.	The Model
5.1	Underlying Assumptions and Constraints
5.1.1	Assumptions
The model makes the following simplifying assumptions :
1.	Frame acquisition is instantaneous.
2.	The channel input buffer is of unlimited size;  bits are clocked out of it at the rate set by the assumed channel without any form of re-arrangement or processing.  Alternatively, this assumption may be phrased as the assumption that buffer control has already limited buffer contents to reasonable values; it is not a function of the model to control buffers.
3.	The delay between a bit being clocked out of the channel input buffer and it being clocked into the channel output buffer is zero, i.e. the “channel delay” component of the model is entirely due to buffer activity.
4.	The channel output buffer is of unlimited size, i.e. the model assumes that the decoder can handle all data streams delivered by the channel.
5.	Decoding of received data is instantaneous; a received frame is available for display at the instant that the last bit in the bitstream that has bearing on the decoding of that frame is clocked out of the channel output buffer to the decoder [due to the instantaneous decode this is equivalent to the instant that that bit is clocked into the buffer from the channel].
6.	Transfer to the display of a decoded frame is instantaneous and may take place at any convenient time following, or coincident with, the time of decoding.
5.1.2	Constraints
5.1.2.1	Bit Allocation to the First Frame
The bit allocation to the first frame of a test sequence is under the direct control of the proposer within the following constraints
•	The bit allocation must not be greater than one second’s worth of bit transmission at the assumed channel bitrate.  e.g. for a nominal 24kbps channel the bit allocation to the first frame cannot exceed 24 kilo bits, for a nominal channel bitrate of 10.0kbps the first frame cannot exceed 10kbits, etc.  This bit allocation must be announced as part of the supporting documentation for a submission.
•	The bit allocation to the first frame is included in the total bit allocation to the sequence [i.e. channel bitrate multiplied by sequence duration].
5.1.2.2	Initial Delay Value
The channel input buffer, at the start of the sequence, shall contain the bit allocation to the first frame.
The assumed processing time for the first frame shall be zero,  i.e. the Encoder Delay for the first frame shall be taken to be zero.
5.2	Components of Delay
5.2.1	Encoder Delay   	DE
5.2.1.1	Encoder Processing Delay 	DEP(m)
DEP(m) is the delay arising from the processing associated with transformation of the bits representing the version of frame m as acquired from the source sequence to the encoded version presented to the channel for transmission.
It is not possible for any model intended to be capable of representing arbitrary codecs to have knowledge of this value.  Each codec will have a different range of values dependent, e.g., on codec complexity or on the nature of the frame where a variety exists.  Within the model the value of this delay is denoted by S(m) where m designates the current frame under consideration.  For each m, S(m) must be specified by the proposer using any appropriate generation function.
Thus
	DEP(m)  S(m)	Equation 1
N.B.	S(m) should be estimated in such a way that it represents the value reasonably to be expected from an implementation intended for use in real applications.  Proposers may be called upon to justify the assigned values during proposal evaluation.
5.2.1.2	Encoder-Wait-For-Data	DEW(m)
This is the delay arising from a need to wait for the acquisition and processing of a frame that occurs later in the source stream than does the current frame m before encoding of m can take place.  This can occur where forward reference is included in the encoding of m.  Where such reference does not occur then DEW(m) is zero ; negative values are forbidden.
Let the latest frame that is required to provide input to m be n.  This delay component can thus be represented by
	DEW(m)  M{[S(n) + DEP(n)] - tS(m)}	Equation 2
where	tS(m) is the time of acquisition of m 
and 	S(n) is the time of commencement of processing of n 
Note that, in Equation 2,  tS(n) should not be substituted for S(n) since there is no guarantee that encoding can be commenced at that time.  All that can be stated is that  S(n) tS(n).  Note also that, even without forward reference, the time of commencement of encoding of a frame can be delayed beyond its acquisition time.
In Equation 2 the function M{} ensures that the delay is positive or zero; the value [tS(n) + DEP(n)] is the time at which the encoded version of n is available.
5.2.1.3	Total Encoder Delay	DE
The total encoder delay is the sum of the above two terms, i.e.
DE(m)  DEP(m) + DEW(m) 
			= S(m) + M{[S(n) + S(n)] - tS(m)}	Equation 3
5.2.2	Channel Delay	DC
5.2.2.1	Discussion
Channel delay is entirely a function of the buffer contents B(m) when the last bit of the frame encoding is loaded into it.  This situation arises from the assumption that completion of decode is not possible before all bits relevant to the frame are available to the decoder.  If partial decode can take place before reception of all the relevant bits then the assumption of instantaneous decode translates to one of instantaneous completion of decoding on reception of the final bit.
There is thus an inescapable delay, which may be zero for an empty buffer, arising from the need for pre-loaded data bits to be transmitted by the channel before the critical final bit of the frame.
On this basis the channel delay can be seen to be given by :
	DC(m)  B(m)/R	Equation 4
It is therefore of high importance to obtain appropriate estimates of B(m) for each m.
The model provides two methods for the computation of B(m), the second being a special case of the first.  These are described below.  Proposers should select the method that best represents their own codec, or should specify (and justify) an alternate method more suited to representing their codec.
5.2.2.2	Scheme 1
This scheme is intended to permit the approximation of a wide range of encoder strategies in terms of the rate of delivery of bits to the channel output buffer during the period of compression.
Specifically, the rate of bit production n’(t) at time t following commencement of encoding is defined by
	n’(t)  k.t	Equation 5
where n’(t) is the rate of bit generation at time t
	k is a proportionality constant 
	t is the elapsed time since the commencement of encoding of the frame
and	 is a constant power value.
A high value of  causes the rate of generation to increase sharply with time, a zero value corresponds to a constant generation rate, and a value in the range  ‑10 corresponds to an early peak followed by a monotonic decline.  The expression therefore allows a range of encoder strategies to be approximated.
Integration of equation (5) indicates that
	n(t)  =  k.t(+1)/(+1)	Equation 6
But at t = S(m)  n(t) must be equal to N(m), the total bit allocation to the frame
Hence
			k  = N(m).( +1)/S(m)(+1)	Equation 7
and therefore	n’(t)  =  N(m).( +1).t/S(m)(+1)	Equation 8
The rate of augmentation of the channel input buffer is given by [n’(t)-R] the difference in loading and emptying rates.  
The number N(q) of bits in the channel output buffer at time q is in principle given by		N(q) = C(m) + 	Equation 9
	= C(m) + k.q(+1)/(+1)  - R.q	Equation 10
where  C(m)	is the number of bits in the buffer at S(m), the time of commencement of encoding of frame m
Note, however, that for certain combinations of the various factors N(q) can be predicted to be negative over an interval, a situation that is impossible in practice.  During such intervals the buffer is in fact empty with the rate of loading less than the channel bitrate.
Two situations of interest exist :
1.	The nominal buffer contents never go negative
2.	The nominal contents do go negative
These may be distinguished by examining the sign of N(q) at the instant Q when the rate of loading equals the rate of emptying, i.e.  when  n’(Q) = R
At this time 			k.Q  = R	Equation 11
thus			Q  = (R/k)1/  	Equation 12
By substitution of Q for q in Equation 10 the sign of N(Q) may be determined.  Since Q is the instant at which, for negative N(Q), buffer augmentation begins to dominate over buffer depletion B(m) may be seen to be given by :
	B(m) = C(m) + 
	= C(m) + N(m) - S(m).R	N(Q) > 0	Equation 13
and	B(m) = 	
	= (k/(+1)).(S(m)(+1) - Q(+1)) - R.(S(m) - Q)	 N(Q)  0	Equation 14
Hence, from Equation 4
	DC(m) = [C(m) + N(m) - S(m).R ]/R	N(Q) > 0	Equation 15
and
	DC(m) = [(k/(+1)).(S(m)(+1) - Q(+1)) - R.(S(m) - Q)]/R
	= [N(m).[1 - (Q/S(m)) (+1)] - R.(S(m) - Q)]/R	N(Q) < 0	Equation 16
5.2.2.3	Scheme 2
In this scheme all bits for the encoded frame m are delivered to the channel input buffer at the end of the period S(m).  
i.e.			DC(m)  [M{C(m) - R.S(m)} + N(m)]/R	Equation 17
5.2.3	Decoder Delay	DD
5.2.3.1	Discussion
Decoder delay potentially arises from two sources.
For the case of forward reference where the referred to frame does not precede the current frame in the data-stream there will be an additional imposed delay to wait for the frame to become available.  This is Decoder-Wait-For-Data, DDW
In the second case it may be desirable to hold back the display of an available, decoded frame to ‘even-out’ the displayed frame rate.  This is Decoder-Wait-For-Display, DDD
The total delay, DD, is the sum of DDW and DDD 
5.2.3.2	Decoder-Wait-For-Data DDW
For frame m
	DDW(m) =  M{tD(n) - tD(m)}	Equation 18
where	tD(i)  	is the time of receipt of the final bit in the encoded representation of frame i at the channel output buffer
and	n 	is the latest (in the data-stream) encoded input frame on which the encoding of frame m depends
Since it should be possible appropriately to order the frames in the data-stream, for all practical codec implementations DDW will probably be zero.
5.2.3.3	Decoder-Wait-For-Display DDD
The computation of DDD is more problematic and involves a degree of freedom on the part of the decoder designer which is difficult to quantify in a model of this type.  However there are constraints that can be specified as follows.
Let the time of decode of frame m, taking account of accumulated delay due to all causes up to and including this time be TD(m)
Let the time of presentation for display of frame i be Tout(i).
Let the frame preceding m in the displayed sequence be p
It is then certainly true that
	DDD(m)    M{Tout(p) - TD(m)}    V1	Equation 19
otherwise the display of m would seem to be possibly prior to that of p in some circumstances.
It would also seem reasonable that     
	DDD(m)     (tS(m) -tS(p))    V2	Equation 20
since otherwise the inter-frame interval is being ‘stretched’ relative to the source sequence value.
The following rule may be discerned :
If  V1  V2 then    DDD  = V1     to avoid too early display
Else
DDD  =       in the range  V1       V2 as appropriate.
 may be determined by the expression
•	  V1  +  .(V2 - V1)	0    1	Equation 21
5.2.4	Total Delay
The total delay experienced for a frame is the arithmetic sum of the individual delay components discussed above.
i.e.	Total Delay  =  DE + DC + DW	Equation 22 
Due to the way that the various terms interact with each other and previous values of delay direct analytic computation is inconvenient.  Suitable computational schemes are presented in sections 6 and 7 below.
6.	Computational Operations — The General Case
6.1	Discussion
The general case, i.e. the full model, can be applied during the generation of encoded demonstration sequences.  However this would potentially involve detailed adjustment of the encoder to the detriment of the development process.
The necessary data can more easily be generated separately from a log of critical information which is readily recorded during the encoding process.  This is the approach described in this document.
6.2	Log of Initial Data
The software emulator used to generate demonstration sequences should be adjusted to record the following data for each encoded frame :
•	The position of the frame in the source sequence, hence its time of acquisition tS 
•	The position of the frame in the encoded sequence
•	The number of bits, N, allocated to the frame by the encoder
•	The position in the source sequence [and hence its time of acquisition] of the latest frame upon which the encoding of the current frame depends
•	The Encoder Processing Delay, S(m), appropriate to the frame [as specified by the developer]
6.3	The Computational Process
	The delay for the first frame may be computed directly since it is solely dependent upon the bit allocation to it and the channel bitrate.  It may be displayed immediately on receipt. The ‘time of acquisition’ of the first frame is equated with time zero.
	The various delay values for the subsequent frames may be determined most conveniently progressively in three ‘passes’.  Each of the first two should be performed in encoded [transmitted] frame order, the third may be performed in source sequence frame order.  
6.3.1	Initialization of variables for the first frame
For the first frame initialize the following values as indicated
Encoder Processing Delay 	DEP		zero
Encoder-Wait-For-Data 	DEW		zero
Final channel input buffer contents	B(0)		frame allocation
Time of acquisition	tS		zero
Time of commencement of processing	S		zero
Time of completion of processing			zero
Channel Delay	CD		B(0)/R
Total frame delay	DT		CD

6.3.2	Pass 1 — Encoder Delay Computation
For each encoded frame m other than the first, taken in transmission order
•	specify a value for the Encoder Processing Delay S(m) using some appropriate scheme.
•	determine the time of acquisition of the frame, tS(m)
•	adopt the later of tS(m) and of the time of completion of processing of the previous frame in the data-stream as S(m), the time of commencement of encoding of m 
•	compute the total encoder delay DE(m) from Equation 3
•	add DE(m) to S(m) obtain the time of completion of processing
6.3.3	Pass 2 — Channel Delay Computation
For each encoded frame other than the first, taken in transmission order
•	if the time of completion of processing of the previous frame in the data-stream is earlier than the time of acquisition of the current frame
	reduce the buffer contents by (the channel bitrate x the time difference); if the nominal buffer contents are negative then make them zero
	otherwise adopt the buffer contents unchanged.
	This is the value C(m)
•	from 
•	the value of C(m) just computed 
•	the duration of encoding of the current frame S(m)
	and
•	the number of bits allocated to the current frame N(m)
	compute and record B(m) the final channel input buffer contents for the current frame
	This may be done using one of the schemes for bit generation rate as presented in the model, or by use of some other appropriate scheme.
•	from the time of commencement of encoding S(m) and the duration of encoding S(m) compute the time of completion of encoding
•	from the final channel input buffer contents for the frame determine the Channel Delay by dividing by the Channel Bitrate
•	by adding the Channel Delay to the time of completion of encoding determine the time of reception tD of the encoded frame by the decoder
6.3.4	Pass 3 — Decoder Delay Computation
For each encoded frame other than the first, taken in source order
•	from the time of display of the previous frame and the time of availability of the current frame determine the time of display of the current frame.
	This may be done by use of the suggested rule in the delay model or by any other appropriate method subject only to the constraint that the times of display of successive frames increase monotonically with time.
•	by subtracting the time of acquisition of the frame from the adopted display time determine the end-to-end delay for the frame.
7.	Computational Operations — The Minimum Delay Case
Where minimum delay is an objective forward reference cannot be allowed.  In this case Encoder-Wait-For-Data is guaranteed zero and all passes may be performed in source frame order.  
Furthermore, since inter-frame references are limited to backwards references all three passes may be combined into one single pass.
8.	Data Presentation
When utilizing the delay model all proposers should, for consistency and ease of comparison with other proposals, present the following data :
For each demonstration sequence
•	A tabulation of the frames actually encoded plus the  end-to-end delays associated with them.
•	The same data presented in graphical form.
•	A statement of the encoder bit generation scheme adopted [plus the values of parameters adopted where the scheme is one of those presented in the model] together with a justification where another scheme has been substituted.
•	A statement of the display delay strategy used [plus the value of  where the scheme presented in the model is adopted].

Annex C - H.26L Requirements Document

1. Introduction
This document provides the requirements for the future video coding standard H.26L activity. This study is conducted under ITU-T Q.15/16 and the set of requirements for the H.26L is defined as the first step of the activity. This requirements document will be attached to the call for proposals of new video coding algorithms suitable for H.26L which will be issued in January 1998.
2. Backgrounds
The ITU-T SG16 Q.15 is currently developing the revision of H.263 the extension of the existing Recommendation H.263 adding more functionalities as well as coding efficiency improvements. The revised Recommendation H.263 is planned to be approved by SG16 meeting in January 1998. 
However, there still be a potential need for the new video coding algorithm which will much improve video quality and will cover more functionalities not supported by other standards. In the following sections, requirements which should be met by the H.26L new generation video coding standard are listed.
3. Applications
H.26L should be a generic video coding standard applicable to very wide range of applications. The possible applications of both real time and non-real time services, which the H.26L will address, may be:
- real-time conversational service (e.g. video phone)
- audiovisual communication over mobile networks
- Internet video applications
- video transmission for sign language and lip-reading communication
- video storage and retrieval service (e.g. video on demand)
- video store and forward service (e.g. video mail)
- multipoint communication over heterogeneous network environments

4. Requirements
The specific requirements for technical items are described in this section. All of these requirements should be met by the final H.26L standard.

The requirements list is structured as specifying general development target for each technical area to clearly show the benefit of H.26L over the earlier standards (such as H.263/H.263++).

9.	Quality Requirements

The main (implicit) assumption in the H.26L work has been that H.26L should be significantly better than H.263. This assumption should be, and has been, extended to cover the best evolved version of H.263, 1998 version of H.263 and H.263++.
The quality requirements should be specified in terms of obtainable visual quality at a given bit-rate. The requirements need not consider how quality is to be evaluated even though this will definitely be a major challenge for the H.26L work. Both visual quality from subjective tests as well as PSNR-like objective measures can be used in development and testing with appropriate common conditions.

The quality target of H.26L is:
The H.26L video codec should provide visual quality comparable to the anchor codec with less than half the bit-rate. At a given bit-rate, the H.26L video codec should enable at least twice the frame rate of the anchor codec without compromising spatial visual quality.
In the above text, the “anchor codec” refers to the best available version of H.263 making the anchor to be a moving target.

It is noted that the following goals should be achieved to obtain a desirable video quality for sign language and lip-reading applications.
•	Coding method can guarantee a certain frame-rate figure.
•	Testing with reference material containing sign language at 25-30 fps CIF, with target bitrate 112kbit/s
•	Testing with reference material containing sign language at 12-15 fps QCIF, with target bitrate 30kbit/s
•	Coding method allows transmission of sign language with an acceptable quality at above reference rates.

10.	Delay Requirements

Delay is a very important consideration in interactive real-time systems. The most important factor affecting the video codec delay is whether the coding algorithm relies on backward prediction. It is necessary that the H.26L codec includes a low-delay mode without any backward prediction. The second major delay source is the frame rate, which is largely independent of video coding algorithms and the frame rate requirement is already addressed in the quality requirements section.
Other delay sources include the rate control mechanism, or more specifically, the variability in allocating bits to individual frames which is usually encoder implementation dependent. The video codec algorithm also has a direct effect on the delay depending on whether the algorithm allows the coding of the frame in small independent parts.

It is not practical to specify absolute delay values for the video codec since this would require considering specific frame rates and bit rates. So, delay requirements in a more general statement is:
The H.26L video codec should facilitate a low-delay mode for interactive real-time communications. No backward prediction from subsequent video frames (implying an additional frame interval delay) are allowed in this mode. Rate control should enable using low buffering delays and flexible trade-off between delay and spatial quality of individual frames.

It is noted that the following is a goal of delay desirable for sign language and lip-reading applications.
•	Low delay, so that the delay requirements for conversational communication stated by SG12 can be met. (0.4 seconds from camera to far end display.)

11.	Complexity Requirements

To be useful for real applications, the H.26L video coding standard should be implementable with reasonable cost using typical platforms of the time. The following factors affect the complexity requirements:
1?	The schedule for H.26L defines the earliest possible completion of the standard around mid-2000.
2?	It is expected that generic software platforms (instead of dedicated video platforms) will be increasingly important as H.26L implementation platforms.
Detailed complexity requirements would include specifying the computational complexity (in terms of number of operations per unit time for some specific platform) as well as the amount of memory (in terms of bytes/words) used by the algorithm.

Definition of complexity requirements in general level as follows is more practical, which is similar to what ITU-T has adopted in speech coding standardization work.
The H.26L implementation complexity, in terms of both computational complexity and memory usage, should be low enough to enable cost-efficient implementations with typical implementation platforms at the time of the completion of the standard. Both software implementations on PC-like platforms as well as dedicated DSP implementations in consumer terminals must be considered. The H.26L video codec should support implementations with scalable complexity in both encoders and decoders providing graceful degradation of video quality with decreasing implementation complexity.

12.	Bit-Rate Requirements

Video coding algorithms can often be applied over a very wide range of bit-rates. This can, and should, also be expected from H.26L. However, it is still useful to specify a range of target bit-rates which can be used by developers in optimizing the performance of their codecs as well as common test conditions in the H.26L work. It is expected that all the other requirements are associated with the target bit-rates.

The following bit-rate requirement comes from the primary target of narrow band video telephony:
The main target bit-rates of the H.26L video codec are in the range of 8-128 kbps. The H.26L codec must also provide significant benefits also beyond that range at least up to 1 Mbps.

13.	Error Resilience Requirements

The error resilience of any video application will depend on the system level specifications and implementation as well as on the video codec itself. Furthermore, the implementation of the video decoder is as important to the error resilience as the bit stream specification. However, error resilience of the codec includes at least the capability to recover from errors (e.g., via Intra updates) as well as the number of synchronization points within one video frame.

Error resilience requirement in general form is:
The H.26L video codec should show graceful degradation of video quality with increasing amount of transmission errors. The codec should provide fast mechanisms for recovering from highly corrupted video frames. Combined error resilience and error concealment in the H.26L video codec should provide acceptable visual quality at random BER values of [10-4] and at burst BER values of [10-3]. The codec should provide the possibility to trade off compression efficiency with error resilience.

14.	Additional Functionalities

It is expected that the H.26L video codec would support several other functions similar to those of H.263. Some of them may need not be considered at an early phase of the development. It is also useful to explicitly require some “self-evident” codec properties provided by present video codecs. These additional functionalities include:

•	The codec should have a “storage mode” with relaxed delay requirements (backward prediction allowed).
•	The codec should support temporal, spatial,  and SNR scalability.
•	The codec should support all important picture formats.
•	The codec should have an Intra coding mode which can be used for providing access points in storage applications.
•	The codec should enable a flexible rate control mechanism.
•	The codec should enable flexible trade-off between spatial and temporal quality.