&%PAGE& - &%PAGE& - Document AVC-65R CCITT SGXV Document AVC-65R Working Party XV/1 May 31, 1991 Experts Group for ATM Video Coding SOURCE : CHAIRMAN TITLE : REPORT OF THE SECOND MEETING OF THE EXPERTS GROUP FOR ATM VIDEO CODING IN PARIS (May 23-31, 1991) Purpose: Report ----------------- PART I - INDEPENDENT SESSIONS 1. General 2. Documentation 3. Tape demonstration 4. Discussion 5. Work plan 6. Actions to other groups 7. Others PART II - JOINT SESSIONS WITH MPEG 1. Requirement Sub-group 2. Test Sub-group 3. Video Sub-group 4. System Sub-group 5. Implementation Sub-group PART I - INDEPENDENT SESSIONS 1. General The second meeting of the Experts Group was held in Paris (France) at the kind invitation of CNET and ENST. The first part of the meeting, held at CNET during May 23-24, consisted of CCITT independent sessions, while the second part, held at ENST during May 27-31, consisted of joint sessions with ISO/IEC JTC1/SC2/WG11 (MPEG) and a closing session. The list of participants appears at the end of this report. At the start of the meeting at CNET, Mr. Jean-Paul Bloch made a welcome address on behalf of the hosting organization. The following changes of the coordinating member were informed of the group; - FRG: Mr. Franz May (Daimler-Benz) has been added as the second member. - Canada: Mr. Denis Lemay (BNR) has replaced Mr. S. Sabri. - Switzerland: Mr. P.A. Probst provisionally replaces Mr. H. Keller until a new member is appointed. At the closing session, Chairman thanked the hosting organizations for the meeting facilities provided and the excellent secretarial support. 2. Documentation (TD2) For this meeting, 43 AVC-numbered documents and 12 temporary documents were available as listed in Annex 1. 3. Tape demonstration (TD4) A number of video tape demonstrations were given with D1 or U-matic to present experimental results as detailed in Annex 2. 4. Discussion 4.1 Report of the meetings relevant to the Experts Group 4.1.1 SGXVIII Matsuyama meeting - November/December 1990 (AVC-24,25) Mr. Biggar reported that coordination among standardization groups involved in video coding study had been initiated by SGXVIII, and that some replies to our questions were given as a liaison statement. It was also reported that "IVS Baseline Document" as in AVC-25 had been created as a vehicle to achieve the coordination mentioned above. 4.1.2 MPEG Berlin meeting - December 1990 (AVC-23,26) Mr. Okubo reported that MPEG had supported to have joint sessions in areas of joint concern with the Experts Group as we proposed. Milestones of the MPEG Phase 2 work and test procedures to find promising video coding algorithms were also reported. It was stressed that we should identify all the technical requirements for the high quality video coding standard from the CCITT point of view. 4.1.3 WPXV/1 Geneva meeting - February 1991 (AVC-27) Mr. Okubo reported the outcome of the meeting of our higher body highlighting the slight extension of our "terms of reference". 4.1.4 CMTT/2 Tokyo meeting - March 1991 (AVC-28) Mr. Carbiere reported that TG CMTT/2 had identified common areas of interest between CMTT/2 and SGXV and that they seeks practical collaboration through the Special Rapporteur. 4.2 Picture format (AVC-29,36,46,59,60,64; TD8) The six documents were presented addressing the following questions; - Do we use a single coding format or multiple ones for the high quality video coding? - What parameter values are possible? How are they related to the existing CIF? How are they related to larger formats such as EDTV and HDTV? - What framework is the best for the service integration on B-ISDN? - Do we take the initiative to adopt the single picture format approach to other groups if proved viable with evidence? During the discussion, various views were expressed as follows; - Discussion of regional boundaries are not appropriate if we consider the globalization of communications. The argument may also be valid for broadcast and video database services. - We are deciding the transmission format which should be distinguished from interface specifications. - Since services are market-driven, "flexibility" is the key word for this format issue. - A single format is desirable for both of inter- and intra-regional communications, but the burden of coding efficiency and hardware should be well evaluated. - Multi-point and bridging capability of a single picture format is also desirable. - Adoption of square-pel shape will permit direct connectivity with computer workstations and encourages desktop videoconferencing. - For high quality applications, the current CIF requires improvement. CMTT is concerned with compatibilities with new EDTV and HDTV with digital input. - The current CIF provides not too bad quality if higher bit rates are allowed. It will satisfy interregional communication needs. - 60 Hz progressive pictures converted from local standards are already degraded. Adding pels at the encoder input and removal at the decoder output does not lead to coding gains. In what time range do progressive cameras and display become widely available? - Progressive scan ameras are questionable but displays have no problems even today. - We are coding "pictures" but not "pels". - We should seek worldwide unification in picture coding format by providing better definition than CCIR601, compatibility with the current CIF and advantages for digital signal processing. - Since picture quality due to frame rate conversion (from interlaced to non-interlaced) is not sufficient, the progressive format solution may be premature if it assumes progressive scan cameras. After the discussion, Mr. Haskell undertook to coordinate a small group for establishing an action plan toward decision at the earliest occasion. The outcome is contained in Annex 3. Participants are requested to make contribution according to this work plan. 4.3 Framework for H.26X 4.3.1 Requirements (Annex 7/AVC-22R; AVC-37,51,57) The following items were recognized as possible requirements for our future video coding study; - Multipoint consideration taking into account B-ISDN capabilities such as listed in 3/AVC-37 - Capability to operate in intraframe mode only may offer advantages in some situations. - Provision for both backward/forward and downward/upward compatibilities and adaptability to future extension Early consideration of the first item and reflection to the coding framework were felt necessary. As to the second item, there was some discussion what tariff structure would be adopted for B-ISDN with respect to the bit rate and communication distance. There are no clear answers at the moment. These items are input to the joint sessions with MPEG from the Experts Group. 4.3.2 Architecture (AVC-34,35,50,55) "Flexible Layering" video coding architecture was addressed in AVC-34,35 which exploits B-ISDN characteristics and provides service integration. The idea was recognized interesting and obtained general support. There was expressed some concern about loss of coding efficiency due to layering which could be counted against service interworking. Since cell loss is one of the key characteristics in ATM networks, it was dealt with in the architecture discussion. Questions were; - What degree of cell loss should we expect for high/low priority cells? - Is simple recovery from rare cell loss sufficient or is sophisticated protection against frequent cell loss required? Some delegates expressed that cell loss ratio of around 10-8 or 10-9 may be obtained for the high priority class, but the total scope remains still ambiguous. AVC-50 listed up various cell loss protection and recovery methods while AVC-55 showed an example of two-layer coding with highly cell loss resilient enhancement layer. Note to 4.3 - The meeting confirmed that the aim of H.26X is to cover a range of video applications, bit rates, resolutions, qualities, and services for B-ISDN, and that compatibility between H.26X and H.261 should be highly respected (see 7.1 and 6.3/AVC-22R). 4.4 VBR vs CBR 4.4.1 Comparison study (AVC-38,41,47,48,56,62) Several studies including simulation experiments were presented on the following topics; - Statistical multiplex gain (AVC-38,41,47) - Picture quality comparison (AVC-38) - Delay and impact of source periodicity based on a single network model and a 30 minute long VBR video data (AVC-62) - Guideline for further simulation works under restrictions of network's UPC (AVC-48,38) - Analysis of allowable information generation at the encoder for a VBR channel (AVC-56) The meeting had a question and answer session, during which it was suggested that we should pay attention to the buffer size in the comparison of VBR/CBR performances and that a test material with noise may be required. It was also noted that Document AVC-62 points out the ambiguity of multiplexing gain, stating that due to statistical multiplexing gain different sources may experience different cell loss ratio which in turn may impact the definition of statistical multiplexing gain. 4.4.2 Statistics (AVC-44,49,61) Some statistics for VBR coded data were presented in the form of bit rates measured with various time windows. It was found that a relatively small buffer makes a significant difference to the shape of source, and that the number of cells per frame for videoconferencing scene with moderate motion and no scene cuts or changes follows a Gamma dsitribution when measured in an open loop VBR coding. It was also addressed for what pictures VBR can be most effective. VBR picture quality advantage in Mr. Biggar's tape demonstration was pointed out to correspond to the illustration in Figure 5/AVC-49. 4.4.3 Network model (AVC-43,61) Four different network models were presented for information. It was clarified that the CLR (Cell Loss Ratio) in the first simplified network model contained in Annex 4 to Document AVC-22R should read Psat, probability of saturation, and that P in AVC-43 should read PVBR. The meeting feels that this model is not sufficient due to the following facts; - The aggregate model does not take into account correlation between arriving frames. - Video source cannot be accurately modeled by a memoryless ON/OFF model. - Document AVC-61 shows that a two-state model which takes into account correlation overestimates cell loss ratio. - A second order AR (autoregressive) model underestimates cell loss ratio. - It does not take into account dynamics of statistical multiplexing (i.e. source periodicity effect). The necessary improvements, however, require further detailed studies. 4.4.4 Framework for further study (TD7) After the discussion of the above items regarding the variable bit rate video coding study, Mr. Verbiest undertook to coordinate a small group to establish a framework for further study. The outcome is contained in Annex 4. Contributions are awaited toward the next meeting. 4.5 Layered coding (AVC-31; AVC-33,35,45,55,58) Impact of layering to the coding efficiency was presented in AVC-31. It was noted that this particular architecture suffered from some ineffectiveness due to VLC and interframe prediction accuracy. Other 5 documents also addressed layered coding, but they were discussed in more directly relevant items. 4.6 Video coding algorithms (AVC-30; AVC-33,45,58) Extensive study results were presented in AVC-30 concerning possible new elements of source coding. The meeting appreciated this information. During the questions and answers, the following facts were clarified; - Line conversion filters with small number of taps are well suited for picture coding. - The quantizer trick described in AVC-30 has been found effective also in the video codec of the Belgian Broadband Experiments. - A self adaptive VLC (the U-VLC) has been defined in the Belgian experiments, whose adaptivity provides a high efficiency. - Difference in picture quality in frames with different quantizers (P1 and P2 pictures on p.6/AVC-30) are noticeable for single frame display but not for sequence display. - One of the reasons for effectiveness of the MPEG1 bi-directional prediction may be better prediction for uncovered backgrounds. 4.7 Compatibilities with other standards 4.7.1 Definition (AVC-32) The meeting supported the proposed definitions for the following terms; - Upward/downward, forward/backward compatibilities - Methods to implement the compatibility - simulcasting, embedded bit stream, syntactic extension, switchable encoder, standard families Members are recommended to use these terms for their future work. During the discussion it was pointed out that "layered coding" corresponds to either of the "embedded bit stream" or the "syntactic extension". 4.7.2 Specific methods (AVC-33,45,58; AVC-35,51,55) Three schemes were presented in AVC-33, 45 and 58 for information which implemented upward/downward (CCIR601 vs CIF or SIF) and forward/backward (H.261/MPEG1 vs H.26X/MPEG2) compatibilities with multi-layered coding by using H.261 or MPEG1 as the base layer. During the discussion, it was pointed out that we need not be restricted by the existing standards, we should rather seek the most optimized solution for the B-ISDN functionalities, the interworking between the N-ISDN and B- ISDN can be implemented through the gateway. Several opposing opinions, such that the cascade connection of H.261 and H.26X may cause additional delay and picture quality degradation, were expressed to this view. The meeting confirmed that the Experts Group should provide means for interworking between H.320 terminals connected to the N-ISDN and H.32X terminals connected to the B-ISDN. Appropriate means to achieve this objective are open at the moment. 4.8 Multimedia multiplexing (AVC-39,42,53) Having heard the study results on multimedia multiplexing methods in the ATM environments, the meeting agreed to use the VCI method as a reference of our future work which separates a multimedia connection into several virtual channels. In order to confirm its suitability, the following items should be further studied; - How to assure cross media synchronization - Penalty in the use of network resource (Note: Tariff is related to this consideration, which is recognized difficult to handle.) - Maximum number of VCIs for a multimedia connection - Existence of user level multiplex to a single bit stream such as N- ISDN audiovisual systems and MPEG systems - Impact of VBR coding and UPC 4.9 Network issues (AVC-24,25,40,52,54; TD10) The meeting endorsed the IVS Baseline Document approach which had been proposed by SGXVIII and confirmed to contribute actively to its upgrading. As a first step, we collected materials from AVC-40 and 52 which should be sent to SGXVIII. We also picked up a few items which should be included in our liaison statement to SGXVIII. Mr. Biggar undertook this drafting work with cooperation of some volunteers. The outcome is contained in Annex 5. AVC-54 addressed the timing recovery for the B-ISDN audiovisual codecs and means of transmitting required timing information. Due to the lack of available time, we could not have detailed discussion. Members are requested to review this document and submit contributions toward the next meeting. 5. Work plan (AVC-34,57) Chairman requested understanding of the members for a change of the work method leading to the first reference model due to the joint work with MPEG. Instead of "informal observation tests", we will use the subjective assessment of promising techniques (Kurihama tests) to quantify the subjective picture quality of candidate algorithms. Submission of simulation results to this Kurihama tests are recommended. It was stressed that the objective of this subjective test is to find promising techniques for further collaborative elaboration. Some concern was expressed whether B-ISDN characteristics such as cell loss can properly be included in the evaluation, and whether some schemes such as layered coding may be precluded. It was clarified that the Kurihama tests are for source coding and necessary adaptation to the transmission channels should be studied and optimized in parallel and subsequently. The meeting agreed to make an input to the joint sessions which contains the following requirements from the communication applications standpoint; - Cell loss resilience - Short coding/decoding delay (`150ms) - Multipoint consideration - When compatible coding is tested, the base layer bit rate should be set as guidance to a value lower than the primary rate, otherwise there is no ways to transport the base layer for interworking with the existing systems. - Test sequences of progressively scanned materials. There may be a danger that coding algorithms are optimized only for interlaced materials. 6. Actions to other groups 6.1 SGXVIII (TD10) The Experts Group sends a liaison statement as contained in Annex 5. 6.2 CMTT/2 (TD9) The Experts Group sends a liaison statement as contained in Annex 5. 6.3 MPEG (TD5,11) The meeting discussed ways of coordinated collaboration with MPEG, related to the document submission to the joint sessions from the Experts group. The followings obtained consensus as its conclusion; - "Proposal" documents should be supported by the group as a whole, and sourced by the Experts Group. It causes confusion if differing proposals are brought independently. - As there have been made no decisions relevant to the joint work, only informational documents, relabeled as "Purpose: Information" if necessary, are submitted this time. Specific documents are AVC-30, 32, 33, 34, 35, 51, 56 and 58. Representatives to MPEG subgroups (Video, System, Requirements, Test, Implementation) were appointed as in Annex 7. The Experts Group sent an input document to the joint sessions, summarizing the discussion in its sole sessions as contained in Annex 8. At the closing session held on May 31, we reviewed the organization of this meeting, particularly the way of collaboration with MPEG we tried the first time here. Some delegates expressed as follows; - Two days for the independent sessions are definitely too short. Sufficient discussion was not possible. - Having a technical session on ATM aspects during the MPEG week may be effective. - The closing session should be held at an early occasion so that the participation in the joint sessions become more flexible. - The current MPEG activities include much of MPEG-1, thus joint sessions are not efficient for the Experts Group. - Method of using liaison statements may be appropriate. Chairman clarified that the joint sessions are not only part of the MPEG meeting, but also part of the Experts Group meeting, and that we are having physically one meeting but it should be considered logically as two meetings, thus the closing session should review the joint sessions as well. He also stressed the importance of working together from the early stage toward joint development of the universal (or generic in MPEG term) high quality video coding standard and the importance of our proposing candidate algorithms to the Kurihama test for this purpose, even though there is some inefficiency due to the mixture of MPEG-1 and MPEG-2 works at the moment. The meeting finally agreed to try again in August this way of meeting organization, but extending the independent sessions to three days. 7. Others 7.1 Status report (TD3,6) The meeting agreed to update the preparatory draft in TD3 by incorporating the achievements obtained at this meeting. Editors who are in charge of updating the document were appointed as in Annex 8. 7.2 Next meetings - 3rd meeting: August 14-16 (independent sessions), August 19-23 (joint sessions with MPEG) in Santa Clara, U.S. - 4th meeting: November 1991 in Yokosuka, Japan - 5th meeting (provisional): January 1991 in Singapore END PART II - JOINT SESSIONS 1. Requirement Sub-group (by Sakae Okubo) 1.1 Documentation 1.1.1 General requirements - 3/016 MPEG91/011,012,021,042,056,060,065,071,075,076,079 1.1.2 Compatibility - 3.4 11)/016 MPEG91/011,017,071,081 1.1.3 Kurihama test conditions - 6.2.1/016 MPEG91/021,049 1.1.4 Test methods for picture quality - 6.2.1 2)/016 MPEG91/019,023,024,025,026,027,028 1.2 Discussion schedule - Requirements (Okubo) Tuesday 9-13, 14-18 - Test methods (Hidaka) Wednesday 10-13, 14-18 - PPD (Okubo) Thursday 9-13 1.3 Agreements 1.3.1 General requirements 1) We will incorporate all the requirements identified in various documents (011,012,042,2/060,071,075,076,079)) into 3 of PPD, as far as they are not contradictory. If there may be any contradictory requirements, they should be solved in the application standards. 2) We have confirmed that these requirements be met by the standard for high quality video coding as a log range target, namely in the course of Committee Draft development. 3) The following tests are to be be considered in the collaboration phase work; - Artificial patterns to check particular elements in the coding algorithm such as presented in MPEG91/021. - Coding performance for progressively scanned materials. For this purpose, provision of test sequences and monitors should be further considered. - Conversion from CMTT to MPEG coding as in MPEG91/065. 4) Layered coding is suggested in MPEG/056,071,076 for realizing compatibility with existing standards, flexibility to the quality/bit rate tradeoff, and flexibility to the differing characteristics of transmission/storage media. 4) An editorial work was carried out by several volunteers to extract appropriate materials in the received contributions which should be reflected in the PPD document (MPEG91/016). The outcome is given as part of the meeting report. 1.3.2 Compatibility 1) We will use the terminology given in MPEG91/081 for future discussion of the compatibility issue. 2) We have confirmed the following previous decisions toward defining the first "Test Model", thus toward developing the standard; - We seek "compatibility" to the maximum extent. - For competition purpose, we will initially concentrate on the picture quality. If we succeed to narrow down the number of candidates from the picture quality measurement, we will apply functionality criteria including compatibility. - Every coding algorithm proposal should describe what compatibility features it has and demonstrate the processed pictures in the claimed compatibility mode(s) such as illustrated in Figure 1(a). 3) The greatest degree of compatibility would be achieved by a core MPEG1 decoder operating on a bit stream in the range of 1.0 - 1.5 Mbit/s. 1.3.3 Kurihama test conditions 1) Selection of test sequences According to the feel for difficulty of coding, the agreed 8 test sequences were ordered as in Figure 2. We gave priority to the four sequences which are to be used commonly for both of 4 Mbit/s and 9 Mbit/s. We selected test sequences for the three cases where 2, 3 and 4 overlapped sequences as in Table 1 according to the guideline to pick up other necessary number of sequences from easy ones for 4 Mbit/s and difficult ones for 9 Mbit/s. The selection of the number of overlapped sequences depends upon the number of proposals, namely the capacity of testing facilities and testing hours. After having reviewed various coded pictures, there were raised some questions whether bit rate values and selected sequences in Table 1 produce sufficient difference among proposed algorithms, particularly at 9 Mbit/s. This matter may be reconsidered if achievements by August indicate no difference at all between original and coded pictures. 2) Resolution For the Kurihama test purposes, the input and output signals are both CCIR 601 4:2:2 ones. Some coding algorithms may subsample the input signal, which should be stated in the coding algorithm description. Different parameters or even different algorithms may be switched for 4 Mbit/s and 9 Mbit/s, but the bit stream should contain all the necessary bits for the purpose so that the decoder can correctly identify these parameters and operate. Note - This conclusion is based on the technical settlements at a joint session between the Video and Requirement/Test subgroups on Wednesday. Since this interpretation differs from the one agreed at the Tuesday session of the Requirement/Test subgroup which was based on the approved report of the Berlin meeting, the final decision supporting the above mentioned settlements was made at the WG11 plenary on Friday, May 31. 3) Coding and decoding delay should be stated. 4) Random access delay addresses "frame (2 fields in a set)". 1.3.4 Materials to be submitted 1) Description - Algorithm including block diagram and syntax diagram - Compatibility feature - Random access feature - Coding/decoding delay - Any other functionalities - VLC, FLC tables employed for coding of classification related overhead, motion vectors, coefficient data, various synchronization words, etc. - Statistics * number of bits and SNR for each frame * cumulative bit count once every 0.4 second (excluding the last 0.2 second) for each sequence * several items (such as motion vector count, luminance and chrominance bit counts, overhead bits) averaged over each sequence following the formats in SM3/RM8 tables as a guideline 2) D-1 tape for subjective tests of normal playback for agreed test sequences at 4 Mbit/s and 9 Mbit/s 3) D-1 tape for demonstration excluding the normal playback pictures - fast forward - fast reverse - compatibilities if claimed - low coding/decoding delay mode pictures if claimed - any other functionalities if claimed 4) Verification of decoded pictures Each picture to be tested at Kurihama should be reconstructed from a coded bit stream. A paper listing should be given for each sequence which indicates the corresponding coded bit stream file in a format such as UNIX "ls -l" output. Decoder executable codes should be made available upon request of MPEG. If this requirement is not met, the proposal can not be considered as part of promising schemes. 5) Implementation aspects - see 5 below of this report Detailed algorithm descriptions are required. Some tables to be filled in may be provided at the next meeting in August 1991. 2. Test Sub-group (by Sakae Okubo) This sub-group dealt with procedures for the Kurihama test which measures subjective picture quality of the proposed video coding algorithms. 1) Test method for picture quality was agreed as described in MPEG91/024,028 (Double stimulus continuous quality-scale method as per CCIR Rec. 500-3). 2) Test sequences and bit rates were agreed as in Table 1/1.3.3 above. 3) Format for the D1 tape format for proposal submission was agreed as in MPEG91/027. 4) Schedule toward the Kurihama test was clarified as follows; - Pre-registration: June 30, 1991. Proposers are requested to send a written indication of participation to Dr. L. Chiariglione. About 26 provisional indications (10 from Europe, 4 from U.S., 12 from Japan) were given at this meeting. - D1 tape submission to JVC Kurihama: October 18, 1991. Tape editing for the subjective tests will follow the tape reception. - Subjective tests: November 18-22 or 21-27, 1991. The meeting also considered the EBU request to provide MPEG-1 processed pictures for picture quality comparison with VHS and other pictures, but concluded it not appropriate for MPEG to provide such materials mainly on the ground that MPEG specifies only those items for proper decoder operation, not such items to guarantee picture quality performance. 3. Video Sub-group (by Gisle Bjoentegaard) The Video group met four times during the week. One session of two hours was devoted to tape demonstrations of coded pictures and oral presentation of accompanying documents. This was the only part devoted to the matters relevant to MPEG2/H.26X The rest of the time was spent on matters relevant to finalizing MPEG1. The key words for this activity are: - Editorial matters concerning the CD (Committee Draft). - Conformance testing. The work was initiated and will be a major point also for the next meeting. - Clearification of the differences between JPEG and MPEG-INTRA. - MPEG report. This is a document to help understanding of the MPEG coding method. The report will be finalized at the next meeting. COMMENTS (from G. Bjoentegaard) to the time schedule for the coming meetings in the Video group: It is foreseen that much time will be spent on MPEG1 matters also at the next meeting. MPEG2/H.26X matters will be more heavily focused after the KURIHAMA test. It is proposed, however, that some time is spent during the next meeting to discuss working methods with particular emphasis on collaboration between MPEG and CCITT after the Kurihama test. 4. System Sub-group (by Barry G. Haskell) The WG11 (MPEG) Systems Committee met in Paris May 27-31, 1991. During this period a number of improvements and simplifications were made to the multiplexing specifications, culminating in Revision 8 of the Committee Draft (CD). In particular, 1) A System target Decoder (STD) was defined for demultiplexing data packets into individual buffers, one for each stream. Decoder buffer sizes are assumed known to the encode/multiplex system, and it is the responsibility of that system to not overflow or underflow the decoder buffers of the STD. 2) A Reference Presentation Model RPM) was defined for audio and video that allows for Presentation Units (PUs) and Presentation Times of these PUs. 3) An STD for variable transmission rates was proposed, including a Target Channel whose rate is known to the encode/multiplex system. As above, the encoder is responsible for managing the decoder buffers. 4) Syntax and semantics were defined that allow for time synchronization, data packetization, different stream types, etc. 5) Prior restrictions were loosened to allow for constant bit-rate per stream, as well as arbitrary interleaving of packets of different types. 6) System Clock Reference data is sent periodically in order to reset or synchronize a System Time Clock. Further study and refinement of these and other aspects of multiplexing continue. 5. Implementation Sub-group The group met in the afternoon of 29 May 1991 and in the morning of the 30th. The following items were on the agenda: 1) Berlin meeting report 2) Implementation issues affecting MPEG-1 3) Verification of the MPEG-1 CD 4) Working method for MPEG-2 and contribution to the Proposal Package Description 5) AOB 5.1 Berlin meeting report (omitted) 5.2 Implementation issues affecting MPEG-1 In the Video Group, Savatier had proposed an amendment to the coding of extended range motion vectors. After confirming that there was a real possibility of such a change being adopted if the Implementation Studies Group supported it, the topic was discussed. It was recognised that implementations would be simplified except that the possible range of motion vectors which a decoder would have to accept would be larger. It was suggested that for bitstreams with the constrained parameters flag set, the vectors could be limited to the range available under the old scheme. With this proviso, the change was supported. There was a short discussion on buffering. The Video Buffer Verifier (VBV) in the CD is only required to be met between sequence headers. A decoder may not be able to play continuously across a sequence boundary if the buffer conditions on each side of it are unfavourable. It was also pointed out that editing of coded video using cut and paste techniques could generate material which did not satisfy the VBV even though the originals did. Failure of systems under these circumstances should not be blamed on decoders. The group wished to ensure that these facts were known and appreciated by MPEG generally and so they were specifically highlighted in the closing plenary on the afternoon of 31 May. 5.3 CD Verification This item was generated by the wish of MPEG to have real time verification of the CD before releasing it for general comment. The implementation studies group agreed with this desire but did not accept that fulfillment of it was the responsibility of the group. They would however act as an information and coordination point. To this end, an analysis was done of the current situation. A decoding system operating in real-time is essential. A real-time encoder would be an advantage but is not essential. The possibility of a decoder was examined. For the video part, no organisation claimed to have a hardware decoder which conformed to the latest version of the CD. Several decoders were known to exist which approached the CD to a greater or lesser extent, but only for one was a date offered as to when it might be fully conformant. This was the programmable decoder decoder being put together by some COMIS partners and the end of August was the target date. For the system part, essentially the demultiplexing function, the only declared activity was again the COMIS project and it was envisaged that this would be ready in the same timeframe as the video decoder. For MPEG audio the situation appeared somewhat better with two hardware decoders identified. One of these was followed up in more detail and this revealed that interfacing it to the COMIS demultiplexer would require some extra interfacing circuitry extending beyond simple level shifters to clock generators/phase locked loops and buffering between intermittent and continuous data rates. It was not known if such work was in progress or planned. On the encoder side, it was thought that coded video could be produced fairly easily by computer simulation, although some minor changes to the CD had been made at the Paris meeting. It had become recommended practice in the Video group to produce bitstreams and several participants had successfully exchanged these. No activity on the system multiplexer was known to be in progress. This was thought to be of low computational complexity (but see later). For the audio encoder both the options of a real-time hardware encoder and an off-line computer simulation appeared available. However, it was found that the simulation programs had not been written to produce coded bitstreams, only decoded audio, and the hardware encoder had never been interfaced to a storage device. A floppy disk or tape cartridge containing a coded audio bitstream did not exist. Another potential problem identified concerned the synchronisation of audio and video by the system multiplexer. Unlike the video specification, the audio one does not have timecodes embedded in the compressed bitstream. Thus there is no method currently available to get the timecodes relating to the uncompressed audio tracks accompanying the video on a D1 recorder to the output side of the audio encoder. This will cause difficulties for the multiplexer which must combine appropriate sections of video with audio. The simple method will be to assume that the separately encoded audio and video bitstreams begin at the same time point and then to work forward by dead reckoning. There was a definite feeling that the issue of hardware verification of the draft CD had been left rather late in the day and information was sought from those with experience of the CCITT H.261 activity on the timescale and procedure adopted there. The validity of such a comparison was questionable because of MPEG's target of a much more highly integrated silicon solution. Nevertheless sufficient time for the verification phase should be built into the MPEG programme. As the Implementation Studies group has no power to produce a hardware demonstrator, the group's view was that it could not accept responsibility for the verification of the CD. It was suggested that this function should fall elsewhere within MPEG. However this was not pursued because shifting the work to another group would not solve the underlying problems and it would still be the same individuals who were involved. 5.4 MPEG-2 complexity analysis The previous meeting in Berlin had come to the conclusion that the complexity assessment procedure used for MPEG-1 should not be repeated for MPEG-2. It had been an objective process which produced hard numbers. However, there were severe doubts about the accuracy of such numbers. There would also be the difficulty of converting these numbers into agreed units which could be combined with the results from the picture quality tests etc. The meeting agreed to use a subjective process in which the proposals would be rank ordered. The first stage would involve assessment by implementation experts working individually. Their rankings would then be compared. Hopefully these would be similar but if not, discussions would take place to reach a common view. It was thought to be too demanding to assess in detail all the candidates as some 20 to 30 are expected. Some preselection will therefore be required. The group felt strongly that simply taking those which did best in the picture quality tests should not be the preselection method. The aim of the complexity assessment would be to provide information to enable MPEG to progress after the November tests. The Implementation Studies group would in effect seek to classify proposals into three groupings: - algorithms which are much simpler than average to implement. Even if their picture quality were not among the best these should not automatically be discarded. The collaborative efforts of MPEG might be able to improve their quality while retaining some or all of their simplicity. - algorithms which are much more complex than average to implement. Even if their picture quality were among the best these should not automatically be retained. MPEG would have to decide whether the extra picture quality was worth the extra complexity and whether other simpler approaches could be improved to be competitive. - algorithms which are of average complexity to implement. For these the implementation complexity would not be a major factor in deciding to retain or discard them. In addition to the descriptions, block diagrams, flow charts etc. which will accompany proposals some other information could be helpful to assessors. It was suggested that the group devise a questionnaire for proposers to complete. Further work on the details of the process were left to be worked out at the next meeting subject to approval of the outline proposed methodology by MPEG in plenary session. 5.5 Other business Astle raised the interesting hypothesis that processing was moving towards architectures employing parallelism and that some algorithms could be more or less favourable for these implementations. This topic was placed on the agenda for the next meeting and written contributions invited. --------------- Annexes to Document AVC-65R Annex 1 Documents for the second meeting of the Experts Group Annex 2 List of tape demonstrations Annex 3 Action plan for ATM picture format Annex 4 Framework for further study on comparison of VBR/CBR coding Annex 5 Liaison statement to SGXVIII Annex 6 Liaison statement to TG CMTT/2 Annex 7 Representatives to MPEG Sub-groups Annex 8 Input to the joint sessions with MPEG Annex 9 Editors for "Status Report" List of Participants of the second meeting of the Experts Group for ATM Video Coding (23 -31 May 1991, Paris) FRG Mr. F. May Daimler-Benz Research CM Mr. G. Zedler DBP Telecom CM Australia Mr. M. Biggar Telecom Australia CM Belgium Mr. L. Elewaut Alcatel Bell Telephone Mr. O. Poncin University of Louvain Mr. W. Verbiest Alcatel Bell Telephone CM Canada Mr. D. Lemay BNR CM USA Mr. P. Alexander PictureTel Mr. B.G. Haskell AT&T Bell Labs Mr. A. Deutermann DIS (CM) Mr. A.J. Tabatabai Bellcore CM Mr. J. Zdepski David Sarnoff Research Center France Mr. G. Eude France Telecom Mr. J. Guichard France Telecom CM Italy Mr. B. Riolfo CSELT (CM) Japan Mr. S. Hattori Mitsubishi Electric Mr. Y. Katayama GCT Mr. K. Matsuda Fujitsu Mr. T. Odaka Toshiba Mr. S. Okubo NTT Chairman Mr. T. Tanaka NTT CM Mr. M. Wada KDD CM Mr. M. Yano NEC Mr. T. Yukitake Matsushita Communication Mr. H. Watanabe NTT Norway Mr. G. Bjoentegaard Norwegian Telecom Mr. H. Sandgrind Norwegian Telecom CM Netherlands Mr. H. Carbiere PTT Research LR (CMTT) Mr. D. Schinkel PTT Research CM UK Mr. M.D. Carr BT Mr. D.G. Morrison BT CM Sweden Mr. H. Brusewitz Swedish Telecom CM CM: Coordinating Member (CM): Substitute for CM LR: Liaison Representative