Page 9 - ITU Journal, ICT Discoveries, Volume 3, No. 1, June 2020 Special issue: The future of video and immersive media

ITU Journal: ICT Discoveries, Vol. 3(1), June 2020

As explorations into immersive media technology continue to thrive, the need for standardization has
not gone unnoticed by the ITU, ISO and IEC. Already, standardization projects including ITU-T’s ILE
work and the Moving Picture Experts Group (MPEG)’s metadata for immersive video (MIV) and video-
based point cloud coding (V-PCC) standards have been used to build systems showcased in some of
the previously discussed papers. To present a more complete picture of immersive standards, other
papers in this issue are dedicated to overviews of emerging video standards.
Streaming of omnidirectional video (a.k.a 360-degree video) content is already available on some large-
scale streaming portals. The MPEG Omnidirectional MediA Format (OMAF) standard creates a
standardized solution for 360-degree video streaming. The paper “Multi-Viewpoint and Overlays in the
MPEG OMAF Standard” reviews the soon-to-be-completed second edition of the MPEG OMAF
standard. It details two major new features of OMAF version 2, i.e., the capability of using overlays
and multiple cameras situated at different locations, and new use cases that they enable.
Advances in image technology can span a much richer domain than traditional two-dimensional
photographic pictures. “JPEG Pleno: Standardizing a Coding Framework and Tools for Plenoptic
Imaging Modalities” provides a detailed review of JPEG Pleno, the upcoming immersive image coding
standard. New image modalities that can be compressed by JPEG Pleno include light fields, holography,
and point clouds; and JPEG Pleno further strives to fulfil important requirements such as high
compression efficiency, random access, scalability, and metadata support. Also in this special issue is
an invited paper discussing the history of the ubiquitous original JPEG standard, which has been woven
into the fabric of our lives. Developed about 30 years ago, the JPEG standard is going strong, with the
volume of its usage being measured in the billions of pictures. Its development was recently recognized
by a Primetime Emmy award in October 2019.

No multimedia experience would be considered immersive if it did not consider our audio experience
as well as visual perception. “A New Set of Directional Weights for ITU-R BS.1770 Loudness
Measurement of Multichannel Audio” discusses immersive audio. Based on the ITU-R BS.1770
multichannel loudness algorithm, a modified version of the loudness algorithm using a new set of
directional weights is reported to perform better than the benchmark, particularly for a system with
more loudspeakers positioned out of the horizontal plane.

This special issue also includes “Overview of MPEG-5 Part 2 - Low Complexity Enhancement Video
Coding (LCEVC),” describing an emerging video standard that deals with the enhancement of
conventional video coding technology. The MPEG LCEVC standard is designed for use in conjunction
with existing video codecs. With a small number of additional tools for encoding the difference between
the original video and a compressed representation, LCEVC can provide a visual quality enhancement
with minimal computational complexity.

Today’s world finds many video codecs in the marketplace. The paper “The Video Codec Landscape
in 2020” observes the highly active area of video codec development in recent years, and aims to analyse
several video coding standards developed by the ITU-T, ISO and IEC, including Versatile Video
Coding (VVC), Essential Video Coding (EVC) and High Efficiency Video Coding (HEVC), as well as
the AV1 design developed in an industry consortium. The comparative study includes discussion of the
respective coding tools, compression efficiency, complexity, and other factors that impact market
deployment.

Nowadays, one can hardly talk about technology advancement without mentioning machine learning.
The third part of this special issue includes two papers related to this exciting breakthrough technology.
The first paper “Data-Driven Intra-Prediction Modes in the Development of the Versatile Video Coding
Standard” details an intra prediction method in the VVC standard that was originally inspired by
machine learning technologies. The method was simplified such that it is implementable with today’s
hardware capabilities, and became known as matrix-based intra prediction (MIP) in VVC. Recognizing
the proliferation of neural networks in many applications, the second paper “Trends and Advancements

– vii –