Page 107 - ITU Journal, ICT Discoveries, Volume 3, No. 1, June 2020 Special issue: The future of video and immersive media

ITU Journal: ICT Discoveries, Vol. 3(1), June 2020

JPEG PLENO: STANDARDIZING A CODING FRAMEWORK AND TOOLS
FOR PLENOPTIC IMAGING MODALITIES

1 2,3 4 5 6 7
Pekka Astola , Luis A. da Silva Cruz , Eduardo A. B. da Silva , Touradj Ebrahimi , Pedro Garcia Freitas , Antonin Gilles ,
11
8
9
12
Kwan-Jung Oh , Carla Pagliari , Fernando Pereira 10,3 , Cristian Perra , Stuart Perry , Antonio M. G. Pinheiro 13,3 ,
16
Peter Schelkens 14,15 , Ismael Seidel , Ioan Tabus 17
1 2 3
Tampere University, Finland, Universidade de Coimbra, Portugal, Instituto Superior Tecnico - Instituto de
4
5
Telecomunicaç es, Portugal, Universidade Federal do Rio de Janeiro, Brazil, Ecole Polytechnique F d rale de Lausanne
6 7 8
(EPFL), Switzerland, Samsung, Brazil, Institute of Research & Technology b-com, Rennes, France, Electronics and
9 10
Telecommunications Research Institute (ETRI), Korea, Instituto Militar de Engenharia, Brazil, Instituto Superior
11 12
Técnico, Universidade de Lisboa, Portugal, University of Cagliari, Italy, University of Technology Sydney, Australia,
13 14 15
Universidade da Beira Interior & Instituto de Telecomunicacoes, Portugal, Vrije Universiteit Brussel, Belgium, imec,
17
16
Leuven, Belgium, Samsung, Brazil, Tampere University, Finland
Abstract – JPEG Pleno is an upcoming standard from the ISO/IEC JTC 1/SC 29/WG 1 (JPEG) Committee. It aims to provide
a standard framework for coding new imaging modalities derived from representations inspired by the plenoptic function.
The image modalities addressed by the current standardization activities are light field, holography, and point clouds, where
these image modalities describe different sampled representations of the plenoptic function. The applications that may ben-
efit from these emerging image modalities range from supporting varying capture platforms, interactive content viewing,
cultural environments exploration and medical imaging to more immersive browsing with novel special effects and more re-
alistic images. These use cases come with a set of requirements addressed by the JPEG Pleno standard. Main requirements
envision high compression efficiency, random access, scalability, error-resilience, low complexity, and metadata support. This
paper presents a synopsis of the status of the standardization process and provides technical insights as well as the latest
performance evaluation results.
Keywords – Holography, JPEG Pleno, light field, plenoptic, point cloud, standardization.

1. INTRODUCTION Moreover, due to the rich description of scenes, often
large amounts of data are required for storing, process-
Modern imaging devices can capture a visual scene not ing and transmission. Current users of plenoptic devices
only as texture information, as in conventional images, predominantlyuse proprietarysolutionsfordata descrip-
but also as geometric scene information. These devices tion and file structure and for the selection of data com-
capture a sampling of the so-called plenoptic function pression tools before storage. Therefore, it is evident that
which represents the amount of light in time and in space, there is a lack of common and recognized tools to repre-
theoretically obtained by positioning a pinhole camera at sent and compress the different plenoptic functions. At
every viewpoint in 3D spatial coordinates, every viewing the same time there are several already existing applica-
angle, and every wavelength. tions, as well as novel applications with great opportuni-
ties in potential near future markets, that can benefit form
In practice, and depending on the capturing device, a sam-
these tools.
pling of the plenoptic function can be described by differ-
ent image modalities. JPEG Pleno particularly supports
point cloud, light field, and holography modalities. Some A common framework for facilitating the acquisition, con-
of the current consumer and industrial applications ex- version, processing, and rendering of these image modal-
ploiting these image modalities support use cases such ities is hence needed for fostering the development of
as: medical and industrial applications adopting holog- novel applications and for providing interoperability be-
raphy for visual inspections or object metrology; gam- tween different systems and technologies (Fig. 1). In
ing and augmented/virtual reality applications that use this context, the JPEG Committee – formally known as
point clouds as a main modality for scene description; ISO/IEC JTC 1/SC 29/WG 1 – is working on new standards
and photography and video creation industries authoring for light field, holography and point cloud compression.
applications that use light fields for computational post- This paper reviews the current status of the JPEG Pleno
production targeting scene refocusing, segmentation, and (ISO/IEC 21794) standardization efforts [1] and is orga-
immersive experience. Depending on the application, nized as follows. The use cases for the JPEG Pleno stan-
conversion from one image modality to another is essen- dard are summarized in Section 2. The requirements that
tial (Fig. 1). have been established are reported in Section 3. The four

© International Telecommunication Union, 2020 85