Page 110 - ITU Journal, ICT Discoveries, Volume 3, No. 1, June 2020 Special issue: The future of video and immersive media

ITU Journal: ICT Discoveries, Vol. 3(1), June 2020

• Different (non-)linear capturing configurations – 4. JPEG PLENO
shall support the representation of content produced
The JPEG Pleno activities are currently at the draft inter-
by various linear/nonlinear configurations and sig-
national standard (DIS) phase for both Part 1 of the stan-
nalling the associated metadata, notably microlens
dard, which addresses the JPEG Pleno framework, and
arrays on sensors, sensor arrays of different regular
for Part 2, which addresses the light field coding tools.
or irregular layouts, rotating sensors/objects;
The deadline for Part 1 and 2 to be submitted for the ap-
• Carriage of supplemental depth maps – should sup- proval stage of the Final Draft International Standard (fi-
port carriage of supplemental depth maps as part of nal draft international standard (FDIS)) is spring 2020.
the codestream or file format. Part 3 and Part 4 activities started in October 2019 and
the expected submission deadline for the FDIS approval is
October 2020. Fig. 3 summarizes the current JPEG Pleno
3.3 Point cloud specific requirements
parts.
Regarding the specific point cloud requirements, the most
recent JPEG Pleno Point Clouds – Use Cases and Require- 4.1 JPEG Pleno Part 1: Framework
ments document [3] focuses on the following properties:
This part defines the file format, named JPL (Jpeg PLeno),
• Geometry scalability – shall allow for decoding the providing a signalling syntax for storing more detailed in-
point cloud geometry using only a part of the full bit- formation and contextualization for plenoptic content of
stream. This shall provide for point clouds with an which codestreams are encapsulated in the file. The com-
increasing number of points, with increasing quality, mittee issued an FDIS ballot text for this part as docu-
for a specified bit depth and increasing bit depth as ment wg1n86056 in February 2020 [9]. The JPL format is
additional layers are decoded; a box-based media file format, which relates back to the
Apple Quicktime format and which belongs to the same
• Attributes scalability – shall allow for decoding point family as the ISO/IEC 14496-12 ISO Base Media File For-
cloud attributes using only a part of the full bit- mat. The actual specification is based on the same syntax
stream. This shall provide point cloud attributes of as the box-based file format of JPEG 2000 in JPEG 2000
increasing quality as additional layers are decoded; Part 1 [10] or Part 2 [11]. The box-based file format acts
as a binary container composed out of boxes, where each
• Random access – shall allow the selective decoding of
box – named as a superbox – can contain other boxes.
a portion of the point cloud, allowing for selectively
decoding a portion of the point cloud corresponding A JPL file will start with a JPEG Pleno Signature box
toavolumein3Dspaceoraportionofthepointcloud and a File Type box, respectively signalling it concerns
visible from a given viewpoint. a JPEG Pleno family file and versioning/compatibility in-
formation. Next, an optional XML box with a catalogue
3.4 Holography specific requirements can be included that provides a directory of the included
plenoptic content. One of the nice features of a JPL file
Regarding the specific holography requirements, the
is that it can contain multiple plenoptic data sets repre-
most recent JPEG Pleno Holography – Use Cases and Re-
sented in different modalities, i.e. point cloud, light field
quirements [4] lists the following additional features:
or holographic data. This allows us to maintain differ-
• Hologram types – shall support the following holo- ent representations of the same content throughout an
gram types: amplitude modulation hologram, phase application chain and/or to fragment a complex data set
modulation hologram, and complex modulation into smaller chunks that can be independently encoded.
holograms; The latter could potentially be interesting in the context
of large point clouds, which are hard to handle as a whole
• Representation format – Shall support one or more due to memory constraints, or which should preferably
of the following representation formats for the sup- be handled in smaller units because individual parts have
ported hologram types: amplitude-only, phase-only, a different semantic interpretation. A particular prob-
amplitude-phase, and real-imaginary; lem of plenoptic data is that a single codestream can be
extremely large compared to a regular image or video
• Field of view – shall support a large field of view; content. Identifying the location of a particular code-
stream in a JPL file might hence result in an excessive file
• Bit depth – shall support holographic data with bit
parsing effort. Therefore, including an XML catalog box
depth ranging from bi-level to 16-bit integer preci-
is a more efficient mechanism that allows for providing
sion.
pointers that can direct the file parser immediately to the
right location in the file to read a particular codestream.
In summary, the requirements above are those defining
Besides the XML catalog box, the JPL file might contain
the Call for Proposals and guiding the whole JPEG Pleno
specification process. other XML boxes, UUID (info) boxes and IPR (Intellectual
Property Rights) boxes, signalling additional information

88 © International Telecommunication Union, 2020