Page 111 - ITU Journal, ICT Discoveries, Volume 3, No. 1, June 2020 Special issue: The future of video and immersive media

ITU Journal: ICT Discoveries, Vol. 3(1), June 2020

Fig. 3 – JPEG Pleno is currently composed of four parts covering the file format, coding of light fields, conformance testing and reference software [9]

about the plenoptic content, eventually vendor-specific sume that these (virtual) cameras are positioned in a pla-
information. nar array in the local coordinate system. Fig. 4 shows
the sub-aperture views from the Bikes lenslet-based light
Thereafter, an optional JPEG Pleno Thumbnail box can be field. View prediction and synthesis algorithms associ-
included that contains a snapshot image of the complete ated with the codec enable accounting for small offsets
plenoptic scene covered described by the JPL file. This of the cameras within the camera array plane. The cam-
will allow the end user to get a quick visual understand- era calibration box signals this information in conjunc-
ing of the contained content without having to decode the tion with an additional translational offset perpendicu-
complete file. This snapshot can be quite advanced and, lar to the camera array plane positioning as well as rota-
for example, represent a multispectral image in case of tional offsets and intrinsic camera parameters – enabling
multispectral plenoptic content. as such more complex camera arrays, where cameras
might even be positioned on a spherical surface. How-
Next, the actual plenoptic content boxes are included.
ever, note that the prediction tools in the codec are only
Separate boxes are defined for light field, point cloud or
based on the assumption of a planar configuration, albeit
holographic content, respectively the JPEG Pleno Light
some small in-plane, translational calibration offsets can
Field, Point Cloud, and Hologram boxes. Currently, only
be accounted for.
the formatting of the first has been defined in JPEG Pleno
Part 2. The committee has issued an FDIS ballot text as
document wg1n87033 in May 2020 [12]. The boxes re-
lated to other modalities are intended to be defined in the
future.
In summary, the JPL format represents a flexible box-
What is interesting to be mentioned at this point is that
based file format that is extensible such that future sup-
JPEG Pleno Part 1 [9] contains also the specification of a
ported modalities and functionalities can be efficiently in-
global and local reference grid system. Individual plenop-
tegrated.
tic data sets can be positioned within the global reference
grid system at a particular position and with a particular
rotation. The local reference grid system is utilized by the
codec engine operating on that elementary plenoptic unit.
In the earlier example of a point cloud segmented in N in-
dividual point clouds, the global coordinates and rotation
of each sub-point cloud would be signalled, but the point
cloud codecs would operate in different local coordinate
systems.

The file format extensions defined in JPEG Pleno Part 2 al-
low for efficient signalling of the codestream elements of
the encoded light field (see Section 4.2), but also of cali-
bration information for the light field. Intrinsically, Part 2
assumes a sub-aperture view based light field represen-
tation where each sub-aperture view corresponds with a
particular position of a pinhole camera. The codecs as- Fig. 4 – Sub-aperture views from Bikes lenslet-based light field.

© International Telecommunication Union, 2020 89