Page 108 - ITU Journal, ICT Discoveries, Volume 3, No. 1, June 2020 Special issue: The future of video and immersive media

ITU Journal: ICT Discoveries, Vol. 3(1), June 2020

Fig. 1 – Conceptual illustration of the JPEG Pleno workflow and file format. A point cloud is encoded an encapsulated in a JPEG Pleno file. Thereafter
it is converted to a hologram deploying a computer-generated holography technique with the original intention to display it on an holographic display.
However, an end user does not have a holographic display at its disposal and converts it to a light field for display on a light field display. Note that one
of the powers of the JPEG Pleno file format is that it allows for storing simultaneously e.g. the holographic and light field data (not illustrated).

current parts of the JPEG Pleno standard are reviewed in files and even measure surface reflection and polariza-
Section 4 and, in addition, future extensions are discussed tion features. Finally, 3D profiling of larger scenes can
that are currently being prepared. Finally, Section 5 con- be achieved by deploying lidar technology, producing pre-
cludes this paper. cise scene descriptions based on point clouds.

2. USE CASES A particular use case is the recording of archaeological
sites, statues and monuments which provide a detailed
The intrinsic 3D nature of plenoptic imaging naturally
register of historical data [6]. Smaller artifacts can be
provides the basis to a large number of applications. In
scanned more accurately utilizing these modalities. Even
the context of JPEG Pleno standardization, multiple use
the 3D texture of paintings can be registered allowing for
cases were identified and analyzed to derive a list of ap-
more in-depth analysis of e.g. the painting technique and
propriate requirements for the representation of the var-
issued materials.
ious addressed modalities. These use cases have been
discussed in several modality specific “Use Cases and Re- Plenoptic data also improves the perception of the real
quirements documents” ’ [2, 3, 4]. Examples of use cases world by machines, providing accurate 3D representa-
are: tions of the surrounding environment. Safe navigation
• photography and video production, of autonomous vehicles is enabled by 3D scanning of the
• virtual, augmented and mixed reality, surrounding environment [7].
• 3D content creation,
An exploding market is 3D visualization provided by
• health applications,
head-mounted displays or holographic/light field dis-
• manufacturing and construction,
plays. These devices require also compact representa-
• retail, robotics,
tions in order to stream the content to these displays and
• surveillance with drones,
support interactivity. The applications here are ubiqui-
• cultural heritage.
tous. Finally 3D and holographic printing are relevant in
For example, in industry, plenoptic representations are
the context of the manufacturing of prostheses, and holo-
deployed in non-destructive testing to build a better un-
graphic prints.
derstanding of the structure of an object under test. Fur-
thermore, lenslet-based plenoptic cameras capture light As shown in Fig. 2, the JPEG Pleno framework provides
fields with a small baseline, but potentially a large num- support for these use cases both at creation or acquisi-
ber of supported viewing angles. A sufficiently large num- tion, or during consumption. Furthermore, a set of tools
ber of pixels under each microlens in both parallax di- based on JPEG Pleno associated metadata allows the user
mensions results in a depth resolution that allows for to interact at different levels in an E2E chain and main-
accurate surface profiling. Alternatively, holographic in- tain even the necessary information during format con-
terferometry [5] can provide nano-precision surface pro- version.

86 © International Telecommunication Union, 2020