Page 56 - ITU Journal, ICT Discoveries, Volume 3, No. 1, June 2020 Special issue: The future of video and immersive media

ITU Journal: ICT Discoveries, Vol. 3(1), June 2020

The first two (H.430.1–H.430.2) Recommendations These commercial services and studies, however,
provide the basic concept of ILE, and ITU-T H.430.3 do not realize ILE because they either could not use
provides service scenarios. ITU-T H.430.4 is the first a real event scene as input, could not process and
to specify the technical aspect of ILE including how show images in real-time, or did not have a real-
to transport additional information such as 3D time transport system to send the necessary
positions of objects in synchronization with the content to remote sites.
audio and visual stream. More Recommendations TECHNOLOGY SUITE FOR ILE
with technical specifications may follow. 3.
This section introduces the Kirari! technology suite.
ITU-T H.430.1 defines the term “ILE,” and specifies
the high-level requirements of ILE. The Kirari! has been developed to provide technology
characteristic experiences ILE provides are as components to realize immersive experiences. This
follows. section also introduces a number of past events
where Kirari! demonstrated its ability to realize
• Watch the event from a remote place in real immersive experiences.
time.
3.1 Kirari!
• Feel the event as if it is happening right there,
with a considerably higher sense of realism. The Kirari! technology suite, which can produce a
number of implementations of ILE, is a suite of
• Share excitement with people around them.
technologies developed by NTT that are necessary
2.2 Related works to reproduce events at remote sites with high
realism [19]. The suite consists of technologies to
Various studies have been conducted to realize
immersiveness, and a number of technologies have capture, measure, process, transport and display an
been applied to commercial services. 3D movie event.
theaters as commercial services have become Figure 1 shows the technology map of Kirari!
popular [6][7][8]. Moreover, vibration, light, air
flow, as well as 360-degree 3D displays have started
to be used in commercial theaters [9][10]. These 3D
theaters require the audience to wear Head
Mounted Displays (HMDs) or 3D glasses.
There have been a number of studies on watching
sports events with HMDs [11][12]. In these studies,
the audience can see from the viewpoint of athletes
or from a free point of view. However, those
approaches have a problem in which the HMDs
prevent the audience from communicating with
those around them while watching the event,
resulting in a difficulty of sharing their emotions
between themselves.
Other studies have focused on improving the
immersiveness and comprehension of scenes by
reproducing vibro-tactile sensations [11][13][14].
These technologies can realize ILE if the sensations
of real events are transported and reproduced in
real-time along with video and audio. Effectively
combining these technologies with Kirari! for Arena
can also further enhance its immersiveness.
Highly realistic experiences without HMDs have Fig. 1 – Kirari! technology suite
been an active research target as well [15][16][17], Real-time image extraction can identify the region
including a number of forms of multi-angle display of an object (e.g. performer on the stage) in an
devices [16][17][18]. image in real-time so that the object image can be
used to reconstruct the scene at the remote site.

34 © International Telecommunication Union, 2020