Page 58 - ITU Journal, ICT Discoveries, Volume 3, No. 1, June 2020 Special issue: The future of video and immersive media

ITU Journal: ICT Discoveries, Vol. 3(1), June 2020

Fig. 4 – Functions of Kirari! for Arena
The Depth Expression & Presentation function uses
a special four-sided display device, which plays a
major role in giving a sense of reality to the Fig. 5 – Four-sided display of Kirari! for Arena
reconstructed images.
The display device is composed of four half-mirrors
as shown in Fig. 5 combined in the shape of a
quadrangular pyramid, and each half-mirror
reflects the image of a flat display installed above it,
facing down. The reflection presents an aerial
virtual image of objects to the audience (Fig. 6). Fig. 6 – Device configuration to show 2D aerial image
Inside the half-mirrors, a physical plate (stage) is
installed as the “floor” of the virtual images. Here, the audience perceives the aerial image as a
real object because the image reflected by the half-
The device uses the principle of Pepper’s ghost mirrors appears to be floating in the air without any
[16][17] to show pseudo-3D images on an arena support. The Depth Expression & Presentation
stage without HMDs. The principle uses the function also processes the object images to
reflection of a half-mirror to create a form of illusion increase the sense of depth.
by showing both the real objects behind the half-
mirror and the reflected image at the same position. To enable the audience to perceive as if a real object
There are other approaches to show 3D images such is there, the object needs to be finely extracted. In
as those that use an array of lenticular lenses [25]. addition, real-time processing is essential to realize
However, the Pepper’s ghost principle was selected a live experience. Real-time object extraction is
for the following reasons. conducted by the Capture & Extraction function.
Kirari! for Arena requires the views from all four The Measurement & Tracking function measures
directions to be shown at the center of the stage, and tracks the 3D position of the object to provide
that is, the four images of the performer should be the depth information (3D position information)
positioned at the same physical spot. Transparent used by the Depth Expression & Presentation
LED panels cannot be used because four of them function.
would have to be placed at the center, intersecting A large-capacity, low-latency media transport
with each other, which is physically impossible. 3D system is indispensable for displaying high-
displays with lenticular lenses can show aerial resolution video streaming at remote sites in real-
virtual images outside of their own physical devices time. Kirari! for Arena requires transport
but still need the devices behind the images. technology that can send four video streams
Therefore, even if the four floating images can be synchronously. The Information Integration &
placed at the center, the devices behind the images Transport function integrates the extracted object
would obstruct the views. The Pepper's Ghost images from the Capture & Extraction function and
principle does not cause such problems and enables the position information from the Measurement &
aerial virtual images to be placed at the center of the Tracking function, and transmits the large amount
stage, intersecting with each other without of data to the Depth Expression & Presentation
obstruction. This principle was also selected function in a low-latency and synchronized manner.
because of its cost efficiency and scalability.

36 © International Telecommunication Union, 2020