Page 55 - ITU Journal, ICT Discoveries, Volume 3, No. 1, June 2020 Special issue: The future of video and immersive media

ITU Journal: ICT Discoveries, Vol. 3(1), June 2020

ARENA-STYLE IMMERSIVE LIVE EXPERIENCE (ILE) SERVICES AND SYSTEMS:
HIGHLY REALISTIC SENSATIONS FOR EVERYONE IN THE WORLD

Jiro Nagao , Kiyoshi Tanaka , Hideo Imanaka
1,2
3
1
1 NTT Service Evolution Labs, Japan, Otsuma Women’s University, Japan, NTT Advanced Technologies, Japan,
2
3

Abstract – Immersive Live Experiences (ILEs) enable audiences at remote sites to feel real-time highly
realistic sensations, as if they are at the event site. This article provides a key feature of an implementation
of ILE services, called Kirari! for Arena, as a use case of arena-style ILE, and its technical elements developed
by NTT Labs. The major functionalities are object extraction from an arbitrary background, object tracking
with depth sensors, low-latency synchronized data transport and four-sided pseudo-3D image presentation
with depth expression. It also provides evaluations on the experience of Kirari! for Arena audiences, as well
as its conformance to International Telecommunication Union, Telecommunication Standardization Sector
(ITU-T) standards for ILE.

Keywords – AI, immersive live experience, MPEG media transport, pseudo-3D images, real-time image
extraction, standardization.

1. INTRODUCTION This article provides a key feature of an
implementation of ILE called “Kirari! for Arena” [1],
The worldwide spread of TV enabled people to that provides an audience at a remote site with the
watch events happening anywhere in the world unprecedented, highly immersive experience of
almost at the same time, which was unimaginable to watching a stage located in the middle of the
people in the past. However, the experiences that audience from all directions. This article
TV can provide are still far from real. Information contributes by providing evaluations of audience
regarding the real size of things, natural sounds at experiences of Kirari! for Arena, as well as its
the site, and the other sides of objects cannot be conformance to International Telecommunication
depicted by a 2D display.
Union, Telecommunication Standardization Sector
Viewing styles of large events have been evolving (ITU-T) standards for ILE.
recently, that is, a number of large music concerts STANDARDIZATION ACTIVITY OF ILE
and sports events have been relayed to remote sites 2.
for public screening. Even if the remote sites are far AND RELATED WORKS
from the real event site, the audience can share their 2.1 ITU-T H.430 standard series
emotions while watching the event. Public
screenings can be held in movie theaters or in front To provide interoperable ILE services worldwide, it
of large public screens. is necessary to establish international standards on
ILE systems, especially their requirements and
If, in addition to video and audio, environmental framework. The ITU-T H.430 series specifies the
information of an event such as vibrations and requirements, framework, service scenarios, and
feelings of excitement is reproduced at the remote MPEG Media Transport (MMT) profile for ILE
site, the experience of remote public screening can [2][3][4][5]. The series of Recommendations is
be further enhanced, resulting in the audience listed as follows.
feeling as if they are at the real event site.
• H.430.1: Requirements of ILE services.
This feeling of being in a highly realistic atmosphere is • H.430.2: Architectural framework of ILE
called immersiveness, and enjoying an event in a remote services.
site in real-time with immersiveness is called an
Immersive Live Experience (ILE). Research and • H.430.3: Service scenarios of ILE including
development to implement ILE has been conducted, and use cases.
a number of ILE-related technologies have been • H.430.4: Service configuration and MMT
standardized to ensure the interoperability between ILE profile for ILE.
service providers and to facilitate the market.

© International Telecommunication Union, 2020 33