Page 39 - Kaleidoscope Academic Conference Proceedings 2022

Standard models and interfaces are also required to generate Outside the ISO/IEC standards suite, ITU-R BT.2133-0
virtual components and facilitate data exchange on XR describes elements required for the transport of immersive
applications. The catalog of published work on this subject audio-visual content using the omnidirectional media format
features a variety of Application Programming Interface (specified in ISO/IEC 23090-2) in IP-based broadcasting
(API) definitions and requirements for representing virtual systems.
3D scenes.
IEEE 1857.9 specifies an efficient coding toolset for
The Khronos group’s XR API suite provides developers with immersive visual content and current work in progress under
tools to create interoperable XR experiences (OpenXR) and the IEEE P2048 series is exploring data formats for
generate 3D scenes/assets (3DCommerce and gITF), and immersive audio and video. The IDEA’s Immersive
high-performance low-latency 3D graphics (Vulkan, Technologies Media Format (ITMF) specification provides
OpenGL and WebGL). a reference for data exchange formats supporting end-to-end
delivery of immersive media.
From the ISO/IEC suite, ISO/IEC 23090-8 defines APIs for
immersive media processing and the extensible 3D (X3D) 3.2 User experience
standards provide a format and architecture for the
representation of 3D scenes/objects (ISO/IEC 19775 and 3.2.1 Accessibility and human factors
19776 series). Specific formats for XR are also defined in
ISO/IEC 18038 to 18040, 18520 and 23488. XR technologies impose very high requirements for usability.
This requirement is even greater for users with specific needs,
Although not published yet, ongoing work in W3C is set to particularly those with sensory impairments or limited
provide developers with APIs to build immersive mobility (where movement is a component of user
applications that can be accessed on the web under the interaction). [12] provides more insight on accessibility
WebXR series. requirements within immersive media and standardization
opportunities in this context.
3.1.3 Data presentation and formats
To support the delivery of optimally accessible XR
The suite of available standards on data formats for experiences, the W3C’s working group note on ‘XR
immersive media has largely been outputs from MPEG and accessibility user requirements’ and ETSI EG 202 848
JPEG and comprises of ISO/IEC 23000-13 and the ISO/IEC outline a set of accessibility guidelines containing accessible
21794, 23005 and 23090 series. The completed seven-part user design principles and architectural considerations for
ISO/IEC 23005 (MPEG-V) set describes an architecture and XR system designers and product developers. CTA 2095
outlines requirements for interoperability between virtual also offers some considerations for XR users with limited
and real worlds. ISO/IEC 23000-13 blends some elements mobility.
from this series to specify AR data formats.
Complementary to this work, the IEEE’s accessibility and
The five-part ISO/IEC 21794 (JPEG Pleno) set complements digital inclusion working group are currently developing
the MPEG-V series by providing a framework and coding testing criteria to measure XR accessibility in IEEE P2843.
tools to facilitate the representation of new image modalities
required to support immersive environments. Currently 3.2.2 Quality aspects
published work in this series specifies the general framework
(Part 1) and coding tools for light field modalities (Part 2). Assessing quality of experience is a crucial factor in
The series also offers guidance on conformance testing evaluating overall user experience. Activities of the ITU,
(Part 3) and implementing reference software (Part 4) for the IEEE and 3GPP have yielded a series of technical
series. The final piece in the series (Part 5), currently under specifications and standards on conducting QoE evaluation
development, is exploring a similar set of coding tools for of XR.
holographic imaging.
Current publications provide guidance on understanding
Completed work in the ISO/IEC 23090 (MPEG-I) series factors influencing user-perceived experience for VR
specifies a set of data formats and coding tools for immersive (ITU-T G.1035) and AR (ITU-T G.1036), outline a set of
media. This work is composed of formats, conformance quality metrics (ISO/IEC 23090-6 and 3GPP TR 26.929) and
testing and reference software implementation for KPIs (3GPP TR 26.998, ITU-T GSTR-5GQoE) for
omnidirectional media (parts 2 and 17), improved features immersive media and describe assessment methodologies for
for versatile video coding (Part 3) and data compression and immersive audio (3GPP TS 26.259 and TS 26.260), 360º
carriage for point clouds (parts 5 and 10). Ongoing activities video (ITU-T P.919), XR telemeetings (ITU-T P.1320), and
on the ISO/IEC 23090 series are looking into specifying perceptual quality of 3D visual content (IEEE 3333.1 series).
coding formats for immersive audio/video [11] and
volumetric, point cloud and versatile video types.

– xxxv –