Page 19 - Kaleidoscope Academic Conference Proceedings 2022

simply using computers and smart devices as gateways is not streaming protocols, and mulsemedia, which are essential
enough, and more immersive spatial computing technologies, technologies for the high QoE metaverse. After that, we will
such as XR which includes AR, MR, and VR, are preferred [4]. discuss the wireless communication challenges and potential
XR uses HMDs which integrate displays, inputs, and various solutions for XR and HTC. Since both of them require high
sensors together. HTC is another way to get access to the reliability, high data rates, low latency, and full connectivity
virtual worlds which can use HMDs and holographic displays to support high QoE metaverse applications, we introduce
[6]. It can display truly immersive 3D objects as holograms generic solutions and also consider their unique
using light field d isplays, w hich can be d irectly o bserved by characteristics. The general research directions to meet the
users with naked eyes. Without HMDs, the experience of living required performances for wireless XR and HTC include the
and working in the virtual worlds becomes very similar to real following aspects.
life. • Reliable high-data-rate low-latency wireless
XR is a spatial computing technology that enables users to communications are required to allow users to move
observe and interact with virtual worlds. XR technologies are freely without constraints. Cables that are tethered to
categorized based on Milgram and Kishino’s some of the existing XR HMDs limit mobility.
Reality-Virtuality Continuum [7], where AR only has simple • Accurate high-fidelity wireless sensing technologies can
virtual content and the majority of the environment in AR is be used to monitor and predict users’ motion and render
real, while VR has fully virtual content, and the user’s content adaptively. Existing XR HMDs use the
observation using VR devices is not related to the user’s integrated head, hand, and eye tracking sensors. The
surrounding real environment. MR comprises mixed real and ultimate goal is to remove HMDs and users can get into
virtual content, and the virtual content can be interactive and the metaverse without wearing any devices. Wireless
dynamic. Although MR and AR have overlapped, MR is sensing of the motion of the head, hand, body, and eyes
considered a broader concept than AR with more interactive can efficiently free the users.
and richer virtual content. • New Internet protocols and wireless networking
HTC is expected to provide truly immersive experiences technologies are needed to provide new services and
for users when getting access to the virtual worlds. HTC is new capabilities to deliver metaverse content with
characterized by the delivery of holograms and other different QoEs.
multi-sensory media (mulsemedia) through wireless and • Computing and content caching need to be moved from
wired networks [6]. A hologram is a recording of a light the core cloud towards the network edge, in order to
field that preserves the original depth and parallax of real 3D reduce the end-to-end latency and support highly
objects. Hologram videos could provide interactive applications.
six-Degrees-of-Freedom (6DoF) immersive viewing This keynote will provide details about research challenges
experiences to users, that is, users can freely move and solutions for communication, sensing, networking,
forward/backward (surging), up/down (heaving), or left/right caching, computation, and metaverse user experiences.
(swaying) to select the favorite viewing angle of a 3D scene
[8]. Furthermore, HTC research aims to stimulate all the III. CONCLUSION
senses of human perception, including sight, hearing, touch, This keynote will introduce metaverse accessing
smell, and taste, to provide a truly immersive experience to technologies including the extended reality (XR) and
users via mulsemedia. Mulsemedia is the media that Holographic-Type Communications (HTC). We classify
stimulates three or more senses, which extends traditional different wireless communications in the metaverse and point
multimedia that usually supports only two senses of sight out the unique requirements for XR and HTC. Then, we
and hearing. HTC systems may collect mulsemedia data discuss research challenges and potential solutions in 6G and
using various sensors at the source and then regenerate the beyond wireless systems. This keynote aims to stimulate
environment using various actuators, displays, and speakers future research around XR and HTC systems.
at the destination. Users can select the senses depending on
the available hardware and privacy and security issues. A REFERENCES
generic HTC system consists of the source, the destination, [1] F. Tang, X. Chen, M. Zhao, and N. Kato, “The roadmap of communication
and the networks. The source uses various sensors to capture and networking in 6g for the metaverse,” IEEE Wireless Communications,
2022.
holographic content, synchronizes multi-sensory data, [2] Y. Cai, J. Llorca, A. M. Tulino, and A. F. Molisch, “Joint
encodes the holographic data, and follows HTC networking compute-caching-communication control for online data-intensive service
delivery,” arXiv preprint arXiv:2205.01944, 2022.
protocols to send data packets. The HTC networks deliver [3] R. Li, K. Makhijani, and L. Dong, “New IP: A data packet framework
source data with guaranteed performance in bandwidth, to evolve the internet,” in 2020 IEEE 21st International Conference on
latency, reliability, etc. which are defined by the High Performance Switching and Routing (HPSR). IEEE, 2020, pp. 1–8.
HTC-enabled use cases. The destination receives and renders [4] I. F. Akyildiz and H. Guo, “Wireless communication research challenges
for Extended Reality (XR),” ITU Journal on Future and Evolving
data for display, utilizes various actuators to regenerate the Technologies, vol. 3, no. 1, pp. 1–15, 2022.
environment at the source, performs synchronization among [5] V. B. S. Kalamkar, N. Nouri, E. Ofek, M. Pahud, J. J. Dudley, J. Hu,
multi-sensory media, and provides useful feedback to the P. O. Kristensson, M. Weerasinghe, K. ˇ C. Pucihar, M. Kljun et al.,
“Quantifying the Effects of Working in VR for One Week,” arXiv preprint
source and the network if necessary. The design of an entire arXiv:2206.03189, 2022.
HTC system requires knowledge from several technical [6] I. F. Akyildiz and H. Guo, “Holographic-type Communication: A New
areas, such as sensors and actuators, computer vision, data Challenge for The Next Decade,” ITU Journal on Future and Evolving
Technologies.
compression, and communication and networking. The rest [7] P. Milgram and F. Kishino, “A taxonomy of mixed reality visual displays,”
of this section highlights current technologies that are most IEICE TRANSACTIONS on Information and Systems, vol. 77, no. 12, pp.
relevant to communications. 1321–1329, 1994.
[8] Z. Liu, Q. Li, X. Chen, C. Wu, S. Ishihara, J. Li, and Y. Ji, “Point cloud
This keynote will provide more details about XR devices, video streaming: Challenges and solutions,” IEEE Network, vol. 35, no. 5,
XR communication and networking, hologram presentation pp. 202–209, 2021.
and display, hologram compression, hologram network

– xv –