Page 117 - Proceedings of the 2018 ITU Kaleidoscope

SMART USAGE OF MULTIPLE RAT IN IOT-ORIENTED 5G NETWORKS: A
REINFORCEMENT LEARNING APPROACH

Ruben M. Sandoval, Sebastian Canovas-Carrasco, Antonio-Javier Garcia-Sanchez, and Joan Garcia-Haro
Department of Information and Communication Technologies, Technical University of Cartagena, Cartagena, Spain

ABSTRACT In these Smart Cities ubiquitous machine-to-machine
(M2M) and machine-to-human (M2H) communications
Smart Cities and Smart Industries are the flagships of the enable cleaner, sustainable and more cost-efficient urban
future IoT due to their potential to revolutionize the way in spaces, ultimately improving citizens’ quality of life. A
which people live and produce in advanced societies. In
these two scenarios, a robust and ubiquitous communication similar concept emerges in the manufacturing and productive
infrastructure is needed to accommodate the traffic sector, where the term Smart Industry (SI) has been coined
generated by the 10 billion devices that are expected by the and received increasing attention in past years. In this
year 2020. Due to its future world-wide presence, 5G is particular communication environment, most of generated
called to be this enabling technology. However, 5G is not a data is derived from M2M interactions, either from
perfect solution, thus providing IoT nodes with different monitoring, actuation, or a combination of both -which gives
Radio Access Technologies (RATs) would allow them to rise to modern autonomous systems (e.g. control and
exploit the various benefits offered by each RAT (such as actuation on industrial plants)-. This computerization entails
lower power consumption or reduced operational costs). By a significant reduction in both cost and safety risks, since a
making use of the mathematical framework of Reinforcement real-time monitoring of critical facilities can be performed
Learning, we have formulated the problem of deciding which without the requirement of additional human resources.
RAT should an IoT node employ when reporting events.
These so-called transmission policies maximize a predefined This increase in the number of control, monitoring and
reward closely related to classical throughput while keeping actuation communication tasks in both cities and industries
power consumption and operational costs below a certain lead to a prominent rise in data generated. Since most IoT
limit. A set of simulations are performed for IoT nodes devices communicate via wireless communications, several
provided with two RATs: LoRa and 5G. The results obtained efforts are being devoted to the development of new Radio
are compared to those achieved under other intuitive Access Technologies (RATs) to alleviate the expected
policies to further highlight the benefits of our proposal. network congestion. Among these RATs, 5G excels as a
promising solution to enable power and cost-efficient
wireless network infrastructures. Long Term Evolution
Keywords – 5G, IoT, Reinforcement Learning, Multi- (LTE), that can be considered to have laid the foundations
RAT, LPWAN, Machine Learning for the 5G revolution, has already had a measurable positive
impact on the performance of cellular networks (one of the
1. INTRODUCTION most popular RATs for IoT). However, the number of IoT
devices is expected to reach 10 billion by the year 2020 [5].
The growing demand for automatization mechanisms in all This sheer increment in data transmitted along with the
quotidian areas has led to the Internet of Things (IoT) particularities of IoT-generated traffic, as discussed below,
paradigm. This technological revolution has enabled the have posed serious doubts about the sustainability of the
interconnection of all sorts of devices, known as things, by current and future cellular networks as an effective RAT for
equipping them with a communication module and a basic the IoT [6], [7].
sensor, memory, and computing unit. It is now extensively
demonstrated that endowing things with some intelligence It is worth mentioning that the traffic generated by the IoT is
has a positive impact on the entire society, leading to a better nothing like traditional phone-generated data, for which
management of critical resources such as water [1], energy cellular networks are very well suited for. This traffic is
[2] and city assets in general [3], [4]. mainly characterized by the transmission of sporadic small
bursts of data that, in most cases, should reach the destination
The progressive integration of different automation systems without an excessive delay. The large volume of aggregated
in urban environments is leading to the so-called Smart City. traffic (considering the expected 10 billion of devices) and
the sporadic nature and on-demand usage of the network,
This work has been supported by the project AIM, ref. TEC2016- implies a heavy signaling overhead that can potentially
76465-C2-1-R (AEI/FEDER, UE). Ruben M. Sandoval and Sebastian
Canovas-Carrasco also thank the Spanish MECD for an FPU (refs. hinder the performance of both, cellular network
FPU14/03424 and FPU16/03530) pre-doctoral fellowship. infrastructure and IoT devices [6], [8], [9].

978-92-61-26921-0/CFP1868P-ART @ 2018 ITU – 101 – Kaleidoscope