Page 124 - ITU Journal, Future and evolving technologies - Volume 1 (2020), Issue 1, Inaugural issue

ITU Journal on Future and Evolving Technologies, Volume 1 (2020), Issue 1

Once multiple slices have been activated in the RAN, paid to the practical implementation aspects of
the cross-slice resource optimization shall ensure these solutions, as it will be further discussed in
that the slice requirements are satisfied over time Section 2. In this respect, departing from 3GPP and
and RAN resources are efficiently utilized. This may O-RAN Alliance specifications, a first contribution of
imply a dynamic modification of the slice this paper is the delineation of the functional
configurations (e.g. specifying the amount of radio framework and information models to be accounted
resources assigned to each slice at each cell, when targeting a practical realization of ML-
adjustment of slice-aware scheduling settings, assisted cross-slice radio resource optimization
configuration of rate limiters, bandwidth parts, solutions for 5G and beyond systems. More
mobility load balancing parameters, access control specifically, the focus is put here on the
priorities, etc.) during its lifetime in order to deal identification of the specific functional components
with the dynamics of the traffic load of the slice and enabling the deployment of ML-based solutions for
with the random propagation effects that lead to RAN management along with the set of information
non-deterministic mapping between radio resource models that have been defined to represent SLAs,
consumption and performance requirements. network slice instances’ characteristics and slicing-
Cross-slice resource optimization has been related configuration parameters of 5G base
identified by 3GPP as a use case in the context of stations. On this basis, a second contribution of this
Self-Organizing Network (SON) feasibility studies paper is the formulation and assessment of a
[15], addressing not only the dynamic allocation of plausible ML-assisted cross-slice radio resource
radio resources to slices but also the distribution of optimization solution that fits within the delineated
other resources such as storage and computing for implementation framework. The solution makes
virtualized implementations. use of Multi-Agent Reinforcement Learning (MARL)
based on the Deep Q-Network (DQN) technique.
The decision-making logic for cross-slice resource
optimization needs to deal with a lot of Illustrative performance results of the proposed
uncertainties and random processes associated solution are provided by means of simulations.
with the variability in traffic generation, device The rest of the paper is organized as follows.
mobility and radio channel conditions, so it is highly Section 2 presents an overview of related works in
difficult to have an accurate a priori statistical order to position the paper in relation to the state-
knowledge of the network resource utilization and of-the-art. Section 3 describes the implementation
delivered performance. For this reason, model-free framework, which is particularized to the proposed
Machine Learning (ML)-based methods, which do ML-assisted cross-slice optimization solution in
not rely on predefined models but are able to learn Section 4 and Section 5 presents some illustrative
and/or predict the particular network dynamics as proof-of-concept results. Finally, our concluding
well as to operate under goal-oriented policies, remarks are wrapped up in Section 6.
become adequate solutions to the problem [16]. RELATED WORK
Besides, the complexity of the problem with a huge 2.
number of variables and conditions (e.g. particular Artificial Intelligence (AI) and more specifically ML
device capabilities, pending traffic, link channel techniques have been applied in the literature for
conditions, resource consumption, etc.) also pushes both slice admission control and cross-slice radio
for the introduction of these sorts of methods. As a resource allocation. In the area of slice admission
result, the system can be in a large number of control, [17] studied an optimal algorithm using
possible states in which the cross-slice resource Semi-Markov Decision Processes (SMDP) and then
allocation needs to determine the optimum capacity proposed an adaptive algorithm based on Q-
sharing among slices. In this case, among the learning. Then, other works have considered deep
possible ML techniques, deep reinforcement Q-learning [18] along with variants for enhancing
learning (RL) schemes become particularly relevant the training process, such as deep dueling neural
because they provide faster convergence under networks [19]. ML tools have also been used for
large state/action spaces in comparison with enhancing the slice admission control with traffic
classical reinforcement learning. prediction, such as in [20], [21], which use Holt-
Winters prediction, or [22], which uses a
While there is a significant amount of work
addressing the cross-slice resource optimization combination of Long Short Term Memory (LSTM)
problem from an algorithmic and performance and dense neural networks for predicting the
assessment perspective, less attention has been resource usage.

104 © International Telecommunication Union, 2020