Page 99 - Proceedings of the 2018 ITU Kaleidoscope

A DEEP REINFORCEMENT LEARNING APPROACH FOR DATA MIGRATION IN
MULTI-ACCESS EDGE COMPUTING

1
1
2
2
1
Fabrizio De Vita ; Dario Bruneo ; Antonio Puliafito ; Giovanni Nardini ; Antonio Virdis ; Giovanni Stea 2
1 University of Messina, Department of Engineering
2 University of Pisa, Department of Information Engineering

ABSTRACT research topic with different solutions already presented in the
literature. In this paper, we will focus on the data migration
5G technology promises to improve the network performance problem taking as reference a MEC system composed of
by allowing users to seamlessly access distributed services a LTE network with some MEC servers attached to the
in a powerful way. In this perspective, Multi-access Edge eNodeb (eNB) base stations. Our intent, is to build a
Computing (MEC) is a relevant paradigm that push data self-adaptive AI-powered algorithm capable to understand
and computational resources nearby users with the final the system status and accordingly migrate users applications
goal to reduce latencies and improve resource utilization. data with the final goal to improve the user Quality of
Such a scenario requires strong policies in order to react Service (QoS). Thanks to machine learning capability to
to the dynamics of the environment also taking into account build complex mathematical models which allow a system
multiple parameter settings. In this paper, we propose a deep to learn intricate relationships among a large number of
reinforcement learning approach that is able to manage data parameters, we believe that such a technique could be an
migration in MEC scenarios by learning during the system enabling technology for the future 5G systems where context
evolution. We set up a simulation environment based on aware network infrastructures must be able to make decisions
the OMNeT++/SimuLTE simulator integrated with the Keras in an autonomous way in order to improve their performance.
machine learning framework. Preliminary results showing In the literature, we found some papers that face this problem
the feasibility of the proposed approach are discussed. with different approaches. The authors in [3] adopts a
traditional machine learning approach using LibSVM toolkit
Keywords - Multi-access Edge Computing, 5G, LTE, Deep to have a forecast on users mobility and implement a proactive
Reinforcement Learning, Data Migration, SimuLTE migration mechanism in order to minimize the downtime
of the system. However, as the author remark, this kind
1. INTRODUCTION of technique is more resource demanding if compared with

Smart services represent the core element in a smart discrete models like Markov Decision Processes (MDP). The
city environment; thanks to IoT diffusion together with approach described in [4], uses a multi-agent reinforcement
the advancement of technology in terms of computation, learning scheme where agents compete among themselves in
nowadays users can access a large number of applications order to establish the best offload policy. In this paper, we
that usually communicate with the Cloud which provides present a deep reinforcement learning approach to deploy an
support to them [1]. However, applications are becoming optimal migration policy in order to improve user QoS. The
more and more resource demanding and they have reached a main difference between our work and the others consist in
level where the cloud paradigm can no more guarantee low the use of a deep learning technique instead of traditional
latencies especially when the distance between it and the user machine learning algorithms or formalisms like MDPs. We
device is very large. In such a context, Multi-access Edge believe that the use of deep learning can be a valid solution
Computing (MEC) can address this problems by moving in order to build a system which is capable to adapt to
data and computational resources nearby the user [2]. 5G changes by understanding the network state and performing
technology has the ambition to realize a framework where actions autonomously. The paper contribution is twofold: we
different technologies can cooperate to improve the overall designed a deep RL algorithm that can be used as a general
performance, also leveraging context-related information and purpose and self-adaptable algorithm to manage complex
real-time awareness of state of the local network (e.g., MEC systems without needing an explicit knowledge of all
congestion, types of services enabled). MEC is regarded the involved aspects; we realized a Deep RL environment by
as a key enabler for the 5G key performance indicators, such integrating SimuLTE and Keras that can be used as a gym
as low latency and bandwidth efficiency. In a 5G system, where different RL approaches can be realized and tested in
MEC is expected to interact with the rest of the network to LTE/5G scenarios.
improve traffic routing and policy control. The adoption of The paper is organized as follows. Section 2 introduces
the MEC paradigm in the new 5G-enabled systems is a hot the 5G and MEC technologies focusing also on the main

978-92-61-26921-0/CFP1868P-ART @ 2018 ITU – 83 – Kaleidoscope