Page 100 - Proceedings of the 2018 ITU Kaleidoscope

‎ 2018 ITU Kaleidoscope Academic Conference‎

challenges which is necessary to address in this context. region area covering a limited number of contiguous cells.
Section 3 contains the problem formulation and the details In these cases, functions are required to maintain service
about the algorithm we designed to train the proposed deep continuity and QoS for UEs using MEC services while in
RL agent. Section 4 provides a description of the simulation mobility. In fact, it may well happen that a UE moves too far
environment that we used to retrieve the data necessary for the away from its MEC server, in which case the communication
system training. Section 5 describes the scenario we realized latency increases beyond reasonable and context awareness
and provides the results we obtained by comparing the policy becomes less effective. In such a scenario, it is preferable
learned by the agent with respect to a simple policy. Finally to move the server-side MEC application to a nearer MEC
Section 6 concludes the paper and gives some details about server. Besides providing functions to achieve this, policies
future developments. are also required in order to understand when it is best to
migrate a MEC application, and where to. These policies may
take into account physical constraints (e.g., the availability
2. MULTI-ACCESS EDGE COMPUTING
of MEC services on particular servers), network-side QoS
parameters (such as latency and communication bandwidth
Multi-access Edge Computing (MEC) is recently being in the cell), server-side QoS parameters (such as available
standardized by the European Telecommunications Standards computation power and storage).
Institute (ETSI) [5]. It consists in placing nodes with
computation capabilities, namely the MEC servers, close to
the elements of the network edge like, e.g., base stations in 3. RL TECHNIQUES FOR MEC
a cellular environment. MEC differs from Fog Computing During the last years, mobile traffic data is increasing and
since it is able to interact with the network elements it is expected to raise in the next future. Such a trend, has
and can gather information on the network environment, lead to the start of a process which aims to improve the
such as resource utilization and users’ location. This network infrastructure in order to address all the problems
information can be obtained via a standard Application related to a such massive amount of data [7]. 5G system is
Programming Interface (API) and allows operators and/or expected to reach far better performance in terms of speed
third-party developers to offer new context-aware services and energy efficiency if compared with the actual Long Term
and applications to the users. The latter can also Evolution (LTE) and 3G technologies; 5G architecture takes
experience lower latency and larger bandwidth with respect advantage of Software Defined Networks (SDN) and Network
to a cloud-based application. Those services are created Function Virtualization (NFV) in order to improve network
within a virtualized environment that allows to optimize management and dynamic resource allocation [8]. In such
the computational load of MEC servers and/or users’ a context, where the network infrastructure is starting to
requirements. Moreover, MEC applications can also migrate become very complex, machine learning can be an useful
between MEC servers to better accommodate mobile users, technique to use in order to face the environment dynamics
e.g. cellular users moving moving to another cell. Connected by finding an optimal strategy to improve the overall system
vehicles, video acceleration and augmented reality are some performance.
of the use cases identified by ETSI that can benefit from the
introduction of MEC [6]. LTE is the cellular technology for 3.1 Reference Scenario
which MEC was first proposed, and will be part (in its current
state, or – more likely – leveraging an evolved physical and The proposed reference scenario consists of a MEC-enabled
MAC layers) of the 5G ecosystem. In an LTE radio access LTE network where eNBs can be directly connected to MEC
network (see Fig. 1), eNBs create radio cells, to which User servers which contain application data, as depicted in Fig.
Equipments (UEs) attach. The entry/exit point for traffic in an 1. Such servers can be used as local repositories by serving
LTE Evolved Packet System (EPS) network is the Packet Data those users attached to the corresponding eNB thus reducing
Network Gateway (PGW). Traffic destined to the UE arrives the network latency as well as the amount of data traveling
at the PGW, where it is tunneled using the GPRS Tunneling on the core network. One of the key issues of this schema is
Protocol (GTP). Tunneled traffic traverses the EPS network. to find an optimal migration schema that opportunely moves
The exit point of the GTP tunnel is on the serving eNB. At the data from one server to another in order to improve a specific
eNB, the packet is extracted from the tunnel and transmitted objective function.
over the air interface. A handover procedure is initiated by a In particular, in this paper we are interested in the
UE to leave a cell and join another, typically when the signal development of a machine learning algorithm based on
strength of the new eNB exceeds the current one’s. In this reinforcement learning (RL) for the deployment of an optimal
case, downstream traffic is steered from the PGW towards the policy which establishes when is necessary to migrate data
new eNB, and the traffic still in transit at the handover may to another eNB depending on the user position and on the
either be discarded or forwarded by the "old" eNB using current state of the network in order to improve the user QoS.
the X2 interface, which connects eNBs in a peer-to-peer RL is a machine learning technique used to observe the
fashion. MEC servers can be deployed at any point in the EPS. dynamics of an environment thus learning an optimal policy
However, it is foreseen that they will be either co-located with with respect to one or more performance indexes. In many
eNBs, in order to minimize the latency, or deployed close to contexts where it is impossible to work with labeled data, RL
them, so that a single MEC server will serve a geographic is the only feasible solution to correctly train a system. In fact,

– 84 –