Page 135 - ITU Journal Future and evolving technologies Volume 2 (2021), Issue 4 – AI and machine learning solutions in 5G and future networks

ITU Journal on Future and Evolving Technologies, Volume 2 (2021), Issue 4

with DS enabled, both in SISO‑RT and MIMO‑RT scenar‑ two beam selection algorithms: one based in RL and a
∘
ios, considering a ray spacing of 0.5 . For SISO‑RT, the run simple baseline. To perform beam‑selection using RL, we
time was 87 longer when enabling the DS compared to not used a Deep Q Network (DQN) [34]. The Stable Baseline
6
using it. For MIMO‑RT this value was even greater: DS in‑ API with default DQN parameters was adopted. The re‑
creased the simulation time more than 600 times. ward is the magnitude of the equivalent channel, as de‑
ined in Eq. (1). The baseline algorithm adopts the fol‑
As described in Table 4, the simulation times depend on lowing heuristic: it simply chooses the beam that points
the number of antenna elements in each Tx‑Rx pair. In‑ to the straight path direction between the BS and the
creasing the number of antenna elements in each Tx‑Rx UAV. For most of the UAV’s path, there is Line‑Of‑Sight
pair signi icantly raises the simulation time. A twelve‑fold (LOS) and this heuristic achieves good results. As ex‑
increase occurs when using = 64 and = 64 ‑ where pected, this strategy does not work well when the link

and are the number of antenna elements in the ULA is Non‑LOS (NLOS), which occurs for the angular range

of the transmitter and receiver, respectively ‑ compared to ∈ [20, 30] degrees.
the baseline case where = 64 and = 2.

The results of this simple experiment is provided in Fig. 8.
Table 3 – Simulation time increase factor for one RT simulation (s012) The bottom plot shows the angle as the UAV takes off
for different ray spacing values, with and without diffuse scattering en‑ ( ∈ [0, 25000]), reaches its destiny and lands ( > 76000).
abled. The baseline time for SISO‑RT is 00:00:11.749 and for MIMO‑ During three time intervals (including a very short one)
RT (with = 64 and = 8) is 00:00:39.654. The time format is the link between the UAV and BS was NLOS. The top plot
(HH:MM:SS.ccc).
shows the magnitude of the equivalent channel | |, in
,
Simulation time increase factor which the ‑th codebook index was chosen at time . The
∘
Ray Spacing ( ) SISO‑RT MIMO‑RT optimum value, obtained by exhaustively trying all =

1 1 0.7 64 indices, is shown together with the values obtained by
0.5 1 1 the DQN (RL) and baseline. While the optimum value is al‑
0.25 2.4 1.5 ways larger than 5 and has an average value of 6.81, both
0.1 11 4 baselineandRLstruggletoreachgoodresultsandachieve
,
0.5 (DS‑enabled) 84.7 412.9 average values E[| |] = 1.7 and 2.3, respectively. It
should be noticed that in this case the RL agent should
choose one among 64 indices having a single input (angle
Table 4 – Simulation time increase factor for one RT simulation (s012) ). In the UFPA Problem Statement for the 2021 ITU AI/ML
considering different numbers of antenna elements in the transmitter in 5G Challenge [6], a richer set of input features will be
and receiver antenna arrays. The baseline time is 00:00:18.437 (with
= 64 and = 2). The time format is (HH:MM:SS.ccc). adopted, allowing not only beam selection but also UAV
path planning.
Simulation time increase factor
64 2 1 5. CONCLUSIONS
64 8 2.2
This paper presented strategies and software for simulat‑
64 64 12 ing 6G systems that represent the surrounding environ‑
ment with images and other types of data. The so‑called
As an illustration of an INLOOP CAVIAR simulation, we de‑ CAVIAR framework bene its from virtual reality tools, em‑
veloped code for the Unreal Engine and AirSim to simu‑ phasizing the physical aspects of the movement of ob‑
late a BS serving a UAV. There are two RL agents: one for jects. This visual information, coupled with MIMO chan‑
determining the UAV trajectory and the other for beam nels generated through RT methods, enables investigat‑
selection. We discuss only the latter agent in this pa‑ ing new AI/ML algorithms in 6G that rely on the environ‑
per. As the UAV lies along its trajectory, the MIMO chan‑ ment and learning from experience.
nel is obtained according to the well‑known geometric We also discussed how semantic segmentation and sen‑
model, with parameters for three multipath components sible RT parameters can improve generated MIMO chan‑
obtained from probability distributions (see, e. g. [27, nels. We advocate that aiming at realistic simulations
16]). This simpler methodology was adopted to speed is the natural path to gain a better understanding on
up the simulations and allow for visualizing the UAV as how ML/AI can make communication systems more ef i‑
it lies. In this speci ic scenario, RT channel responses are cient. The effort along the direction of larger and realistic
not used due to the required simulation time. The BS used datasets is important for properly evaluating ML‑based
a ULA with = 64 antennas, while the UAV uses a single

antenna. A DFT codebook is adopted. algorithms, and to avoid unfair comparisons to conven‑
tional signal processing.
At each time , the UAV informs its position to the BS,
which can then calculate the Angle of Arrival (AoA) of 6 https://stable‑baselines.readthedocs.io/en/master.
the beam at the UAV. This angle is used as the input for

© International Telecommunication Union, 2021 119