Page 64 - ITU Journal Future and evolving technologies Volume 2 (2021), Issue 4 – AI and machine learning solutions in 5G and future networks

ITU Journal on Future and Evolving Technologies, Volume 2 (2021), Issue 4

Fig. 1 – Illustration of the proposed RL‑based beamforming (b) for MMIMO systems and the network layout (a) of 5G dense Urban‑eMBB cells, in which
∗( )
the BS of target small cell #0 with 0 ( ) mobile users chooses the optimal beamforming 0 (c) to mitigate interference from the neighboring
small cells at time slot , and 0 ( ) users return estimated SINRs ( ) to the BS.
ℎ
The rest of the paper is organized as In Section 2, In RF, the BS shapes beamforming for the UE by con‑
related work on RL‑based ICI mitigation are presented. iguring weights on AEs according to AOA ⟨ , ⟩ [15],

The system model and our proposed dynamic Q‑learning where is the azimuth and is the vertical angle of the
scheme are presented in Section 3 and Section In Sec‑ ℎ UE. The weights on the ℎ AE in a row can be repre‑
tion simulation results are presented and the conclu‑ sented as
sion follows in Section 6.
= − 2 ℎ sin , ∈ 1∗ (1)
2. RELATED WORK where is the row AE distance. And the ℎ AE in the
ℎ
column can be obtained by
ICI control is a key issue in 5G MMIMO systems, intensive
resear carried t address Surveys = − 2 cos , ∈ ∗1 (2)
have been carried out on ICI mitigation techniques in L TE
downlink networks [8, 9], and research on ICI coordina‑
where is the column AE distance. From (1) and (2), the

tion techniques in 5G UFMC systems [10, 11]. inal beamforming weights for the ℎ UE can be derived
by
RL‑based approaches in
Π = Ψ Ω (3)
mitigation For Q‑learning‑
scheme coordi‑ where
issue cooperative multi‑agent prob‑ Ω = [ , , … , ],
1

2
{
lem to improve the performance of the cellular systems Ψ = [ , , … , ] .
2
1

is proposed in An RL‑based power control scheme
Since the inal weights in (3) depend on ⟨ , ⟩, the im‑

for ultra‑dense small cells to improve network through‑
plementation complexity for ⟨ , ⟩ estimation gets high

put energy presented [13],
as the perfect CSI needed, which is usually affected by ICI.
in which the BS selects the downlink transmit power to
manage interference. A dynamic RL‑based ICI coordina‑
Search‑based beamforming
tion algorithm as developed in [14] smartly of loads traf‑
ic to open access picocells and then improves the system
To mitigate the ICI with a low complexity, a search‑based
throughput.
beamforming algorithm is reported in [15], which uses
MC to search the optimal weights rather than AOA estima-
3. SYSTEM MODEL tion in (3). In MC beamforming, the best weights are ob‑
tained by searching ⟨ , ⟩ in all possible angles to min‑

ICI is caused by multiple sources transmitting signals with imize the ICI, i.e.
the same subcarrier and being received by a receiver A ∗ ∗ ( ) ( )
⟨ , ⟩ ⟵ arg ( < |ℎ , )

user receives signals from the serving cell and neighbor‑ ⟨ , ⟩ (4)
ing cells but at different power levels due to the path loss. . . − < , ≤

where is the probability of SINRs weaker than the tar‑

AOA‑based beamforming ( ) ( )
get given the channel ℎ and UE density , and

The Angle‑Of‑Arrival(AOA)‑based beamforming is usually ℎ ℎ
used in 5G MMIMO systems, where the BS is con igured the SINR in (4) for the UE located on the cell can be
expressed by [15]
with an antennas array composed of AEs, and numbers of
AEs are arranged as per row and per column [2]. −
, ,
, = (5)
+ ∑ =1, ≠ −
0

48 © International Telecommunication Union, 2021