Page 48 - ITU Journal Future and evolving technologies Volume 2 (2021), Issue 6 – Wireless communication systems in beyond 5G era

ITU Journal on Future and Evolving Technologies, Volume 2 (2021), Issue 6

The PSG input column vector at the t h iteration is
de ined as follows: D
( )
⎡ n ( − ∶ ) ⎤ Norm.
⎢ 0 0 ⎥ −1
i = ⎢ r ( − ∶ − 1) ⎥ , (9)

1
0
⎢ … ⎥
⎣ r −1 ( − ∶ − 1) ⎦

Fig. 2 – Structure of the PSG.
th
where ( ) is the systematic symbol, n ( − ∶ )
0
0
is a column vector of length + 1 which contains noise Functions and will be parameterized using DNNs.
0
samples from the sequence n of (1), r ( − ∶ − 1) The structure of Fig. 2 corresponds to a recurrent ar‑

0
( = 0, … , − 1) is a column vector of length which chitecture, and therefore, we will consider the following

contains noise samples from the sequence r of forward‑ three recurrent architectures to model it: RNNs, GRUs

th
channel noise samples that corrupt the symbol of each and LSTMs.
parity symbol sequence, that is:
2.3.1 RNN
r ≜ ( ( ), … , −1 ( )), (10)
0

When modeled with an RNN, the function (⋅) in (13) is
th
where ( ) ( = 0, … , −1) is the sample of v in (5) de ined as follows:

and , … , are arbitrary positive integers ( can be 0),

0
0
hereafter called the encoder input extensions. We note that (i , h −1 ) = tanh(Wh −1 + Yi + b), (14)

the Deepcode [1] encoder can be recovered as a special
0
case by setting = 0 and = … = = 1, which where W is a state‑transition matrix of size × , Y
0
1
0
0
means that, in each iteration, only a single noise sample is an input‑state matrix of size × ( is the length of

0
for each systematic or parity check symbol is used. The vector i ), and b is a bias vector of length . W, Y and b
buffers in the DEF encoder contain the systematic sym‑ are obtained by NN training.
bol sequence x and the corresponding forward‑noise se‑
quence n of (1). Those sequences are generated during 2.3.2 GRU
0
the irst encoding phase and used by the PSG in the second With a GRU, the function (⋅) of (13) is de ined as follows:
phase.
(i , h −1 ) = (i , h −1 ) ∘ (1 − (i , h −1 ))

0

2.3 Parity Symbol Generator (PSG) + h −1 ∘ (i , h −1 ). (15)

The core functionality of the DEF encoder is the compu‑ The function (⋅) in (15) is de ined as follows:
tation of the parity check symbols, which is performed by 0
the block denoted (PSG) (see Fig. 1). PSG computes the (i , h −1 ) = tanh((W h + b ) ∘ (i , h −1 )
−1
0

ℎ

th
th
parity symbol sequence p based on the modula‑ + Y i + b ). (16)

tion symbol and a subset of the past forward‑channel

outputs. The functions (⋅) in (15) and (⋅) in (16) are de ined as
th
Fig. 2 shows the structure of the PSG. In the encoding follows:
th
iteration, the PSG generates a parity symbol sequence
p which consists of real parity symbols obtained as fol‑ (i , h −1 ) = (W h + Y i + b ) (17)

−1

lows: (i , h −1 ) = (W h + Y i + b ) (18)

−1

p = Norm( (h )), (11)

)
where ( ) ≜ (1 + − −1 denotes the sigmoid function.
where h , a real vector of arbitrary length , denotes the In equations (15)‑(18), matrices W , W , W , Y , Y , Y

0

PSG state at time instant , while function (⋅) consists of a and vectors b , b , b , b are obtained by NN training.
lineartransformationappliedtothePSGstateh obtained ℎ

as follows: 2.3.3 LSTM
(h ) = Ah + c, (12)

where A has size × and c has length . The above As for LSTM, the function (⋅) of (13) is de ined as follows:
0
matrices W, Y, A and vectors b, c are obtained by NN train‑ (i , h ) = (i , h ) ∘ tanh(s ) (19)
ing. The Norm(⋅) function normalizes the PSG output so −1 1 −1
that each parity symbol has zero mean and unit variance. where s is the cell state at time instant . The cell state

The PSG state h is recursively computed as provides long‑term memory capability to the LSTM NN,

h = (i , h −1 ), (13) whereas the state h provides short‑term memory capa‑

bility. The cell state is recursively computed as follows:
where function (⋅) will be discussed below, and i is de‑

ined in (9). As for the initialization, we set h as the all‑ s = (i , h −1 ) ∘ s −1

2
0
zero vector. + (i , h −1 ) ∘ (i , h −1 ). (20)

4
3
36 © International Telecommunication Union, 2021