Page 101 - ITU Journal Future and evolving technologies Volume 2 (2021), Issue 5 – Internet of Everything

ITU Journal on Future and Evolving Technologies, Volume 2 (2021), Issue 5

Fig. 7 – Bot tests: access to the platform

experience a smooth interaction. For instance, using a vir‑ meaning to the sentence so as to be able to guarantee the
tual machine instance with 1vCPU and 512 Mb of RAM, best match even with requests that are not well formu‑
the latency values of the system to process and provide lated, albeit with a lower score than the optimal one.
a response for a single request are between 100 ms and
200 ms, to which the delay introduced by the network In Fig. 9 we can see the scores of the low of requests that
should be added. This brings to an overall round trip time have been submitted to the bot during the con iguration
of less than a second. Furthermore, thanks to the ductil‑ phase in two distinct cases. The irst case, called “Best”,
ity of the serverless system, by appropriately con iguring was produced by submitting to the bot the sentences for‑
the load balancing rules, when all the service instances mulated as similar as possible to how they were inserted
are occupied a new instance can be started to automati‑ into the intents, trying to make them as close as possi‑
cally lighten the load of others. ble to natural language. The second case, called “Worst”,
on the other hand was formulated using the synonym of
the keywords and looking for a grammatical form quite
7.4 Analysis of the questions matching scores different from the one used in the previous case. Simi‑
larly, in Fig. 10, the same analysis was performed for the
We have also analyzed the relevance of the questions sub‑ second test, where device setting and data request were
mitted to the bot with the patterns inserted in the intents, performed. Sentences 4 and 7 in 9, sentences 4 and 10
created on the Dialog low platform. The score calculated in 10 are cases in which the match between sentences is
by DialogFlow was used for this purpose. This evalu‑ not accurate. This phenomenon is governed both by the
ates the level of con idence of the question submitted to
number of synonyms that have been associated with the
the bot with the example ones present in the platform.
entities, and by the level of similarity between the various
This con idence level is calculated based on the state of
intents implemented and their length. For example, if you
the conversation and exploiting the Term Reinforcement
have two intents that trigger two different events but are
techniques. These techniques allow for a greater weight
very similar in natural language, the classi ier will be less
to certain words through their repetition or the use of
accurate about which one to choose. In Table 1 we also
synonyms. Score values range from 0.0 (completely un‑
show the average values which demonstrate that there is
certain) to 1.0 (completely certain). In the proposed im‑
not a big difference between the “Best” and “Worst” cases;
plementation, once a question is evaluated, there are two
indeed, in both cases it was possible to con igure the bot,
possible outcomes: a) if the question achieves a con i‑ request data and set the devices smoothly without any is‑
dence match score greater than or equal to the classi i‑ sue about possible request misunderstanding. Obviously,
cation threshold setting, the higher con idence intent is the better the intents are constructed, the easier it will be
triggered; b) if no intent meets the threshold, no match is to get accurate matches by submitting questions that are
returned. In this case the threshold was set to 0.7. The apparently different but express the same concept.
score plotted in Fig. 9 and Fig. 10 indicates the quality of
the match between the ideal question (the one contained Table 1 – Comparison between the average values of the scores obtained
in the intent) with the real question (the one generated for the two considered scenarios
by the user). Obviously, the sentences inserted within
the intent are constructed, with the help of the entities, Platform access 0.956 0.844
in such a way so as to be as general as possible, so they Device con iguration 0.925 0.819
are not strictly meaningful sentences but rather they are
composed only of the words actually necessary to give a

© International Telecommunication Union, 2021 89