Page 17 - Methodology for measurement of Quality of Service (QoS) Key Performance Performance Indicators (KPIs) for Digital Financial Services

From a QoS perspective, and therefore also for net- 4.1.4 Failure information in top-level views
work testing, the degree of network involvement should Depending on DFS implementation, collection of infor-
be considered to be mission-critical. mation needed to perform a DFS money transfer may
The number of data exchanges through the network involve data transfers through the network. In the hier-
is much higher in type B than in type A. As the overall archical phase model, such steps are described by re-
success of the MoMo transaction depends on the suc- spective sub-phases of the set-up phase.
cess of each of those steps, the MoMo success rate has While it is formally possible to define respective KPI
a stronger dependency on network performance than from these sub-phases, this may not be the best choice.
in type A. Secondly, the collection of information items It would increase the number of KPI vastly. This may
involves human interaction, i.e. typing. This extends the weaken the value of each KPI and obscure the function
time during which the network needs to perform well of a KPI as indicator of quality from the user’s perspec-
considerably which plays a role in mobile scenarios. tive. When benchmarking services, each contender
On the other hand, Type B implementations enable can be the “test winner” in some category if there are
collection of more information about the network’s enough KPI in the portfolio. In the end, this decreases
performance as each step in the information-gather- transparency instead of creating it. Therefore, the set
ing phase provides an information source for respec- of KPI should be as small as possible, with each KPI
tive indicators. This topic will be discussed in detail in carrying a strong meaning with a clear relation to user
subsequent clauses. Briefly, the question is if it makes perception.
sense to define KPIs for every possible combination of Moreover, a KPI is essentially an isolated quantity. A
events—which is technically possible but may obscure phase consists of individual steps or sub-phases which
things rather than provide insights. occur in a given sequential order. With KPI for each sub-
In this context, not only network quality should be phase, information about this sequential order is not
considered but also mobile device related effects, like visible anymore. Therefore, a single KPI describing the
running out of battery power. Annex B provides an overall success (or failure) rate of that phase, plus de-
overview of elements for checking. tail information about unsuccessful cases is more useful.
Of course, this discussion does not change the ne- Such detail information would then consist of informa-
cessity of using the actual implementation of the ser- tion at which step of the sequence the failures have oc-
vice. From a QoE point of view, there is no choice—the curred. If required, statistics on such causes can be cre-
whole transaction has to be taken into account if the ated or further processed to KPI-like indicators, i.e. this
test results are assumed to describe the customer per- way is still open if required. The advantage as compared
spective properly. to the primary use of KPI to convey this information is
that the information about failure causes is preserved
4.1.3 Phase definition on the transaction level and can be used to create addi-
4.1.3.1 Top-level phases tional diagnostic insight.
Set-up: Preparations for the actual transfer: In the set of DFS KPI, the Money Transfer completion
rate is a very good example for this approach. With the
• Activation of service. abstract model described in Event and action flow, and
• Input of required information (destination account, the practical example shown in clause 4.2, this approach
amount of money to be transferred, reference, cre- is demonstrated as follows.
dentials to enable the transfer e.g. password or PIN). The information required to perform a DFS transac-
tion is prompted sequentially. After the user has entered
Usage: a value, it is transmitted to the service, which in effect
• Performance of actual money transfer (including ser- triggers the prompt for the next item of information. In
vice-related transfer of information on A and B side). order to make this happen, two data transfers are re-
quired. As seen from the A party’s mobile device, this is:
■ NOTE: The set-up phase may or may not include
access to functions within the service. Typically, the – Sending an item of information, via the transport net-
information required for a money transfer consist of work, to the service, and
several items of information. These items can be col- – Receiving the next item from the service.
lected on the A party side and sent in one block of
data, or can be sent one after another. From a diag- As seen from the A Party device, there are two ways this
nostic point of view, these variants will have different sequence can be interrupted.
appearance and relation to transport network prop- a) Sending an information item can fail, with a failure
erties. However, from an end-to-end related function- information; this can be a temporary failure when a
al view the actual mode is not relevant. retry takes place, or a permanent failure when e.g.
the maximum number of retries or a time-out condi-
tion is reached.

Methodology for measurement of Quality of Service (QoS) Key Performance Indicators (KPIs) for Digital Financial Services • 15Methodology for measurement of Quality of Service (QoS) Key Performance Indicators (KPIs) for Digital Financial Services • 15