Page 29 - Methodology for measurement of Quality of Service (QoS) Key Performance Performance Indicators (KPIs) for Digital Financial Services

- The app should have a built-in time synchronization The result of this pilot testing and evaluation is a set
with network or GPS time (which may need modi- of procedural insights and recommendations as well as
fication of the phone, i.e. “rooting” to get required guidance useful for design and performance of future
access rights), or at least captures GPS (combined testing. The basic expectations, with respect to QoE as-
with procedures to make sure that there is at least a sessment of MoMo services and correlation with carrier
minimum level of GPS data capture) in order to have network performance, have been field-tested and found
a high-precision time source on board. to be valid. These results provide a functional method-
ology as well as a clear path to further extension and
- Captured time stamps, along with information on refinement.
the measurement team and other respective data, is The existing use case P2P needs to be extended, e.g.
stored locally as well as automatically updated to a for the following topics:
server location.
• Areas with non-optimal radio coverage.
If for any reason a solution involving paper logs and
manual transfer has to be used, the following improve- • Mobility aspects.
ments are recommended: • Better statistical relevance of the data base.
- Use Excel® templates which contain a set of built-in Additional use cases should be elaborated (besides
initial checks and create visual warnings, e.g. if time- P2P). However, care has to be taken to correctly isolate
stamps are inconsistent (based on expected ranges the payment process from the application in which it
or relations between entries. may be embedded. Also with any new use case taken
into consideration, it has to be analyzed which events
- If budget allows, prescribe a four-eyes method for or trigger points are accessible. This again may vary
data transfer.
depending on who is planning to conduct the testing.
- Further improve paper logs by visual elements which Possible ways can be to do a “friendly” testing with all
reduce the risk of “eye slips”. stakeholders involved may enable access to internal
trigger points; it could also be testing by a third party
(e.g. the regulator) which however may turn out to be
11 CONCLUSIONS AND WAY FORWARD significantly more difficult.
An important type of use case is G2P, i.e. payments
The methodology described in the present document of governmental bodies to induvial. From a testing per-
provides all means to conduct and evaluate QoS mea- spective, this would also provide a good basis as real
surements on Mobile Money services. The current focus money flows, which are a necessity in such kind of tests,
is on person-to-person money transfers, but the overall can rather easily be controlled in order to create a most-
framework has been designed with extension to other ly circular type of transfer with respectively moderate
use cases in mind. need with respect to used capital.
With respect to the actual execution of tests, all-man- Studies are underway which seem to indicate
ual testing and data acquisition has been deliberately that users under certain circumstances prefer dedi-
chosen to provide maximum transparency on the pro- cated hardware solutions over an app on the smart-
cedures, despite restrictions in accuracy. As expected, phone. With the advent of Internet-of-Things (IoT)
the manual processes exposed various ways data can and Low-Power-Networks (LPN) being rolled-out a
be compromised, in particular where information is new class of DFS solutions may appear on the mar-
transferred between different media. Respective con- ket which are using dedicated hardware in the context
sistency checking procedures have been designed and of LPN enabled IoT devices. Dedicated hardware DFS
tested, and a broad range of experience and ways to solutions would have the potential to reduce human
handle such errors has been created. errors on the users’ side of DFS. The methodology laid
From the robustness and data quality point of view, out in the present report, while in principle sufficiently
automated systems are encouraged for testing, similar wide in scope, will require a thorough review in order
to common practice in most field quality assessments. to explicitly include this class of solutions.
Where this cannot be done for practical reasons (i.e.
budget restrictions), at least technically supported time-
taking should be used. A practical way would be to have
a multi-step time-taking tool with automatic upload of
acquired data.

Methodology for measurement of Quality of Service (QoS) Key Performance Indicators (KPIs) for Digital Financial Services • 27Methodology for measurement of Quality of Service (QoS) Key Performance Indicators (KPIs) for Digital Financial Services • 27