Page 27 - Methodology for measurement of Quality of Service (QoS) Key Performance Performance Indicators (KPIs) for Digital Financial Services

TABLE 8-1: Categorization of impact of mobile network and DFS infrastructure performance on end to end
DFS QoE
WELL-PERFORMING DFS FUNCTIONALITY POORLY PERFORMING DFS FUNCTIONALITY
Well-performing High level of overall QoE, only vulnerable to local Mobile network performance not relevant/not
mobile network or temporal impairments of each component visible
Poorly performing Overall DFS QoE strongly depends on mobile Low level of overall QoE, no clear dominance of
mobile network network performance each component
FIGURE 8-1: Generic model of the elements involved in the interaction between the A party and the DFS system

Device Mobile Network Service specific
Service specific; Non-DFS 3rd party
Non-DFS Mobile infrastructure infrastructure
Device Air
Network
Common interface Common
Testcase Device Path (physical) Path Mobile Network DFS specific
control Service specific; DFS specific 3rd party
(manual) DFS infrastructure infrastructure

quence of test cases for different services. The purpose Most of these use cases have parameters such as
is to evaluate the general condition of the network. The the amount of data transferred. Choice of parameters
intensity of these tests is however moderate in order to is made in a way to avoid overloading the transport
not stress the network too much. network. This relates to pauses between test cases as
Also, some basic network parameters as well as GPS well as use case specific parameters, e.g. data volume
information are taken continuously. However, the ex- transferred in upload or download, and selection of the
tent of these passive data is limited. On purpose, in this web sites used for testing.
methodology only unmodified (“out of the box”) mobile
devices are used. 8.4 Monitoring
The following parameters are recorded: Some baseline data should be collected for assessment
• Signal strength. of packet data performance. It is recommended to also
run a monitoring device under good radio conditions
• Type of network (Radio network technology, RAT). (or via Wi-Fi connected to a fixed-line connection)
• Cell identity (as far as the device supports this). which accesses the same server (for UL/DL) or web site
• GPS position and speed.
respectively.
If more information is desired, modifications to the By analyzing the performance, times where the serv-
phones are unavoidable. Such an extension of the meth- er or web site itself is down (or its performance is de-
odology is for further study. graded) can be easily identified.
In the following clauses, considerations about the de-
sign of this sequence and the accompanying method-
ological considerations are described. 9 DATA VALIDATION AND PROCESSING

8.3 Test cases for transport network background 9.1 Plausibility and validity checks
testing The tables in the following subclauses are meant to be
Scenarios for testing the transport network in the back- checklist templates, e.g. validated items would receive
ground have to be selected and defined on a country- respective check marks.
by-country basis.
As an example, the following test cases can be used: 9.1.1 Tests on DFS data
• SMS. ❏ Are backup records (photos of filled-in sheets) com-
• USSD. plete?
• Web browsing (to a live and a reference page).
• http download and upload. ❏ Check time spans for electronic data (Excel tables
from primary data) vs. backup copies (range checks,
These test cases—with respective guard times and ad- i.e. first and last transaction on each data log sheet).
ditional pauses to achieve a desired density of tests—
are repeated cyclically. ❏ Check timestamps of DFS data against respective lo-
cation logs

Methodology for measurement of Quality of Service (QoS) Key Performance Indicators (KPIs) for Digital Financial Services • 25Methodology for measurement of Quality of Service (QoS) Key Performance Indicators (KPIs) for Digital Financial Services • 25