Page 28 - Methodology for measurement of Quality of Service (QoS) Key Performance Performance Indicators (KPIs) for Digital Financial Services

Does the timestamp range match the time window This was not entirely unexpected; manual mode was
recorded for that location? deliberately chosen to provide the widest possible an-
gle of view and maximum transparency of the data ac-
❏ Check timestamps of background measurement data quisition process.
against respective location logs
The weaknesses of manual operation are manifest in
Does the timestamp range match the time window some main fields:
recorded for that location? Time-taking of the transaction, with typical time
❏ Decide on necessity to exclude time ranges. scales in the order of a few seconds, introduces quanti-
zation errors at best where times have to be read from
Does the location log indicate special events and con- displays with a typical resolution of one second. Other
ditions, which set the need to exclude data from the effects come on top, such as potential errors due to time
set?
offsets when using different devices. Time-synchroniza-
❏ Visualize timestamps of transactions: Are there any tion of such devices can only provide limited protection
gaps or unusually dense transactions during a period as this compensation is itself quantized to one second
of time? If yes, validate reasons. steps (unless modified devices are being used).
Transferring readings to paper logs open up addi-
❏ (further check items to be added)
tional sources of error due to handwriting.
Further transferring of paper logs to electronic
9.1.2 Tests on background test data
means—which is a prerequisite of data processing—is
❏ If GPS data are available, does the location indicated, again prone to reading errors.
and the GPS location match? All process steps are essentially dull and repetitive,
and are therefore vulnerable to human errors.
❏ Visualize timestamps of transactions: Are there any Typical error patterns are:
gaps or unusually dense transactions during a period
of time? If yes, validate reasons. a) Reading from the display: subsequent timestamps
10:49:58 10:50:02 logged as 10:49:02
❏ (further check items to be added)
b) Transferring from paper logs: number switches such
9.1.3 Cross tests between data (after import) as 1<>5, 2<->3, 2<->5, 3<->5, 1<-7, 4<->9 depending
on handwriting.
❏ Validate time stamps of DFS and background data
for consistency. c) Transferring from paper logs: eye-jumping to the line
above or below the actual one
❏ Validate consistency between network unavailabili-
ty in DFS and background data. A possible consis- d) Transferring from paper logs: Number-switching, e.g.
tency problem exists if background data indicate 12:30:14 ->13:20:14
network unavailability but DFS transactions work Transferring from paper logs: simple typing errors.
during a given timespan. If such periods of time Of course, it is possible to extend the manual estab-
exist, mark them in the database and seek further lished data quality assurance procedures, which can
clarification. prevent or eliminate errors. However, this is a signifi-
❏ (further check items to be added) cant cost driver and therefore needs to be considered
against automation or partially automation of the data
9.2 Additional processing acquisition process. Some suggestions how this could
With respect to some KPI definitions, additional check be done are given subsequently.
procedures may be done.
10.2 Recommended measures
Examples are: Fully automated DFS transactions would eliminate all
1. Check consistency of accounts throughout a se- of the above mentioned sources of error. If budgetary
quence of information SMS. conditions allow, this would be the method of choice.
It should however kept in mind that careful—and peri-
2. Check for “false negatives” (ref. Money Transfer False odically repeated — validation of automated solutions
Negative Rate MTFNR) by comparing account bal- is part of the design of such a solution and therefore
ance against transaction results. needs to be considered in a cost assessment.
The next best solution — in case budgetary or other
considerations lead to the decision to not use full auto-
10 LESSONS LEARNED
mation—is tool-assisted time-taking. A respective tool
would have the following basic functionality:
10.1 Overview
The manual capture of DFS TA has shown to be a major - Android app with a simple and user-friendly user
weak point in the campaign. interface, e.g. showing a group of buttons with one
button per timer flag.

26 • Methodology for measurement of Quality of Service (QoS) Key Performance Indicators (KPIs) for Digital Financial Services