Page 221 - ITU Kaleidoscope 2016

ICTs for a Sustainable World

a high number of experiments to have statistically Table 1. Pros and cons of experimentation stair’ steps
representative samples to model the behavior of both radio Experimentation level Pros Cons
part and TCP. Apart from that, the emulated environments -Repeatability -Synthetic fadings
Simulated environment -Cheapest option -Faked UEs
do not allow UEs’ real movement, cutting down the -Parameters gathering -Hard modeling
possibility of fulfilling many studies in this regard.
-Real UEs
-Statistics required
Emulated testbed -Easy LTE configuration -No real movement
2.1.3. Controlled deployment -Radio info. collection
-Real movement
Controlled deployments are a very good approach to Controlled deployment -Ad-hoc patterns -Speed/space
experiment with real equipment and ad-hoc experiments out -Air transmission limitation
of the wild. Nowadays, many facilities are available due to -Real speeds -Limited parameters
Real-world deployment -Real patterns study
the efforts made by federations such as Fed4Fire [15]. -Ability to study realism -Data quota
Our steps in relation with controlled deployments were

devoted to study the behavior of TCP under different 2.2. Standardization bodies and efforts
mobility patterns and speeds in a deployment with real
Regarding the definition of QoS/QoE measurements and
equipment and therefore, having a real-world-alike testbed
under controlled circumstances. Following cost- methods, the standardization efforts have been manifold
effectiveness, we were able to launch and study such [19]. The IP Performance Metrics (IPPM) WG is working
on methods to measure the bandwidth. In [20] Mathis
experiments in iMinds’ LTE facilities (LTE w-iLab.t [16])
explains how TCP causes self-inflicted congestion as a
in Zwijnaarde, Ghent. We performed TCP-based
result of numerous factors including TCP's circular
measurements with different mobility pattern helped by
Roomba robots with LTE dongles on top of them, enabling dependencies between data rate, loss rate and Round-Trip
Time (RTT) that inflict zero predictive value for any TCP-
to design, assign and follow a mobility path during the
based speed test. Due to this, there is currently an ongoing
experiment in an almost fully-real environment.
remarkable effort towards the definition of a measurement
Apart from all the positive features that have been
highlighted regarding this kind of facilities, it has to be said mechanism that could overcome these and other TCP
constraints, also being carried out by IPPM WG: Model
that LTE transmission are done by air in a well prepared
Based Metrics for Bulk Transport Capacity [7].
environment allowing a proper study of TCP and LTE
events. Even though the movement is real, the space The work regarding model-based metrics at IP level was
limitation of these small-scale deployments may well extended and it was given a deeper insight of the statistical
provoke to perform under limited speeds for procedure in this regard [21]. The goal is to provide end-
feasibility/suitability/security reasons. users with an ‘as accurate and Congestion Control
Algorithm (CCA) independent as possible’ estimation. So
2.1.4. Real-world deployments far this alternative seems so promising that IETF IPPM WG
and IETF Large-Scale Measurement of Broadband
There is nothing more representative than performing
Performance (LMAP) WG [22] have standardized a
experiments in real world deployments, world-wide if framework for large-scale measurement platforms.
possible. However, the number of available ones is very
However, doubts persist around the measurement proposal
little and the possibility to have access to one of them is
[7] regarding its accuracy, technical implementations
very unlikely. Even though there exist works [17] that
feasibility, statistical meaningfulness and the extent to
involve real nodes, they are capable of measuring simplified which it will resemble the actual performance end-users
static scenarios and the impact that background traffic could
perceive. Moreover, the method requires the input of
have on foreground flows, but they lack the involvement of
parameters such as Target Data Rate, Target RTT and
every phenomenon (i.e. self-induced delay due to the
Target Maximum Transmission Unit (MTU) to start
mobility, position according to the eNodeB and so forth) . analyzing a certain path.
So solve this equation MONROE [18] presents pure-realist
As for ITU-T, it has been working in the last couple of
deployment approach. It allows performing experiments in
years in the definition of new frameworks [3-4] and testing
static and mobile nodes (buses, trains and trucks) with up-
methodology [5] for Internet speed measurement in both
to-date equipment.
Apart from the aforementioned strengths, real-world fixed and mobile networks. Concretely, the last framework
is also aligned to the draft specification TS 103 222-1 [6]
deployments give trustworthy insights of movement impact,
from European Telecommunications Standards Institute
cross-traffic loads impact and they open the gate to study
(ETSI) and the output RFC 7594 from IETF LMAP [22]
traffic patterns and detect deficiencies, either due to
where the main entities of the framework are described.
coverage, equipments or resource lack. As for the
What this work amongst standardization bodies evidences is
weaknesses, the possibility to gather information with high
that there is an actual need for collaborating towards the
granularity is limited due to the impact of greedy
definition of a common methodological technique to
background processes on overall performance and so is the
evaluate QoS/QoE performance in both fixed and mobile
ability to infer inter-layer performance. Moreover, since the
networks.
utilized amount of data is under contract, there could be a
In relation to QoS measurement methodologies, we
limitation of data quota depending on the operator.
made a simulation and real-world distributed servers’ study

– 203 –