| Work group:
|
Q16/11 (Presentation Web page is available here)
|
| Title:
|
Test specifications for protocols, networks and services for emerging technologies, including benchmark testing and federated testbeds
|
| Description:
|
1 Motivation
WTSA Resolution 76 "Studies related to conformance and interoperability testing, assistance to developing countries, and a possible future ITU Mark programme" resolves that ITU-T Study Group 11 continues to coordinate the Sector's activities related to the ITU Conformance and Interoperability (C&I) programme across all study groups and continue to undertake activities within the C&I programme, including pilot projects on conformance/interoperability testing.
ITU T is producing a large number of Recommendations. To achieve interoperability and conformity, one of the important aspects of the ITU C&I programme concerns the development and maintenance of testing frameworks and methodologies.
It is essential that conformance and interoperability testing methodologies used by all study groups engaged in testing are aligned and consistent with each other. To achieve interoperability on a global scale, the ITU-T Recommendations must be developed and maintained with conformance and interoperability in mind according to the relevant methodology.
The objective of conformance testing is to determine how completely and correctly the requirements stated in the Recommendation have been met by the implementation. On the contrary, in interoperability testing, the objective is to determine if two or more implementations of the same Recommendation communicate and correctly exchange information with each other. It is generally assumed that the conformity of an implementation has been tested prior to perform an interoperability testing assessment.
The recent trends (such as IMT-2020, IMT-2030, IoT technologies) may lead to many changes with regards to the extant network architecture that will require higher network performance. This in turn would impact on the specifications of the terminal unit as in CPE, mobile units, phones, etc.
Most of the telecom operators are implementing various emerging technologies and migrate from circuit-switched to the packet-switched networks, trying to provide their services using "all over IP" concept. As a result, operators face some issues which are in general related to the compatibility and interoperability of the used ICT equipment and the interconnection of IP-based networks (e.g., VoLTE, ViLTE, VoNR, ViNR, IMT-2020, IMT-2030), which, among others, will be used for roaming/nomadic services. The conformance and interoperability testing of network-network interfaces (NNI) against ITU-T Recommendations may help operators to be sure that their network solutions are ready for interconnection. This interconnection approach may be also used for future packet-based networks, e.g., IMT-2020, IMT-2030 network and beyond.
Also, new ICTs, networks and industry-oriented applications are becoming increasingly complex to test using standalone testbeds. Federated testbeds bring sustainability in fostering environments for quick innovations and testing of complex technologies and use cases, and for enabling quicker time to market for products and services.
This Question is responsible for Q.3900-Q.4099-series (testing for next generation networks), Q.1912.x-series, X.290-series (except X.292), X.Suppl.4, X.Suppl.5 and Z.500-series.
2 Question
Study items to be considered include, but are not limited to:
- What is the testing methodology for testing emerging technologies?
- What are the existing ITU-T Recommendations which include test suites?
- What is the architecture of the testbed or testing facilities to be used for testing emerging technologies?
- Which technologies that are being developed for the ICT market require conformity and interoperability testing (taking into account market needs)?
- What kind of test suites are needed for testing interconnection of IP-based networks (e.g., IMT-2020, IMT-2030 network and beyond)?
- What kind of equipment could be benchmark tested?
- What kind of test procedures could be used for benchmark testing?
- What type of traffic could be simulated for benchmark testing?
- What design objectives need to be benchmark tested?
- How parameters/technologies/services may be tested remotely?
- What APIs need to be developed for implementing federated testbeds?
- What user requirements for Testbed-as-a-Service (TaaS) need to be developed?
- How federated testbeds reference model needs to be updated based on market needs?
- What guidelines need to be developed for implementing federated testbeds?
- What is the methodology for testing IMT-2020 and IMT-2030 mobile devices?
- What is the methodology for testing services which require ultra-low latency?
3 Tasks
Tasks include, but are not limited to:
- study the testing methodology for testing emerging technologies;
- identify the existing ITU-T Recommendations which include test suites;
- identify the architecture of the testbed or testing facilities to be used for testing emerging technologies;
- identify market driven ICT technologies, which require conformity and interoperability testing;
- develop test suites to be used for testing interconnection of IP-based networks (e.g., IMT-2020, IMT-2030 network and beyond);
- determine the types of equipment, which could be benchmark tested;
- develop the test procedures for benchmark testing;
- identify the type of traffic to be simulated for benchmark testing;
- define the design objectives that need to be benchmark tested;
- determine the parameters/technologies/services which may be tested remotely;
- develop the requirements for APIs to be used for federated testbeds;
- develop user requirements for TaaS;
- update federated testbeds reference model;
- develop guidelines on implementation of federated testbeds;
- develop a methodology (guide) that would extend current experiences and testing approaches to IMT-2020 and IMT-2030;
- determine the methodology for testing IMT-2020 devices;
- determine the methodology for testing services which require ultra-low latency.
An up-to-date status of work under this Question is contained in the SG11 work programme (https://www.itu.int/ITU-T/workprog/wp_search.aspx?sp=18&q=16/11).
4 Relationships
Recommendations:
- Q, Y, H, G, E, I, M, P, X, Z and F-series
Questions:
- All ITU-T SG11 Questions
Study Groups:
- ITU T SG3 on policy issues
- ITU T SG12 on parameters and requirements of QoS
- ITU T SG13 on future networks (e.g., SDN, NFV), cloud computing, IMT-2020, machine learning
- ITU T SG15 on core and access technologies
- ITU-T SG17 on testing languages, including TTCN-3
- ITU T SG21 on multimedia services, applications and e-health
- All other ITU-T SGs that are involved with activities on C&I, IMT-2020 networks and beyond, machine learning
- ITU-D SG2 Q4/2
Other bodies:
- ETSI (especially ETSI TC INT)
- IETF
- IEEE
WSIS action lines:
- C5, C6
Sustainable Development Goals:
- 9
|
| Comment:
|
Continuation of Q16/11
|
|
Rapporteur:
| Mr. | Cedric | CRETTAZ |
|
Associate rapporteur:
| Mr. | Martin | BRAND |
|
|
