Manual for measuring ICT access and use by households and individuals

2020 Edition

Acknowledgements

This edition of the Manual for Measuring ICT Access and Use by Households and Individuals is based on the 2009 and 2014 editions, which were prepared by Sheridan Roberts, consultant to the ITU (2009), and José L. Cervera, DevStat- Servicios de Consultoría Estadística (2014), with contributions by Juan Muñoz, consultant to the ITU. This revision was prepared by José L. Cervera, DevStat- Servicios de Consultoría Estadística, and Bouazza Bouchkhar, consultant to the ITU and revised by Michael Minges, also consultant to the ITU.

The work was coordinated and supervised by the ICT Data and Analytics Division of the Digital Knowledge Development Hub Department within the Telecommunication Development Bureau of ITU. Substantive contributions were provided by Christopher Jones, Esperanza Magpantay and Martin Schaaper.

The revision of this Manual was subject to an extensive consultation process through the ITU Expert Group on ICT Household Indicators (EGH). The mandate of the EGH is to revise the core indicators on ICT household access and ICT individual use, of the Partnership on Measuring ICT for Development, and to revise the ITU Manual. EGH. The proposed modifications for the third revision were presented to the EGH in September 2019 and approved by the EGH in December 2019. The Manual was launched in May 2020.

Various references were consulted for this Manual, especially publications from the Partnership on Measuring ICT for Development, International Telecommunication Union, United Nations Statistics Division, OECD, Eurostat, International Labour Organization and United Nations Educational, Scientific and Cultural Organization. Special acknowledgement is made to national bodies and international organizations for the many examples used in the Manual.

The desktop publishing and cover design was carried out by the ITU Publication Production Service (PUBL).

Original language of publication: English.

ISBN:

978-92-61-30851-3 (Paper version)
978-92-61-30861-2 (Electronic version)
978-92-61-30871-1 (EPUB version)
978-92-61-30881-0 (Mobi version)

Please consider the environment before printing this report.

© ITU 2020

Some rights reserved. This work is licensed to the public through a Creative Commons Attribution-Non-Commercial-Share Alike 3.0 IGO license (CC BY-NC-SA 3.0 IGO).

Under the terms of this licence, you may copy, redistribute and adapt the work for non-commercial purposes, provided the work is appropriately cited. In any use of this work, there should be no suggestion that ITU endorse any specific organization, products or services. The unauthorized use of the ITU names or logos is not permitted. If you adapt the work, then you must license your work under the same or equivalent Creative Commons licence. If you create a translation of this work, you should add the following disclaimer along with the suggested citation: “This translation was not created by the International Telecommunication Union (ITU). ITU is not responsible for the content or accuracy of this translation. The original English edition shall be the binding and authentic edition”. For more information, please visit https://creativecommons.org/licenses/by-nc-sa/3.0/igo/

Preface

Welcome to this third edition of the ITU Manual for Measuring ICT Access and Use by Households and Individuals. The aim of this Manual is to support countries in their efforts to collect and disseminate information and communication technology (ICT) statistics, based on internationally-agreed definitions and standards. It is designed as a practical tool to guide countries in their ICT data production, serving as a basic reference when preparing, designing and implementing ICT household surveys. To assist our members and partners make best use of it, ITU is also developing complementary training material.

The impressive growth in digital infrastructure and uptake over the past decade has led to increasing demand for accurate and comparable ICT data and statistics. Household surveys are a vitally important source of such statistics, offering invaluable insights into how and where people access and use ICTs, and the impact of ICTs on their lives. Household survey-based data are essential for monitoring national and international ICT-related development goals and targets, including those of the World Summit on the Information Society (WSIS) and the Broadband Commission for Sustainable Development, as well as the UN Sustainable Development Goals.

Since ITU first published this Manual in 2009, the digital technology sector has evolved almost beyond recognition. The Internet is now accessed through a multitude of devices, including mobile phones, tablets and similar handheld computers. The impressive spread of mobile broadband networks has brought online access to people in areas where fixed infrastructure is limited – for example, outside major urban areas, especially in developing countries. And the world over, more and more young people are growing up using digital technologies.

The second edition of the Manual, published in 2014, extended the list of ICT indicators and added a full chapter on coordination of the national statistical system in the area of ICT statistics. However, the continued rapid growth and evolution of the global information society demands continuous review of our current ICT indicators and their definitions. A key component of ITU’s statistical work therefore involves the development and revision of the international standards used to monitor the progress of countries’ transformation into information societies.

In this third edition, we continue the trend of expanding the list of ICT indicators, while taking stock of the experiences of developed and developing countries alike in the implementation of ICT surveys. In addition, we have also introduced new measurement topics, reflecting the evolving nature of digital access and use.

The revisions in this new edition are carried out through the ITU’s Expert Group on ICT Household Indicators (EGH), created in 2012 following a recommendation by the 10th World Telecommunication/ICT Indicators Meeting in Mauritius in 2011. The principal mandate of the EGH is to review the core indicators on ICT household access and individual use, and to revise and update this Manual.

The ICT household indicators covered by this Manual are collected by ITU through an annual questionnaire sent to Member States. They are part of an extensive core list of ICT indicators, developed under the framework of the Partnership on Measuring ICT for Development, and widely used around the world. Since its inception in 2004, this partnership has achieved international recognition as the key initiative concerned with promoting and improving the availability of ICT statistics globally. Its work has been repeatedly endorsed by the United Nations Statistical Commission, and is referred to in the outcome documents of the WSIS and in a number of resolutions adopted by the UN Economic and Social Council. As an active member of the Partnership, ITU has contributed significantly to developing and revising the core list of indicators; in particular those on ICT infrastructure and access, and on ICT access and use by households and individuals.

I am confident that this Manual will serve as an indispensable resource for all those involved in the production of accurate and reliable ICT household statistics.

Doreen Bogdan-Martin
Director
Telecommunication Development Bureau (BDT)
International Telecommunication Union

Foreword

ICT statistics have been on the agenda of the international statistical community for many years and are receiving increasing attention due to the impact these technologies have on the economy and society in general. The Partnership on Measuring ICT for Development has been leading a process, in cooperation with national and international stakeholders, to develop statistical standards and relevant methodologies pertinent to ICT measurement.

Starting from its 38th session, held in 2007, the United Nations Statistical Commission has been endorsing the core list of ICT indicators. The core list, which was developed by the Partnership, includes indicators on ICT infrastructure and access; ICT access and use by households and individuals; ICT use by businesses and the ICT (producing) sector. At its 43rd session in 2012, the Commission endorsed a revised and extended core list of ICT indicators, which also included new indicators on ICT in education and e-government, and asked that countries use the list as a reference for the production of ICT-related statistics. The revision and expansion of the list also underline the fast pace of ICT development and the need for statistics to keep up and provide relevant data. The Commission consequently urged the Partnership on Measuring ICT for Development to update the list of indicators regularly and assist countries in their capacity building efforts for ICT data collection. The core list was further updated in 2016 and endorsed by the Commission in the same year.

This Manual is an update of the 2009 and 2014 publications with the same title, dedicated to the collection of ICT statistics at the household level, reflecting the need for statistical methods and concepts to keep up with changes in the area of information and communication technologies.

This Manual complements the UNCTAD Manual on the Production of Statistics on the Information Economy, which covers ICT statistics collected through business surveys and is based on the Partnership core list of indicators on ICT use by businesses. The two manuals provide an important set of tools at the disposal of national statistical offices for use in their ICT data collection programmes.

The publication is intended to assist national statistical offices in collecting and compiling ICT statistics. It provides updated guidance on the collection, processing, evaluation and dissemination of ICT household statistics and will be a useful reference for ICT data producers worldwide.

I wish to congratulate the ITU on continuing to take a leading role in the development of internationally comparable indicators on ICT infrastructure and ICT access and use by households and individuals.

Stefan Schweinfest
Director
United Nations Statistics Division

Table of Contents

Acknowledgements

Preface

Foreword

Index of boxes, tables and figures

Boxes

Tables

Figures

Chapter 1. Introduction

Conceptual framework for the Information society

International work on measuring ICT

International Telecommunication Union

United Nations Conference on Trade and Development

UNESCO Institute for Statistics

Organisation for Economic Co-operation and Development

Eurostat

United Nations Department of Economic and Social Affairs

United Nations Environment Programme Secretariat of the Basel Convention

International Labour Office

United Nations University

World Bank

United Nations Regional Commissions

ICT statistics to measure progress towards the Sustainable Development Goals

More general work by international organizations in standard setting

Scope and structure of the Manual

Chapter 2. Coordination among national stakeholders in ICT measurement

Stakeholders in the national statistics system and coordination of the production of statistics

Models and mechanisms for coordination

Issues in coordination among producers

Multiyear planning as a mechanism for coordination

Mechanisms for user consultation

Relationships with data providers

Chapter 3. Planning and preparation for ICT household surveys

ICT household survey planning

Budget and management issues

Other general preparatory work

Staff training and selection

Chapter 4. Statistical standards and measurement topics for ICT household statistics

Core ICT household indicators

Household ICT access core indicators

Household ICT expenditure core indicator

Individual ICT use core indicators

Classifications for ICT household statistics

Individual characteristics

Household characteristics

Cross-classification of data

Other classificatory variables

Detailed information on core ICT household indicators

Other measurement topics related to ICT household statistics

Cybersecurity: trust in the online environment and child online protection

The social and economic impacts of ICT access and use by households and individuals

Internet of Things (IoT)

Gender-sensitive indicators

Time-related issues

Chapter 5. Data sources and collection techniques for ICT household statistics

Data sources: surveys, administrative data and Big Data

Types of surveys

Surveys not specific to ICT

Stand-alone ICT surveys

Advantages and disadvantages of using stand-alone ICT surveys

Administrative sources

Data collection techniques

Personal face-to-face interviews

Personal telephone interviews

Self-enumeration

Internet-based surveys

Chapter 6. Question and questionnaire design for ICT household surveys

General principles of questionnaire design for household surveys

ICT model questions

Reference period

ICT concepts that may be difficult to understand

Questionnaire logic

Chapter 7. Sampling for ICT household surveys

Scope and coverage for households and individuals

Target populations and sample frames

Statistical units

Sample design and selection

Risks of other sample selection methods

Chapter 8. Data processing for ICT household statistics

Data entry

Data editing

Microedits

Macroedits

Imputation for missing data (non-response)

Weighting of data

Calculating and reporting ICT household indicators

Chapter 9. Data quality and evaluation for ICT household statistics

Statistical quality

Sampling error

Non-sampling error

Assessment of data quality

Evaluation

Chapter 10. Dissemination of ICT household data and metadata

Data dissemination

Tabulation plans for ICT indicators

Data visualization

Metadata reporting and dissemination

Data collection and dissemination of ICT statistics by ITU

Annex 1. Core list of ICT Indicators (as of 2016)

Core indicators on ICT infrastructure and access

Core indicators on access to, and use of, ICT by households and individuals

Core indicators on use of ICT by enterprises

Core indicators on the ICT sector and trade in ICT goods

Core indicators on ICT in education

Core indicators on e-government

Annex 2. Model questionnaire for measuring ICT access and use by households and individuals

Notes and instructions to questionnaire designers

Model questionnaire for measuring ICT access and use by households and individuals

Annex 3. Examples of imputation and weighting

Imputation for missing data

Tree-based algorithms

Weighting records

Use of modern software

Annex 4. ITU Questionnaire on Information and Communication Technology (ICT) Access and Use by Households and Individuals

ICT Access by urban/rural and household composition (Table 1a of the questionnaire)

ICT Usage by sex and urban/rural (Table 2a of the questionnaire)

ICT Usage by age and sex (Table 2b of the questionnaire)

ICT Usage by highest education level attained/received and sex (Table 2c of the questionnaire)

ICT Usage by labour force status and sex (Table 2d of the questionnaire)

ICT Usage by occupation (Table 2e of the questionnaire)

Annex 5. Glossary of terms and abbreviations

Bibliography

Index of boxes, tables and figures

Boxes

Box 1. The Partnership on Measuring ICT for Development

Box 2. The Global e-Waste Statistics Partnership

Box 3. The Regional Centre for Studies on the Development of the Information Society of Brazil

Box 4. Centre for Statistics and Monitoring of the Information Society- Russian Federation

Box 5. UNECE/EUROSTAT Generic Law on Official Statistics

Box 6. National Coordination for ICT Statistics: Philippines

Box 7. Cooperation between the Information Technology Authority and the National Centre for Statistics and Information in the Sultanate of Oman

Box 8. Partnership between the National Communications Authority and the Ghana Statistical Service

Box 9. Information society statistics in the Philippines

Box 10. National Observatory of Telecommunications and the Information Society in Spain

Box 11. Qatar: ICT Observatory & ICT Directory

Box 12. An example of supervision and monitoring in ICT household surveys

Box 13. The 2019 National ICT Household Survey-Philippines to monitor development plans

Box 14. Oman: measuring barriers to household access to a computer

Box 15. Washington Group on Disability Statistics Short Set of questions

Box 16. Urban and rural areas

Box 17. India: definition of rural and urban areas

Box 18. Choice of topics in European ICT surveys

Box 19. Brazil: measuring online cultural activities

Box 20. Canada: Questions on Cybersecurity

Box 21. Canada: measuring IoT in household surveys

Box 22. Nepal: surveys used to collect household ICT access data

Box 23. Kenya: Inclusion of ICT Questions in the Population Census, 2019

Box 24. Inclusion of ICT questions in the Palestinian Housing Conditions Survey 2015 and the Multipurpose Household Survey of Costa Rica

Box 25. Saudi Arabia: Households and Individuals ICT Access and Usage Survey

Box 26. Community Survey on the usage of ICT among households and individuals- Luxembourg

Box 27. Use of a combination of techniques to collect ICT household data

Box 28. Lebanon: use of PDAs in Labour Force and Households’ Living Conditions Survey

Box 29. Malta: ICT Usage by Individuals and Households

Box 30. Cognitive interviewing as a tool to evaluate questions: a practical case from Brazil

Box 31. How to ask the question on Internet access by type of access

Box 32. In some countries, many use the Internet without realizing

Box 33. Hong Kong, China: question on use of e-Government services, 2008

Box 34. Luxembourg: Question on e-commerce

Box 35. Canada- Question on digital skills

Box 36. Uganda- Questions on ICT skills

Box 37. Interviewer instructions for ICT household surveys of Canada and Hong Kong, China

Box 38. ICT gap between generation- Curaçao experience

Box 39. Luxembourg: Community Survey on the usage of ICT among households and individuals

Box 40. Definition of household in France

Box 41. Honduras: change in statistical units in household surveys

Box 42. Definition of household used in Australia and Hong Kong, China

Box 43. Greece: stratification of the sample of the ICT household survey

Box 44. AfterAccess: Challenges of collecting robust data for policymaking in the Global South

Box 45. Example of a consistency check

Box 46. Australia: example of post-stratification

Box 47. Using R to tabulate ICT survey data in Brazil

Box 48. Sampling error of the value of an ICT household indicator

Box 49. Use of disseminated microdata on ICT e-commerce in Spain

Tables

Table 1. SDG targets related to ICT

Table 2. Reference material for household surveys and standards

Table 3. List of indicators on access to, and use of, ICT by households and individuals (“ITU ICT HH indicators”)

Table 4. Classification of ICT equipment and services, based on COICOP 2018

Table 5. ISCO major groups: 1988 and 2008

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23

Table 7. Types of surveys where ICT questions have been included between 2014 and 2017

Table 8. Surveys used by countries of the Latin American and Caribbean region to measure household access to, and individual use of ICT

Table 9. Mode of data collection, survey vehicle and response rate for countries conducting the Eurostat community survey, 2017

Table 10. Summary of data collection methods

Table 11. Structure and logic of a model questionnaire/module for collecting ICT household data

Table 12. Micro and macroedits for ICT household statistics

Table 13. Example of data reporting: partial table

Table 14. Example tabulation for household ICT access core indicators

Table 15. Example tabulation for barriers to household Internet access

Table 16. Example tabulation for ‘whole population’ individual ICT use indicators

Table 17. Example tabulation for the location of Internet use

Table 18. Example tabulation for ICT skills of individuals

Table 19. Metadata associated with ICT household surveys

Figures

Figure 1. Information society conceptual framework

Figure 2. Stakeholders in the ICT statistics system

Figure 3. Scheme of the Generic Statistical Business Process Model (GSBPM)

Figure 4. ITU ICT HH indicators by type of statistical unit (households and individuals)

Figure 5. Example of data visualization

Chapter 1. Introduction

1. The availability of information and communication technology (ICT) infrastructure – and the uptake in its use – continue to grow. The Internet, in particular, is transforming society, with a growing amount of information and a large number of tools available. These advances, together with the recognition that ICTs are a driver of social and economic development, have driven a need for reliable, comprehensive and comparable statistics to support government and industry policy decisions. While demand for data on access to, and use of, ICT has grown, and many developing countries have made significant progress in the measurement of ICT access and use, the availability and quality of statistics derived from household surveys is still limited in terms of coverage and frequency.

2. Measuring ICT for development has been emphasized in the World Summit on the Information Society (WSIS). The Summits have highlighted the importance of measuring progress towards the information society through internationally comparable statistical indicators and have called upon countries and international organizations to allocate appropriate resources for the provision of ICT statistics required for the analysis of the changing information society.

3. In September 2015, the 2030 Agenda for Sustainable Development was agreed at the United Nations Sustainable Development Summit. This new framework for international cooperation to promote sustainable development between 2015 and 2030 is composed of 17 Sustainable Development Goals (SDGs), 169 targets, and some 232 indicators. While none of the goals is specifically about ICTs, several targets refer to digital technology. The 2030 Agenda for Sustainable Development also recognizes that “The spread of information and communications technology and global interconnectedness has great potential to accelerate human progress, to bridge the digital divide and to develop knowledge societies". Sustainable Development Goal 9, which concerns industrialization, innovation and infrastructure, recognizes the importance of ICTs and establishes Target 9.c, to “significantly increase access to information and communications technology and strive to provide universal and affordable access to the Internet in least developed countries by 2020”.

4. Measuring ICT access and use by households and individuals is key to monitoring the progress of countries towards becoming information societies. This Manual will assist countries to achieve that by enabling the production of high quality and internationally comparable data.

5. The Manual focuses on household surveys as instruments to investigate access to, and use of, ICT by households and individuals.[1] In covering many of the statistical difficulties in this field, the Manual aims to assist national statistical offices with limited capacity (in particular, in developing economies) in their ICT household data collection endeavours, while ensuring comparability between statistics collected by all countries through adherence to globally agreed core ICT indicators and associated statistical standards.

6. The intended audience for this Manual are national institutions in charge of collecting, processing and disseminating ICT statistics and indicators. In most countries, this is the responsibility of National Statistical Offices (NSOs), which routinely collect social, economic, environmental and other official statistics. In some countries, agencies other than NSOs are involved in the production of ICT statistics, such as ministries for telecommunications, telecommunication regulators, etc. However, it is recommended that NSOs contribute significantly to this task, as they have the expertise (interviewers, statisticians, IT experts), the statistical infrastructure (household and business frames) and, moreover, the role of coordinating the National Statistical System. The issues related to inter-institutional coordination are further discussed in this Manual.

7. This Manual and the associated training course are a major part of ITU’s technical assistance work in this area of statistics, which in 2020 will be extended to an online training course on the ITU Academy online platform.[2] In this regard, ITU responds to the mandate of its member countries through the World Telecommunication Development Conference (WTDC). At the 2010 conference, Resolution 8 covered the collection and dissemination of information and statistics, and included clause 11 (“to provide technical assistance to the relevant national authorities for the collection of ICT statistics, in particular by means of national surveys”) and clause 12 (“to develop training material and conduct specialized training courses on information society statistics for developing economies.”) (ITU, 2010a).[3] Resolution 131 of the Plenipotentiary Conference (rev. Dubai 2018) echoed this by instructing the Director of the Telecommunication Development Bureau “to hold, on a regular basis, regional seminars and training events for developing countries, in order to raise the level of knowledge and skills in the collection and processing of ICT indicators.”

8. The first edition of the Manual, of 2009, introduced standards for the collection of ICT statistics at the household and individual level, including the definition of the core ICT indicators and the concepts needed to understand the object of measurement.

9. The 2014 edition updated the previous one by:

• Revising the core list of indicators and changing the presentation of the indicators following the recommendation of the ITU Expert Group on ICT Household Indicators (EGH);

• Adding indicator HH16;

• Updating definitions, classifications and examples;

• Creating tables for each core indicator of the most relevant metadata such as definitions of technical terms, clarifications and methodological issues, model questions, disaggregation and classifications, core indicator calculation, and policy relevance;

• Extending the conceptual framework and international work carried out on ICT measurement;

• Including a new chapter on national coordination of ICT statistics;

• Giving extra emphasis to the collection of ICT statistics through existing household surveys, providing examples of country practices.

• Including some revisions in the areas of survey methodology, sampling design, and data collection standards specific to the core indicators.

10. Since 2014, a number of indicators have been revised and new indicators have been developed. This revised edition of the Manual updates the list of indicators (adding indicators HH17 to HH23) and their breakdowns to further reflect the decisions of the EGH and the experiences of the countries and international organizations (principally Eurostat and the OECD) which already have experience in this area of official statistics. Advances in data collection methods (such as the use of electronic questionnaires, the combination of survey and administrative data and the use of Big Data sources) are also considered in this edition. Country examples are updated, with new ones selected for the purposes of illustrating the diversity of practices.

Conceptual framework for the Information society

11. Conceptually, the information society is a complex set of topics, entities, actions and relationships. OECD (2009 and 2011), through its Working Party on Indicators for the Information Society (WPIIS)[4], has developed a broad conceptual framework covering the information society measurement areas of ICT demand and supply, ICT infrastructure, ICT products, and electronic content and media. Further work by OECD, the G20, UNCTAD and other institutions has led to the conceptual definition of the “digital economy” to encompass the economic impact of ICT[5]. Figure 1 provides an adapted and simplified view of the conceptual framework and shows how ICT household statistics fit into the ‘demand side’ of the framework.

Figure 1. Information society conceptual framework

12. The supply side provides ICT infrastructure and products (goods and services). ICT infrastructure has quickly evolved since the first edition of the Manual, with the increased affordability of devices, the development of mobile broadband connections to the Internet (LTE and the emergence of 5G), Internet of Things (IoT) and cloud computing, and the ubiquitous presence of ICT-enabled services[6] in such diverse areas as human communication, education, tourism, health, finance, etc.

13. Once these are in place, it is necessary to develop the skills and knowledge to use them and to transform them into powerful tools for social and economic development. Sound public policy is necessary to ensure that maximum benefit is obtained from ICT and to facilitate the transition towards an information society. Relevant policies can include national strategies, legislation, regulation and appropriate incentives. The ultimate goal is that the efficient and effective use of ICT is reflected in social and economic benefits (impact) for society.

14. The recognition that ICT can be a development enabler, if applied and used appropriately, is critical to countries that are moving towards information or knowledge-based societies (ITU, 2009a, UNCTAD, 2019). The 2030 Agenda recognizes that “the spread of information and communication technology and global interconnectedness has great potential to accelerate human progress, to bridge the digital divide and to develop knowledge societies". In this process, a careful look at the social dimension of ICT impact is of key importance. This includes aspects related to the digital divide (inequality in access to, and use of, ICTs), social interaction through ICT and, more generally, characteristics of access to technologies and their use by different demographic and social groups. Policy-makers can maximize the benefits of ICT for their citizens by promoting equality and security of access and use, the skills required for use, the availability of ICT infrastructure, and the affordability of ICT services. Broadband Internet access and use in particular are enablers of the digital economy, and can be a tool for inclusive and sustainable development by facilitating access to education (e.g. through massive, open online courses), access to health services (e.g. remote diagnosis, remote surgery), financial inclusion (e.g. provision of mobile banking), helping monitor the environment (e.g. weather measurements, humanitarian relief after natural disasters) (Broadband Commission for Sustainable Development, 2018). ICT (and other) skills determine the effective use that is made of ICTs, and are critical to leveraging the full potential of ICTs for socio-economic development. Economic growth and development will remain below potential if economies are not capable of exploiting new technologies (ITU, 2009a).

15. Therefore, it is essential to take stock of the infrastructure and products available, their coverage, their benefits and shortcomings, the level of equality or inequality of their availability, their level of penetration, access and use in societies and economies, and level of required and available skills. That is, collection of data on ICT in society is necessary to monitor the progress towards achieving an information society and to provide the information needed by policy-makers to guide that progress.

16. The component of main interest for this Manual relates to ICT access and use by households and individuals, although all elements of the framework are relevant to some degree. For instance, households use ICT infrastructure and interact with businesses and government. The measurement of ICT demand by businesses and ICT supply, is covered in the UNCTAD Manual for the Production of Statistics on the Information Economy (UNCTAD, 2009)[7], as well as in the methodological recommendations to measure trade in ICT services and provision of ICT-enabled services (UNCTAD, 2015) . Measurement of ICT demand by schools is covered in the UNESCO Institute for Statistics (UIS) Guide to Measuring Information and Communication Technologies (ICT) in Education (UIS, 2009). Similarly, the Framework for a set of e-government core indicators covers the measurement of ICT demand and use in government (Partnership and UNECA, 2012).

17. It is important, at the outset, to distinguish what we mean by ICT access and ICT use. ICT access refers to availability of ICTs (working devices and services) for use by any member of the household at any time, independently of whether the device is owned or not by the household. Use of ICT refers to use by one or more individuals of the household, whether at home or elsewhere.

18. Household statistics on ICT access and use are mainly produced by national statistical offices (NSOs) using traditional household surveys. These can be broadly categorized as household surveys that are either dedicated to measuring ICT access and use or surveys that include some questions or modules on ICT access and use. NSOs[8] have particular expertise in conducting household surveys. Their role in ICT household statistics is considered in Chapter 2, which discusses the importance of relationships between data producers, data users and data providers.

International work on measuring ICT

19. During the past decade, several international organizations have been involved in the development of statistical standards for measurement of ICT infrastructure, access and use by different sectors of the economy and society.

20. The coordination of the work of international organizations with respect to ICT statistics is done through the Partnership on Measuring ICT for Development.

21. The Partnership on Measuring ICT for Development (Box 1) is a multi-stakeholder initiative consisting of 14 international and regional organizations involved in ICT measurement. It was established following the Geneva phase of the World Summit on the Information Society in 2003[9] and officially launched in 2004, with the overall objective of improving the availability and quality of internationally comparable ICT statistics.

22. One of the main achievements of the Partnership was the development of a core list of ICT indicators, with their corresponding definitions and other metadata, in close consultation with other stakeholders, mainly NSOs. At its meetings of 2007, 2012, 2014, 2016 and 2018, the United Nations Statistical Commission (UNSC) endorsed the Partnership core list of ICT indicators and its revisions, and encouraged countries to use it in their data collection programmes. The core list, with the revisions and additions presented in this Manual, includes more than 60 indicators, covering ICT infrastructure and access, ICT access and use by households and businesses, the ICT (producing) sector, trade in ICT goods and services, ICT in education, e-government and electronic waste. The main purpose of the core list is to help countries produce high quality and internationally comparable ICT statistics. The indicators have associated statistical standards, including concepts, definitions, model questions, classificatory variables, and guidance on scope and statistical units. This edition of the Manual updates standards for ICT household statistics.

23. The work of the Partnership has been recognized by the Economic and Social Council (ECOSOC) through several resolutions. The United Nations Statistical Commission validates the statistical development work of the Partnership, thus guaranteeing that standards are coherent with those of other areas of official statistics. Every two years, the Secretary-General presents to the Statistical Commission a report on international activities in the area of ICT statistics (see Partnership for Measuring ICT for Development 2012, 2014, 2016 and 2018).

24. ECOSOC Resolution 2008/3 recommends that the Partnership considers the creation of benchmarks and indicators, including impact indicators, for further consideration and decision by the UN Statistical Commission, in order to track progress towards the attainment of the specific goals and targets set out in the outcome documents of the World Summit on the Information Society, particularly section B of the Plan of Action adopted in Geneva. Resolution 2009/7 recognizes its institutional strengthening and the creation of the working group to measure the economic and social impact of information and communication technologies. Resolutions 2011/16 and 2012/5 call upon the Partnership to further its work on measuring the impact of information and communication technologies, particularly in developing economies, by creating practical guidelines, methodologies and indicators. They encourage governments to collect relevant data at the national level on ICTs, to share information about country case studies and to collaborate with other countries in capacity-building exchange programmes. The 2013 resolution encouraged member states to provide information to the Partnership, so as to contribute to its final assessment report on the achievement of the WSIS targets.[10] Led by ITU, the Partnership prepared the Final WSIS Targets Review: Achievements, Challenges and the Way Forward (Partnership on Measuring ICT for Development, 2014a). This report analyzed and discussed the achievements made on each one of the ten WSIS targets. The report was launched in June 2014, at the WSIS+10 high-level meeting on the overall review of the WSIS, in June 2014.

25. In addition to establishing a core list of indicators, the Partnership and its members are involved in a number of activities that support their mission of achieving internationally comparable and reliable ICT statistics. These include the dissemination of national ICT data, development of statistical manuals and the provision of capacity-building services (training and technical assistance missions) to enable statistical agencies to collect the data necessary to compile the core indicators.

Box 1. The Partnership on Measuring ICT for Development

Launched:

June 2004 at UNCTAD XI (Sao Paulo, Brazil).

Current members:

ITU, OECD, Eurostat, UNCTAD, UNESCO Institute for Statistics (UIS), ILO, four UN Regional Commissions (UNECLAC, UNESCWA, UNESCAP, UNECA), the World Bank, UNDESA, UNEP/Secretariat of the Basel Convention, and the United Nations University Vice-Rectorate in Europe Sustainable Cycle Programme (UNU-ViE SCYCLE).

Objectives:

Defining a core list of ICT indicators and methodologies to collect these indicators;

Helping developing economies collect ICT statistics, particularly through capacity building and hands-on training for national statistical offices; and

Collecting and disseminating information society statistics in a number of formats, including global reports and databases.

Memorandum of Understanding:

Signed by all partners in order to further strengthen their commitment and to provide guidelines to potential new members.

Structure:

A Steering Committee (currently consisting of ITU, UNCTAD and UN-DESA) plus task groups. Some task groups have members that are not members of the Partnership.

Core indicators:

A first edition of the core list of ICT Indicators was released during the Tunis phase of the World Summit on the Information Society (WSIS) in 2005. It was the result of an intensive consultation process with statistical agencies and policy-makers that was facilitated by members of the Partnership. The core list was composed of 41 ICT indicators on infrastructure, access and use by households and businesses, the ICT (producing) sector and trade in ICT goods (Partnership, 2005). The list has been revised and added to over time and includes more than 60 indicators as of June 2019. The complete revised core list can be found in Annex 1.

The Partnership also studied the integration of ICT statistics into the monitoring framework of the 2030 Agenda for Sustainable Development.

Capacity building:

The capacity-building work of the Partnership is undertaken by its members independently but coordinated through the Partnership. Activities include the conduct of training courses and workshops, as well as the production of technical material (of which this Manual, as well as its previous editions of 2009 and 2014, are examples). Other methodological manuals include those on statistics of the Information Economy (UNCTAD, 2007 and 2009), e-government (Partnership and UNECA, 2013), use of ICT in education (UIS, 2009). OECD (2011) and Eurostat (2013) cover broader areas of information society measurement. Materials from the workshops and training courses designed for NSIs are available online.[1]

More information on the Partnership, its members and its activities can be found at: http://www.itu.int/ITU-D/ict/partnership/index.html.

[1] https://www.itu.int/en/ITU-D/Statistics/Pages/events/tech.aspx

26. The Partnership ensures that the competencies of each partner organization are used and overlaps are avoided. The work of Partnership members in the area of ICT measurement is described below.

International Telecommunication Union

27. The International Telecommunication Union (ITU) has a long history in setting standards for telecommunication statistics. Of most relevance to the Manual is the important role played by ITU in standardizing definitions of telecommunication/ICT indicators. Over 90 ICT indicators produced by ITU are defined in the Handbook for the Collection of Administrative Data on Telecommunications/ICT (2020), which is updated and reviewed regularly. ITU reviews these indicators through the Expert Group on Telecommunication/ICT Indicators (EGTI),[11] which was created in May 2009. The EGTI has a mandate to revise the list of ITU supply-side indicators, as well as to discuss outstanding methodological issues and new indicators. EGTI is open to all ITU members and experts in the field of ICT statistics and data collection. It works through an online discussion forum and occasional face-to-face meetings. EGTI reports back periodically to the World Telecommunication/ICT Indicators Symposium (WTIS). Some of the definitions in the Handbook are used for many of the technical terms (mobile cellular network, Internet access technologies, etc.) used in the Partnership’s core indicators on access to, and use of, ICT by households and individuals.

28. In 2003, closely linked to the WSIS and the recognized need to measure the information society, ITU expanded its statistical work from defining and collecting (primarily administrative) data in the area of telecommunication and ICT infrastructure from regulatory authorities, into the area of household statistics and started to collect data on household ICT indicators from national statistical offices.[12] As an active member of the Partnership,[13] ITU has contributed to the development of the core list of ICT household indicators, including their definitions, consultations with stakeholders and the preparation of relevant methodological documents.

29. This Manual is a further contribution by ITU towards the availability of comparable ICT household statistics based on internationally agreed standards. Revisions to the indicators included in the Manual have been discussed in the Expert Group on ICT Household Indicators (EGH).[14] The EGH was established in May 2012 to review the statistical indicators for measuring ICT access and use by households and individuals. It is open to all ITU members, and to ICT experts and statisticians familiar with data collection on these indicators. EGH works through an online discussion forum and holds annual face-to-face meetings. Reports of the EGH will be presented periodically to the World Telecommunication/ICT Indicators Symposium (WTIS).

30. Based on the Handbook and the Manual, ITU assists governments in developing economies in their ICT data collection and dissemination efforts. Support is provided for the production of statistics in the areas of ICT infrastructure, access, and use by household and individuals. Technical workshops are carried out at the national and regional levels to exchange experiences and discuss methodologies, definitions, survey vehicles and other issues related to the collection of ICT statistics.

31. As part of the global statistical system of the UN, ITU collects statistics covering the telecommunication/ICT sector for about 200 economies worldwide, in line with other specialized agencies that produce statistics covering their respective fields of operations. ITU statistics include indicators outlined in the Handbook and are usually collected from national telecommunication regulatory agencies, ministries or specialized agencies by means of regular questionnaires. Indicators on access to, and use of, ICTs by households and individuals are also collected annually, through a questionnaire sent to national statistical offices (see Annex 4).

32. ITU disseminates the data that it collects in a number of forms. The electronic database, World Telecommunication/ICT Indicators (WTI) database, provides an important historical perspective of the ICT industry, with annual time series dating back to 1960 and extending until 2019. The database includes over 180 indicators, covers about 200 economies and is widely respected as the world’s leading source of ICT statistics. The Measuring Digital Development series features national data. A wealth of information is provided for free at: http://www.itu.int/en/ITU-D/Statistics/Pages/stat/default.aspx on popularly requested telecommunication/ICT statistics.

United Nations Conference on Trade and Development

33. The United Nations Conference on Trade and Development (UNCTAD) has collected data from developing economies since 2004 on the use of ICT by businesses and on the ICT (producing) sector, based on the core list of ICT indicators. UNCTAD has contributed to the development of the core list of ICT business indicators, including their definitions, consultation with stakeholders and preparation of methodological documents. The measurement of ICT use by businesses and of value added and workforce of the ICT sector are outlined in the UNCTAD 2009 Manual for the Production of Statistics on the Information Economy (UNCTAD, 2009), a revised version of which will be released in 2020.

34. The results from the data collection of UNCTAD are used in their research and analysis, such as the Digital Economy Report (bit.ly/2019DER) and are disseminated through UNCTAD's statistics portal (http://unctadstat.unctad.org/). UNCTAD also compiles and publishes data on international trade (imports and exports) of ICT goods, ICT services, and digitally-deliverable services. UNCTAD provides technical assistance to developing economies on the measurement of the digital economy, including through training courses and advisory services. More information on UNCTAD’s measurement work is available on their website at: https://unctad.org/en/Pages/DTL/STI_and_ICTs/ICT4D-Measurement.aspx.

UNESCO Institute for Statistics

35. The UNESCO Institute for Statistics (UIS) is responsible for developing and collecting indicators on ICT access and use in education. UIS has led the work of developing the core list of ICT indicators in education, including definitions and preparation of methodological documents, particularly the Guide to Measuring Information and Communication Technologies (ICT) in Education (UIS 2009). UIS has implemented data collections on ICT in education in Latin America and the Caribbean, in five Arab states, in Asia and in Sub-Saharan Africa, to gather internationally comparable data. The surveys were developed with the International Working Group on ICT Statistics in Education (WISE), which was established by the UIS. The working group provides invaluable insight into the design and implementation of the survey instruments and related methodologies. This work also entails close collaboration with UNESCO’s Communication and Information Sector and other strategic institutional partners.

36. The UIS is currently the custodian agency for the SDG indicator 4.a.1 on schools with access to certain facilities or services, which include access to the Internet and availability of computers for pedagogical purposes. The necessary data to compute that indicator is collected in the UIS Survey of Formal Education annually.

Organisation for Economic Co-operation and Development

37. The OECD undertakes a wide range of efforts to measure and understand the digital transformation, particularly through its Working Party on Measurement and Analysis of the Digital Economy and Committee on Statistics and Statistical Policy.

38. The OECD Going Digital Toolkit[15] presents 33 key indicators that provide an overview of countries’ state of digital development. The indicators are aligned with the OECD Going Digital Policy Framework (OECD, 2020), which helps governments and stakeholders to develop an integrated approach to policy making in the digital age and to shape policies for an inclusive digital future. Complementary indicators and interactive data visualisation features allow the richness of data in the underlying OECD databases to be explored.

39. Measuring the Digital Transformation: A Roadmap for the Future (OECD, 2019a) benchmarked member countries and key partner economies across over 180 indicators, identified measurement gaps, and developed the medium-term Going Digital Measurement Roadmap.  These indicators, and ongoing measurement work, cover many aspects of the digital transformation such as artificial intelligence (AI), the Internet of Things, digital security and privacy, consumer trust in online environments, skills in the digital age, barriers to trade in digital services, and the future of work.

40. Many of these indicators rely on surveys implementing internationally agreed definitions, such as the OECD definition of e-commerce set out in the Guide to Measuring the Information Society (OECD, 2011), and benefit from OECD Model surveys of ICT usage in business and in households (OECD, 2015a); OECD, 2015b). These are periodically reviewed and revised by the OECD Working Party on Measuring and Analysing the Digital Economy.

41. Two key contributions to delivering the Going Digital Measurement Roadmap are the OECD Framework for Digital supply-use tables (OECD, 2019b) and the OECD-WTO-IMF Handbook on Measuring Digital Trade (OECD, 2019c), which will help to make the digital transformation visible in economic statistics.

Eurostat

42. The Statistical Office of the European Union (Eurostat) works closely with the NSOs from its Member States, and other participating countries, to develop and run the annual EU “Community survey on ICT usage in households and individuals” (as well as surveys on ICT usage and e-commerce in enterprises). The harmonized approach based on a common legal instrument (in addition to a model questionnaire) is very effective and provides detailed and highly comparable datasets. Eurostat produces model questionnaires and methodological manuals[16] dealing with measurement of ICT access and use (for example, Eurostat, 2018). Between 2006 and 2013, Eurostat carried out several comparative studies with NSOs to analyse the impact of ICT on the economy through the linking of microdata from different surveys, that is, ICT, innovation and structural business surveys (Eurostat, 2008).

United Nations Department of Economic and Social Affairs

43. The UN Department of Economic and Social Affairs (UNDESA), since 2003, has collected data on the use of ICT by governments in the provision of e-services. UNDESA tracks and monitors the progress in e-government by 193 member states and publishes the findings biennially as the UN E-government Survey (for example, UNDESA, 2018), which features the -Government Development Index- EGDI as a tool to measure countries’ and cities’ use of ICT to deliver public services. UNDESA also contributes its expertise in the area of e-government measurement through the Partnership Task Group on e-Government (TGEG), discussed below.

United Nations Environment Programme Secretariat of the Basel Convention

44. The UN Environment Programme Secretariat of the Basel Convention (UNEP/SBC) provides expertise on e-waste issues. UNEP/SBC leads the Partnership Task Group on Measuring E-waste (TGEW), which aims to support the compilation of reliable data on e-waste as a basis for political decision making and further action on the environmentally sound management of used and end-of-life ICT equipment. It is currently developing a framework document for monitoring e-waste based on internationally defined indicators.

International Labour Office

45. The International Labour Office (ILO) joined the Partnership at a later stage. In 2014, the technical note “Issues in the development of internationally harmonized measures of employment related to ICT” established standards for defining employment in the ICT sector and employment in ICT specialist occupations. The note describes the collection of data and metadata from labour force surveys (LFS) and establishment surveys, covering aspects such as sex disaggregation, correspondence to international classifications by economic activity (ISIC Rev.4) and by occupations (ISCO-08), coverage of the surveys, sample sizes, levels of disaggregation up to which estimates are reliable.

United Nations University

46. The United Nations University (UNU) has been addressing issues associated with the production, usage and final disposal of ICT since 2000. Its Vice-Rectorate in Europe Sustainable Cycle Programme (UNU-ViE SCYCLE) has a special focus on e-waste related issues. UNU-ViE SCYCLE has conducted a number of detailed and standardized studies to qualify and quantify e-waste arising in various countries. Moreover UNU hosts the multi-stakeholder initiative Solving the E-waste Problem (StEP). In 2017, ITU, UNU and the International Solid Waste Association formed the Global e-Waste Statistics Partnership, which aims to build capacity in countries to produce reliable and comparable e-waste statistics.

Box 2. The Global e-Waste Statistics Partnership

The Global E-waste Statistics Partnership (GESP) was founded in 2017 by the International Telecommunication Union (ITU), the United Nations University (UNU), and the International Solid Waste Association (ISWA). The objectives of the Partnership are to monitor developments of e-waste over time, and to help countries to produce e-waste statistics. The initiative will inform policy makers, industries, academia, media and the general public by enhancing the understanding and interpretation of global e-waste data and its relation to the Sustainable Development Goals (SDGs).

The Partnership:

• Collects and improves worldwide e-waste statistics.

• Enhances the understanding and interpretation of global e-waste data.

• Improves the quality of e-waste statistics by guiding countries and building national capacity through e-waste statistics training.

• Raises awareness and communicates the data to the general public and relevant stakeholders.

• Some of the key achievements of the Partnership in 2017/2018:

• Publication of the Global E-waste Monitor 2017: Within hours of its publication the report reached 400+ million audience. It was featured by leading media outlets and nearly all international news services, including National Geographic, Washington Post, New York Times, and many others. It was covered in over 80 countries, in 25 languages, and in close to 2,000 news items.

• Regional capacity building workshops in East Africa, Latin America, and the Arabian States: Over 180 people from 40 countries were trained.

• Publication of the second edition of “E-waste Statistics: Guidelines for classification, reporting and indicators”.

• The Global E-waste Statistics Partnership, launched globalewaste.org, an open source portal that visualizes e-waste data and statistics globally, by region and by country, for policy-makers, industry, academia and the public.

Source: https://globalewaste.org/about-us/

World Bank

47. The World Bank’s work on ICT indicators is mainly carried out through its ICT Sector Unit and Development Data Group. It consolidates ICT indicators from ITU and other sources, including various household and business surveys conducted by the World Bank, and publishes the “Little Data Book[17] on Information and Communication Technology” in partnership with ITU. These provide country specific key ICT data (about 30 indicators) and indices. ICT indicators are also featured in the World Bank’s triennial report series Information and Communications for Development. The World Bank contributed to the work of the Partnership through the Task Group on Database Development, which looked at the dissemination of data for the core list of ICT indicators collected by different members of the Partnership.

United Nations Regional Commissions

48. The UN Regional Commissions have the mandate to promote a regional perspective to global discussions while introducing global concerns at the regional and subregional levels. UN Regional Commissions contribute to the Partnership with their strengths, such as ground presence, knowledge about regional particularities and close relationships with their member countries. In addition, they coordinate statistical activities in their respective regions and liaise directly with NSOs.

UN Economic Commission for Africa

49. The UN Economic Commssion for Africa (UNECA) is the regional coordinator of statistical activities in Africa and in 1990 launched the Addis Ababa Statistical Action Plan for Africa. It also hosts the secretariat of the Advisory Board for Statistics in Africa (ABSA) made up of representatives from member states, regional organizations and partners working on statistics in Africa.

50. UNECA led the Partnership Task Group on e-Government (TGEG), which developed and released the e-government core indicators, in Framework for a set of e-government core indicators (Partnership and UNECA, 2012). The Framework and the Manual for measuring e-government (Partnership and UNECA, 2013). Can be used by countries when collecting data for the e-government core indicators.

UN Economic Commission for Latin America and the Caribbean

51. The UN Economic Commission for Latin America and the Caribbean (ECLAC) acts as the secretariat of the Statistical Commission of the Americas and the technical secretariat of the Digital Agenda for Latin America and the Caribbean (eLAC2020). ECLAC assists its member countries in the compilation, analysis and dissemination of data and indicators on access to, and use of, ICTs, through capacity building, technical assistance and the provision of conceptual frameworks. This has promoted the harmonization of statistics and facilitated monitoring ICT policies in Latin America and the Caribbean, especially through the Working Group on Measurement of Information and Communication Technologies[18] of the Conference of Statisticians of the Americas, which can be considered as a good example of collaboration of NSOs at the regional level, organizing training workshops (including online seminars) and discussing methodological tools. ECLAC is leading a data laboratory that seeks to boost innovation in the use of alternative data sources and their combination with official statistics for the measurement of the digital economy. In this framework ECLAC opted for the use of public data available on the web that were captured through web crawling and web scraping techniques, in addition to the use of APIs (Application Programming Interfaces). Topics analysed include online labour markets and digital skills, prices of technological goods, crowdfunding, e-commerce, cryptocurrency trends and social network analysis reviewing content about the Sustainable Development Goals (SDGs).

UN Economic and Social Commission for Western Asia

52. The UN Economic and Social Commission for Western Asia (ESCWA) is acting as the secretariat of the Statistical Commission for Western Asia. The Information and Communication Technology Division (ICTD) at ESCWA plays a major role with member countries in measuring the information society and their transformation towards a knowledge-based society. In this regard, ICTD produces periodical profiles of the information society for each of the member countries and for Western Asia as a whole (see for example the publication “Arab Horizon 2030: Digital Technologies for Development”)[19]. In addition, the ICT and Statistics divisions assist member countries with methodological work related to collecting ICT statistics using household and business surveys, capacity building and technical cooperation in ICT statistics.

UN Economic and Social Commission for Asia and the Pacific

53. The UN Economic and Social Commission for Asia and the Pacific (ESCAP) is the regional coordinator of statistical activities and is committed to a resilient Asia and the Pacific founded on shared prosperity, social equity and sustainability. ESCAP publishes reports on ICT issues based on statistics and has developed materials for advocacy for ICT statistics as a contribution to evidence-based ICT policy-making (ESCAP, 2018), which can also be used by other countries.

ICT statistics to measure progress towards the Sustainable Development Goals

54. In March 2017, the Statistical Commission agreed upon the global framework of indicators for the Sustainable Development Goals (SDGs) that had been developed by the Inter-Agency and Expert Group on Sustainable Development Goal Indicators. Of the 232 indicators included in the framework, 7 ICT indicators have been included and ICTs have been recognized as a key development enabler. Following this, the Partnership established in 2017 a task group on ICT for the Sustainable Development Goals. This Task Group has produced a thematic list of ICT indicators that could be used to measure ICT availability and its use in sectors relevant to the SDGs that are not covered in the global framework of indicators.[20]

55. While none of the SDGs focuses exclusively on ICT, four of them explicitly mention ICTs: Goal 4 (target 4.b), Goal 5 (target 5.b), Goal 9 (target 9.c) and Goal 17 (target 17.8). (see Table 1).

Table 1. SDG targets related to ICT

Goal

Target

4. Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all

Target 4.b: By 2020, substantially expand globally the number of scholarships available to developing countries, in particular least developed countries, small island developing States and African countries, for enrolment in higher education, including vocational training and information and communications technology, technical, engineering and scientific programmes, in developed countries and other developing countries

5. Achieve gender equality and empower all women and girls

Target 5.b: Enhance the use of enabling technology, in particular information and communications technology, to promote the empowerment of women

9. Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation

Target 9.c: Significantly increase access to information and communications technology and strive to provide universal and affordable access to the Internet in least developed countries by 2020

17. Strengthen the means of implementation and revitalize the Global Partnership for Sustainable Development

Target 17.8: Fully operationalize the technology bank and science, technology and innovation capacity-building mechanism for least developed countries by 2017 and enhance the use of enabling technology, in particular information and communications technology

Source: SDG Indicators Metadata Repository (https://unstats.un.org/sdgs/metadata/)

56. However, to measure the contribution of ICT to sustainable development, the SDG monitoring framework will not produce sufficient information. In particular, additional ICT indicators would be useful to measure progress towards SDGs 1, 2, 8, 13, and 16. In total, the Partnership on Measuring ICT for Development has identified 30 other targets that would benefit from additional indicators on usage, affordability, quality of access and ICT skills, to measure the impact of ICT in progress towards SDGs.[21]

More general work by international organizations in standard setting

57. A number of international organizations are active in developing standards for household surveys more generally. These are detailed in Table 2 and are likely to be relevant to countries measuring ICT household statistics. Some refer to standardized methodologies and survey vehicles, while others refer to major classifications on which ICT household classifications are based.

58. At the international level, the coordination of the statistical work of agencies has been facilitated since 2002 by the Committee for the Coordination of Statistical Activities (CCSA).[22] Among other things, it maintains a Global Inventory of Statistical Standards, which includes those on ICT statistics.

Table 2. Reference material for household surveys and standards

Publishing entity

Title

Short description

International Labour Organization (ILO, 1993)

International Standard Classification of Occupations (ISCO)

ISCO is a classification that organizes jobs into a clearly defined set of groups according to the tasks and duties undertaken in the job.

International Labour Organization (ILO, 2013)

International Classification of Status in Employment (ICSE)[23]

ICSE classifies jobs held by persons, with respect to the type of explicit or implicit contract of employment of the person with other persons or organizations.

International Household Survey Network (IHSN, 2013)[24]

Maintenance of a catalogue of developing economies’ household surveys, and the development of tools for metadata management.

UNCTAD and International Labour Organization (2015)

Global Assessment of Sex-Disaggregated ICT Employment Statistics:

Data availability and challenges on measurement and compilation[25]

United Nations Educational, Scientific and Cultural Organization (UNESCO, 2011)

International Standard Classification of Education (ISCED)[26]

ISCED is a classification that describes the educational attainment of individuals. The current version (ISCED, 2011) organizes educational attainment into eight levels from early childhood education to doctoral level.

United Nations Statistics Division (2005a)

Household Sample Surveys in Developing and Transition Countries[27]

Guidance on conducting household surveys in developing and transition economies, including sample design, survey implementation, non-sampling error, survey costs and data analysis.

United Nations Statistics Division (UNSD, 2005b)

Designing Household Survey Samples: Practical Guidelines^27

Provides a practical reference tool for those involved in designing and implementing household sample surveys.

United Nations Statistics Division (UNSD, 2017)

Principles and Recommendations for Population and Housing Censuses Revision 3

Information on statistical standards as well as on the conduct of population censuses. In particular, it recommends the collection of statistics on household access to ICT as a ‘core topic’.[28]

United Nations Statistics Division (updated as of June 2019)

SDG Metadata repository[29]

Official list, definitions and metadata for the Sustainable Development Goals, their targets and indicators.

Washington Group on Disability Statistics (established by the UN Statistical Commission)

Implementation guidelines[30]

Methodological guidelines for the internationally comparable collection of data on disability status within censuses and household surveys. A set of questions has been proposed, asking whether people have difficulty performing basic universal activities (walking, seeing, hearing, cognition, self-care and communication).

World Bank (2013)

Living Standards Measurement Study (LSMS) Surveys

The LSMS provides survey tools that are applicable to any complex household survey. The LSMS has a broad objective of improving the quality of household statistics in developing economies, with a more specific goal being to develop methods to monitor progress in raising living levels in developing economies.[31]

Scope and structure of the Manual

59. The Manual focuses on the core list of ICT indicators on household ICT access and use developed by the Partnership and collected by ITU at the international level, in particular those produced by NSOs through household surveys. All the core indicators can be found in Annex 1.

60. Many aspects of survey design and processing are not specific to ICT household statistics. While this Manual broadly covers those aspects, it leaves it for other publications to provide detailed methodological advice for conducting household surveys or including ICT-related questions in Population and Housing Censuses. References are provided to such publications, the main ones being from the United Nations Statistics Division and the World Bank:[32]

• Household Sample Surveys in Developing and Transition Countries (UNSD, 2005a);

• Designing Household Survey Samples: Practical Guidelines (UNSD, 2005b);

• Living Standards Measurement Study (LSMS) Surveys (World Bank, 2013); and

• Principles and Recommendations of Population and Housing Censuses (UNSD, 2017) Rev 3.

61. Following the recommendations of the 10th World Telecommunications/ICT Indicators Meeting (WTIM-12), Chapter 2 of the Manual focuses on the coordination of the compilation of ICT household statistics at the national level, which is considered a major issue for the development of ICT statistics. The chapter looks at the roles of data producers, users and providers/respondents in a national context and presents models and mechanisms for their coordination.

62. Chapter 3, Planning and preparation for ICT household surveys, describes the range of preparatory activities for the measurement of access to, and use of, ICT, especially using sample surveys. It explains planning, budgeting and other preparatory work.

63. Chapter 4, Statistical standards and measurement topics for ICT household statistics, addresses statistical standards for ICT household statistics. It introduces the core ICT household indicators and associated classificatory variables that describe household and individual characteristics. In order to assist countries that wish to collect statistics beyond the core list, the chapter considers other ICT household measurement topics, such as e-commerce, trust in the online environment and cybersecurity, child online protection, impacts of ICT access and use, ICT skills, and gender-sensitive indicators. It also discusses time-related issues such as survey frequency, reference periods and the value of time series.

64. Chapter 5, Data sources and collection techniques for ICT household statistics, provides information on the types of surveys and data collection techniques that statistical agencies may use to measure access to, and use of, ICTs by households and individuals. The use of Big Data as a source for ICT statistics is explored, without providing specific recommendations due to the novelty of the approach.

65. Chapter 6, Question and questionnaire design for ICT household surveys, explores the principles of good question and questionnaire design and issues relevant to ICT household measurement. It presents model questions for the core indicators and a logical structure for a stand-alone questionnaire or a module of questions for measuring the core indicators to be inserted in a broader household survey.

66. Chapter 7, Sampling for ICT household surveys, considers issues of survey scope and coverage, target populations and survey frames, statistical units to be used, and sample design and selection.

67. Chapter 8, Data processing for ICT household statistics, deals with the conduct of surveys and processing of survey data, including data entry, editing, imputation and data weighting. It also discusses how the core ICT household indicators are calculated.

68. Chapter 9, Data quality and evaluation for ICT household statistics, discusses issues relating to data quality, including sampling and non-sampling error, data quality assurance frameworks, reporting and evaluation.

69. Chapter 10, Dissemination of ICT household data and metadata, looks at dissemination of ICT household data and metadata and presents basic tabulation plans. It also describes the data collection and dissemination efforts of ITU.

70. There are five annexes as follows:

• Annex 1 provides the entire Partnership’s core list of ICT indicators (as of June 2019).

• Annex 2 presents the ITU model questionnaire (for a stand-alone survey or a module in an existing household survey) for collecting statistics for the core indicators on household access to, and individual use of, ICT.

• Annex 3 provides examples of imputation and weighting, and complements Chapter 8.

• Annex 4 provides an extract from the questionnaire ITU sends to countries to report on the core ICT household indicators.[33] Note that this is a questionnaire to collect aggregated data from countries, and should not be confused with the model questionnaire proposed in Annex 2 intended to collect data from households and individuals.

• Annex 5 is a glossary of terms and abbreviations.

71. The Manual concludes with a bibliography, including references to online documents and websites.

[1] The Manual generally uses the term “ICT household statistics” to refer to statistics on both household access to, and individual use of, ICT.

[2] The ITU Training Course on Measuring ICT Access and Use by Households and Individuals is available at: http://www.itu.int/en/ITU-D/Statistics/Pages/capacitydev/default.aspx.

[3] For more details of WTDC’s resolution 8, see: http://www.itu.int/ITU-D/conferences/wtdc/2010/pdf/WTDC10_DraftPreliminaryReport.pdf

[4] Later renamed to the Working Party on Measurement and Analysis of the Digital Economy (WPMADE).

[5] See for instance UNCTAD’s Digital Economy Reports, https://unctad.org/en/Pages/DTL/STI_and_ICTs/ICT4D-Report.aspx.

[6] The statistical measurement of ICT-enabled services from the supply side (i.e. enterprises providing services to other enterprises, households, individuals and institutions through networks) is recent. Work to reconcile the supply and demand sides, in the case of services delivered to households, may require further methodological development and country experiences in statistical measurement.

[7] At the time of edition of this Manual, a revision of the UNCTAD Manual is also being undertaken.

[8] The term NSO as used in this Manual is taken to include all government agencies that collect official statistics. Where a national statistical system is decentralized, there may be several official statistical agencies in a country. NSOs are usually government funded and responsible for providing high quality, standardized statistical data to government, industry and the public. They may also be responsible for coordinating the national statistical system.

[9] For more information on the Geneva and Tunis phases of the World Summit on the Information Society, see ITU (2005).

[10] For more information on all ECOSOC’s resolutions related to the “Assessment of the progress made in the implementation of and follow-up to the outcomes of the World Summit on the Information Society”, see: http://www.un.org/en/ecosoc/docs/docs.shtml.

[11] See http://www.itu.int/ITU-D/ict/ExpertGroup/default.asp.

[12] The first ITU questionnaire on access to, and use of, ICT by households and individuals was sent to all national statistical offices in October 2005.

[13] Including a member of the Partnership Steering Committee.

[14] Available to registered members. Registration is possible via http://www.itu.int/net4/ITU-D/forum/expertgrouponhouseholds/forum/yaf_login.aspx?returnurl=%2fnet4%2fITU-D%2fforum%2fexpertgrouponhouseholds%2fforum%2f.

[15] Available via www.oecd.org/going-digital-toolkit

[16] See: https://circabc.europa.eu/faces/jsp/extension/wai/navigation/container.jsp for a repository of all questionnaires and manuals.

[17] https://www.itu.int/en/ITU-D/Statistics/Documents/publications/ldb/LDB_ICT_2018.pdf

[18] https://rtc-cea.cepal.org/en/working-group/measurement-information-and-communications-technologies/2018-2019.

[19] https://www.unescwa.org/unbis/ict-indicators.

[20] https://www.itu.int/en/ITU-D/Statistics/Documents/intlcoop/partnership/Thematic_ICT_indicators_for_the_SDGs.pdf

[21] https://sustainabledevelopment.un.org/content/documents/14826ict.pdf

[22] http://unstats.un.org/unsd/accsub-public/workpartner_ccsa.htm

[23] See: http://www.ilo.org/public/english/bureau/stat/isco/index.htm (ILO, 2013) and http://www.ilo.org/global/statistics-and-databases/statistics-overview-and-topics/status-in-employment/current-guidelines/lang--en/index.htm (ILO, 1993).

[24] The International Household Survey Network (IHSN) is a partnership of international organizations. Its mission is “to improve the availability, accessibility, and quality of survey data within developing economies, and to encourage the analysis and use of this data by national and international development decision makers, the research community, and other stakeholders”. (IHSN, 2013). For further information, see: https://ihsn.org/.

[25] https://unctad.org/en/PublicationsLibrary/tn_unctad_ict4d04_en.pdf

[26] http://uis.unesco.org/sites/default/files/documents/international-standard-classification-of-education-isced-2011-en.pdf (UNESCO, 2011).

[27] See http://unstats.un.org/unsd/demographic/sources/surveys/default.htm.

[28] Including household access to radio, TV, fixed-line telephone, mobile cellular telephone, personal computer, Internet access (landline and mobile connections, access from elsewhere other than home) and reasons for not having access. These correspond to the household access core ICT indicators (HH1-HH4 and HH6This refers to any member of the household and is not a core ICT indicator. The Partnership recommends that such information is collected from individuals (note that the results are not equivalent). The UNSD publication can be found here: https://unstats.un.org/unsd/demographic-social/Standards-and-Methods/files/Principles_and_Recommendations/Population-and-Housing-Censuses/Series_M67rev3-E.pdf

[29] https://unstats.un.org/sdgs/metadata/

[30] http://www.washingtongroup-disability.com/publications/implementing/

[31] See: http://www.worldbank.org/LSMS/ (World Bank, 2013).

[32] In addition, the International Household Survey Network (http://www.ihsn.org/home/index.php?q=tools/overview) provides methodological materials for different social surveys across the world.

[33] Updated as of 2018.

Chapter 2. Coordination among national stakeholders in ICT measurement

72. This chapter considers the roles of stakeholders in the national statistics system and issues relating to coordination and planning of ICT data collection and dissemination at the national level. It then describes models and mechanisms for coordination that have been used in different national statistical systems.

73. Given the cross-cutting nature of ICTs, permeating all sectors of society, statistical data collection and dissemination may be fragmented. For example, ICT infrastructure data may be collected by the ministry responsible for telecommunication or the national regulatory agencies of the telecommunication sector, use of ICT in businesses may be collected by the National Statistical Office (NSO) in the framework of business surveys, use of ICT in schools may be collected by the ministry of education, and access and use of ICT by households may be collected by the NSO. Because of the variety of existing ICT data sources, coordination and cooperation among data producers are fundamental to the production of high-quality official ICT statistics. Otherwise, there is a risk that published data will be inaccurate or inconsistent, leading to incorrect policy decisions. Other benefits of coordination include reduction of the overall response burden for data providers, avoidance of duplication of effort and optimization of the efficient use of resources. It also helps to identify existing data gaps, to harmonize objectives and priorities between the different stakeholders, and to improve monitoring and evaluation of the ICT statistical production process. Coordination among statistical agencies within countries is one of the ten Fundamental Principles of Official Statistics,[1] adopted by the UN Statistical Commission in 1994. The objective of this chapter is to describe ways and means of establishing a national coordination mechanism to ensure the efficient and timely production of ICT indicators.

Stakeholders in the national statistics system and coordination of the production of statistics

74. This section describes the different stakeholders in the national ICT statistics system, more generally, and in particular on ICT household statistics. It considers the types of mechanisms that could be put in place for ensuring coordination, including planning statistical activities.

75. In a national context, there are three main groups of stakeholders involved in the ICT statistics system (see Figure 2). They are:

• data producers, especially NSOs, but in some countries also the national telecommunication regulatory authorities, sector ministries, and non-official sources such as private companies, universities and research centres;

• data users, including policy-makers from sector ministries and regulatory authorities dealing with ICT and/or telecommunications, and other users, such as international organizations, private businesses, academia, media and the general public; and

• data providers/respondents, mainly individuals from sampled households for the purposes of this Manual[2].

Figure 2. Stakeholders in the ICT statistics system

Source: Cetic.br

76. Coordination is needed among producers of ICT statistics, between producers and users, and between producers and data providers who are the source of primary information. The first group (data producers), and in particular NSOs, is the main audience for this Manual.

77. ICT household statistics are demanded by a variety of users. Since survey-based ICT statistics is a relatively new field in most developing economies, the initiative to produce ICT data often originates from a demand by users, in particular policy-makers such as ministries responsible for ICT and telecommunication regulation agencies. Government policy-makers responsible for telecommunications and ICT policy will usually be the most influential data users, although users from business, the non-profit sector and academia may also have important input and their experience should be taken into consideration. Mapping the users (i.e. identifying the different users and their needs) is a fundamental step in planning for data collection.

78. In line with the Fundamental Principles of Official Statistics, the 10th World Telecommunication/ICT Indicators Meeting (2012)[3] recognized that coordination among statistical agencies (NSOs) and other stakeholders within countries is crucial in improving the availability and quality of ICT statistics. Coordination should cover the phases of planning, producing and disseminating ICT statistics. In particular, coordination among NSOs, telecommunication regulatory authorities and ministries responsible for ICT policies is a prerequisite for the development of a quality ICT household statistics system. Other institutions dealing with social issues (such as education and health) should be involved where necessary for developing sector policies based on the adoption and use of ICT.

79. The multiplicity and diversity of producers of ICT statistics require strong coordination among them. In addition, users may face problems in assessing the relative merits of different sources, therefore, the establishment of relationships with users is another important component of the national coordination system. Finally, the relationship with information providers (individuals and households) is to be considered. The main topics relating to coordination among stakeholders are:

• the coordination between data producers covering legal, technical and resource allocation aspects;

• the responsiveness of producers to users’ needs for information, ensured by user consultation, established user-producer relationships and forums, user-orientated dissemination policies (mainly of NSOs), and efficient techniques; and

• the concern of data producers for a sound relationship with information providers (households and individuals), with respect to their privacy and minimization of response burden.

80. National statistical offices (NSOs) play a central role in the production of official statistics in all countries. They are generally entitled by law to collect primary information from individuals, households, business and other organizations for the production of statistics, subject to scientific principles and legal requirements of confidentiality. In the area of ICT statistics, they usually collect data through household and business sample surveys, occasionally through population and economic censuses, or from administrative data provided by other institutions. NSOs operate under statistical legislation which sets up governance of the national statistical system, providing for mechanisms for coordination, the protection of confidentiality, the obligation of response, and establishing a survey programme, which may run over a number of years (referred to in this Manual as a ‘multiyear programme’). In most countries, NSOs adhere to the UN Fundamental Principles of Official Statistics.[4] NSOs have different levels of resources, but in general, they have the necessary expertise for data collection and analysis, and often possess a network of field offices with the capacity for mobilizing enumerators and supervisors for the implementation of large, nationwide surveys.

81. National telecommunication regulatory agencies (NRAs) are responsible for authorizing operating licenses for the telecommunication sector. In many countries, they are mandated to analyse and monitor the sector, thus explaining their interest in the production and dissemination of ICT indicators. NRAs maintain a register of licenses that can be used to identify sources of telecommunication/ICT indicators (e.g. service providers). In general, data are collected for administrative purposes, but in some countries, NRAs have carried out ICT surveys (with varying levels of coordination with NSOs). These institutions have substantial expertise in technical matters related to the telecommunication sector.

82. Sector ministries (or sector government agencies and multi-stakeholder organizations in some countries) responsible for telecommunications, science and technology, or innovation collect administrative data in different areas, and may be a source for ICT indicators. As with NRAs, in some countries they have carried out ad hoc surveys of businesses and households on access to, and use of, ICT, with different levels of coordination with the NSO. These institutions may have substantial expertise in technical matters related to ICT, but in general they will be more focused on the analysis and use of information.

83. Other line ministries may also have an interest in collecting ICT-related indicators. They include ministries of education (statistics on ICT in education and ICT skills), ministries of labour (statistics on ICT sector production and employment) and ministries of health (statistics on ICT in health). So far, data collection in those areas is limited, particularly in developing economies, while at the same time demand for data is growing.

84. In most cases, NRAs and ministries of ICT can be considered as users but also as producers of ICT data. The coordination with the NSO is of the utmost importance. In some countries, specific agencies other than the ones mentioned above have been mandated or created to collect and disseminate national ICT statistics. This could be the case where NSOs have little subject matter experience, where resources from other sources are available, or where countries have a particularly strong interest in monitoring their ICT development. Examples include Brazil (see Box 3). These agencies have various levels of cooperation and coordination with NSOs.

Box 3. The Regional Centre for Studies on the Development of the Information Society of Brazil

The Regional Centre for Studies on the Development of the Information Society – Cetic.br – has consolidated its position as an international reference centre for the production of indicators and statistics on the use of information and communication technologies in Brazil. In 2012, the Brazilian government signed a pioneering agreement with UNESCO to set up Cetic.br, hosted by NIC.br, the entity managing the domain .br. It is UNESCO’s first center of studies on the information society. Cetic.br publishes periodic specialized surveys and reports – often in collaboration with the IBGE (NSO of Brazil) on the use of ICT by several segments of society. The results of these surveys are critical to monitor and assess the social and economic impact of ICTs and also to allow comparison between the realities in Brazil with other countries.

Cetic.br regularly publishes books containing articles and data analyses for surveys such as: ICT Households, ICT in Education, ICT in Culture, ICT Kids Online Brazil, ICT in Health, ICT Enterprises, ICT Electronic Government. Cetic.br also publishes Sectoral Studies and Internet Sector Overview Reports focusing on emergent topics such as Big Data for Development, Artificial Intelligence and Ethics etc. Cetic.br’s surveys are based on international reference models, such as the methodological references and data collection instruments defined by the UN’s Partnership on Measuring ICT for Development, Eurostat documents, OECD and UNCTAD.

Cetic.br also continued to seek innovative solutions for data production – exploring new methodological tools for data collection and sharing and using new sources such as Big Data to complement traditional data sources in the production of public statistics. New functions were thus created within Cetic.br’s organizational structure to better accommodate data production innovation. Cetic.br intensified its actions to promote online access to microdata on the ICT Households and Kids Online surveys, providing access to the data based on specific interests, and thus reinforcing the Center’s commitment to disseminating information and knowledge and expanding the reach of its survey methodologies and results.

Source: https://cetic.br

85. Other potential producers of ICT statistics and indicators include private companies (in the ICT sector or providing market analyses) and research centres and universities (see Box 4 for the case of the Russian Federation). While private organizations are often very successful in publishing results and receiving the attention of the media, they do not always adhere to the principles of official statistics, to international statistical standards or may not have the capacity to carry out nationally representative large surveys. Universities and research centres may suffer the same limitation, but are well placed to provide in-depth analysis of the data.

Box 4. Centre for Statistics and Monitoring of the Information Society- Russian Federation

The Centre for Statistics and Monitoring of the Information Society of the Russian Federation, an academic institution, elaborates indicators and modern tools for statistical and sociological surveys in line with international standards and performs monitoring and analysis of the information society in Russia on the basis of statistical, sociological and expert data.

Key projects implemented by the Centre include work for federal governmental bodies (Ministry of Communications and Mass Media, Federal Service for State Statistics and others), companies (RIO Center, Rostelecom and others), as well as international organizations (European Commission, UNIDO and others). International projects involving the Centre include Statistical Monitoring of Processes of the New Economy Formation/European programme (TACIS), National Institute of Statistics, Italy (ISTAT); Development of Methodologies of ICT Statistics for Russia: Implementation of International Standards, United Nations Industrial Development Organization (UNIDO), European Commission.

In all its research and development, the Centre carefully implements international approaches and standards into the Russian statistical practice and adapt them to the tasks of the national statistics of the information society.

The results of its scientific research as well as statistical and analytical papers are published in Russian and English, containing unique data on the production and use of information and communication technologies and are an important source of information for federal and regional executive bodies as well as academic and educational communities. They include the annual statistical digest Information Society Indicators which is issued by the Higher School of Economics of the University of Moscow in conjunction with the Ministry of Communications and Mass Media and the Federal Service for State Statistics.

With the support of the European Commission and the United Nations Industrial Development Organization (UNIDO), a monograph “Information Society Statistics in Russia: Harmonization with International Standards” was published, dedicated to the problems of development of internationally harmonized statistics of information society in leading countries and Russia. It includes the key methodical concepts and data of Russian and international statistics, as well as unique developments of the HSE ISSEK.

Source: https://issek.hse.ru/en/dep_infoob

Models and mechanisms for coordination

86. This section presents models and mechanisms for coordination among producers, between users and producers, and between producers and informants. The main mechanisms introduced are: inter-institutional commissions or working groups, multiyear programming mechanisms and mechanisms for consultation with users. With respect to the relationship between ICT statistics producers and informants, the issues of data protection and confidentiality, and response burden reduction are addressed.

87. National statistical legislation may provide for coordination mechanisms among data producers (see Box 5). These could include: formal inter-institutional commissions, technical working groups, bilateral arrangements for the distribution of tasks, multiyear planning of statistical operations and arrangements for funding the production of ICT statistics. In some countries, more informal arrangements can work, especially in the initial phases of collection of ICT statistics. Examples of informal arrangements can include conversations between technical staff, or joint work on specific phases of the design.

Box 5. UNECE/EUROSTAT Generic Law on Official Statistics

UNECE and Eurostat have developed a Generic Law on Official Statistics to serve as a model for modernizing the legislation of the national statistical system in countries of the Eastern Europe, Caucasus and Central Asia region (but can be applicable to other countries).

The Generic Law reflects the Fundamental Principles of Official Statistics. It defines the National Statistical System, gives a coordinating role to the NSO, promotes strategic and operational programming of the statistical activity, the management of quality and the protection of confidentiality. It defines as well the functioning of other statistical producers than the NSO and how they should coordinate their activity.

Source: http://www.unece.org/index.php?id=45114

88. In many countries, a national statistical council (or equivalent) has been established by statistical legislation, with representatives of all ministries and statistical agencies, where the methodology and quality of statistical operations are discussed together with general aspects such as statistical ordinances, legislation and arrangements for data exchange between administrations, protection of confidentiality and response burden.[5] With respect to ICT statistics, the national statistical council:

• could provide a forum to discuss the major needs in ICT statistics and the distribution of tasks among the different statistics producers. For this, a subject matter working group may be established, where possible under the umbrella of the national statistical council, and detailed methodological aspects of ICT statistics discussed. The working group could include representatives of the NSO, the ministry responsible for ICT, the telecommunication regulation agency – as both producers and users of ICT statistics – and possibly researchers and experts in ICT issues.

• should ensure that an appropriate legal framework enables the collection of ICT statistics, ensures their status as official data and ensures their funding.

• should review the multiyear programming of surveys, to accommodate ICT questions in planned surveys when possible, or provide for the implementation of specific ICT surveys (see below).

89. When a national statistical council (or equivalent) does not exist or is not operational (for administrative or other reasons), an inter-institutional task force or working group may play the role, at a more technical level, of a forum to discuss implementation details for ICT statistics. This working group may be attached to the NSO, or if this institution does not have the main responsibility in producing ICT statistics, to a high-level administrative unit in the government (such as the ministry responsible for ICT or the presidential office).

Box 6. National Coordination for ICT Statistics: Philippines

The task of generating ICT statistics in the Philippines resides within a government-led, interagency mechanism initiated through the enactment of Republic Act No. 10625 (RA 10625) or the “Philippine Statistical Act of 2013”.

The law provides for the establishment of interagency statistical committees (IACs) by the Philippine Statistics Authority (PSA) for the purpose of coordinating and resolving agency and sectoral concerns on statistical matters. The IACs serve as forums for discussion of the issues raised by concerned producers, users and other stakeholders of statistics[1].

Under this mechanism, the Interagency Committee on ICT Statistics (IAC-ICTS) was established. The IAC-ICTS is Chaired by the Department of Information and Communications Technology (DICT) and Co-Chaired by the PSA. Through this cooperative framework, the domain of ICT statistics is able to lay the foundations for ICT statistics development, to address current ICT data gaps and challenges.

A key result of this cooperative framework is the National ICT Household Survey (NICTHS) which was conducted in 2019. The NICTHS is the first-ever national survey on ICT being conducted at the household level, addressing data gaps in ICT access and use and incorporating indicators from Philippine ICT plans and policies as well as ITU core ICT household indicators. The survey is a crucial source of key indicators of households’ ICT use in barangays for evidence-based planning and policy formulation using the PSA Master Sample.

With the successful implementation of the NICTHS in 2019, the next step for the IAC-ICTS is the institutionalization of the NICTHS for implementation every two years, thus ensuring the sustainability and supply of key metrics to measure ICT development and diffusion in the Philippines.

Source: Department of Information and Communications Technology, Philippines

https://psa.gov.ph/content/interagency-committees-statistics

[1] https://psa.gov.ph/content/interagency-committees-statistics

90. In the context of measuring the SDGs, many countries have established inter-institutional working groups to “localize” the indicators to their national statistical system. These working groups may provide an occasion to discuss ICT indicators, as some of them are used to monitor the SDGs.

Issues in coordination among producers

91. The role of different government agencies in data collection is generally established by national statistical legislation, which also sets up the processes of planning the statistical activity. In most countries, there is a multiyear statistical programme that specifies which operations are to be carried out in the medium-term (generally, 4 or 5 years) and which government institutions (NSO or other) are responsible for their implementation.

92. At the technical level, the following issues may be discussed among statistical producers:

• Distribution of roles in ICT data collection;

• establishment of common definitions of ICT indicators and relevant classifications, based on international standards but adapted whenever necessary to national conditions;

• establishment of population frames for households and business surveys;

• agreements on procedures for the preparation and dissemination of data and metadata, including appropriate joint publications; and

• time scheduling of different data collections in order to optimize the use of resources and reduce response burden.

93. The distribution of activities in ICT data collection is generally related to access to respondents and the responsibilities of the involved institutions. Indicators on ICT infrastructure, tariffs and subscriptions are often provided by NRAs, which have access to administrative data from operators. Statistics compiled from survey data from businesses and households are usually produced by NSOs, but in some countries they have also been produced by other institutions such as NRAs, ministries for ICT, or other entities charged with this task. Data on access to, and use of, ICT in schools or health facilities may be collected by ministries of education or health respectively.

94. In some countries, ICT data collection has been initiated by government institutions other than NSOs, for instance, ministries or other governmental agencies that promote ICT uptake, and regulatory agencies. While these institutions may have technical expertise on the subject matter, their ability to conduct a household survey based on sound methodology is likely to be limited unless they closely collaborate with NSOs, which are likely to have access to statistical infrastructure, such as a master sample frame (e.g. a list of randomly selected households that is representative of the whole population), access to a country-wide network of experienced interviewers, skilled staff with expertise in data treatment and analysis, and the legal mandate for these types of activities. In addition, there is uncertainty regarding the sustainability of data collection if conducted by the regulator or ministry, as it is not usually included in their regular programme of work and often done just to gauge the size of the market at one point in time. The risk of obtaining low quality and non-comparable statistics (with international data or with other national household surveys) should not be underestimated. In addition, coordination in such cases can cover the dissemination phase (e.g. joint publications) and the use of resources (joint funding, data collection network, IT equipment, etc.). Box 7 describes the cooperative arrangements for collecting data on access to, and use of, ICT by businesses, households and individuals in the Sultanate of Oman.

Box 7. Cooperation between the Information Technology Authority and the National Centre for Statistics and Information in the Sultanate of Oman

The Information Technology Authority (ITA) of Oman has established a collaboration protocol with the National Centre for Statistics and Information (NCSI), the NSO of the Sultanate, to implement surveys on access to, and use of, ICT by businesses (2011) and households and individuals (2013). In both cases, the samples are extracted from the business register or the enumeration area listing provided by the NCSI, while the questionnaire is designed by ITA. Data collection is carried out either by staff of the NCSI or by an external contractor, under the supervision of ITA. Data processing is partially carried out with resources from the NCSI (data cleaning, calculation of sample weights) and from ITA (data analysis and tabulation). Dissemination is carried out by ITA.

International comparability is ensured by the use of international standards in the preparation of the questionnaire and the compilation of indicators, which include the core ICT indicators developed by the Partnership on Measuring ICT for Development.

Source: ITA Oman, https://www.ita.gov.om/ITAPortal/MediaCenter/Document_detail.aspx?NID=66

Box 8. Partnership between the National Communications Authority and the Ghana Statistical Service

The National Communications Authority (NCA) in an effort to meet the increasing data needs on ICT indicators from all stakeholders, both domestic and global, in partnership with the Ghana Statistical Service (GSS) is conducting in 2019 a nationwide survey on ICT access, usage, skills and digital divide at the household level. The objective of the survey is to provide a database that would contribute to planning, implementation and evaluation of policies and programs, to promote the rapid development of the ICT industry in Ghana. The ICT survey would be a nationwide household-based survey, using the Ghana Census of Agriculture (GCA) sampling frame (which would cover about 300 enumeration areas ). It is estimated that 6000 households and 18,000 individuals would be covered in the survey, across all the ten regions of the country. The use of tablets, through Computer-Assisted Personal Interviews (CAPI) techniques, ensures high quality data and timely completion of the survey. Stakeholders involved include the Ministry of Communications, National Information Technology Agency (NITA), and the Ghana Investment Fund for Electronic Communications (GIFEC). The survey is expected to be completed by December 2019.

Source: National Communications Authority, Ghana

95. It is strongly recommended that ICT statistics are collected by NSOs (defined to include all government statistical agencies, as described in Chapter 1) or in close collaboration with them. There are numerous benefits to this, including the capacity of NSOs to carry out nationwide, representative household surveys and their links with national and international statistical systems, some of which may include ICT questions. Most NSOs are also supported by legislation designed to protect data and, in many cases, to mandate provision of data, thereby enhancing response rates. They have the necessary technical experience in data collection and generally provide credibility of the official statistics they produce. In some countries, NSOs will also have a coordination role that confers advantages in terms of technical and legal coordination, as well as resource allocation. In addition, like other sectoral statistics, ICT data collection could be integrated in regular household surveys, thereby ensuring sustainability of data collection.

96. The expertise available in ministries of ICT and other agencies should be taken into account when designing data collection instruments. In particular, decisions on inclusion of ICT topics, the adaptation of international recommendations to national standards (such as the types of Internet connection available in the country), and the analysis of results should be carried out in close collaboration with specialized agencies.

97. The process of “nationalizing” SDG indicators, undertaken in many countries under the leadership of NSOs, can be taken as a model for coordination among statistical producers. Indeed, on the basis of the internationally agreed list of SDG indicators, countries have adapted the definition and sources of SDG indicators to their national reality, by establishing thematic groups covering each SDG. Since SDG Goals 4 (quality education), Goal 5 (gender equality), Goal 9 (industry, innovation and infrastructure) and Goal 17 (partnership) mention ICT issues, the thematic groups established for those goals may discuss the production of ICT indicators.

Multiyear planning as a mechanism for coordination

98. Most national statistical systems are governed by a multiyear programme for the production and dissemination of official statistics. Such a programme should include and describe the implementation arrangements for ICT statistics. The range of statistical operations should cover the different domains, such as ICT infrastructure statistics, surveys of businesses and households (alternatively the inclusion of ICT access and use modules in other surveys). See Box 9 for an example of multiyear programming of ICT statistics in the Philippines.

99. Many developing economies have developed National Strategies for the Development of Statistics (NSDS),[6] which are short-term or medium-term programming instruments in the national statistical system. Countries preparing or updating their NSDS should consider the explicit inclusion of ICT surveys in their multiannual planning in order to ensure coordination with other household surveys and sustainability over time.

Box 9. Information society statistics in the Philippines

The improvement of the state of ICT in the country by expanding the deployment of ICT infrastructure addressing the gaps in digital connectivity, continuing the enhancement of the country’s e-government and establishing reforms in the policy and regulatory frameworks are parts of the Philippine Development Plan (PDP) 2017–2022. The PDP 2017-2022 aims to lay down a strong foundation for inclusive growth, a high-trust society, and a globally-competitive knowledge economy. It is intended to be the first of four medium-term plans geared towards the attainment of the long-term vision called “Ambisyon Natin 2040”.

The Philippine Statistical System (PSS) is a decentralized system that ensembles all statistical organizations at all administrative levels, its personnel and the national statistical development program. One of its main components for an effective and efficient national statistical system is the management and coordination mechanism within the government.

The Philippine Statistics Authority (PSA), as lead agency, has to coordinate and monitor the implementation, periodic assessment and updating of the PSDP.

In the formulation of the PSDP, the PSS takes into consideration the long-term plan “AmBisyon Natin 2040” and the Philippine Development Plan (PDP) 2017-2022 to respond to the new and emerging statistical requirements of the government and private sector.

Among other domains, the PSDP presents plans and methods of action to accurately monitor, track, and measure the impact of ICT, through timely and relevant statistics. An Interagency Committee on Information and Communications Technology Statistics (IAC-ICTS) was establish to coordinate the production of statistics in the ICT domain. ,

A number of key developments in ICT statistics in the PSDP 2018–2023 included institutional, methodological and implementation activities, among them:

• Inclusion of e-commerce indicators on establishment and household surveys;

• Inclusion of the ICT indicators and statistical activities in the system of official statistics;

• Creation of technical working groups under the IAC-ICTS to discuss plans to improve the generation of ICT statistics (e.g. household ICT use and access, business and e-commerce, education, ICT use and access of national government agencies, etc.);

• Formulation of an Information Society Statistics Framework and adoption of official concepts and definitions on ICT Statistics for statistical use;

• Development of guidelines for ICT data collection, production, and dissemination;

• Development of a methodology on the estimation of satellite accounts for the information economy;

• Conduct of surveys that will generate ICT related statistics to address the requirements of development plans such as the PDP 2017-2022, the SDGs, ITU Core ICT Indicators, eGov Master Plan;

• Updating of the 2009 PSIC (Philippine Standard Industrial Classification) to include new/emerging ICT industries.

Sources: http://www.neda.gov.ph/wp-content/uploads/2017/12/Abridged-PDP-2017-2022_Final.pdf

https://psa.gov.ph/system/files/kmcd/PSDP%202018-2023%20-%20Final.pdf?width=950&height=700&iframe=true

100. A multiyear plan should specify which institutions are responsible for each statistical operation. The list of core ICT indicators (see Chapter 4) can be used as a basis for dialogue between statistics producers, to agree on the distribution of roles among stakeholders for the collection of ICT statistics. Different domains of interest (ICT infrastructure, ICT access and use by households and enterprises, ICT in education, ICT in government, ICT waste, etc.) may be covered by different institutions. The distribution of tasks with respect to ICT statistics can be based on the domain of interest or on the type of operation (statistics from administrative registers, surveys, censuses). It is important that definitions and concepts are as coherent as possible across statistical operations, even if the institutions responsible are different.

101. The planning process for the collection of statistics on the access and use of ICT by households and individuals should take into account a time frame based on the programming of household surveys, population and housing censuses and business surveys:

• Population and Housing Censuses (PHC) are generally carried out only once in every decade (years ending on 0 or 1), due to their high cost. They are necessary to update population distribution figures, as well as the statistical infrastructure for household surveys in terms of geographical distribution of the population and exhaustive listing of dwellings. The use of PHC for collecting ICT statistics has limitations due to low frequency and a small number of possible questions, though it could be noted that it is the best source for data disaggregation by geographic location (such as results at the province or state level) or by specific population (such as people with disabilities).

• Large household surveys that take place with low frequency (such as income and expenditure surveys) require intensive labour input from NSOs and may absorb most human resources, especially in low-resource NSOs. They may include modules on the access and use of ICT (see Chapter 5 for a comparison between stand-alone surveys and modules in existing surveys).

• Smaller household surveys with higher frequency (such as living conditions surveys or labour force surveys) may provide a good vehicle to investigate rapidly changing phenomena such as ICT adoption.

102. For other ICT statistics, a multiyear programme should consider the frequency and coverage of existing statistical operations (such as business surveys), the expertise and competencies of different institutions and the resources available. In any case, NSOs should avoid implementing ICT-related statistical operations without properly programming their implementation in the frame of a programme that takes into consideration the needs of ICT statistics users, in particular, policy-makers. In addition to multi-stakeholder mechanisms, bilateral agreements should be arranged between producers (e.g. between the NSO and the NRA/Ministry of ICT) for detailed implementation aspects of the survey, including the funding of operations. Different survey activities could be undertaken by different actors. For instance, data collection may be undertaken and supervised by the NSO using external funding, while questionnaire design, testing and sample design could be directly provided by the NSO.

Mechanisms for user consultation

103. It is recommended that representatives of data users work closely with data collection agencies (NSOs and others) to ensure the relevance of ICT statistics programmes, that is, that they satisfy users’ needs and are useful for decision-making, Relationships between users and producers should be explicitly considered in national statistical coordination mechanisms.

104. Users from the government sector (e.g. ministries for telecommunication/information society) should take into account their information and data needs when drafting national ICT strategies. This would allow statistics producers to better identify the needs of institutional users.

105. In order to identify users, especially from the private sector, it may be useful for data collection agencies to make a list of existing business associations with an interest in ICT, such as:

• ICT sector associations (for instance, ICT manufacturing industry associations, telecommunication associations);

• ICT market studies providers;

• Universities and ICT-related academic institutions, and

• Organizations with a more general profile (such as chambers of industry and commerce).

106. The dialogue with users can be implemented in stages, first establishing the purpose and the process, then clarifying the information demand. Issues that may be relevant in a detailed dialogue with users include the definition of target populations, questionnaires, level of data disaggregation, level of accuracy needed, frequency of data collection and timeliness. Mechanisms for the coordination between users and producers can have different degrees of formalization. In countries where ICT data collection is new, the mechanisms may be rather informal in the first phase. As ICT data production becomes more regular, more formal mechanisms could be put in place.

107. Informal mechanisms for consultation with users that can be considered are:

• organizing open events to present survey plans and results (when available) to motivate users to provide feedback for the next surveys;

• carrying out user needs and satisfaction surveys, asking about specific dimensions of data quality, such as, topics and concepts, detail of breakdown, timeliness and access; and

• participating in ICT sector and academic events to present, promote and get feedback on statistical plans with respect to the issues discussed.

108. Formal mechanisms for consultation with users can be set up at the high, decision-making level, or at the technical level. At the decision-making level, in most countries, a national statistical council comprising representatives of statistical producer agencies and a variety of users (such as academia, chambers of commerce, business associations, trade unions, NGOs and the media) is established by statistical legislation and has a consultative role within the national statistical system. ICT household statistics may be discussed in the framework of the national statistical council (probably together with other ICT statistics).

109. At the technical level, and particularly with users from government institutions (sector ministries, NRA), a useful mechanism for coordination is the establishment of a working group to discuss technical issues that may affect the future use of statistics. These could include survey scope and coverage, level of disaggregation and accuracy, and dissemination formats and channels. Participants to this type of working group may also include representatives from research centres, universities and the ICT sector (ICT market analysts in particular).

110. A useful tool for informing users about the quality of ICT statistics is the dissemination of quality reports; this is discussed in Chapters 9 and 10.

111. In some countries, a national observatory for the information society has been established (see Spanish experience described in Box 10). This type of entity collects and collates data from different sources, prepares specific publications (such as sector reports) and disseminates ICT indicators via a centralized website. The arrangements for such an observatory may include the participation of users and producers in its governing or advisory bodies. Other examples include the Regional Centre for Studies on the Development of the Information Society (Cetic.br),which holds regular consultation meetings with data users before planning and designing future ICT surveys (see Box 3).

Box 10. National Observatory of Telecommunications and the Information Society in Spain

The ONTSI (National Observatory of telecommunications and the information society) is the Spanish observatory of the economy and digital society. It is under the responsibility of the Ministry of Economic Affairs and Digital Transformation, its purpose being to generate knowledge of value for public policies – as well as for business and citizen intervention – around technological development and its different impacts on the economy, employment, public services, rights, security, quality of life and equality between people. Also, its objective is to become the reference centre for the analysis and monitoring of the information society in Spain in collaboration and coordination with the public and private sectors and to monitor public policies in the ICT sector.

For this, it carries out studies and indicators, analyzes policies and strategies, evaluates programs, analyzes trends, identifies good practices and processes, disseminates and exchanges knowledge in relation to these fields.

The main statistical sources for information include the surveys on ICT access and use in households and in businesses carried out by the National Statistical Institute (INE), economic data from telecommunication operators collected by the said Ministry, price information, telephone network and broadband coverage data, the infrastructure and equipment indicators from the National Commission of Markets and Competition (CNMC), other data on ICT sector from Ministry of Employment and Social Security Social Security and as well as ad hoc studies by ONTSI and other data from Eurostat and European Commission.

Source: https://www.ontsi.red.es/es/indicadores.

Box 11. Qatar: ICT Observatory & ICT Directory

Qatar ICT Observatory

Qatar’s ICT Observatory is an online central repository of data and statistics relating to the country’s dynamic ICT landscape. It is accessible to businesses, entrepreneurs, researchers, policy makers, government entities and the general public.

The ICT Observatory is a trusted source of information from various sources – including specialized surveys on ICT usage; data from licensed telecoms operators; international reports and aggregated data from other government entities in Qatar.

Users can view indicators, identify trends, review charts and tables and browse and download analytical reports from the ICT Observatory for further analysis and use.

The ICT Observatory is part of the government-wide effort to provide the public with more open access to data in order to drive innovation and further increase government transparency and accountability. It will streamline ICT indicators across Qatar and support national ICT policymaking and progress evaluation, reduce the costs of research and data provision, and promote informed discussion on ICT-related topics.

The Observatory is available at: http://ictobservatory.qa/en/index.html

Source: http://www.motc.gov.qa/en/news-events/news/all-qatar%E2%80%99s-ict-indicators-single-platform.

Qatar ICT Directory

The Ministry of Transport and Communications launched the “Qatar ICT Directory” in collaboration with the Ministry of Economy and Commerce. The Directory is part of MOTC’s efforts to enhance the private sector’s footprint in the market by ensuring their competitiveness regionally and globally, as well as helping public and private sectors and business community find best ICT service providers and products.

The Qatar ICT Directory has developed a database of ICT companies in the State of Qatar that will help government and businesses select suitable ICT companies. It also helps better understand Qatar’s ICT landscape. The Directory has a fast search feature where users can search for companies, services and products. Users can also smoothly review a list of all ICT companies and includes an advanced search to find specific results.

MOTC spares no effort to support the ICT market in Qatar, providing the key requirements of operating small and medium businesses to enhance and diversify their activities towards a robust and diversified economy.

Source: http://www.motc.gov.qa/en/sectors/digital-society/digital-industry-development/qatar-ict-directory

Relationships with data providers

112. Data providers (respondents) are very important players in the statistical system. In the ICT household statistics system, respondents are individuals in households. Without their cooperation, data would be inadequate in terms of either/both quality and quantity. It is important that NSOs recognize the contribution of respondents and put the necessary effort into gaining their trust and cooperation, by developing close collaborative relationships with them. At the most obvious level, this entails making the respondent’s job easier for them by providing coherent and understandable survey material, including questionnaires and instructions. This is further discussed in Chapter 6.

113. The three main issues with respect to primary data providers, which data producers should take into account, are: cooperation in providing response, reduction of response burden and protection of confidentiality. As the response burden decreases, the probability of non-response will also decrease, thus improving the quality of the aggregated data by reducing possible biases due to non-response.

114. Examples of actions that can improve the cooperation of respondents in providing responses are the following:

• providing public (and, whenever possible, personalized) information to respondents before the interview on the objectives and importance of the survey, and the future use of the results (for instance, where the first stage of sampling is by geographic area, there could be localized advertising in areas that have been selected in a household survey);

• carefully designing the interview process in terms of questionnaire design, filter questions, language used;

• training interviewers in the subject matter (ICT issues) so that concepts are clearly transmitted while minimizing the risk of interviewer bias; and

• carefully planning the timing of interviews, asking for the most suitable schedule for visiting the household.

115. As a rule, it is important to minimize the burden on respondents of surveys. The potential benefits to NSOs of doing this include higher response rates and better data quality. This issue is also addressed in Chapter 6. Examples of actions that can reduce the burden of response are the following:

• carefully selecting the sample of households so that samples for different surveys do not overlap;

• limiting questionnaires to content that reflects the requirements of data users;

• controlling the duration of interviews;

• alternating topics covered in the questionnaire among different years; and

• using auxiliary information whenever possible (such as administrative registers).

116. While many NSOs work in a legal framework that makes provision of statistical data mandatory,[7] and provides for sanctions to non-respondents, cooperation may be better if such legislation is used sparingly.

117. The legal framework of NSOs will also generally ensure the confidentiality of data provided by individuals. It is very important that protection of individual statistical data is assured and is communicated to respondents.

118. Measures to protect confidential data include the confidentialization of survey registers and their secure storage, as well as legal measures to ensure adherence to confidentiality requirements by the staff of data producer institutions (e.g. formal commitment, sanctions in case of breach).

119. There is a current debate on the access by NSOs to ICT-related data collected by private enterprises such as mobile operators. While the potential of using mobile phone data (Call Data Records, CDR), data from online purchases, etc. is recognized in the context of the modernization of official statistics, the regulations with regard to private data confidentiality prevent access to detailed data from private operators. Pilot exercises of producing statistical data from micro-aggregated telecom data have taken place in many regions, but there is no experience in the routine use of such data for the production of official statistics[8]. The statistical legislation, which in most countries enforces the protection of personal and firm data collected for statistical purposes, could be considered in the framework of agreements between telecom operators and NSOs.

[1] See: https://unstats.un.org/unsd/dnss/hb/E-fundamental%20principles_A4-WEB.pdf

[2] Individuals also provide data to the telecommunication operators such as telephone companies, both as requisites for contracting the services, and “organically”, while using the services. For example, records of calls are stored by telephone companies and can eventually be used internally, with the safeguards of privacy, for the production of statistics for marketing or management purposes. The use of such data for the production of ICT statistics by NSOs is still in an exploratory phase, but may constitute a valuable source of data in the future.

[3] See: http://www.itu.int/en/ITU-D/Statistics/Pages/events/wtim2012/default.aspx.

[4] https://unstats.un.org/unsd/dnss/hb/E-fundamental%20principles_A4-WEB.pdf

[5] See Box 6 for the case of the Philippines.

[6] For a more complete description and materials for the preparation of a NSDS, please see the dedicated PARIS21 website at: https://nsdsguidelines.paris21.org/node/741

[7] The legal basis of a large number of NSOs can be found here: http://unstats.un.org/unsd/dnss/kf/LegislationCountryPractices.aspx.

[8] A pilot project “Innovative ways to utilize Big Data as a new data source for ICT indicators” was implemented by ITU, to explore the possibility of using mobile phone data to complement ICT household indicators by proposing an additional list. See: https://www.itu.int/en/ITU-D/Statistics/Documents/events/wtis2016/BigData_Tiru.pdf

Chapter 3. Planning and preparation for ICT household surveys

120. Survey planning and preparation is an obvious prerequisite to survey success and cost minimization. As discussed in the previous chapter, widespread and extensive consultation and coordination with policy-makers and other stakeholders (users and producers of statistics) will help to ensure that the final product is of optimum relevance to users’ needs. It will also help build support for the project that may assist with fund-raising or publicity.

121. This chapter looks at planning considerations, budget and cost issues, and other preparatory activities. More detailed information about planning household surveys can be found in the UNSD handbook Household Sample Surveys in Developing and Transition Countries (UNSD, 2005a). For a useful checklist of quality aspects of the survey planning process, readers can consult the European Self-Assessment Checklist for Survey Managers (DESAP).[1] This chapter focuses more on specific aspects of ICT household survey preparation.

122. A useful model for considering all the phases of a statistical operation is given by the UNECE Generic Statistical Business Process Model (GSBPM, see Figure 3).[2] The GSBPM is one of the cornerstones of the High-Level Group for the Modernization of Statistics (HLG-MOS)[3]. Its first full version was released in 2009 and has since been adopted by the statistical offices of the most advanced countries in the field. It has proven to be very useful for laying out and describing all the phases to produce statistical information. The GSBPM is intended to guide the planning of surveys and other statistical operations by systematically considering all processes and the workflow from initial preparatory steps to dissemination, documentation and archiving. The model includes preparatory activities starting from the identification of information needs, to final activities such as the dissemination of statistics and evaluation of specifics parts of the process whenever necessary. Most importantly, it facilitates the development of a fully detailed strategy to produce such information.

Figure 3. Scheme of the Generic Statistical Business Process Model (GSBPM)

123. The general business processes identified by the GSBPM are:

• Specify needs: used when new statistics are identified or when feedback from current statistics requires a review of them. Its activities are related to precise identification of statistical needs (such as areas of ICT use not covered so far by surveys), preparation of solutions for them and proposals of business cases to meet those needs;

• Design: the statistical processes are related to development and design as well as to research work to define outputs, methodologies and such. It includes all the design elements needed to define or redefine the metrics that the business case asks for. The metadata and procedures to be used in the following phases are specified at this point.

• Build: the outputs from the “Design” processes are assembled and configured in this case to create the complete operational environment to run the process. New services are also created in response to gaps in the existing catalogue of services sourced from within the organization and externally. These new services are constructed so that they can be reused when necessary or possible. For example, the dissemination of anonymized microdata from ICT surveys for further use by researchers.

• Collect: gathering all the necessary information and load it into the proper environment for further processing. This process may include validation of data set formats, but never the transformation of data, which is done in the process phase.

• Process: processing of input data and their preparation for analysis. The processing of the data makes it so it can not only be analysed, but also disseminated as statistical outputs. The activities can be carried out in parallel to those in the “Analyse” process and may commence before the “Collect” one.

• Analyse: statistical outputs are produced and examined in detail. Statistical content for publications, reports, etc. is prepared, and it ensures that outputs are adequate prior to dissemination. It includes sub-processes and activities that enable statistical analysts to understand the data and the statistics produced.

• Disseminate: manages the release of statistical products to users. Activities related with assembling and releasing the products via different channels so that users can access them. This can include presentation of the results of ICT surveys to forums of users.

• Evaluate: in this last process, the purpose is to evaluate specific instances in the statistical process. This can be done at the end or be ongoing during the statistical production process. Once the evaluation of the specific instance is complete, a range of qualitative and quantitative inputs is drawn and potential improvements are identified and prioritized.

124. Applying this methodology to the statistical business processes in the field of ICT statistics (and other domains) has several benefits that cannot be overlooked. The first one is that the standardization of terminology creates efficiency savings as well as it makes comparisons internationally much easier. Secondly, its implementation allows the adherence to the standard framework for benchmarking in statistics and hence, it facilitates the use of common tools and methods that again, result in more efficiency savings. Also, the GSBPM includes tools to manage the quality of the process much better, making for better and more reliable data. Finally, it provides a clear and easy way to understand information for data producers and users.

125. The broad assumption made in this Manual is that ICT household surveys will generally be conducted by NSOs (broadly defined to include all government agencies that collect official statistics). NSOs will generally be more experienced in conducting household surveys than other organizations and will have access to the statistical infrastructure required (including skilled staff, survey frame information, conceptual and methodological knowledge and computer systems). As some country examples show, ICT surveys have also been implemented by other institutions, but this requires that they have the subject-matter and statistical expertise and infrastructure.

ICT household survey planning

126. A worthwhile discipline in planning any statistical survey is to constantly review the purpose for which the eventual results will be used and what types of public policy or other decisions will rely on the results (GSPM phase “Evaluate”). As it is common for the release of statistics to trigger further demand for data, planners need to contemplate whether, and when, they are prepared to undertake follow-up data collection. Given the rapid change in the state of ICT, there is likely to be a demand for regular surveys (whether stand-alone ICT household surveys or questions included in other household surveys),[4] and this should be considered in the context of a multiyear work programme (see Chapter 2).

127. A likely outcome of the planning stage is that priorities will be reassessed and changes made to aspects of the survey, such as its purpose and objectives, and subsequent stages such as survey design.

128. A household survey is generally a complex and costly undertaking. Good planning will almost certainly lead to a better result – in terms of data quality, cost and timeliness. Broad areas to consider early in the planning stage are:

• Mechanisms for cooperating with policy-makers and other data users. As discussed in Chapter 2, close cooperation with policy-makers and other data users is strongly recommended in order to improve the relevance of the survey results and to optimize survey questions by determining the purpose of the survey (see below). Consultation mechanisms (such as a national working group on ICT statistics) may already exist in the area of ICT statistics. If they do not, then they should be considered at the planning stage (GSBPM phase “Specify needs”). Possible mechanisms were introduced in Chapter 2.

• Establishment of a management and planning structure, for example, using an interdisciplinary survey team. Team members may include external representatives from consultative bodies or equivalent.

• Purpose and data content of the survey. It is important to always have the purpose of the survey in mind and to review it if necessary. It is assumed that the purpose of the survey will be formulated with the input of policy-makers and other major users and will be centred on the most important, and measurable, needs of policy-makers. The purpose will lead on to data specifications and may extend to proforma data tabulations. Whatever the detail, the purpose needs to be clear – and clearly understood – by the statisticians and data users involved in the survey. It may also be communicated to respondents in some form in order to gain their cooperation, for example at the time of interview or in publicity material before and during the survey implementation. It is also important that statisticians closely collaborate with ICT market experts to fully understand the different items for measurement, such as: types of devices, types of connection to the Internet, ICT uses, ICT skills, service packages (e.g. bundles including fixed and mobile telephone together with cable TV services). It is equally important that ICT experts contribute to the “localization” of the questionnaire, i.e. to provide examples of the national market for ICT services (such as prevalent social media, browsers and email software, online commerce sites, etc.) which could be provided as examples in the questionnaires.

• Availability of alternative sources of data. A survey is a costly statistical operation. Before planning a specific survey on ICT, or including ICT-related questions in an existing survey, alternative potential data sources should be investigated. For instance, if topics on household ICT were included in a recent household living conditions survey, it may be unnecessary to collect information about them.

• Estimation of resources required. Significant resources (human and financial) are required to implement a survey. The budget available to conduct a survey may be a constraining factor that needs to be considered early in the planning stage. The possibility of obtaining additional budget from users interested in particular topics should be ascertained early in the process as it might affect the development of the survey. For example, if one user is very interested in use of ICT by people in a particular region, then that user may provide funds to boost the sample in that region and therefore enable more detailed output. Survey budgeting is discussed in more detail below. With respect to human resources, the need for trained field staff (in ICT concepts besides their general training as interviewers) and skilled statisticians may represent further constraints. Training issues are also discussed below.

129. The result of the above steps could be a business case seeking approval and possibly funding by the relevant authorities (e.g. a national statistical council or a coordination board) for an ICT household survey to be conducted or included in a national statistical plan.

130. More detailed planning will include consideration of procedures and outputs of the data collection, including the choice of data collection and processing methodologies. Issues to be considered here with respect to ICT household surveys are:

• Available survey vehicle. A decision needs to be taken fairly early in the planning stage on the survey vehicle to be used. The options are to include a modest set of questions in an existing multipurpose household survey or to conduct a stand-alone ICT household survey (also known as a ‘dedicated’ survey). There are various advantages of each and these are explored in Chapter 5.

• Adherence to existing statistical standards.[5] A major focus of this Manual is the core list of ICT household indicators (see Chapter 4 and Annex 1). Those indicators have associated statistical standards and statisticians are strongly encouraged to use those standards so that resulting output is internationally comparable as well as consistent over time in a national context. The core indicator standards also incorporate other international statistical standards, for example, the classification of occupations, labour force status and educational attainment. Statistical standards are described in Chapter 4.

• Timetable. The release of data should generally occur as quickly as possible after the date of the survey and the reference period (without compromising data quality), particularly given rapid developments in the area of ICT. Statistical resources, such as interviewers and other staff, may only be available in a specified time frame and will have costs associated with the time they are employed on the survey. For these reasons, it is very important that the planning stage includes a detailed timetable of activities involved in the survey cycle and that milestones are adhered to. An example of a broad timetable that could be adapted can be found in Chapter 2 of Designing Household Survey Samples: Practical Guidelines (UNSD, 2005b).[6]

• Legal and related issues. There could be a range of legal and related issues that need to be considered. These include legal obligations of the data collection agency (which may include confidentiality constraints on data release, for example, a prohibition on release of data identifying individuals); respondents’ legal obligations (e.g. to take part in the survey); and practices that may not be prescribed in law but are policy of the data collection agency (e.g. minimization of respondent burden).

• The survey scope (the ‘target population’ of interest) and units. When planning a survey, it is important to define the target population to be covered. For instance, is information required for both urban and rural areas? In some countries, ICT penetration is very low in rural areas and therefore collecting ICT data may not be cost-effective.[7] It may be very expensive to survey some populations, for instance, those living in remote locations, so decisions need to be made at the planning stage about such populations.[8] In respect of individuals, are there age scope considerations, for instance, is there a particular interest in children – or older people? In some countries, different methods are used to survey persons living in private households and those living in collective households (such as temporary workers living on construction sites). Chapter 7 covers scope and units in some detail.

• Classificatory data required. In general, users require detailed breakdowns by particular characteristics (e.g. age of individuals, geographic area, labour force status, sex or educational attainment). These should be established at the planning stage as they may have implications for sample design and size (and therefore cost). Classifications are considered in Chapter 4.

• Survey design. Survey design issues and procedures include statistical standards to be used, the availability or development of a survey frame, data sources, data collection techniques, sample design according to the available budget and output requirements, and questionnaire design and testing. These issues are examined in chapters 5, 6 and 7.

• Survey implementation. Survey implementation issues and procedures include data collection, data processing requirements (such as editing and estimation), data capture, development and testing of computer systems, and consideration of skills and training required. Data processing is covered in chapter 8.

• Post-survey processes. Post-survey processes include data tabulation and dissemination, metadata dissemination, archiving, documentation and evaluation. These are very important elements of the survey process, especially as they are the most visible to users. Post-survey processes are mainly covered in chapters 9 and 10.

Budget and management issues

131. It is rarely possible to achieve good results without significant cost but it is certainly possible to incur significant costs and yet obtain poor quality results if the survey is not well planned. Having an experienced and knowledgeable project manager and paying careful attention to planning for every phase before commencing operations will generally be very cost effective. The project manager position would usually be full-time and would cover all phases of the survey (design, implementation and dissemination).

132. Costs include wages and salaries, ICT costs and administrative costs. Depending on the costing policy of the organization carrying out the survey, overhead costs (fixed and/or variable), may need to be added to the budget.

133. It is necessary, at the outset, to itemize and estimate costs associated with the survey. A draft budget sheet has been developed by UNSD[9] and could be adapted by countries based on their own costing data.

134. Budgeting needs to be carried out carefully in order to avoid the most common difficulties, which include:

• underestimating known costs (for instance, because allowance is not made for things going wrong and therefore staff resources required are underestimated);

• omitting some costs (e.g. costs of unexpected publicity); and

• ignoring or underestimating overhead costs (these can be significant and will include direct and indirect overhead costs).[10]

135. Given the likelihood of delays, it is useful to include some extra budget (and other resources such as staff time) for unforeseen events or delays.

136. Often trade-offs are required to fit the survey to the available financial and other resources. Trade-offs could include a reduction in sample size (usually resulting in larger sampling error), removal of some level of detail, or removal of some questions or topics. Ideally, such decisions would be made together with policy-makers and other major data users to ensure that their data needs are still adequately addressed.

137. Mechanisms to reduce costs should be considered and include:

• Use of ‘economies of scale’, for instance, including ICT questions in a larger survey (the marginal cost is likely to be lower than the cost of running a stand-alone survey). This will be discussed later, considering the possibility of ICT survey modules.

• Use of technology to reduce costs, for instance, the integration of computer-based quality control[11] into fieldwork should be considered. This includes the use of computer assisted personal or telephone interviewing for data collection, and computer assisted data entry while in the field (in the case of pen and paper interviews). Such techniques are important for improving data quality and should also reduce costs by reducing or eliminating further data editing. Careful consideration of other factors is necessary, for instance, the costs of computer systems development and maintenance.

• Use of free software for data processing, such as R[12], thus benefitting from the existence of libraries of statistical routines for the analysis of survey data.

• Use of standard tools provided (generally for free) by international organizations for the dissemination (including visualization) and documentation of ICT household data and indicators. Examples of such tools are NADA (National Data Archive,[13] used to document surveys and provide access to microdata), PxWeb[14] and PC-AXIS[15] (developed by a consortium of NSOs led by Statistics Sweden, to disseminate aggregate data and geographical information) and REDATAM[16] (mainly used by Latin American countries to disseminate population and housing census data, including some ICT variables: presence of computer, access to the Internet, access to fixed and mobile telephone).

Other general preparatory work

138. Preparatory work covers a range of activities and arguably continues throughout the survey cycle. Planning and budget issues were discussed above. In subsequent chapters, we will look at data sources, collection methods, questionnaire design and survey design. In this chapter, we look at staff training which is only discussed briefly in other parts of the Manual.

Staff training and selection

139. It can be seen from the information presented above that staff with diverse skills and expertise will be required for different aspects of the survey. Skills relevant to the following areas will be required: survey management, survey design, computer systems development, interviewing, data entry, data editing and coding, data imputation and estimation, data analysis, survey documentation, archiving and dissemination (including publication writing).[17] Although field work managers and computer programmers may belong to different departments within the organization, they would ideally be recognized as members of the survey team.

140. Staff selection and training will run parallel with survey planning, questionnaire design and sample selection. It will often be a phased activity, for instance, staff involved in survey planning and development are likely to be employed first. While specialized staff may be available in an organization, they will usually need to be trained on the specifics of the survey.

141. It is better to make training an inclusive activity. For instance, interviewers and their supervisors often have valuable ideas for the operational phases of the survey and will be more committed to a quality result if they have had some involvement in documentation and the setting of field procedures.

142. While many staff involved in a particular survey will be skilled and will require minimal training, others may need significant training. Of particular importance is the training of interviewers, some of whom may be inexperienced. An important cause of bias is poor handling of respondents by interviewers, for instance, asking leading questions or suggesting some judgement of responses (for instance, by their tone of voice or facial expression). Training and briefings, as well as survey material, should be focused on avoiding such problems. Elements of training may include classroom training, interviewer manuals and field work in the presence of experienced interviewers or supervisory staff. Training is perhaps the most underestimated component of survey implementation. Although it is difficult to provide specific recommendations on the minimum time needed for training, it should probably be measured in weeks and not in days.

143. It is important that staff involved in a particular process (e.g. data collection) receive the same, or similar, training to avoid bias. Providing sufficient time for training is also key to ensuring high quality information.

144. The performance of all staff should be monitored closely, especially in the early stages of the survey. Any instances of underperformance, or behaviour that might induce statistical bias, need to be addressed as quickly as possible. Box 12 illustrates a particularly relevant problem in supervision.

Box 12. An example of supervision and monitoring in ICT household surveys

A typical problem of questionnaire design and supervision is found when a positive answer to a question triggers other questions. This situation could encourage the interviewer (and possibly the interviewee) to provide a negative answer. For instance, the question on the use of Internet in the model questionnaire for measuring ICT access and use by households and individuals (Annex 2) may represent such a case. If answered positively, it requires that the questions on type of device used and type of connection are answered as well; otherwise, the interview is over. The only way to solve this problem is to increase the level of supervision and monitoring. There are no possible solutions within the questionnaire design.

Supervision implies random control visits in which certain questions are asked again in a randomly chosen subset of the households. It is generally considered good practice to perform this kind of supervision in around 15–20 per cent of households in the sample, although unfortunately the levels actually found in practice are usually much lower. As mentioned, the question on the use of Internet is a natural candidate for this kind of supervision.

Monitoring can be performed through tabulation of the data while interviews are being performed in the field. Odd trends can be detected, such as a particular interviewer whose questionnaires have noticeably low rates of Internet use (that is, with a tendency to insert “No” as the answer to the said question).

145. Given that some questions on ICT access and use are somewhat technical, it could be advantageous to employ people who are more attuned to ICT, for instance young adults and people with proven ICT skills. Clearly also, it is important to provide training in the ICT concepts and terms used in questionnaires.[18] As mentioned above, the collaboration with ICT experts is crucial to understand complex technological issues such as types of devices, types of connections, ICT services, etc.

146. A most important prerequisite for training is the availability of training and/or procedures manuals for each broad class of staff, including interviewers, supervisors and data entry staff. Such manuals should ideally be prepared before the survey commences and may continue to be useful references during the survey. Training manuals should clearly explain the purpose of the survey and be quite explicit about tasks to be performed by staff.[19]

147. Training resources that can be used before undertaking a household ICT survey include:

• this Manual (available in the six UN official languages),

• e-learning on the Manual,[20] and

• face-to-face training courses organized by ITU.

[1] See: https://ec.europa.eu/eurostat/documents/64157/4373903/07-Checklist-for-Survey-Managers_DESAP-EN.pdf/ec76e3a3-46b5-409e-a7c3-52305d05bd42

[2] See: https://statswiki.unece.org/display/GSBPM/GSBPM+v5.1 Note that the terminology used in this model is not necessarily the same as used in this Manual, but the areas covered are similar.

[3] https://www.unece.org/stats/mos.html

[4] These may be in the form of distinct question modules or questions distributed throughout a questionnaire.

[5] The word ‘standards’ is used here in a broad context covering topics, questions, concepts, classifications and definitions.

[6] Time-table of Household Survey Activities for Country X; the same chapter has a cost worksheet that may also be useful in outlining the steps in the survey process.

[7] If electricity is not available, it is unlikely that most ICTs are widely used there (the possible exception to this is the mobile telephone).

[8] However, note the importance of providing available data and metadata in relation to the non-availability of ICT services in unsurveyed areas. Some data dissections, such as urban/rural, are important for measuring the digital divide and where a survey measure is not available, an alternative based on other data (for instance, infrastructure data) could be included in the data dissemination. It is also very important to document any scope limitations so that users are able to make relevant comparisons. This subject is further dealt with in Chapter 10.

[9] See UNSD (2005a, Chapter IV).

[10] Direct overhead costs are those that are proportional to units used (e.g. staff salary overheads such as contributions to pension funds). Indirect overhead costs are those that are not proportional to units used but may be significant at a ‘whole of organization’ level. They include things like building costs.

[11] Computer-based quality control will be discussed in Chapter 8.

[12] https://www.r-project.org/

[13] https://nada.ihsn.org/

[14] https://www.scb.se/en/services/statistical-programs-for-px-files/px-web/

[15] https://www.stat.fi/tup/tilastotietokannat/px-tuoteperhe_en.html

[16] See http://www.cepal.org/redatam/ for the main page. Some census data are available online for Latin America, Caribbean, Asian and African countries.

[17] The last is quite a specialized skill, needing general writing ability as well as an understanding of data and an ability to describe and interpret data clearly and unambiguously.

[18] Technical definitions are included with the core indicators.

[19] UNSD (2005a, Chapter IV) discusses this in more detail.

[20] https://www.itu.int/en/ITU-D/Statistics/Pages/capacitydev/default.aspx.

Chapter 4. Statistical standards and measurement topics for ICT household statistics

148. Survey planning can be simplified, and output improved, by the use of national and international standards covering definitions of indicators, model questions, concepts, units, scope and classifications. In respect of ICT household statistics, the standards described in this Manual are both specific and general. Specific standards are those recommended by the Partnership on Measuring ICT for Development and endorsed by the United Nations Statistical Commission (UNSC). General standards include definitions and classifications of labour force status, occupation, educational attainment and age.

149. Adherence to both specific and general standards governing ICT statistics should be a primary aim and will ensure that collected data have maximum usefulness and are as comparable as possible with the output of other countries. In a national context, use of standards may enable comparability with other datasets and enhance comparability with historical data.

150. In this chapter, we will consider specific standards pertaining to ICT household statistics and relevant international classifications. Methodological standards, such as survey procedures, scope and statistical units are discussed in the following chapters.

Core ICT household indicators

151. The core list of ICT indicators was formally released by the Partnership in 2005 as Core ICT Indicators (Partnership, 2005). At its meetings of 2007, 2012, 2014, 2016 and 2018, the United Nations Statistical Commission (UNSC) endorsed the Partnership core list and its revisions. The Partnership and its members are continuously improving the core list, in consultation with member countries, based on data collection experiences and in light of technological change.

152. Part of the core list is under the responsibility of the ITU. This includes the indicators on ICT infrastructure and those indicators on ICT access and use by households and individuals (hereafter “ITU ICT HH indicators”), to which this Manual refers. The Partnership core list of indicators on ICT access and use by households and individuals is a subset of the list of ITU ICT HH indicators that are the subject of this Chapter.

153. The mechanism for revisions of the ITU ICT HH indicators include consultations with the Expert Group on ICT Household Indicators (EGH), established after the recommendation of the 9th World Telecommunication/ICT Indicators meeting (WTIM-11). The revised list is then endorsed by the World Telecommunication/ICT Indicators Symposium (WTIS, replacing the WTIM).

154. The list of ITU ICT HH indicators as of 2018 is shown in Table 6. The complete Partnership core list of ICT indicators can be found in Annex 1.

155. There are 23 ICT HH indicators in the ITU ICT HH list. The reference indicator (HHR1, Proportion of households with electricity) was dropped from the list and is referred to in this Manual as a useful variable to cross-tabulate ICT access indicators. The main purpose of the ITU ICT HH list is to assist countries to produce high quality and internationally comparable ICT household statistics. The indicators have associated standards and metadata including definitions, model questions, classificatory variables, scope and statistical units.

156. The indicators added by the EGH in 2013 were the following: Proportion of households with multichannel television, by type (HH13), Barriers to household Internet access (HH14), Individuals with ICT skills, by type of skills (HH15) and Household expenditure on ICT (HH16). Indicators on other important topics (namely Individuals using the Internet by type of portable device and network used to access the Internet, Internet security, and Children and youth online protection) were discussed by the EGH but were not adopted at this time.

157. In 2014-2015, after the recommendations of the EGH and the World Telecommunication/ICT Indicators Symposium (WTIS), three new indicators (HH17, HH18 and HH19) were included. These are: Proportion of individuals using the Internet, by type of portable device and network used to access the Internet (HH17); Proportion of individuals who own a mobile phone (HH18) and Proportion of individuals not using the Internet, by type of reason (HH19).

158. In 2018, after the recommendations of the EGH, the WTIS endorsed a revised list, by adding four new indicators (HH20 to HH23) on e-commerce, as well as by adding the smart telephone as a new sub-category for three indicators (HH3, HH10 and HH18). The new indicators are: Proportion of individuals who purchased goods or services online, by type of good and service purchased (HH20); Proportion of individuals who purchased goods or services online, by type of payment channel (HH21); Proportion of individuals who purchased goods or services online, by method of delivery (HH22) and Proportion of individuals who did not purchase goods or services online, by type of reason (HH23).

159. Some ICT HH indicators are used to monitor the SDGs, as well as national development plans when they are based on the SDGs (see Box 9 on The Philippines). These are marked in Table 3.

Table 3. List of indicators on access to, and use of, ICT by households and individuals (“ITU ICT HH indicators”)

Indicator Number

Indicator name

Used for monitoring SDGs

HH1

Proportion of households with a radio

HH2

Proportion of households with a television

HH3

Proportion of households with telephone

HH4

Proportion of households with a computer

HH5

Proportion of individuals using a computer

HH6

Proportion of households with Internet

√

HH7

Proportion of individuals using the Internet

HH8

Proportion of individuals using the Internet, by location

HH9

Proportion of individuals using the Internet, by type of activity

HH10

Proportion of individuals using a mobile cellular telephone

HH11

Proportion of households with Internet, by type of service

HH12

Proportion of individuals using the Internet, by frequency

HH13

Proportion of households with multichannel television, by type

HH14

Barriers to household Internet access

HH15

Individuals with ICT skills, by type of skills

√

HH16

Household expenditure on ICT

HH17

Proportion of individuals using the Internet, by type of portable device and network used to access the Internet

HH18

Proportion of individuals who own a mobile phone

√

HH19

Proportion of individuals not using the Internet, by type of reason

HH20

Proportion of individuals who purchased goods or services online, by type of good and service purchased

HH21

Proportion of individuals who purchased goods or services online, by type of payment channel

HH22

Proportion of individuals who purchased goods or services online, by method of delivery

HH23

Proportion of individuals who did not purchase goods or services online, by type of reason

Box 13. The 2019 National ICT Household Survey-Philippines to monitor development plans

The Philippine Statistics Authority (PSA) granted clearance to the implementation of the 2019 National ICT Household Survey. The survey is proposed by the Department of Information and Communications Technology (DICT) and will be conducted by the Philippine Statistical Research and Training Institute (PSRTI).

The survey aims to:

• gather ICT data at the household and individual level to support the monitoring of performance indicators in the Sustainable Development Goals (SDGs), Philippine Development Plan (PDP) Results Matrix, DICT plans, and international benchmarking indices

• gather critical data on ICT demand towards improvement of public service delivery as well as growth of potential markets in the digital economy (e.g. e-commerce)

• provide a national baseline for core ICT household indicators

Source: http://www.psa.gov.ph/content/psa-approves-conduct-2019-national-ict-household-survey-0

160. To simplify understanding, the following discussion of the indicators distinguishes the access indicators applying at the household level, the household ICT expenditure indicator (HH16), and the use indicators applying to individuals.

Figure 4. ITU ICT HH indicators by type of statistical unit (households and individuals)

Household ICT access core indicators

161. Indicators HH1, HH2, HH3, HH4, HH6, HH11, HH13 and HH14 refer to household access (and barriers to access) to ICT equipment and services, not to use of those products by individual household members. In order for a household to have access to ICT equipment or services, it should be able to be used, that is, equipment is in working condition at the time of interview and ICT services are in operation.

162. In order for a household to have access to ICT equipment or services, the equipment/service should generally be available for use by all members of the household at any time, regardless of whether it is actually used. ICT equipment may or may not be owned by the household. Household surveys traditionally ask about the availability of assets in the household, including TV, electricity, refrigerator, piped water, etc. A similar principle has been adopted for ICT equipment and services, that is, they should be available for use by household members at home, regardless of whether they are used. They can be taken away from home occasionally, but the basic principle is that they are usually available for use by all household members at home.

163. For many developing economies, there are significant barriers to access to, and use of, ICT. Indicator HH14 measures barriers to Internet access by households. Consideration could also be given to including other household and/or individual barriers questions in ICT household questionnaires. For example, countries may be interested in investigating the reasons why households do not have computers (not included in the core list). Box 14 shows the reasons included in the ICT Household Survey of Oman, 2013.

Box 14. Oman: measuring barriers to household access to a computer

The Information Technology Authority of Oman included, in its 2013 questionnaire, a question on the reasons why the interviewed household does not own a computer. The question is worded as follows:

What are the reasons for the household not owning a computer? (select all that apply)

• Cannot afford one

• Household members have access to computers at work, school or Internet café

• Household has no need for a computer

• No one in the household knows how to use a computer

• Harm for health

• Waste of time

• Illiteracy

• Need special equipment/software

• Other

Source: ITA (2013) questionnaire.

164. The access indicators are presented as the proportion of households with [equipment, service].[1] With the exception of HH14, indicator values are calculated by dividing the number of in-scope households with [equipment, service] by the total number of in-scope households. For HH11 (Internet access by type of service), output for each type of Internet service category can also be presented as the proportion of households with Internet. Similarly, for HH13 (multichannel TV by type of service), output for each type of service category can also be presented as the proportion of households with TV. For HH4, where countries ask about the type of computer, output for each type of computer category can be presented as the proportion of households with computer. For HH14 (barriers to household Internet access), output for each type of barrier category should be presented as the proportion of households without Internet.

165. Sub-indicators can be constructed using the classificatory variables, household composition and size. These are detailed later in this chapter.

Household ICT expenditure core indicator

166. Indicator HH16 (Household expenditure on ICT) is intended to measure the expenditure undertaken by households on ICT equipment and ICT services. HH16 can be presented as the amount, or proportion, of household expenditure spent on ICT. For international comparison, HH16 may be presented together with other economic indicators such as the distribution of household income or per capita income. Demographic indicators such as the number of households or average household size should be considered for the analysis of this indicator.

167. In general, indicator HH16 is collected in household expenditure surveys and therefore not necessarily included in ICT household surveys. Further discussion about this source can be found in Chapter 3.

168. When broken down by socio-economic classificatory variables, HH16 can provide information on differences in ICT consumption patterns of households. As with other consumption indicators, the analysis of the distribution of the variable provides more information than summary statistics (such as the mean or the median).

169. Product and service classifications used to classify household expenditure should be used. For this purpose, the UN COICOP 2018 classification (Classification of Individual Consumption According to Purpose) (UNSD, 2018) is recommended. With the development of multi-purpose devices such as smartphones, the boundary between communication and audio-visual, photographic and information processing equipment is problematic. Other difficulties include the measurement of expenditure when products or services in different categories are bundled (sold as a package at a discounted price).

170. Table 4 presents a proposal for use of the COICOP classification for ICT equipment and services for the purposes of measuring ICT expenditure.

Table 4. Classification of ICT equipment and services, based on COICOP 2018[2]

COICOP code

ICT equipment and services categories

08

Information and communication

08.1 Information and communication equipment

• Fixed telephone equipment: telephones, radio-telephones, telefax machines, telephone-answering machines and telephone loudspeakers..

• Mobile telephone equipment: mobile telephone handsets, including devices with several functions; and smartphones..

• Information processing equipment: personal computers, printers, scanners, monitors, projectors, augmented reality (AR) and virtual reality (VR) head mounts, modems, routers, network switches and the like, keyboards, mice, digitizers; tablets; calculators, including pocket calculators; typewriters and word processors (device); toner and ink cartridges, laser printer drums, typewriter ribbons ; and telefax and telephone-answering facilities running on computers.

• Equipment for the reception, recording and reproduction of sound and vision:

- Television sets, video cassette players and recorders, digital video recorders, DVD players, Blu-ray players, Ultra HD Blue-ray players, streaming boxes, TV aerials of all types;

- radio receivers (radio sets, digital radio sets, Internet radio sets, satellite radio sets, car radios, radio clocks, two-way radios, walkie-talkies, amateur radio receivers and transmitters);

- portable and non-portable CD players;

- portable and non-portable sound players;

- stereo equipment and CD radio cassette recorder;

- turntables, tuners, amplifiers, cassette decks, microphones and speakers, DJ equipment, karaoke systems;

- audio and video systems for cars

- set-top boxes, satellite receivers, IPTV receivers, TV converter boxes;

- digital media players;

- headphone, earplugs and wireless/Bluetooth headsets.

Table 4. Classification of ICT equipment and services, based on COICOP 2018 (continued)

COICOP code

ICT equipment and services categories

• Unrecorded recording media:

- CDs (R and RW);

- DVDs (R and RW);

- Blu-ray discs (R and RE);

- video cassettes;

- audio tapes, cassettes, DAT;

- external hard drives and solid state disks, NAS (network attached storage);

- USB keys/flash drives;

- SD cards, compact flash, etc.;

- magnetic data tapes;

- other magnetic recording media;

- other optical recording media;

- other recording media (phase-change recording media, holographic recording media, molecular recording media).

08.2 Software excluding games computer software packages, such as operating systems, applications, programming languages, etc.

• software subscriptions and use of online software;

• apps. .

08.3 Information and communication services

• Fixed communication services:

- installation and subscription costs of personal telephone equipment;

- telephone calls from a private line or from a public line (public telephone box, post office cabin, etc.);

- local, regional, national and international calls;

- telephone calls from hotels, cafés, restaurants and the like.

• Mobile communication services:

- local, regional, national, and international calls, including voice and video calls;

- messages, including voice, written (SMS) and image (MMS) messages, subscription fees for other messengers;

- additional calling features, such as voice mail and call display, whether sold separately or bundled with the mobile local service plan;

- voice and messaging mobile phone plans that also include limited data;

- mobile phone voice, text and data plans;

- other mobile telephone services

- costs of telephone equipment if included in subscription costs;

- mobile phones included in a package, i.e. prepaid or post-paid packages, generally tied to a specific operator for a certain period of time, if not separately priced.

Table 4. Classification of ICT equipment and services, based on COICOP 2018 (continued)

COICOP code

ICT equipment and services categories

• Internet access provision services and net storage services:

- Internet access services provided by operators of wired, wireless or satellite infrastructure;

- cloud storage, file hosting and web hosting services;

- subscriptions for email services.

- Includes also:

- activation and installation fees and monthly rate.

• Bundled telecommunication services:

- telephony/Internet/TV packages;

- any combination of telecommunication package.

• Repair and rental of information and communication equipment:

- The cost of materials is included only if the materials are not separately invoiced. Includes:

- repair of all information and communication equipment;

- rental of telephones, telefax machines, telephone-answering machines and telephone loudspeakers;

- rental of wireless telephone equipment;

- rental of Internet access provision equipment;

- rental of telegraphy, telex, telefax, radiotelephony, radiotelegraphy and radio telex equipment.

• Other information and communication services:

- telegraphy, telex and telefax services;

- VoIP (Voice over Internet Protocol) provision (nomadic use);

- TV and radio licenses;

- subscription to cable TV, satellite TV, IPTV, and Pay-TV;

- streaming services;

- online videorecorder services (web-based DVR services);

- VOD services (video on demand);

- rental or subscription of CDs, video tapes, DVDs, Blu-ray discs, software (excluding game software).

- radiotelephony, radiotelegraphy and radiotelex services;

- software installation services;

- rental/lease fees for a decoder, television set-top boxes, etc.

09.1

Other recreational goods

09.2.1 Games, toys and hobbies:

• video-game software; video-game computers that plug into a television set; video-game cassettes and video-game CD-ROMs, video-game downloads;

• game apps;

• gamepads, joysticks, racing wheels and other accessories for video gaming;

• electronic games.

Source: https://unstats.un.org/unsd/classifications/business-trade/desc/COICOP_english/COICOP_2018_-_pre-edited_white_cover_version_-_2018-12-26.pdf

Individual ICT use core indicators

171. Indicators HH5, HH7, HH8. HH9, HH10, HH12, HH17, HH18, HH19 refer to use of (or reasons for not using) ICT equipment and services by individual household members. HH15 measures the skills of individuals by examining the activities they have carried out on digital devices. HH20, HH21, HH22 and HH23 refer to e-commerce activities of individuals. The suggested reference period,[3] which has been revised from the previous edition of this Manual, is the last three months.

172. Five individual use indicators (HH5, HH7, HH10, HH18) are presented as the proportion of in-scope individuals using ICT equipment or the Internet, while HH19 is expressed in terms of those who do not use the Internet. The other four use indicators (HH8, HH9, HH12 and HH17) break down Internet use (by location, Internet activities undertaken and frequency of use respectively). HH15 (ICT skills) should be calculated as the proportion of all individuals (in order to be device-agnostic). Indicators on e-commerce (HH20 to HH22) break down online purchases by individuals (and can be presented as percentages of all individuals purchasing online goods or services), while HH23 is calculated as a proportion of all in-scope individuals.

173. For HH8 (location of use), HH9 (Internet activities undertaken), HH12 (frequency of use) and HH17 (type of device), the indicators may be calculated as both the proportion of in-scope individuals and the proportion of individuals using the Internet.

174. In general, it is recommended that countries report the data in absolute numbers, and explicitly mention the denominator when presenting the indicators as percentages or proportions.

175. Reasons for not using the Internet are measured through indicator HH19. The reasons can be external to the individual (availability, cost, lack of local content), lack of authorization) or due to her/his knowledge, concerns or skills.

176. Indicators HH20 to HH23 refer to e-commerce activities of individuals, and reasons for not doing so. They were added to the list of core indicators after the recommendation of the EGH.

177. For individual members of households, e-commerce presents an alternative method of purchasing (and increasingly selling) goods and services for private use. According to the long-standing OECD statistical standard for measuring e-commerce, it is the method by which an order is placed or received, rather than the payment or channel of delivery, which determines whether a transaction is an e-commerce transaction. The definition is:

“...the sale or purchase of goods or services, conducted over computer networks by methods specifically designed for the purpose of receiving or placing of orders. The goods or services are ordered by those methods, but the payment and the ultimate delivery of the goods or services do not have to be conducted online. An e-commerce transaction can be between enterprises, households, individuals, governments, and other public or private organisations. To be included are orders made over the web, extranet or electronic data interchange. The type is defined by the method of placing the order. To be excluded are orders made by telephone calls, facsimile or manually typed e-mail.”

178. The current ICT indicators on e-commerce measurement for the household sector focus on use of the Internet for purchasing rather than sales transactions. This can provide useful information on, for example, business-to-consumer e-commerce, often not obtained through business surveys. While indicator HH9 includes the activities of purchasing and selling of goods or services, indicators HH20, HH21 and HH22 detail the online purchase (by type of good or service and payment and delivery methods). Surveys of ICT use in households may collect additional information, including the nature of goods and services purchased or sold, the monetary value of those purchases or sales, or whether a product was purchased from overseas.

179. There are both conceptual and data collection challenges associated with measuring the value of e-commerce. For example, in respect of individuals reporting the value of Internet purchases, there are issues of understanding the definition of e-commerce (for instance, distinguishing purchases from payments[4]). In population segments where the volume of e-commerce is small (e.g. older people, regions with weak Internet coverage), the relatively small volume of e-commerce activity with respect to total household expenditures may lead to issues related to the size of sampling error, if this is too large. In addition, there are recall issues for respondents in relation to the value of e-commerce purchases (that is, they may not be able to report reliable information on the value of those purchases). As in-app purchases[5] become more common, some e-commerce purchases may be quickly forgotten. Indeed, as Internet transactions become more common, the recall problem is likely to worsen.

180. It has to be recalled that the core ICT indicators include other indicators on e-commerce in the business sector, namely, the proportion of businesses receiving orders over the Internet (B7) and proportion of businesses placing orders over the Internet (B8). These are usually collected via business surveys[6].

Classifications for ICT household statistics

181. Specific subpopulations are generally of more interest to policy-makers than the whole population. The need for detailed data on subpopulations has been clearly stated in the 2030 Agenda for Sustainable Development, by coining the expression “leave no one behind”. Classifications of units to define subpopulations are therefore generally part of a statistical framework. This section discusses the classifications recommended for use with the core indicator statistical units, households and individuals.

182. It is unlikely that small subpopulations will be sufficiently well represented in a sample survey to enable reliable estimates. It will therefore be difficult to obtain ICT indicators for these subpopulations from sample surveys unless they are specifically designed to cover them. In some household surveys, limitations to scope may also exclude certain subpopulations, for example, those in remote areas or living in collective households. Where ICT indicator data are collected from a census, then small subpopulations are likely to be covered, unless there are scope limitations that exclude them.

Individual characteristics

183. In terms of ICT policies, the following information about individuals may be of interest: children and youth, the elderly, persons with disabilities, socio-economic groups (e.g. defined by level of education, labour force status and occupation ) and groups with certain ethno-cultural characteristics. The latter includes those using a particular language, nomads or persons living in areas to which access is difficult, immigration status (e.g. temporary migrant workers),. In addition, the gender perspective is increasingly considered in all social and economic topics, and recommended in particular for studies on access to, and use of, ICT.

184. Standard questions on age and sex used as classificatory variables allow dissection of indicators on the use of ICT by individuals, identifying groups such as children, youth, the elderly and women. Data for these classificatory variables are usually collected in the survey that collects ICT data or, less commonly, are available from the survey frame.

185. For many developing economies, there are socio-economic problems that create barriers to use of ICT by individuals. These problems are diverse and broadly cover lack of opportunity and lack of ability. They include illiteracy and other linguistic limitations, socio-cultural barriers, lack of ICT and other skills, lack of confidence or awareness and low income. Some classificatory variables that might address these issues include level of education, status in the labour force and occupation. The issue of ICT skills has been further analysed and is discussed in the description of indicator HH15.

186. There are several individual characteristics classifications recommended by this Manual; they are: sex, age, highest education level attained, labour force status and occupation. Classifications for these categories are based on international standards and are described below.

Sex

187. Sex (male or female)[7] of every individual should be recorded, as sex disaggregation of data is a fundamental requirement for social statistics and in particular for the analysis of the gender gap in the use of ICT. For policy purposes as well as for monitoring SDG#5 (“Achieve gender equality and empower all women and girls”), all core ICT use indicators should be broken down by sex, in order to maximize information on any digital gender gap.

Age

188. Age is a strong determinant of ICT use so a common age cut-off and categories are important. It is recommended that countries use the following ranges: under 5; 5–9; 10–14; 15–24; 25–34; 35–44; 45–54; 55–64; 65–74 and 75 and over.[8] Countries are particularly encouraged to collect data for children, where this is feasible, to provide indicators on child online protection.[9]

189. The above age ranges are consistent with UNSD recommendations on age ranges of individuals, which are: under 1; 1–4; 5–9; 10–14; 15–19; 20–24; 25–29; 30–34; 35–39; 40–44; 45–49; 50–54; 55–59; 60–64; 65–69; 70–74; 75–79; 80–84; 85–89; 90–94; 95–99; and 100 and over.[10]

Highest education level attained

190. ITU research finds that within the online population, important differences exist in terms of the types of online activity engaged in by Internet users. Education levels seem to influence the type of activity in which users engage, with implications for their potential gains. Many Internet users, in particular those with lower levels of education and income, make very limited use of the Internet and are not able to exploit its full potential. In developing countries, the Internet is still used mainly for communication and entertainment purposes. In developed countries, citizens use the Internet to a greater degree for reading newspapers, magazines, and books, interacting with government, and performing banking and e-commerce services (ITU, 2016).

191. Categories are based on the United Nations Educational, Scientific and Cultural Organization (UNESCO) International Standard Classification of Education, the latest revision of which occurred in 2011 (UNESCO, 2011). ISCED codes are defined for educational programmes (ISCED-P) and for educational attainment (ISCED-A), providing parallel classifications of levels of an education programme and of educational attainment (i.e. highest educational level that the individual has completed). Categories at the first-digit levels for educational attainment can be found below. In general, countries have adapted the classification and created correspondence tables between ISCED and their national educational level classifications.

192. ISCED-A 2011 levels are as follows:[11]

• level 0: Less than primary,

• level 1: Primary education,

• level 2: Lower secondary education,

• level 3: Upper secondary education,

• level 4: Post-secondary non-tertiary education,

• level 5: Short-cycle tertiary education,

• level 6: Bachelor or equivalent,

• level 7: Master or equivalent,

• level 8: Doctoral or equivalent.

193. This Manual recommends that the ISCED level categories are aggregated for reporting and comparison purposes as follows:

• primary education or lower (ISCED levels 0, 1),

• lower secondary education (ISCED level 2),

• upper secondary education or post-secondary non-tertiary education (ISCED levels 3,4),

• tertiary education (ISCED levels 5, 6), and

• post-tertiary education (ISCED levels 7, 8).

194. When analyzing the use of ICT by educational level, it may be relevant to cross-classify data by age group. Where official age ranges are defined for educational levels (e.g. 6–12 for primary education in some countries), these should be considered in the analysis at the national level. For international comparability purposes, presenting ICT indicators by ISCED level and age group may provide a better understanding of ICT use according to education (e.g. the proportion of Internet users among children aged 10–14 with primary education may be much higher than that of those aged 40–44 with the same educational attainment). It is not recommended that countries calculate estimates for each combination of age interval and educational level, unless the sample is large enough to provide for accurate estimates (e.g. Eurostat disseminates data on the proportion of persons with lower secondary education attainment only for those aged 15 or more, and with tertiary education attainment only for those aged 20 and more).

Labour force status

195. Categories for labour force status used to break down ICT indicators at the individual level are based on the International Labour Organization (ILO) International Classification of Status in Employment (ICSE-93),[12] with additional categories for those who are unemployed or outside the labour force. The categories are:

• employee (including permanent employees, fixed-term employees, short-term and casual employees, paid apprentices, trainees and interns);

• self-employed (includes the four categories: employers, own-account workers, members of producers’ cooperatives, and contributing family workers);

• workers not classifiable by status (for whom insufficient relevant information is available, and/or who cannot be included in the preceding categories);

• unemployed; and

• outside the labour force.

196. The ICSE-93 categories map to ICT indicators categories for those in the labour force. The additional labour force categories recommended by this Manual are Unemployed and Outside the labour force (which includes individuals who are not economically active, that is those who are neither employed nor unemployed). For policy purposes, it may be relevant to further break down the Outside the labour force category to differentiate those attending an educational institution (students) from others (such as, those performing household duties, retired, or not active for reasons such as illness).

197. After the 20th International Conference of Labour Statisticians, a revision of the ICSE (ICSE-18)[13] has been proposed for testing and implementation by the countries in household surveys. This classification adds a self-identified status in employment used to further filter the respondent to the relevant classification module. This self-identification helps classify respondents as self-employed, employees and contributing family workers. The ISCE-18 presents a categorization based on the type of authority (dependent vs independent workers), ISCE-18-A, whose categories are:

• Independent workers

o A Employers

- 11 Employers in corporations

- 12 Employers in household market enterprises

o B Independent workers without employees

- 21 Owner-operators of corporations without employees

- 22 Own-account workers in household market enterprises without employees

• Dependent workers

o C Dependent contractors

- 30 Dependent contractors

o D Employees

- 41 Permanent employees

- 42 Fixed-term employees

- 43 Short-term and casual employees

- 44 Paid apprentices, trainees and interns

o E Contributing family workers

- 51 Contributing family workers

198. As the ISCE-18 is in testing phase, it is recommended that countries use a classification of labour force status consistent with that of the labour force survey in place.

Occupation

199. For those in the labour force, occupation categories should be based on ISCO major groups where possible.[14] The major groups for the 1988 and the 2008 versions of ISCO are shown in Table 5 ISCO is the International Standard Classification of Occupations and is maintained by the ILO. According to the ILO, “ISCO is a tool for organizing jobs into a clearly defined set of groups according to the tasks and duties undertaken in the job”.[15] ISCO-88 is being replaced by ISCO-08. There have been some changes at the level of interest for ICT statistics (the major group level).[16]

200. Occupation will not be relevant for some age groups (those likely to be under and over the usual working age range) and those outside the labour force. A ‘not applicable’ category would therefore be a useful addition to tables showing occupation.

Table 5. ISCO major groups: 1988 and 2008

Major group

1988 (ISCO-88)

2008 (ISCO-08)

1

Legislators, senior officials and managers

Managers

2

Professionals

Professionals

3

Technicians and associate professionals

Technicians and associate professionals

4

Clerks

Clerical support workers

5

Service workers and shop and market sales workers

Service and sales workers

6

Skilled agricultural and fishery workers

Skilled agricultural, forestry and fishery workers

7

Craft and related trades workers

Craft and related trades workers

8

Plant and machine operators and assemblers

Plant and machine operators, and assemblers

9

Elementary occupations

Elementary occupations

0

Armed forces

Armed forces occupations

Source: ILO, http://www.ilo.org/public/english/bureau/stat/isco/intro.htm.

ICT skills

201. Following a proposal stemming from the 5th Meeting of the EGH in September, 2017, a sub-group (“EGH-Skills”) was created within the EGH to improve the guiding documents related to the measurement of ICT skills based on ICT household data.

202. HH15 can be tabulated against socio-economic individual characteristics, or can be used to break down other ICT indicators. For instance, the relationship between indicators HH9 (activities carried out using the Internet) and HH15 (ICT skills) was acknowledged, so that the breakdown of HH9 by categories of response of HH15 can shed light on the level of competence required to carry out each activity.

203. EGH-Skills proposed to adopt the European Commission’s Digital Competence Framework for Citizens (DigComp 2.0[17]) as the conceptual framework to be used going forward to help guide the measurement of ICT skills. The framework has five major areas of skills measurement: Information and data literacy, Communication and collaboration, Digital content creation, Safety, and Problem-solving, so that the competences are categorized into these areas. The competences are device-agnostic, in the sense that the characteristic activities could be performed on devices other than traditional home and laptop computers (e.g. via smartphones).

204. The possibility of creating one aggregate ICT skills indicator (e.g. basic, moderate and advanced user) has been considered as premature. Countries interested in developing such indicators may refer to the EU Digital Economy and Society Index[18] which combines competence domains and skills levels.

Disability status

205. Persons with disabilities (PwD) face difficulties in everyday life, and this includes the use of Internet. Visually impaired people, for instance, have difficulties in reading the screens of devices. However, ICT offers new possibilities for PwD, such as accessing online training, using voice-recognition software to manage a computer, etc. ICTs, when accessible and available, can serve as critical enablers that allow persons with disabilities the opportunity to fully and effectively participate, on the basis of equality, in all aspects of society and development. ICTs can help persons with disabilities have greater access to knowledge and independent living. The collection of indicators on the use of ICT by PwD responds to the needs of ensuring inclusive access[19].

206. Internationally comparable data on disabilities and PwD can be collected using the statistical standards proposed by the Washington Group on Disability Statistics. The Washington Group Short Set was designed as a core set of questions for self-reporting of functional limitation at the individual level. The questions can be included in any survey that collects data at the individual level.

207. The prevalence of certain disabilities in the target population will influence the representativeness of the sample for measuring ICT use by PwD. Countries may wish to increase the sample size of particular disabled groups if relevant to their ICT policies.

Other individual characteristics

208. Additional individual classifications on socio-demographic/cultural characteristics might also be of interest. They include level of literacy, ethnicity, languages spoken and language skills.[20]. For these other individual characteristics, it is advised to follow the UN Statistical Commission recommendations for Population Censuses.

Box 15. Washington Group on Disability Statistics Short Set of questions

The Washington Group Short Set comprises questions on six core functional domains: seeing, hearing, walking, cognition, self-care, and communication. The questions are worded as follows:

Introduction: The next questions ask about difficulties you may have doing certain activities because of a HEALTH PROBLEM.

1. Do you have difficulty seeing, even if wearing glasses?

2. Do you have difficulty hearing, even if using a hearing aid?

3. Do you have difficulty walking or climbing steps?

4. Do you have difficulty remembering or concentrating?

5. Do you have difficulty (with self-care such as) washing all over or dressing?

6. Using your usual language, do you have difficulty communicating, (for example understanding or being understood by others)?

Each question has four response categories, which are read after each question. The response categories capture the full spectrum of functioning from mild to severe. 1. No, no difficulty 2. Yes, some difficulty 3. Yes, a lot of difficulty 4. Cannot do it at all.

Source: http://www.washingtongroup-disability.com/wp-content/uploads/2016/12/WG-Document-4-The-Washington-Group-Short-Set-on-Functioning-Question-Specifications.pdf

Household characteristics

209. There are three household characteristics classifications recommended by this Manual. They are:

• household composition (households with children under 15 and households without children under 15)

• household size (number of household members, including those outside any age scope imposed) and

• rural or urban households.

210. Household composition is relevant to measuring the digital divide in households with children – who are ‘digital natives’ in many countries (see Chapter 4 of ITU, 2013a).

211. Other characteristics may be derived from those of the household ‘head’ or ‘reference person’. These include sex, level of education, status in the labour force and occupation, which have a high explanatory role in socio-economic status, and therefore, may be useful for analysing the conditions for household access and of individual use of ICT within the household. In particular, female-headed households may be of policy interest. This is further discussed in para. 233 as a breakdown for producing gender-sensitive indicators.

212. Of particular interest, countries that have a notable rural/urban divide or a strong regional structure (e.g. federal states) may be interested in using a geographical classification. The international comparability of what is defined as rural or urban is limited, and countries have their own definitions based on the size, density or administrative status of localities.[21] Population size and density are usually proxies for the rural/urban classification (see Box 16). Recently, within the Global Strategy for Agricultural and Rural Statistics (GSARS, 2018), the UN Food and Agriculture Organization (FAO), in collaboration with other international organizations, has proposed guidelines for the definition of rural areas on the basis of density of population grids of 1 square km with less than 300 inhabitants per square km and outside an urban centre[22]. The complexity of geographic classifications is illustrated by the approach in India (see Box 17), which combines geographic, demographic, administrative and socio-economic criteria to define urban and rural areas. It is recommended that countries provide metadata on their definition of rural and urban areas when providing national data to ITU.

Box 16. Urban and rural areas

Because of national differences in the characteristics that distinguish urban from rural areas, the distinction between the urban and the rural population is not yet amenable to a single definition that would be applicable to all countries or, for the most part, even to the countries within a region. Where there are no regional recommendations on the matter, countries must establish their own definitions in accordance with their own needs.

The traditional distinction between urban and rural areas within a country has been based on the assumption that urban areas, no matter how they are defined, provide a different way of life and usually a higher standard of living than are found in rural areas. In many industrialized countries, this distinction has become blurred and the principal difference between urban and rural areas in terms of the circumstances of living tends to be a matter of the degree of concentration of population. Although the differences between urban and rural ways of life and standards of living remain significant in developing countries, rapid urbanization in these countries has created a great need for information related to different sizes of urban areas.

Hence, although the traditional urban-rural dichotomy is still needed, classification by size of locality can usefully supplement the dichotomy or even replace it where the major concern is with characteristics related only to density along the continuum from the most sparsely settled areas to the most densely built-up localities.

Density of settlement may not, however, be a sufficient criterion in many countries, particularly where there are large localities that are still characterized by a truly rural way of life. Such countries will find it necessary to use additional criteria in developing classifications that are more distinctive than a simple urban rural differentiation. Some of the additional criteria that may be useful are the percentage of the economically active population employed in agriculture, the general availability of electricity and/or piped water in living quarters and the ease of access to medical care, schools and recreation facilities. For certain countries where the facilities noted above are available in some areas that are still rural since agriculture is the predominant source of employment, it might be advisable to adopt different criteria in different parts of the country. Care must be taken, however, to ensure that the definition used does not become too complicated for application to the census and for comprehension by the users of the census results.

Source: https://unstats.un.org/unsd/demographic/sconcerns/densurb/densurbmethods.htm.

Box 17. India: definition of rural and urban areas

The Ministry of Statistics and Programme Implementation of India uses several demographic, administrative and socio-economic variables to define urban and rural areas.

Urban areas are defined as (a) all places with a Municipality, Corporation of Cantonment and places notified as town area, (b) all other places that satisfy the following criteria: a minimum population of 5000, at least 75 per cent of the male working population is non-agriculturist and (iii) a density of population of at least 400 per square kilometer. However, there are urban areas which do not possess all the above characteristics uniformly. Certain areas are treated as urban on the basis of their possessing distinct urban characteristics, overall importance and contribution to the urban economy of the region.

The rural sector covers areas other than the urban areas. The rural areas are composed of whole villages as well as part of villages.

The lists of census villages as published in the Primary Census Abstracts constitute the rural areas, and the lists of cities, towns, cantonments, non-municipal urban areas and notified areas constitute the urban areas.

Source: Ministry of Statistics and Programme Implementation. http://mospi.nic.in/Mospi_New/upload/nsso/concepts_golden.pdf?status=1&menu_id=49.

Cross-classification of data

213. In terms of output, many countries will want to cross-classify some of the above variables (for instance, age by sex[23]). This can produce information that is very useful for analytical purposes. However, it should be noted that cross-classified output is usually quite detailed and would therefore generally require higher sample sizes to support reliable estimates. Cross-classification is more likely to be feasible for higher-level indicators, such as individual use of the Internet. When disseminating the results of cross-classification of ICT indicators, sample sizes and accuracy measures should also be included (see Chapter 9 for more information on quality indicators).

214. The questionnaire used by ITU to collect data from countries (see extract in Annex 4) proposes the following cross-classifications:

• household composition by rural/urban,

• rural/urban by sex,

• age by sex,

• educational attainment by sex,

• status in the labour force by sex, and

• occupation by sex.

Other classificatory variables

215. For many countries, an income variable will also be of interest. Because both variables household income and individual income are problematic from a collection and international comparability viewpoint, they are not included as recommended classificatory variables for the ICT indicators. However, because of their policy relevance, they are described in this Manual. They are regarded as important classificatory variables because of the strong correlation between income and access to/use of ICT. To overcome the difficulty of measuring them, there is a diversity of approaches typically used by statistical agencies (household/individual, monthly/annual, gross/net etc.). Eurostat adopted a quartile approach for measuring household income from 2006. It entails either collecting income in ranges corresponding to quartiles (based on other survey data) or collecting income data in other ways and converting them to quartiles for output purposes. It is expected that the quartile approach would allow better comparability between countries choosing to use this classificatory variable (and within a country, possibly better comparability over time). ECLAC has adopted quintiles of household per capita income for surveys by countries of Latin America and the Caribbean (LAC).[24] As most of the LAC household surveys that include ICT questions have also collected information on household income, it is possible to compare differences in the domestic income divide.[25]

216. The type of dwelling may be an interesting classification variable for household data, as it is generally related to the geographical location, the existing urban infrastructure (e.g. electricity, mobile network coverage, Internet cable) and the level of income. Countries interested in such breakdown may use the types recorded in social surveys and the Population and Housing Census (UNSD, 2017).

Detailed information on core ICT household indicators

217. Table 6 presents the core indicators on access to, and use of, ICT by households and individuals along with standards that countries should apply to their compilation. The ICT concepts that are used in the definition of indicators are standardized with ITU definitions and UNSD recommendations.

218. The detailed information for each indicator included in Table 6 includes:

• definitions,

• clarifications and methodological issues,

• suggested model question/s,

• relevant disaggregations and classifications enabling the construction of sub-indicators,

• method of calculation, and

• notes on policy relevance, justifying the need for collecting statistics to compile the indicator, and help in understanding the importance of the indicator in identifying and monitoring aspects of the digital divide.

219. Considerations for questionnaire design for the collection of primary data to compile the indicators are discussed in Chapter 6, while the ITU questionnaire collecting aggregate data from countries is presented in Annex 4.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23

Indicator HH1: Proportion of households with a radio

Definitions:

This is the proportion of households that have a radio.

A radio is defined as a device capable of receiving broadcast radio signals, using common frequencies, such as FM, AM, LW and SW. A radio may be a stand-alone device, or it may be integrated with another device, such as an alarm clock, an audio player, a mobile telephone or a computer.

Clarifications and methodological issues:

‘Household with a radio’ means that a radio is generally available for use by all members of the household at any time, regardless of whether it is actually used. The radio may or may not be owned by the household, but should be considered a household asset.

Household is defined in Chapter 7.

With respect to the previous edition, more devices with radio functionality are now included in the definition. For time series compatibility, countries may want to split the question to include more than one response category.

Equipment should be in working condition at the time of the survey.

Model question:

Does this household have a radio? Yes/No

Disaggregation and classification:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Breakdown by household characteristics, such as household composition and size, and whether the household has access to electricity.

• Breakdown by characteristics of the head of the household/household reference person, such as sex, level of education, occupation or status in the labour force.

• Other breakdowns or classifications, where relevant variables or questions are used in the questionnaire, such as household income.

Calculation:

The number of in-scope households with a radio is calculated by aggregating the weighted responses (see Chapter 8).

The proportion of households with a radio is expressed as a percentage and is calculated by dividing the number of in-scope households with a radio by the total number of in-scope households, and then multiplying the result by 100.

• HH1% = [(number of in-scope households with a radio) / (total number of in-scope households)]*100

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH1: Proportion of households with a radio

Policy relevance:

Radio and television have been for a long time the most widespread ICTs in the world. They are generally reliable and are particularly useful for those parts of the world where Internet-based ICTs are not affordable, or available.

For many people, especially in developing economies, radios, while an ‘old ICT’, remain the only communication device. Their importance as an affordable and accessible mass medium for rural and marginalized urban communities has been highlighted. Radios are largely able to overcome literacy and language barriers and, according to UNESCO, radio is the medium that reaches the widest audience worldwide – radios remain “widely accessible, relatively cheap and very simple to use” and radio is “the medium that can carry any message to any place at any time – even without electricity. In situations of conflict and natural disaster, shortwave radio provides a lifeline of information that can save lives”.

Especially where Internet access is not available or affordable, radios play a very important role in delivering education or services in the areas of agriculture and health. Country projects that rely on the availability of radios in households will find the information provided by this indicator very useful.

However, as mobile phones are increasingly available, many users (especially the young) may prefer to watch or listen to content on their mobile devices.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH2: Proportion of households with a television

Definitions:

This is the proportion of households that have a television (TV).

A television (TV) is a device capable of receiving broadcast television signals, using popular access means such as over-the-air, cable and satellite. A television set is typically a stand-alone device, but it may also be integrated with another device, such as a computer or a mobile telephone.

Clarifications and methodological issues:

‘Household with a television’ means that a TV is generally available for use by all members of the household at any time, regardless of whether it is actually used. The TV may or may not be owned by the household, but should be considered a household asset.

Household is defined in Chapter 7.

With respect to the previous edition, TV functionality integrated in other devices is now included in the definition. For time series compatibility, countries may want to split the question to include more than one response category, such as ‘stand-alone TV’ or ‘TV embedded in another device’.

Equipment should be in working condition at the time of the survey.

Note: Indicator HH13 refers to the proportion of households with multichannel television.

Model question:

Does this household have a television? Yes/No

Disaggregation and classification:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Breakdown by household characteristics, such as household composition and size, and whether the household has access to electricity.

• Breakdown by characteristics of the head of the household/household reference person, such as sex, level of education, occupation or status in the labour force.

• Other breakdowns or classifications, where relevant variables or questions are used in the questionnaire, such as household income.

Breakdown by type of TV service is considered in a new indicator, HH13.

Calculation:

The number of in-scope households with a TV is calculated by aggregating the weighted responses (see Chapter 8).

The proportion of households with a TV is expressed as a percentage and is calculated by dividing the number of in-scope households with a TV by the total number of in-scope households, and then multiplying the result by 100.

• HH2% = [(number of in-scope households with a TV) / (total number of in-scope households)]*100

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH2: Proportion of households with a television

Policy relevance:

Televisions are sometimes referred to as an ‘old’ ICT but remain important as a source of information, and as a means of expressing national identity. Moreover, they are widely available globally. Some developed countries no longer collect this indicator, due to its almost universal prevalence, although it could start to decline in some advanced countries. This is a meaningful comparison, since both require access to electricity in order to function properly, and both represent an expense for household budgets in acquiring the equipment/service and/or for any subscription charges.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH3: Proportion of households with telephone

Definitions:

This is the proportion of households that have a telephone.

A fixed telephone line refers to a telephone line connecting a customer’s terminal equipment (e.g. telephone set, facsimile machine) to the public switched telephone network (PSTN) and which has a dedicated port on a telephone exchange. This term is synonymous with the terms main station or Direct Exchange Line (DEL) that are commonly used in telecommunication documents. It may not be the same as an access line or a subscription.

A mobile (cellular) telephone refers to a portable telephone subscribing to a public mobile telephone service using cellular technology, which provides access to the PSTN. This includes analogue and digital cellular systems and technologies such as IMT-2000 (3G) and IMT-Advanced. Users of both postpaid subscriptions and prepaid accounts are included.

A smart telephone (or smartphone) refers to a mobile handset that is used as the person’s primary phone device which has smart capabilities, including Internet-based services, and performs many of the functions of a computer, including having an operating system capable of downloading and running applications, also those created by third-party developers. Users of both postpaid subscriptions and prepaid accounts are included.

Clarifications and methodological issues:

‘Household with a mobile (or smart) telephone’ means that the mobile (or smart) telephone is generally available for use by all members of the household at any time, regardless of whether it is actually used. The mobile (or smart) telephone may or may not be owned by the household but should be considered a household asset (as has traditionally been the case for fixed telephone).

Household is defined in Chapter 7.

A household can be considered as having access to a mobile (or smart) telephone when it is able to receive and make calls from inside, or within the near vicinity of, the house (for example, the garden of the house). If all members (with the exception of young children) have a mobile phone, then it is considered that the household has a mobile (or smart) phone.

Equipment should be in working condition at the time of the survey.

Model questions:

For fixed telephone: Does this household have a fixed telephone line? Yes/No

For mobile telephone: Does this household have a mobile telephone? Yes/No

If YES: does this household have a smartphone? Yes/No

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Breakdown by household characteristics, such as household composition and size, and whether the household has access to electricity.

• Breakdown by characteristics of the head of the household/household reference person, such as sex, level of education, occupation or status in the labour force.

• Disaggregation by type of mobile telephone (smartphone, other mobile phone).

• Other breakdowns or classifications, where relevant variables or questions are used in the questionnaire, such as household income.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH3: Proportion of households with telephone

Calculation:

The following sub-indicators can be calculated:

• Proportion of households with a fixed telephone (regardless of whether they have a mobile telephone)

• Proportion of households with a mobile telephone (regardless of whether they have a fixed telephone)

• Proportion of households with any telephone (fixed and/or mobile)

• Proportion of households with a smartphone

• Proportion of households with a fixed telephone only

• Proportion of households with a mobile telephone only

• Proportion of households with both fixed and mobile telephone.

The number of in-scope households with a given type of telephone is calculated by aggregating the weighted responses for each case (see Chapter 8). Separate variables in the micro database need to be created for calculating each sub-indicator using the two questions included in the survey questionnaire.

Sub-indicators are calculated as follows:

• The proportion of households with a fixed telephone is calculated by dividing the number of in-scope households with a fixed telephone by the total number of in-scope households. The result is then multiplied by 100 to be expressed as a percentage.

• The proportion of households with a mobile telephone is calculated by dividing the number of in-scope households with a mobile telephone by the total number of in-scope households. The result is then multiplied by 100 to be expressed as a percentage.

• The proportion of households with any telephone is calculated by dividing the number of in-scope households with access to any telephone (fixed and/or mobile) by the total number of in-scope households. The result is then multiplied by 100 to be expressed as a percentage.

• The proportion of households with fixed telephone only is calculated by dividing the number of in-scope households with a fixed telephone only by the total number of in-scope households. The result is then multiplied by 100 to be expressed as a percentage.

• The proportion of households with mobile telephone only is calculated by dividing the number of in-scope households with a mobile telephone only by the total number of in-scope households. The result is then multiplied by 100 to be expressed as a percentage.

• The proportion of households with both fixed and mobile telephone is calculated by dividing the number of in-scope households with both a fixed and mobile telephone by the total number of in-scope households. The result is then multiplied by 100 to be expressed as a percentage.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH3: Proportion of households with telephone

The sub-indicators are expressed algebraically as follows:

• HH3%fixed = [(number of in-scope households with a fixed telephone) / (total number of in-scope households)]*100

• HH3%mobile = [(number of in-scope households with a mobile telephone) / (total number of in-scope households)]*100

• HH3%smartphone = [(number of in-scope households with a smartphone) / (total number of in-scope households)]*100

• HH3%any = [(number of in-scope households with a fixed and/or mobile telephone) / (total number of in-scope households)]*100

• HH3%fixed only = [(number of in-scope households with a fixed telephone only) / (total number of in-scope households)]*100HH3%mobile only = [(number of in-scope households with a mobile telephone only) / (total number of in-scope households)]*100

• HH3%both fixed and mobile = [(number of in-scope households with both fixed and mobile telephone) / (total number of in-scope households)]*100

Policy relevance:

A telephone provides shared, two-way communication services to a household and therefore allows members of a household to be reached, or to make a call, without having to travel any distance. It is an important basic device to stay in contact with people and to make calls in the case of an emergency. Basic telephone services remain relatively affordable and are usually free for incoming calls, which makes them relatively accessible to low income households. At the same time, almost anyone can use a telephone since it does not require specific literacy or ICT skills.

Mobile cellular telephones are becoming the predominant method of communication in many countries. Although fixed telephone lines have now been surpassed by mobile telephony globally, they are still an important affordable communication medium. Furthermore, they provide a basis for Internet access in most economies, whether through dial-up, Integrated Services Digital Networks (ISDN), or Digital Subscriber Line (DSL) services. Smartphones allow for access to contents and services through the Internet, and high computational capacities.

HH3 will help monitor trends in fixed and mobile telephony over time. In developing economies, while fixed-telephone networks are often limited to urban areas, mobile-cellular network coverage has reached about 91 per cent of their population, effectively overcoming a major infrastructure barrier.

Many countries have identified specific goals for the delivery of universal service, which refers to the availability of basic telecommunication services for every household. To achieve universal service, many governments have set up universal service funds and identified universal service obligations, which oblige operators to provide basic services (often at set or limited prices) to a certain percentage of households, and in particular those in rural and remote areas, or to households with low income levels. Examples include the 2002 Universal Service Obligation in India[26] and the 1997 United States Universal Service Fund.[27]

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH4: Proportion of households with a computer

Definitions:

This is the proportion of households that have a computer.

A computer refers to a desktop computer, a laptop (portable) computer or a tablet (or similar handheld computer).

• Desktop: a computer that usually remains fixed in one place; normally the user is placed in front of it, behind the keyboard.

• Laptop (portable) computer: a computer that is small enough to carry and usually enables the same tasks as a desktop computer; it includes notebooks and netbooks but does not include tablets and similar handheld computers.

• Tablet (or similar handheld computer): a tablet is a computer that is integrated into a flat touch screen, operated by touching the screen rather than (or as well as) using a physical keyboard.

It does not include equipment with some embedded computing abilities, such as smart TV sets, and devices with telephony as their primary function, such as smartphones.

Clarifications and methodological issues:

HH4 has been revised since the last edition of this Manual and now includes tablets (and similar), which are more recent devices.

‘Household with a computer’ means that the computer is generally available for use by all members of the household at any time, regardless of whether it is actually used. The computer may or may not be owned by the household, but should be considered a household asset.

Household is defined in Chapter 7.

For time series compatibility, countries may want to split the question to include more than one response category, such as desktop computer, laptop (portable) computer and tablet (or similar handheld computer).

Equipment should be in working condition at the time of the survey.

Model question:

Does this household have a computer (desktop, laptop, tablet or similar)? Yes/No

Note that if the question is split by type of computer, respondents should select all that apply.

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Breakdown by household characteristics, such as household composition and size, and whether the household has access to electricity.

• Breakdown by characteristics of the head of the household/household reference person, such as sex, level of education, occupation or status in the labour force.

• Disaggregation by type of computer: desktop, laptop, tablet or similar.

• Other breakdowns or classifications, where relevant variables or questions are used in the questionnaire, such as household income.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH4: Proportion of households with a computer

Calculation:

The number of in-scope households with a computer, or a given type of computer, is calculated by aggregating the weighted responses (see Chapter 8).

The proportion of households with a computer is expressed as a percentage and is calculated by dividing the number of in-scope households with a computer (or a given type of computer) by the total number of in-scope households and then multiplying the result by 100.

• HH4% = [(number of in-scope households with a computer) / (total number of in-scope households)]*100

• HH4%desktop = [(number of in-scope households with a desktop computer) / (total number of in-scope households)]*100

• HH4%laptop = [(number of in-scope households with a laptop computer) / (total number of in-scope households)]*100

• HH4%tablet = [(number of in-scope households with a tablet or similar) / (total number of in-scope households)]*100

Alternatively, percentages can be also presented for each type of computer as the percentage of in-scope households with a computer by type of computer, that is, the denominator is the total number of households with a computer of any type.

Policy relevance:

The availability of a computer at home allows people to carry out basic computing tasks, and store and process information, including word processing. The availability of a computer at home is also important for household members to improve their ICT skills.

A computer is also an important device to access and use the Internet, in particular advanced applications and services (which are usually more difficult to use on a smartphone). Moreover, there is growing evidence of the strong association between school performance and home access and use of computers.[28]

Information on the proportion of households with a computer is critical in identifying households without a computer. Classificatory variables – such as ‘household income’, ‘rural/urban’ and other geographical classifications – can provide data to inform policies targeting households without a computer (e.g. computer subsidy schemes).[29] Such detailed information at household level is unavailable through other data sources.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH5: Proportion of individuals using a computer

Definitions:

This is the proportion of individuals who used a computer from any location in the last three months.

A computer refers to a desktop computer, a laptop (portable) computer or a tablet (or similar handheld computer).

• Desktop: a computer that usually remains fixed in one place; normally the user is placed in front of it, behind the keyboard.

• Laptop (portable) computer: a computer that is small enough to carry and usually enables the same tasks as a desktop computer; it includes notebooks and netbooks but does not include tablets and similar handheld computers.

• Tablet (or similar handheld computer): a tablet is a computer that is integrated into a flat touch screen, operated by touching the screen rather than (or as well as) using a physical keyboard.

It does not include equipment with some embedded computing abilities, such as smart TV sets, and devices with telephony as their primary function, such as smartphones.

Clarifications and methodological issues:

HH5 has been revised since the last edition of this Manual and now includes tablets (and similar), which are more recent devices.

This indicator refers to use of a computer by individual household members.

The suggested reference period is the last three months. Country practices vary, but ideally, reference periods should be aligned in order to obtain comparable data. Countries changing their reference period may wish to split the reference period in order to obtain comparable time series.

Individual is discussed in Chapter 7.

As for HH4, for time series compatibility, countries may want to split the question to include more than one response category, such as desktop computer, laptop (portable) computer and tablet (or similar handheld computer).

Model question:

Have you used a computer (desktop, laptop, tablet or similar) from any location in the last three months? Yes/No

Note that if the question is split by type of computer, respondents should select all that apply.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH5: Proportion of individuals using a computer

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Classification by sex.

• Classification by age: countries can use the following age groups for reporting on individuals’ age: under 5; 5–9; 10–14; 15–24; 25–34; 35–44; 45–54; 55–64; 65–74 and 75 and over.

• Classification by highest education level attained: countries can use the ISCED 2011 classification for reporting on individuals’ level of education: primary education or lower (ISCED 0,1); lower secondary education (ISCED 2); upper secondary or post-secondary non-tertiary education (ISCED 3, 4); tertiary education (ISCED 5, 6); and post-tertiary education (ISCED 7, 8).

• Classification by labour force status: countries can use the following categories for reporting on individuals’ labour force status: employee; self-employed; workers not classifiable by status, unemployed; and outside the labour force.

• Classification by occupation: countries should use the ISCO 2008 categories where these are in use (if not, use ISCO-88 per Table 5 earlier in this chapter) for reporting on individuals’ occupation: managers; professionals; technicians and associate professionals; clerical support workers; service and sales workers; skilled agricultural, forestry and fishery workers; craft and related trades workers; plant and machine operators, and assemblers; elementary occupations; and armed forces occupations. (However, note that armed forces personnel may be out of scope of household surveys.)

• By type of computer: desktop, laptop, tablet or similar.

• Other breakdowns or classifications, where relevant variables or questions are used in the questionnaire, such as individual income.

Calculation:

The number of in-scope individuals using a computer is calculated by aggregating the weighted responses (see Chapter 8).

The proportion of individuals using a computer is expressed as a percentage and is calculated by dividing the total number of in-scope individuals using a computer by the total number of in-scope individuals, and then multiplying the result by 100. The indicator can be split by type of computer, similarly to HH4.

• HH5% = [(number of in-scope individuals using a computer) / (total number of in-scope individuals)]*100

Policy relevance:

Using a computer is an increasingly important life skill. It allows people to carry out basic computing tasks, and store and process information, including word processing. Using a computer also allows people to improve their ICT skills and become more familiar with advanced computing functionalities.

A computer is also an important means of using the Internet, in particular advanced applications and services (which are usually more difficult to use on a smartphone).

Classificatory variables for individuals using a computer – such as age, sex, education level or labour force status – can help identify digital divides, for example, a gender gap or a socio-economic divide. This information can feed policy analysis and contribute to the design of targeted policies to overcome digital divides.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH6: Proportion of households with Internet

Definitions:

This is the proportion of households with Internet access at home.

The Internet is a worldwide public computer network. It provides access to a number of communication services including the World Wide Web and carries e-mail, news, entertainment and data files, irrespective of the device used (not assumed to be only via a computer − it may also be by mobile telephone, tablet, PDA, games machine, digital TV etc.). Access can be via a fixed or mobile network.

Clarifications and methodological issues:

‘Household with Internet access’ means that the Internet is generally available for use by all members of the household at any time, regardless of whether it is actually used. The connection and devices may or may not be owned by the household but should be considered household assets.

If one member of the household has a mobile phone with connection to the Internet and makes it available for all members, then it should be considered that the household has access to the Internet.

Household is discussed in Chapter 7.

An Internet connection in the household should be working at the time of the survey.

Model question:

Does this household have Internet? Yes/No

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Breakdown by household characteristics, such as household composition and size, and whether the household has access to electricity.

• Breakdown by characteristics of the head of the household/household reference person, such as sex, level of education, occupation or status in the labour force.

• Other breakdowns or classifications, where relevant variables or questions are used in the questionnaire, such as household income.

• Breakdown by type of connection is considered a separate indicator (HH11: Proportion of households with Internet, by type of service)

Calculation:

The number of in-scope households with Internet is calculated by aggregating the weighted responses (see Chapter 8).

The proportion of households with Internet is expressed as a percentage and is calculated by dividing the number of in-scope households with Internet by the total number of in-scope households, and then multiplying the result by 100.

• HH6% = [(number of in-scope households with Internet) / (total number of in-scope households)]*100

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH6: Proportion of households with Internet

Policy relevance:

Having Internet at home is a key indicator of information society progress because Internet access at home allows citizens to benefit from the availability of ICT services and applications on the Internet in the privacy and security of their home. For instance, for the delivery of e-government services it is critical knowing whether citizens have Internet access at home.

Home Internet access has some advantages over Internet access at other locations because users are free to choose the Internet activity they want to carry out. Moreover, home Internet access is usually not limited in terms of time, or purpose, and is potentially available to all members of the household, even those lacking ICT skills (because other household members can help them, or carry out certain activities for them, for example, carrying out research or finding information).

A number of studies, including the OECD’s PISA studies (for example, OECD (2010), highlight the positive relationship between ICT access and use at home and educational performance, with those that have Internet access at home achieving higher grades and faring better academically.

ITU estimates that almost 60 per cent of households have Internet access at home in 2018, up from less than 20 per cent in 2005. In developing countries, almost half of all households have Internet access at home, a considerable increase compared with 8.4 per cent in 2005. A growing number of countries include Internet access (usually broadband access) as a target in their national ICT or broadband plans.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH7: Proportion of individuals using the Internet

Definitions:

This is the proportion of individuals who used the Internet from any location in the last three months.

The Internet is a worldwide public computer network. It provides access to a number of communication services including the World Wide Web and carries e-mail, news, entertainment and data files, irrespective of the device used (not assumed to be only via a computer − it may also be by mobile telephone, tablet, PDA, games machine, digital TV etc.). Access can be via a fixed or mobile network.

Clarifications and methodological issues:

This indicator refers to use of the Internet by individual household members.

The suggested reference period is the last three months. Country practices vary, but ideally, reference periods should be aligned in order to obtain comparable data. Countries changing their reference period may wish to split the reference period in order to obtain comparable time series.

It may happen that an individual is not aware of the use of Internet when carrying out certain activities (e.g. posting on social media).This is especially the case for “zero-rated services”, i.e. Internet services not paid by the user. Interviewers may probe the response to the question on use of Internet after the response to questions regarding activities.

Individual is discussed in Chapter 7.

Model questions:

Have you used the Internet from any location in the last three months? Yes/No

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Classification by sex.

• Classification by age: countries can use the following age groups for reporting on individuals’ age: under 5; 5–9; 10–14; 15–24; 25–34; 35–44; 45–54; 55–64; 65–74 and 75 and over.

• Classification by highest education level attained: countries can use the ISCED 2011 classification for reporting on individuals’ level of education: primary education or lower (ISCED 0,1); lower secondary education (ISCED 2); upper secondary or post-secondary non-tertiary education (ISCED 3, 4); tertiary education (ISCED 5, 6); and post-tertiary education (ISCED 7, 8).

• Classification by labour force status: countries can use the following categories for reporting on individuals’ labour force status: employee; self-employed (includes employers, own account workers, members of producers’ cooperatives and contributing family workers); workers not classifiable by status, unemployed; and outside the labour force.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH7: Proportion of individuals using the Internet

• Classification by occupation: countries should use the ISCO 2008 categories where these are in use (if not, use ISCO-88 per Table 5 earlier in this chapter) for reporting on individuals’ occupation: managers; professionals; technicians and associate professionals; clerical support workers; service and sales workers; skilled agricultural, forestry and fishery workers; craft and related trades workers; plant and machine operators, and assemblers; elementary occupations; and armed forces occupations (noting that armed forces personnel may be out of scope).

• Other breakdowns or classifications, where relevant variables or questions are used in the questionnaire, such as individual income.

Calculation:

The number of in-scope individuals using the Internet is calculated by aggregating the weighted responses (see Chapter 8).

The proportion of individuals using the Internet is expressed as a percentage and is calculated by dividing the total number of in-scope individuals using the Internet by the total number of in-scope individuals, and then multiplying the result by 100.

• HH7% = [(number of in-scope individuals using the Internet) / (total number of in-scope individuals)]*100

Policy relevance:

Internet user uptake is a key indicator tracked by policy-makers and analysts as an indication of a country’s progress towards becoming an information society. Classificatory variables for individuals using the Internet – such as age, sex, education level or labour force status – can help identify digital divides in individuals using the Internet. This information can contribute to the design of targeted policies to overcome those divides.

According to ITU, in developed countries, slow and steady growth saw the percentage of population using the Internet increase from 51.3 per cent in 2005 to 80.9 per cent in 2018. In developing countries Internet use has increased from 7.7 per cent in 2005 to 45.3 per cent at the end of 2018.

HH7 is one of the SDG indicators, used to monitor SDG 17 (Indicator 17.8.1).

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH8: Proportion of individuals using the Internet, by location

Definitions:

This is the proportion of individuals who used the Internet from specified locations in the last three months.

The Internet is a worldwide public computer network. It provides access to a number of communication services including the World Wide Web and carries e-mail, news, entertainment and data files, irrespective of the device used (not assumed to be only via a computer − it may also be by mobile telephone, tablet, PDA, games machine, digital TV etc.). Access can be via a fixed or mobile network, including wireless access at a WiFi ‘hotspot’.

Access via a mobile device should be classified to the appropriate location or to ‘while commuting, in transport or walking’, that is, while mobile.

Locations of Internet use are defined as follows:

• Home

• Work: where a person’s workplace is located at his/her home, then he/she would answer yes to the home category only

• Place of education: applies only to students. Teachers and others who work at a place of education would report ‘work’ as the place of Internet use. Where a place of education is also made available as a location for general community Internet use, such use should be reported in the Community Internet access facility category

• Another person’s home: the home of a friend, relative or neighbour

• Facility open to the public: use at a facility open to the public regardless of payment, type of connection or nature of the facility. Common examples are libraries, telecenters, cafes, restaurants, and shopping malls

o Of which: Community Internet access facility: Internet use at community facilities such as public libraries, publicly provided Internet kiosks, non-commercial telecentres, digital community centres, post offices, other government agencies; access is typically free and available to the general public

• While commuting, in transport or walking: use of the Internet while moving between places, commuting, or on the street. The emphasis is on the act of moving, not on the device being used. Beyond mobile networks, it might also cover connection on public transportation systems and public Wi-Fi.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH8: Proportion of individuals using the Internet, by location

Clarifications and methodological issues:

Individuals should be asked about all locations of Internet use. The survey question used by countries should specify multiple responses and should not, for example, ask about the most frequently used location(s).

In the 2014 edition of this Manual, there was a distinction between community and commercial facilities. However, in many cases it is impossible for a survey respondent to understand what a public facility is and what is not, therefore this distinction is no longer present. Similarly, payment for a connection is also not considered to be a decisive factor for allocation to a category.

Community access is part of the category “facilities open to the public”. However, due to its high policy relevance it has been included as a subcategory for international comparison.

The category “While commuting, in transport or walking” is a clarification from the previous item “In mobility” to stress the fact that the focus is not on the device, but on the aspect of using the Internet while in mobility.

It is also important to understand that the provider of the Internet connection is not important, only the location is. For example, a person using their own data bundle at work, should still report “at work” as the appropriate category.

At international level, reporting should be done according to these categories. At national level, countries might find it relevant to further split some of the categories, considering local needs from data users (example: public squares, libraries, public transportation systems, etc.; or paying vs. free access for each category). Other alternatives would be adapting to local examples or using a follow-up question.

The suggested reference period is the last three months. Country practices vary, but ideally, reference periods should be aligned in order to obtain comparable data. Countries changing their reference period may wish to split the reference period in order to obtain comparable time series.

Individual is discussed in Chapter 7.

Model question:

Where did you use the Internet in the last three months? Respondents should select all locations (see above) that apply.

Countries may also ask about location of use as a series of yes/no questions, each referring to one location of use.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH8: Proportion of individuals using the Internet, by location

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Classification by sex.

• Classification by age: countries can use the following age groups for reporting on individuals’ age: under 5; 5–9; 10–14; 15–24; 25–34; 35–44; 45–54; 55–64; 65–74 and 75 and over.

• Classification by highest education level attained: countries can use the ISCED 2011 classification for reporting on individuals’ level of education: primary education or lower (ISCED 0, 1); lower secondary education (ISCED 2); upper secondary or post-secondary non-tertiary education (ISCED 3, 4); tertiary education (ISCED 5, 6); and post-tertiary education (ISCED 7, 8).

• Classification by labour force status: countries can use the following categories for reporting on individuals’ labour force status: employee; self-employed (includes employers, own account workers, members of producers’ cooperatives and contributing family workers); workers not classifiable by status, unemployed; and outside the labour force.

• Classification by occupation: countries should use the ISCO 2008 categories where these are in use (if not, use ISCO-88 per Table 5 earlier in this chapter) for reporting on individuals’ occupation: managers; professionals; technicians and associate professionals; clerical support workers; service and sales workers; skilled agricultural, forestry and fishery workers; craft and related trades workers; plant and machine operators, and assemblers; elementary occupations; and armed forces occupations (noting that armed forces personnel may be out of scope).

• Other breakdowns or classifications, where relevant variables or questions are used in the questionnaire, such as individual income.

Calculation:

The number of in-scope individuals using the Internet from a specific location is calculated by aggregating the weighted responses for each location (see Chapter 8).

Proportions are expressed as percentages and are calculated by dividing the number of in-scope individuals using the Internet from a specific location by either the total number of in-scope individuals using the Internet (see HH7) or by the total number of in-scope individuals, and then multiplying the result by 100.

Examples:

The percentage of Internet users using the Internet at home is calculated as:

• HH8%Internet users home = [(number of in-scope individuals using the Internet at home) / (total number of in-scope individuals using the Internet)]*100

• The percentage of in-scope individuals using the Internet at home is calculated as:

• HH8%Individuals home = [(number of in-scope individuals using the Internet at home) / (total number of in-scope individuals)]*100

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH8: Proportion of individuals using the Internet, by location

Policy relevance:

HH8 can provide important information for policy-makers because it helps them to identify possible ways of increasing Internet access. A country where many people rely on public Internet access facilities could, for example, ensure that these are widely available and affordable. Conversely, if public Internet facilities exist in a country but are not much used, it could mean that there are constraints on use or that users have more convenient options. Countries may wish to track changes in location of Internet use, such as from other locations to home Internet access. Home Internet access is likely to be preferred where it is possible (e.g. it is private, safer and may be less limited in terms of purpose or time).

Classificatory variables can provide useful information on differences in the location of use of the Internet by men/women, children/adults, employed/unemployed, etc. These data may inform targeted policies to reduce digital divides within a country, such as a gender gap or a socio-economic divide in individuals using the Internet.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH9: Proportion of individuals using the Internet, by type of activity

Definitions:

This is the proportion of individuals who undertook one or more activities using the Internet for private (defined as non-work) purposes from any location in the last three months. Internet activities are classified in groups of similar activities, and are defined as follows:

Access to information

• Getting information about goods or services

• Seeking health- related information (on injury, disease, nutrition etc.).

• Getting information from general government organizations

• Using services related to travel or travel-related accommodation

• Downloading software or applications (includes patches and upgrades, either paid or free of charge)*

• Reading or downloading newspapers, magazines or electronic books in a digital format

Communication, civic participation and collaboration

• Sending or receiving e-mail*

• Making calls (telephoning over the Internet/VoIP using Skype, WhatsApp, Viber, iTalk, etc.; includes video calls via webcam)

• Participating in social networks (creating user profile, posting messages or other contributions to Facebook, Twitter, Instagram, Snapchat, etc.)

• Making an appointment with a health practitioner via the Internet (i.e. website, app, software)

• Interacting with general government organizations (downloading/requesting forms, completing/lodging forms online, making online payments and purchasing from government organizations etc.)

General government organizations should be consistent with the SNA93 (2008 revision) concept of general government. According to the SNA “… the principal functions of government are to assume responsibility for the provision of goods and services to the community or to individual households and to finance their provision out of taxation or other incomes; to redistribute income and wealth by means of transfers; and to engage in non-market production.” (General) government organizations include central, state and local government units.

• Taking part in consultations or voting via the Internet to define civic or political issues (urban planning, signing a petition etc.)

• Accessing or posting opinions via any device on chat sites, blogs, newsgroups or online discussions (e.g. on civic or political issues, general interest topics) that may be created by any individual or organization

Electronic commerce, trade, and transactions

• Purchasing or ordering goods or services (purchase orders placed via the Internet whether or not payment was made online; excludes orders that were cancelled or not completed; includes purchasing of products such as music, travel and accommodation via the Internet)

• Selling goods or services (via eBay, Mercado libre, Facebook, Amazon, Alibaba, etc.)

• Internet banking (includes electronic transactions with a bank for payment, transfers, etc. such as M-Pesa, or for looking up account information; excludes electronic transactions via the Internet for other types of financial services such as share purchases, financial services and insurance)

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH9: Proportion of individuals using the Internet, by type of activity

Learning

• Doing an online course (in any subject)

• Consulting wikis (Wikipedia etc.), online encyclopedias or other websites for formal or informal learning purposes

Professional life

• Looking for a job or sending/submitting a job application (includes searching specific websites for a job; sending/submitting an application online)

• Participating in professional networks (professional networks are also seen in the broader context of social networking and have the same requirement of profile creation, contributing through messaging or chat, or uploading text or audio-visual content files; examples of professional or business networks are LinkedIn, Xing, Bark, Opportunity and Jobcase)

Entertainment, digital content consumption

• Listening to web radio (either paid or free of charge)

• Watching web television (either paid or free of charge)

• Streaming or downloading images, movies, videos or music; playing or downloading games (either paid or free of charge)

Digital content creation

• Uploading self/user-created content to a website to be shared (text, images, photos, videos, music, software, etc.)

• Using storage space on the Internet to save documents, pictures, music, video or other files (e.g. Google Drive, Dropbox, Windows Skydrive, iCloud, Amazon Cloud Drive)

• Using software run over the Internet for editing text documents, spreadsheets or presentations

Clarifications and methodological issues:

* The categories marked with * overlap with categories asked in HH15 (Individuals with ICT skills, by type of skills) and can therefore be removed from HH9 if HH15 forms part of the survey.

Note that these activities are restricted to private purposes and therefore exclude activities such as purchasing over the Internet undertaken as part of a person’s job or teleworking.

Individuals should be asked about all Internet activities. The survey question used by countries should specify multiple responses and should not, for example, ask about the most frequent activities undertaken. Activities are not mutually exclusive, that is, there is overlap between some categories.

When collecting data on these activities, some of them may need rewording and examples provided according to the local context.

The suggested reference period is the last three months. Country practices vary, but ideally, reference periods should be aligned in order to obtain comparable data

Individual is discussed in Chapter 7.

Countries may wish to extend activities to some non-private purposes, such as teleworking (working from one’s home either as an employee or as a self-employed person).

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH9: Proportion of individuals using the Internet, by type of activity

Model question:

For which of the following activities did you use the Internet for private purposes (from any location) in the last three months?’

Respondents should select all activities (see above) that apply.

Countries may ask about activities as a series of yes/no questions, each referring to one activity.

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Classification by sex.

• Classification by age: countries can use the following age groups for reporting on individuals’ age: under 5; 5–9; 10–14; 15–24; 25–34; 35–44; 45–54; 55–64; 65–74 and 75 and over.

• Classification by highest education level attained: countries can use the ISCED 2011 classification for reporting on individuals’ level of education: primary education or lower (ISCED 0, 1); lower secondary education (ISCED 2); upper secondary or post-secondary non-tertiary education (ISCED 3, 4); tertiary education (ISCED 5, 6); and post-tertiary education (ISCED 7, 8).

• Classification by labour force status: countries can use the following categories for reporting on individuals’ labour force status: employee; self-employed (includes employers, own account workers, members of producers’ cooperatives and contributing family workers); workers not classifiable by status, unemployed; and outside the labour force.

• Classification by occupation: countries should use the ISCO 2008 categories where these are in use (if not, use ISCO-88 per Table 5 earlier in this chapter) for reporting on individuals’ occupation: managers; professionals; technicians and associate professionals; clerical support workers; service and sales workers; skilled agricultural, forestry and fishery workers; craft and related trades workers; plant and machine operators, and assemblers; elementary occupations; and armed forces occupations (noting that armed forces personnel may be out of scope).

• Other breakdowns or classifications, where relevant variables or questions are used in the questionnaire, such as individual income.

Calculation:

The number of in-scope individuals using the Internet for a specific activity is calculated by aggregating the weighted responses for each activity (see Chapter 8).

Proportions are expressed as percentages and are calculated by dividing the number of in-scope individuals using the Internet for a specific activity by either the total number of in-scope individuals using the Internet (see HH7) or by the total number of in-scope individuals, and then multiplying the result by 100.

Examples:

The percentage of Internet users who undertook Internet banking is calculated as:

• HH9%Internet users banking = [(number of in-scope individuals who used the Internet for banking) / (total number of in-scope individuals who used the Internet)]*100

The percentage of in-scope individuals using the Internet for Internet banking is calculated as:

• HH9%Individuals banking = [(number of in-scope individuals who used the Internet for banking) / (total number of in-scope individuals)]*100

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH9: Proportion of individuals using the Internet, by type of activity

Policy relevance:

The indicator on the types of Internet activities undertaken by individuals is a key indicator in tracking the information society because it allows policy-makers, businesses and analysts to understand how Internet access is changing the way that people do business, learn, buy, communicate and interact with others, including governments.

This indicator is important for the formulation of policies since it is an indication of the demand for certain online services and applications. For example, it helps government organizations understand the demand for e-government information and services. Questions on e-goods and services will help businesses and others understand the degree to which users are adopting e-commerce, etc.

HH9 also provides useful information on the sophistication of Internet use and the degree of ICT skills that Internet users have, in conjunction with HH15. Classificatory variables can provide further information on differences in the Internet activities undertaken by men/women, children/adults, employed/unemployed, etc. This information may be used, for instance, to design policies to promote e-commerce and extend e-government services to particular target groups, in order to ensure transition towards an inclusive information society.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH10: Proportion of individuals using a mobile cellular telephone

Definitions:

This is the proportion of individuals who used a mobile telephone in the last three months.

A mobile (cellular) telephone refers to a portable telephone subscribing to a public mobile telephone service using cellular technology, which provides access to the PSTN. This includes analogue and digital cellular systems and technologies such as IMT-2000 (3G) and IMT-Advanced. Users of both postpaid subscriptions and prepaid accounts are included.

A smart telephone refers to a mobile handset that is used as the person’s primary phone device which has smart capabilities, including Internet-based services, and performs many of the functions of a computer, including having an operating system capable of downloading and running applications, including those created by third-party developers. Users of both postpaid subscriptions and prepaid accounts are included

Clarifications and methodological issues:

This indicator refers to the use of a mobile (or smart) telephone by individual household members for communication (therefore, not only as a clock, playing games or listening to music).

Use of a mobile (or smart) telephone does not necessarily mean that the telephone is owned or paid for by the individual but should be reasonably available through work, a friend or family member, etc. It excludes occasional use, for instance, borrowing a mobile (or smart) telephone to make a call.

The suggested reference period is the last three months. Country practices vary, but ideally, reference periods should be aligned in order to obtain comparable data. Countries changing their reference period may wish to split the reference period in order to obtain comparable time series.

Individual is discussed in Chapter 7.

Model questions:

Have you used a mobile telephone in the last three months? Yes/No

Have you used a smartphone in the last three months? Yes/No

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Classification by sex.

• Classification by age: countries can use the following age groups for reporting on individuals’ age: under 5; 5–9; 10–14; 15–24; 25–34; 35–44; 45–54; 55–64; 65–74 and 75 and over.

• Classification by highest education level attained: countries can use the ISCED 2011 classification for reporting on individuals’ level of education: primary education or lower (ISCED 0,1); lower secondary education (ISCED 2); upper secondary or post-secondary non-tertiary education (ISCED 3, 4); tertiary education (ISCED 5, 6); and post-tertiary education (ISCED 7, 8).

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH10: Proportion of individuals using a mobile cellular telephone

• Classification by labour force status: countries can use the following categories for reporting on individuals’ labour force status: employee; self-employed (includes employers, own account workers, members of producers’ cooperatives and contributing family workers); workers not classifiable by status, unemployed; and outside the labour force.

• Classification by occupation: countries should use the ISCO 2008 categories where these are in use (if not, use ISCO-88 per Table 5 earlier in this chapter) for reporting on individuals’ occupation: managers; professionals; technicians and associate professionals; clerical support workers; service and sales workers; skilled agricultural, forestry and fishery workers; craft and related trades workers; plant and machine operators, and assemblers; elementary occupations; and armed forces occupations (noting that armed forces personnel may be out of scope).

• By type of mobile telephone, for example, to distinguish smartphone use.

• Other breakdowns or classifications, where relevant variables or questions are used in the questionnaire, such as individual income.

Calculation:

The number of in-scope individuals using a mobile cellular telephone is calculated by aggregating the weighted responses (see Chapter 8).

The proportion of individuals using a mobile telephone is expressed as a percentage and is calculated by dividing the total number of in-scope individuals using a mobile telephone by the total number of in-scope individuals, and then multiplying the result by 100.

• HH10% = [(number of in-scope individuals using a mobile cellular telephone) / (total number of in-scope individuals)]*100

Policy relevance:

The proportion of individuals who use a mobile telephone is an important indicator to measure the uptake of mobile cellular technology. The indicator complements the widely available indicator on the number of mobile cellular subscriptions and does not have the same shortcomings as the latter: HH10 provides information on the number and type of people who use a mobile telephone, whereas the mobile cellular subscription indicator measures the number of subscriptions.

HH10 is particularly useful for developing economies, where ICT access and use remains limited.

Classificatory variables for individuals using a mobile telephone – such as age, sex, education level or labour force status – can help identify digital divides, for example, a gender gap or a socio-economic divide. This information can feed policy analysis and contribute to the design of targeted policies to overcome those divides.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH11: Proportion of households with Internet, by type of service

Definitions:

This is the proportion of households with access to the Internet, by type of service.

The Internet is a worldwide public computer network. It provides access to a number of communication services including the World Wide Web and carries e-mail, news, entertainment and data files, irrespective of the device used (not assumed to be only via a computer − it may also be by mobile telephone, tablet, PDA, games machine, digital TV etc.). Access can be via a fixed or mobile network.

The broad types of Internet services to be identified are the following:

• Fixed narrowband network: includes analogue modem (dial-up via standard telephone line), ISDN (Integrated Services Digital Network), DSL (Digital Subscriber Line) at advertised download speeds below 256 kbit/s, and other forms of access with an advertised download speed of less than 256 kbit/s

• Fixed broadband network: refers to technologies at advertised download speeds of at least 256 kbit/s, such as DSL, cable modem, high speed leased lines, fibre-to-the-home/building, powerline and other fixed broadband

• Terrestrial fixed broadband network: refers to technologies at advertised download speeds of at least 256 kbit/s, such as WiMAX, fixed CDMA

• Satellite broadband network (via a satellite connection), at advertised download speeds of at least 256 kbit/s

• Mobile broadband network (at least 3G, e.g. UMTS) via a handset

• Mobile broadband network (at least 3G, e.g. UMTS) via a card (e.g. integrated SIM card in a computer) or USB modem

Clarifications and methodological issues:

As households can use more than one type of access service, multiple responses are possible.

Household is defined in Chapter 7.

Proposed categories and technical terms will probably not be used directly in household questionnaires. They are recommended for reporting purposes and international data harmonization. The questions and response categories at the country level (in the household questionnaire) need to be adapted according to plans and services offered by operators and terminologies that are more familiar to users. Furthermore, according to local context, additional information could be collected to better identify the correct type of access.

WiFi access available inside the home to redistribute a fixed broadband signal is categorized as fixed broadband. If information is collected by access technology (e.g. cable modem, DSL, fibre-to-the-home/building, other fixed broadband), then WiFi should be classified to the type of technology available in the household.

The Internet connection(s) selected should be working at the time of the survey.

Model question:

What type/s of Internet services are used for Internet access at home? Respondents should select all that apply (see above).

Countries may ask about services as a series of yes/no questions, each referring to one type of service.

This question can present difficulties for both respondents and interviewers. The response options should be based on the commercial packages or plans offered in the country at the time of the interview – possibly using the brand names used by service providers, where these can be recoded to the different types of networks. Grouping into the above categories should be done by specialists during the coding phase of data processing.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH11: Proportion of households with Internet, by type of service

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

Disaggregation by household characteristics:

• Breakdown by region, such as geographical areas, urban/rural.

• Breakdown by household characteristics, such as household composition and size, and whether the household has access to electricity.

• Breakdown by characteristics of the head of the household/household reference person, such as sex, level of education, occupation or status in the labour force.

• Other breakdowns or classifications, where relevant variables or questions are used in the questionnaire, such as household income.

Calculation:

The number of in-scope households with Internet access by a given type of access is calculated by aggregating the weighted responses for each type of access (see Chapter 8).

Proportions are expressed as percentages and are calculated by dividing the number of in-scope households with a given type of Internet access by either the total number of in-scope households with Internet access (HH6) or by the total number of in-scope households, and then multiplying the result by 100.

Examples:

The percentage of households with Internet access using a fixed broadband network should be calculated as:

• HH11%households with access, fixed broadband = [(number of in-scope households with fixed broadband Internet access) / (total number of in-scope households with Internet access)]*100

The percentage of in-scope households accessing the Internet via a fixed broadband network should be calculated as:

• HH11%all households, fixed broadband = [(number of in-scope households with fixed broadband Internet access) / (total number of in-scope households)]*100

Policy relevance:

Information on the type of Internet access that households use enables monitoring of the uptake of broadband Internet access. It is also important in tracking the transition from fixed Internet access, to wireless access. Fixed–mobile substitution has an impact on the definition and regulation of broadband markets, because it determines to a great extent the competitive forces acting in the market. Quantitative data on fixed–mobile substitution is becoming an increasingly significant input for evidence-based market regulation.

The breakdown by technology allows policy-makers to evaluate the speed and the quality of Internet connection, as each technology has different characteristics and offers different speeds. Linked to the difference in speed and quality, this will also allow for an evaluation of the differences in the potential of these technologies and what types of activities can/cannot be carried out over the Internet. For example, some countries are trying to increase the number of households that are connected via ftth since fibre offers particularly high speeds and quality, including for future applications and services.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH12: Proportion of individuals using the Internet, by frequency

Definitions and notes:

This is the frequency of Internet use by individuals who used the Internet from any location in the last three months.

The Internet is a worldwide public computer network. It provides access to a number of communication services including the World Wide Web and carries e-mail, news, entertainment and data files, irrespective of the device used (not assumed to be only via a computer − it may also be by mobile telephone, tablet, PDA, games machine, digital TV etc.). Access can be via a fixed or mobile network.

Frequency of use categories are as follows:

• At least once a day: once a working day for respondents who only (or most frequently) use the Internet from work or school, etc.

• At least once a week but not every day

• Less than once a week.

Clarifications and methodological issues:

It is recommended that countries collect this information in respect of a typical period; therefore, respondents should ignore weekends (if they only use the Internet at work or school etc) and breaks from their usual routine, such as holidays.

Multiple responses are not possible.

The suggested reference period is the last three months.

Individual is discussed in Chapter 7.

Model question:

How often did you typically use the Internet during the last three months (from any location)?

Respondents can only respond to one category (see above).

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Classification by sex.

• Classification by age: countries can use the following age groups for reporting on individuals’ age: under 5; 5–9; 10–14; 15–24; 25–34; 35–44; 45–54; 55–64; 65–74 and 75 and over.

• Classification by highest education level attained: countries can use the ISCED 2011 classification for reporting on individuals’ level of education: primary education or lower (ISCED 0,1); lower secondary education (ISCED 2); upper secondary or post-secondary non-tertiary education (ISCED 3, 4); tertiary education (ISCED 5, 6); and post-tertiary education (ISCED 7, 8).

• Classification by labour force status: countries can use the following categories for reporting on individuals’ labour force status: employee; self-employed (includes employers, own account workers, members of producers’ cooperatives and contributing family workers); workers not classifiable by status, unemployed; and outside the labour force.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH12: Proportion of individuals using the Internet, by frequency

• Classification by occupation: countries should use the ISCO 2008 categories where these are in use (if not, use ISCO-88 per Table 5 earlier in this chapter) for reporting on individuals’ occupation: managers; professionals; technicians and associate professionals; clerical support workers; service and sales workers; skilled agricultural, forestry and fishery workers; craft and related trades workers; plant and machine operators, and assemblers; elementary occupations; and armed forces occupations (noting that armed forces personnel may be out of scope).

• Other breakdowns or classifications, where relevant variables or questions are used in the questionnaire, such as individual income.

Calculation:

The number of in-scope individuals using the Internet with a specific frequency is calculated by aggregating the weighted responses for each frequency (see Chapter 8).

Proportions are expressed as percentages and are calculated by dividing the number of in-scope individuals using the Internet with a specific frequency either by the total number of in-scope individuals using the Internet (see HH7) or by the total number of in-scope individuals, and then multiplying the result by 100.

Examples:

The percentage of Internet users who used the Internet at least once a day is calculated as:

• HH12%Internet users at least once a day = [(number of in-scope individuals using the Internet at least once a day) / (total number of in-scope individuals who used the Internet)]*100

The percentage of in-scope individuals using the Internet at least once a day is calculated as:

• HH12%Individuals at least once a day = [(number of in-scope individuals who used the Internet at least once a day) / (total number of in-scope individuals)]*100

Policy relevance:

HH12 provides policy-makers with an understanding of the pervasiveness of the information society as it will help them distinguish between occasional Internet users and those who use the Internet intensively. Measuring intensity of use is becoming more relevant as Internet access spreads, and the policy focus shifts from “what proportion of people use the Internet?” to “how much are people using the Internet?”. The indicator is also useful in tracking progress over time and monitoring the evolution of Internet behavior. Classificatory variables can provide further information on differences in the intensity of Internet use among men/women, children/adults, employed/unemployed, etc. These data may inform targeted policies to promote Internet use among low-usage groups, and thus contribute to a more inclusive information society.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH13: Proportion of households with multichannel television, by type

Definitions:

This is the proportion of households with multichannel television (TV) and by type of multichannel service. Multichannel TV services are as follows:

• Cable TV (CATV): multichannel programming delivered over a coaxial cable for viewing on television sets

• Direct-to-home (DTH) satellite services: TV services received via a satellite dish capable of receiving satellite television broadcasts

• Internet-protocol TV (IPTV): multimedia services such as television/video/audio/text/graphics/data delivered over an IP-based network managed to support the required level of quality of service, quality of experience, security, interactivity and reliability; it does not include video accessed over the public Internet, for example, by streaming. IPTV services are also generally aimed at viewing over a television set rather than a personal computer.

• Digital terrestrial TV (DTT): the technological evolution from analogue terrestrial television, providing capability for significantly more channels

Clarifications and methodological issues:

National questionnaires should reflect services available in the country and describe them using local terminology, such as brand names.

As households can use more than one type of multichannel TV service, multiple responses are possible.

Household is discussed in Chapter 7.

The TV service(s) selected should be working at the time of the survey.

Model question:

Does this household have any of the following television services?

Respondents should select all services (see above) that apply.

Countries may ask about multichannel services as a series of yes/no questions, each referring to one service.

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Breakdown by household characteristics, such as household composition and size, and whether the household has access to electricity.

• Breakdown by characteristics of the head of the household/household reference person, such as sex, level of education, occupation or status in the labour force.

• Other breakdowns or classifications, where relevant variables or questions are used in the questionnaire, such as household income.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH13: Proportion of households with multichannel television, by type

Calculation:

The number of in-scope households with any type of multichannel TV service is calculated by aggregating the weighted responses for any multichannel TV service (see Chapter 8).

The proportion of households with any type of multichannel TV service is expressed as a percentage and is calculated by dividing the number of in-scope households with multichannel TV (of any type) by either the total number of in-scope households with TV (from HH2) or by the total number of in-scope households, and then multiplying the result by 100.

The number of in-scope households with a given type of multichannel service is calculated by aggregating the weighted responses for each multichannel TV service (see Chapter 8).

Proportions are expressed as percentages and are calculated by dividing the number of in-scope households with a given type of multichannel TV service by either the total number of in-scope households with TV (from HH2) or by the total number of in-scope households, and then multiplying the result by 100.

Examples:

Percentage of households with a TV having cable TV:

• HH13%households with TV, cable = [(number of in-scope households with cable TV) / (total number of in-scope households with a TV)]*100

Percentage of in-scope households with cable TV:

• HH13%all households, cable TV = [(number of in-scope households with cable TV / (total number of in-scope households)]*100

Policy relevance:

This indicator is useful in tracking the move from analogue to digital TV and the adoption of multichannel services. Since some multichannel and digital TV services do not require a subscription (e.g. free-to-air satellite services and most digital terrestrial television broadcasts), nationally representative household surveys and censuses are the main vehicles for obtaining reliable data on multichannel and digital TV adoption. For instance, countries doing the digital terrestrial television switchover use household surveys as a means of monitoring the process – because they can keep track of households equipped with set-top boxes or digital TVs. Similar data cannot be captured from subscription-based administrative data sources. A distinction of technologies will further allow policy-makers to monitor the current regulation of different TV platforms, particularly in view of convergence. Indeed, because of the historical development of TV platforms, several technologies currently competing in the same TV markets may be subject to different regulation. Reliable data on the evolution and uptake of new TV platforms will provide insights about the extent to which these platforms are complementing or substituting traditional TV services. Hence, these data will be a relevant input for future revisions of the audiovisual regulatory framework.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH14: Barriers to household Internet access

Definitions:

This measures the barriers to Internet access for households without Internet access. It is expressed as a proportion of households without Internet access.

Barriers (that is, reasons for not having Internet) are:

• Do not need the Internet (not useful, not interesting, lack of local content)

• Have access to the Internet elsewhere

• Cost of the equipment is too high

• Cost of the service is too high

• Privacy or security concerns

• Internet service is not available in the area

• Internet service is available but it does not correspond to household needs (e.g. quality, speed)

• Cultural reasons (e.g. exposure to harmful content)

• No electricity in the household

• Other reason, specify

Clarifications and methodological issues:

Household is defined in Chapter 7.

As households can have more than one barrier (reason for not having Internet), respondents should select all that apply.

Model question:

Why does this household not have Internet access? Respondents should select all reasons (see above) that apply.

Some countries may ask about barriers as a series of yes/no questions.

This question should be asked of households with no access to the Internet at home.

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Breakdown by household characteristics, such as household composition and size, and whether the household has access to electricity.

• Breakdown by characteristics of the head of the household/household reference person, such as sex, level of education, occupation or status in the labour force.

• Other breakdowns or classifications, where relevant variables or questions are used in the questionnaire, such as household income.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH14: Barriers to household Internet access

Calculation:

The number of in-scope households with a given barrier to Internet access is calculated by aggregating the weighted responses for each TV service (see Chapter 8).

Proportions are expressed as percentages and are calculated by dividing the number of in-scope households with a given barrier by the total number of in-scope households without Internet access, and then multiplying the result by 100.

Example:

Percentage of households without Internet access for which cost of equipment is too high:

• HH14%cost too high = [(number of in-scope households for which cost of equipment is too high) / (total number of in-scope households without Internet access)]*100

Policy relevance:

HH14 is useful in countries where home Internet access is relatively low because it will help policy-makers to identify policy tools to overcome barriers to higher penetration rates. If, for example, many citizens feel that the cost of the equipment or services is too high, specific actions could be taken to reduce costs. If Internet services are not used because they are not available, more efforts can be made to expand Internet infrastructure. Other specific programmes could tackle the awareness of the benefits of the Internet, the creation of local content or the development of ICT skills.

In countries where Internet access is high, this indicator is also relevant because barriers to household Internet access may be less evident and thus require detailed data (in some cases collected at subnational level) to inform policy-makers. Indeed, connecting the last subscribers usually requires more targeted policies than those aiming at larger shares of the population.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH15: Individuals with ICT skills, by type of skills

Definitions:

This refers to ICT skills, defined for the purpose of this indicator as having undertaken certain activities in the last three months, independent of the device(s) used.

Activities to measure ICT skills are as follows:

• Using copy and paste tools to duplicate or move data, information and content in digital environments (e.g. within a document, between devices, on the cloud)

• Sending messages (e.g. e-mail, messaging service, SMS) with attached files (e.g. document, picture, video)

• Using basic arithmetic formulae in a spreadsheet

• Connecting and installing new devices (e.g. a modem, camera, printer) through wired or wireless technologies

• Finding, downloading, installing and configuring software and apps

• Creating electronic presentations with presentation software (including text, images, sound, video or charts)

• Transferring files or applications between devices (including via cloud-storage)

• Setting up effective security measures (e.g. strong passwords, log-in attempt notification) to protect devices and online accounts

• Changing privacy settings on your device, account or app to limit the sharing of personal data and information (e.g. name, contact information, photos)

• Verifying the reliability of information found online

• Programming or coding in digital environments (e.g. computer software, app development)

Clarifications and methodological issues:

Individual is discussed in Chapter 7.

Most individuals will have carried out more than one activity and therefore multiple responses are expected.

The tasks are broadly ordered from less complex to more complex, although there is no requirement for a respondent to select simpler tasks before selecting a more complex task.

Model question:

Which of the following activities have you carried out in the last three months (independent of the device used)?

Respondents should select all that apply (see above).

Some countries may ask about tasks as a series of yes/no questions.

This question is asked of all individuals.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH15: Individuals with ICT skills, by type of skills

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Classification by sex.

• Classification by age: countries can use the following age groups for reporting on individuals’ age: under 5; 5–9; 10–14; 15–24; 25–34; 35–44; 45–54; 55–64; 65–74 and 75 and over.

• Classification by highest education level attained: countries can use the ISCED 2011 classification for reporting on individuals’ level of education: primary education or lower (ISCED 0, 1); lower secondary education (ISCED 2); upper secondary or post-secondary non-tertiary education (ISCED 3, 4); tertiary education (ISCED 5, 6); and post-tertiary education (ISCED 7, 8).

• Classification by labour force status: countries can use the following categories for reporting on individuals’ labour force status: employee; self-employed (includes employers, own account workers, members of producers’ cooperatives and contributing family workers); workers not classifiable by status, unemployed; and outside the labour force.

• Classification by occupation: countries should use the ISCO 2008 categories where these are in use (if not, use ISCO-88 per Table 5 earlier in this chapter) for reporting on individuals’ occupation: managers; professionals; technicians and associate professionals; clerical support workers; service and sales workers; skilled agricultural, forestry and fishery workers; craft and related trades workers; plant and machine operators, and assemblers; elementary occupations; and armed forces occupations (noting that armed forces personnel may be out of scope).

• Other breakdowns or classifications, where relevant variables or questions are used in the questionnaire, such as individual income.

Calculation:

Indicator HH15 is calculated as the proportion of in-scope who have carried out each activity. The indicator is expressed as a percentage.

For instance, the percentage of individuals having used copy and paste tools to duplicate or move data, information and content in digital environments can be calculated as:

• HH15copied or moved a file= [(number of in-scope individuals who used copy and paste tools) / (number of in-scope individuals)]*100

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH15: Individuals with ICT skills, by type of skills

Policy relevance:

ICT skills determine the effective use that is made of ICTs. The information from HH15 may therefore assist in making the link between ICT usage and impact. Currently, there is little data available for measuring ICT-specific skills, and hence researchers and policy-makers must rely on proxy indicators to measure this important enabler of ICT development.

The conceptual framework adopted for this skills question is the European Commission’s Digital Competence Framework for Citizens (DigComp 2.0). The framework has five major areas of skills measurement:

• Information and data literacy

• Communication and collaboration

• Digital content creation

• Safety

• Problem-solving

The guiding principles of this framework helped to complete the structure and effectiveness of questions HH9 and HH15.

HH15 is an appropriate way to measure and track the level of proficiency of individuals. This information could be used, for example, to adapt ICT literacy courses in schools, identify barriers to certain uses of computers as well as potential applications and services that could be accessed over the Internet. Classificatory variables can provide further information on the differences in ICT skills among men/women, children/adults, employed/unemployed, etc. These data may be used to inform targeted policies to improve ICT skills, and thus contribute to an inclusive information society.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH16: Household expenditure on ICT

Definitions:

This measures the percentage of total household expenditure that is expended on ICT goods and services as follows (see Table 4 for complete detail):

• 08.1 Information and communication equipment: fixed telephone equipment, mobile telephone equipment, information processing equipment (personal computers, printers, scanners, monitors, etc); Equipment for the reception, recording and reproduction of sound and vision (TV sets, digital video recorders, radio receivers, CD players, stereo equipment, etc.), recording media (CDs, DVDs, USD keys, etc.)

• 08.2 Software excluding games computer software packages, such as operating systems, applications, programming languages, etc.

• 08.3 Information and communication services: fixed and mobile communication services, Internet access provision, bundled telecommunication services, repair and rental of ICT equipment, other ICT services)

• 09.2.1 Games toys and hobbies: video game software, game apps, gamepads joysticks, etc., electronic games.

Clarifications and methodological issues:

The 2018 UN Classification of Individual Consumption According to Purpose (COICOP) is used as the basis of the classification presented above and to define the scope of ICT goods and services.

It is expected that data would be collected from a household budget survey. Ideally, the reference period would be a year, but this is likely to vary depending on the nature of countries’ existing budget surveys.

Model question:

As this indicator will usually be derived from household budget survey, no model question is proposed.

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Breakdown by household characteristics, such as household composition and size, and whether the household has access to electricity.

• Breakdown by household income, for example, by income quintile or quartile, according to the practices of the country.

Calculation:

The indicator is calculated as the percentage of total household expenditure that is expended on ICT goods and services as defined above.

Policy relevance:

Information on the percentage) of income that households spend on ICT can be compared to expenditure on other services (such as electricity, food, etc.). The indicator provides an indication of the importance of ICTs because it shows how much households are prepared to spend on ICT. At the same time, it can be linked to the price of ICT equipment and services and help governments identify ways of reducing the cost and increasing the affordability of ICT.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH17: Proportion of individuals using the Internet, by type of portable device and network used to access the Internet

Definitions:

This is the proportion of individuals who used the Internet using a portable device. The Internet is a worldwide public computer network. It provides access to a number of communication services including the World Wide Web and carries e-mail, news, entertainment and data files, irrespective of the device used (not assumed to be only via a network or via other wireless networks (e.g. WiFi).

a. Mobile phone

a1) via mobile cellular network

a2) via other wireless networks (e.g. WiFi)

b. Tablet

b1) via mobile cellular network, using a USB key/dongle or integrated data SIM card

b2) via other wireless networks (e.g. WiFi)

c. Portable computer (laptop, notebook, netbook)

c1) via mobile cellular network, using a USB key/dongle or integrated data SIM card or

mobile cellular telephone as modem

c2) via other wireless networks (e.g. WiFi)

d. Other portable devices (e.g. portable games consoles, watches, e-book readers etc.)

Clarifications and methodological issues:

This indicator refers to use of the Internet by individual household members.

The suggested reference period is the last three months. Country practices vary, but ideally, reference periods should be aligned in order to obtain comparable data Countries changing their reference period may wish to split the reference period in order to obtain comparable time series.

Individual is discussed in Chapter 7.

Model questions:

Have you used the Internet in the last three months using…?

a. Mobile phone

a1) via mobile cellular network (Yes/No)

a2) via other wireless networks (e.g. WiFi) (Yes/No)

b. Tablet

b1) via mobile cellular network, using a USB key/dongle or integrated data SIM card (Yes/No)

b2) via other wireless networks (e.g. WiFi) (Yes/No)

c. Portable computer (laptop, notebook, netbook)

c1) via mobile cellular network, using a USB key/dongle or integrated data SIM card or mobile cellular telephone as modem (Yes/No)

c2) via other wireless networks (e.g. WiFi) (Yes/No)

d. Other portable devices (e.g. portable games consoles, watches, e-book readers etc.) Yes/No

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH17: Proportion of individuals using the Internet, by type of portable device and network used to access the Internet

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Classification by sex.

• Classification by age: countries can use the following age groups for reporting on individuals’ age: under 5; 5–9; 10–14; 15–24; 25–34; 35–44; 45–54; 55–64; 65–74 and 75 and over.

• Classification by highest education level attained: countries can use the ISCED 2011 classification for reporting on individuals’ level of education: primary education or lower (ISCED 0,1); lower secondary education (ISCED 2); upper secondary or post-secondary non-tertiary education (ISCED 3, 4); tertiary education (ISCED 5, 6); and post-tertiary education (ISCED 7, 8).

• Classification by labour force status: countries can use the following categories for reporting on individuals’ labour force status: employee; self-employed (includes employers, own account workers, members of producers’ cooperatives and contributing family workers); workers not classifiable by status, unemployed; and outside the labour force.

• Classification by occupation: countries should use the ISCO 2008 categories where these are in use (if not, use ISCO-88 per Table 5 earlier in this chapter) for reporting on individuals’ occupation: managers; professionals; technicians and associate professionals; clerical support workers; service and sales workers; skilled agricultural, forestry and fishery workers; craft and related trades workers; plant and machine operators, and assemblers; elementary occupations; and armed forces occupations (noting that armed forces personnel may be out of scope).

Calculation:

The number of in-scope individuals using the Internet for a particular type of device and network is calculated by aggregating the weighted responses (see Chapter 8).

The proportion of individuals using the Internet for a particular type of device and network is expressed as a percentage and is calculated by dividing the total number of in-scope individuals using the Internet for a particular type of device and network by the total number of in-scope individuals using the Internet (see HH7) and then multiplying the result by 100.

HH17device, network% = [(number of in-scope individuals using the Internet using device, network) / (total number of in-scope individuals using the Internet)]*100

Policy relevance:

Internet user uptake is a key indicator tracked by policy-makers and analysts as an indication of a country’s progress towards becoming an information society. Classificatory variables for individuals using the Internet – such as age, sex, education level or labour force status – can help identify digital divides in individuals using the Internet.

HH7 is also one of the SDG indicators, used to monitor SDG 17 (Indicator 17.8.1). As HH17 is a breakdown of HH7, it may be used to better understand the type of networks and devices used.

This information can contribute to the design of targeted policies to overcome those divides.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH18: Proportion of individuals who own a mobile phone

Definitions:

This is the proportion of individuals who own a mobile telephone.

An individual owns a mobile cellular telephone if he/she has a mobile cellular phone device with at least one active SIM card for personal use. It includes mobile cellular phones supplied by employers that can be used for personal reasons (to make personal calls, access the Internet, etc.) and those who have a mobile phone for personal use that is not registered under his/her name. It excludes individuals who have only active SIM card(s) and not a mobile phone device.

A smart telephone (or smartphone) refers to a mobile handset that is used as the person’s primary phone device which has smart capabilities, including Internet-based services, and performs many of the functions of a computer, including having an operating system capable of downloading and running applications, also those created by third-party developers.

An individual owns a smart telephone if he/she has a smart telephone device with at least one active SIM card for personal use. It includes smart telephones supplied by employers that can be used for personal reasons (to make personal calls, access the Internet, etc.) and those who have a smart telephone for personal use that is not registered under his/her name. It excludes individuals who have only active SIM card(s) and not a smart telephone device.

Clarifications and methodological issues:

The suggested reference period is the last three months. Country practices vary, but ideally, reference periods should be aligned in order to obtain comparable data. Countries changing their reference period may wish to split the reference period in order to obtain comparable time series.

Model questions:

Do you own a mobile phone? Yes/No

Do you own a smartphone? Yes/No

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Classification by sex.

• Classification by age: countries can use the following age groups for reporting on individuals’ age: under 5; 5–9; 10–14; 15–24; 25–34; 35–44; 45–54; 55–64; 65–74 and 75 and over.

• Classification by highest education level attained: countries can use the ISCED 2011 classification for reporting on individuals’ level of education: primary education or lower (ISCED 0,1); lower secondary education (ISCED 2); upper secondary or post-secondary non-tertiary education (ISCED 3, 4); tertiary education (ISCED 5, 6); and post-tertiary education (ISCED 7, 8).

• Classification by labour force status: countries can use the following categories for reporting on individuals’ labour force status: employee; self-employed (includes employers, own account workers, members of producers’ cooperatives and contributing family workers); workers not classifiable by status, unemployed; and outside the labour force.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH18: Proportion of individuals who own a mobile phone

• Classification by occupation: countries should use the ISCO 2008 categories where these are in use (if not, use ISCO-88 per Table 5 earlier in this chapter) for reporting on individuals’ occupation: managers; professionals; technicians and associate professionals; clerical support workers; service and sales workers; skilled agricultural, forestry and fishery workers; craft and related trades workers; plant and machine operators, and assemblers; elementary occupations; and armed forces occupations (noting that armed forces personnel may be out of scope).

Calculation:

The number of in-scope individuals owning a mobile phone is calculated by aggregating the weighted responses (see Chapter 8).

The proportion of individuals owning a mobile phone is expressed as a percentage and is calculated by dividing the total number of in-scope individuals owning a mobile phone by the total number of in-scope individuals, and then multiplying the result by 100.

HH18% = [(number of in-scope individuals owning a mobile phone) / (total number of in-scope individuals)]*100

Policy relevance:

Sustainable Development Goal 5 (SDG5) includes the indicator “Proportion of individuals who own a mobile telephone, by sex” (5.b.1). Mobile phone ownership, in particular, is important to track gender equality since the mobile phone is a personal device that, if owned and not just shared, provides women with a degree of independence and autonomy, including for professional purposes. A number of studies have highlighted the link between mobile phone ownership and empowerment, and productivity growth. Existing data on the proportion of women owning a mobile phone suggest that the proportion of women who own a mobile phone is lower than for men. This indicator highlights the importance of mobile phone ownership, to track and to improve gender equality, and to help design targeted policies to overcome this gender divide.

The indicators is used to monitor SDG Target 5.b: “Enhance the use of enabling technology, in particular information and communications technology, to promote the empowerment of women”

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH19: Proportion of individuals not using the Internet, by type of reason

Definitions:

This measures the barriers to Internet use by individuals. It is expressed as a proportion of individuals who do not use the Internet. The reasons for not using the Internet are:

• Do not need the Internet (not useful, not interesting)

• Do not know how to use it

• Cost of Internet use is too high (service charges, etc.)

• Privacy or security concerns

• Internet service is not available in the area

• Cultural reasons (e.g. exposure to harmful content)

• Don’t know what Internet is

• Not allowed to use the Internet

• Lack of local content

• Other reason, specify

Clarifications and methodological issues:

The suggested reference period is the last three months. Country practices vary, but ideally, reference periods should be aligned in order to obtain comparable data. Countries changing their reference period may wish to split the reference period in order to obtain comparable time series.

Model question:

[for those individuals answering “No” to the question on use of Internet]

What are the reasons for not having used the Internet?

(see categories above, multiple response).

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Classification by sex.

• Classification by age: countries can use the following age groups for reporting on individuals’ age: under 5; 5–9; 10–14; 15–24; 25–34; 35–44; 45–54; 55–64; 65–74 and 75 and over.

• Classification by highest education level attained: countries can use the ISCED 2011 classification for reporting on individuals’ level of education: primary education or lower (ISCED 0,1); lower secondary education (ISCED 2); upper secondary or post-secondary non-tertiary education (ISCED 3, 4); tertiary education (ISCED 5, 6); and post-tertiary education (ISCED 7, 8).

• Classification by labour force status: countries can use the following categories for reporting on individuals’ labour force status: employee; self-employed (includes employers, own account workers, members of producers’ cooperatives and contributing family workers); workers not classifiable by status, unemployed; and outside the labour force.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH19: Proportion of individuals not using the Internet, by type of reason

• Classification by occupation: countries should use the ISCO 2008 categories where these are in use (if not, use ISCO-88 per Table 5 earlier in this chapter) for reporting on individuals’ occupation: managers; professionals; technicians and associate professionals; clerical support workers; service and sales workers; skilled agricultural, forestry and fishery workers; craft and related trades workers; plant and machine operators, and assemblers; elementary occupations; and armed forces occupations (noting that armed forces personnel may be out of scope).

Calculation:

The number of in-scope individuals not using the Internet for any reason is calculated by aggregating the weighted responses (see Chapter 8).

The proportion of individuals not having used Internet for a certain reason is expressed as a percentage and is calculated by dividing the total number of in-scope individuals not having used Internet for a certain reason by the total number of in-scope individuals who do not use the Internet (which can be calculated from HH7), and then multiplying the result by 100.

HH19reason% = [(number of in-scope individuals not having used Internet for a certain reason) / (total number of in-scope individuals who do not use the Internet)]*100

Policy relevance:

One increasingly cited reason for the lack of Internet access in households is that there is “no need”. This suggests that non-users are either not aware of the information and applications available over the Internet, or that insufficient content is made available of relevance for specific user groups. Lack of confidence, knowledge and skills is another important and frequently cited barrier, pointing to the importance of raising levels of education in order to enable people to benefit from online opportunities. ITU research finds that education levels are one of the most important indicators as to whether or not people are Internet users, both in developed and in developing countries.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH20: Proportion of individuals who purchased goods or services online, by type of good and service purchased

Definitions:

This indicator measures the specific goods and services purchased online by individuals. It is expressed as a proportion of individuals who purchased goods or services over the Internet.

The following response categories of goods and services purchased online are (multiple choices possible):

• Books, magazines or newspapers

• Clothing, footwear, sporting goods or accessories

• Computer equipment or parts (including peripheral equipment)

• Computer or video games

• Computer software (includes upgrades and paid apps; not games)

• Cosmetics

• Financial products (including shares and insurance)

• Food, groceries, alcohol or tobacco

• Household goods (e.g. furniture, toys, etc.; excluding consumer electronics)

• ICT services (excluding software)

• Medicine

• Movies, short films or images

• Music products

• Photographic, telecommunications or optical equipment

• Tickets or bookings for entertainment events (sports, theatre, concerts, etc.)

• Travel products (travel tickets, accommodation, vehicle hire, transport services etc.)

Clarifications and methodological issues:

The suggested reference period is the last three months. Country practices vary, but ideally, reference periods should be aligned in order to obtain comparable data. Countries changing their reference period may wish to split the reference period in order to obtain comparable time series.

The purchase orders should have been placed via the Internet, independently whether the payment is made or not online.

Model questions:

[for individuals answering “yes” to the option “purchasing or ordering goods or services” of the question on Internet usage (see Indicator HH9)]:

What types of goods or services did you buy or order over the Internet for private use in the

last 3 months?

(see categories above)

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH20: Proportion of individuals who purchased goods or services online, by type of good and service purchased

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Classification by sex.

• Classification by age: countries can use the following age groups for reporting on individuals’ age: under 5; 5–9; 10–14; 15–24; 25–34; 35–44; 45–54; 55–64; 65–74 and 75 and over.

• Classification by highest education level attained: countries can use the ISCED 2011 classification for reporting on individuals’ level of education: primary education or lower (ISCED 0,1); lower secondary education (ISCED 2); upper secondary or post-secondary non-tertiary education (ISCED 3, 4); tertiary education (ISCED 5, 6); and post-tertiary education (ISCED 7, 8).

• Classification by labour force status: countries can use the following categories for reporting on individuals’ labour force status: employee; self-employed (includes employers, own account workers, members of producers’ cooperatives and contributing family workers); workers not classifiable by status, unemployed; and outside the labour force.

• Classification by occupation: countries should use the ISCO 2008 categories where these are in use (if not, use ISCO-88 per Table 5 earlier in this chapter) for reporting on individuals’ occupation: managers; professionals; technicians and associate professionals; clerical support workers; service and sales workers; skilled agricultural, forestry and fishery workers; craft and related trades workers; plant and machine operators, and assemblers; elementary occupations; and armed forces occupations (noting that armed forces personnel may be out of scope).

Calculation:

The number of in-scope individuals having purchased a type of good or service is calculated by aggregating the weighted responses (see Chapter 8).

The proportion of individuals having purchased a type of good or service is expressed as a percentage and is calculated by dividing the total number of in-scope individuals having purchased a type of good or service by the total number of in-scope individuals, and then multiplying the result by 100.

HH20type% = [(number of in-scope individuals purchased a type of good or service) / (total number of in-scope individuals)]*100

Policy relevance:

E-commerce is relevant for enterprises as it offers opportunities to reach new markets. For consumers, e-commerce offers convenience (accessing providers all over the world, comparing prices, buying from home or any other location, and at any time). Policies may target the promotion of e-commerce while preserving certain modes of traditional commerce (e.g. proximity commerce). Understanding the behavior of consumers helps companies design new goods or services, new forms of delivery or additional services.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH21: Proportion of individuals who purchased goods or services online, by type of payment channel

Definitions:

This indicator measures the payment channels used by individuals when making purchases online. It is expressed as a proportion of individuals who purchased goods or services over the Internet.

The following response categories of types of payment channels are (multiple choices possible):

• Cash on delivery

• Credit card online

• Debit card or electronic bank transfer online

• Mobile money account (account connected to the mobile phone number)

• Online payment service (e.g. PayPal, Google Checkout)

• Prepaid gift card or online voucher

• Points from rewards or redemption program (e.g. Air Miles)

• Other (e.g. bank check by post, etc.)

Clarifications and methodological issues:

The suggested reference period is the last three months. Country practices vary, but ideally, reference periods should be aligned in order to obtain comparable data. Countries changing their reference period may wish to split the reference period in order to obtain comparable time series.

Model questions:

[for individuals answering “yes” to the option “purchasing or ordering goods or services” of the question on Internet usage (see Indicator HH9)]:

How did you pay for the goods or services you bought over the Internet for private use in the last 3 months?

(see categories above)

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH21: Proportion of individuals who purchased goods or services online, by type of payment channel

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Classification by sex.

• Classification by age: countries can use the following age groups for reporting on individuals’ age: under 5; 5–9; 10–14; 15–24; 25–34; 35–44; 45–54; 55–64; 65–74 and 75 and over.

• Classification by highest education level attained: countries can use the ISCED 2011 classification for reporting on individuals’ level of education: primary education or lower (ISCED 0,1); lower secondary education (ISCED 2); upper secondary or post-secondary non-tertiary education (ISCED 3, 4); tertiary education (ISCED 5, 6); and post-tertiary education (ISCED 7, 8).

• Classification by labour force status: countries can use the following categories for reporting on individuals’ labour force status: employee; self-employed (includes employers, own account workers, members of producers’ cooperatives and contributing family workers); workers not classifiable by status, unemployed; and outside the labour force.

• Classification by occupation: countries should use the ISCO 2008 categories where these are in use (if not, use ISCO-88 per Table 5 earlier in this chapter) for reporting on individuals’ occupation: managers; professionals; technicians and associate professionals; clerical support workers; service and sales workers; skilled agricultural, forestry and fishery workers; craft and related trades workers; plant and machine operators, and assemblers; elementary occupations; and armed forces occupations (noting that armed forces personnel may be out of scope).

Calculation:

The number of in-scope individuals having purchased a type of good or service is calculated by aggregating the weighted responses (see Chapter 8).

The proportion of individuals having paid an online purchase through certain channel is expressed as a percentage and is calculated by dividing the total number of in-scope individuals having paid an online purchase through certain channel by the total number of in-scope individuals, and then multiplying the result by 100.

HH21channel% = [(number of in-scope individuals having paid an online purchase through certain channel) / (total number of in-scope individuals)]*100

Policy relevance:

E-commerce is relevant for enterprises as it offers opportunities to reach new markets. For consumers, e-commerce offers convenience (accessing providers all over the world, comparing prices, buying from home or any other location, and at any time). Policies may target the promotion of e-commerce while preserving certain modes of traditional commerce (e.g. proximity commerce). Understanding the behavior of consumers helps companies design new goods or services, new forms of delivery or additional services.

Modes of payment offered by suppliers are important to consumers, as they facilitate or restrict e-commerce. From the consumer viewpoint, the possibility of using a diversity of payment modes (including credit) makes it easier to access a wider range of goods and services.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH22: Proportion of individuals who purchased goods or services online, by method of delivery

Definitions:

This indicator measures the delivery method for online purchases. It is expressed as a proportion of individuals who purchased goods or services over the Internet.

The following response categories of types of delivery methods are (multiple choices possible):

• Delivery directly to the buyer using regular postal services or other forms of delivery

• Picked up from a point of sale or service point

• Online/electronic delivery by downloading from a website or through an application, software or other device (e.g. in-app purchases, streaming services etc.).

Clarifications and methodological issues:

The suggested reference period is the last three months. Country practices vary, but ideally, reference periods should be aligned in order to obtain comparable data. Countries changing their reference period may wish to split the reference period in order to obtain comparable time series.

Model questions:

[for individuals answering “yes” to the option “purchasing or ordering goods or services” of the question on Internet usage (see Indicator HH9)]:

How did you receive the goods or services you bought over the Internet for private use in the last 3 months?

(see categories above)

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Classification by sex.

• Classification by age: countries can use the following age groups for reporting on individuals’ age: under 5; 5–9; 10–14; 15–24; 25–34; 35–44; 45–54; 55–64; 65–74 and 75 and over.

• Classification by highest education level attained: countries can use the ISCED 2011 classification for reporting on individuals’ level of education: primary education or lower (ISCED 0,1); lower secondary education (ISCED 2); upper secondary or post-secondary non-tertiary education (ISCED 3, 4); tertiary education (ISCED 5, 6); and post-tertiary education (ISCED 7, 8).

• Classification by labour force status: countries can use the following categories for reporting on individuals’ labour force status: employee; self-employed (includes employers, own account workers, members of producers’ cooperatives and contributing family workers); workers not classifiable by status, unemployed; and outside the labour force.

• Classification by occupation: countries should use the ISCO 2008 categories where these are in use (if not, use ISCO-88 per Table 5 earlier in this chapter) for reporting on individuals’ occupation: managers; professionals; technicians and associate professionals; clerical support workers; service and sales workers; skilled agricultural, forestry and fishery workers; craft and related trades workers; plant and machine operators, and assemblers; elementary occupations; and armed forces occupations (noting that armed forces personnel may be out of scope).

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH22: Proportion of individuals who purchased goods or services online, by method of delivery

Calculation:

The number of in-scope individuals having received goods or services purchased over the Internet via a certain delivery mode is calculated by aggregating the weighted responses (see Chapter 8).

The proportion of individuals having received goods or services purchased over the Internet via a delivery mode is expressed as a percentage and is calculated by dividing the total number of in-scope individuals having received goods or services purchased over the Internet via a delivery mode by the total number of in-scope individuals, and then multiplying the result by 100.

HH22deliveryl% = [(number of in-scope individuals having received goods or services purchased over the Internet via a delivery mode / (total number of in-scope individuals)]*100

Policy relevance:

E-commerce is relevant for enterprises as it offers opportunities to reach new markets. For consumers, e-commerce offers convenience (accessing providers all over the world, comparing prices, buying from home or any other location, and at any time). Policies may target the promotion of e-commerce while preserving certain modes of traditional commerce (e.g. proximity commerce). Understanding the behavior of consumers helps companies design new goods or services, new forms of delivery or additional services.

Certain goods or services are digital and can be delivered online (e.g. music, computer games, ICT services) while others can only be delivered physically. The accessibility or remoteness of a location may determine the possibility of e-commerce from that location. For instance, postal services delivering online purchases may not regularly serve certain locations. The diversity of delivery modes enhances consumer access to goods and services.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH23: Proportion of individuals who did not purchase goods or services online, by type of reason

Definitions:

This measures the reasons for individuals to not purchase goods or services online. It is expressed as a proportion of individuals who use the Internet but do not purchase goods and services online.

The following response categories of reasons for not purchasing online are (multiple choices possible):

• Not interested

• Prefer to shop in person

• Security concerns (e.g. about giving personal details)

• Privacy concerns (e.g. about giving personal details)

• Technical concerns (e.g. about websites, payment or delivery)

• Trust concerns (e.g. about warranties, receiving or returning products)

• Lack of confidence, knowledge or skills

Clarifications and methodological issues:

The suggested reference period is the last three months. Country practices vary, but ideally, reference periods should be aligned in order to obtain comparable data. Countries changing their reference period may wish to split the reference period in order to obtain comparable time series.

Model questions:

[for individuals answering “no” to the option “purchasing or ordering goods or services” of the question on Internet usage (see Indicator HH9)]:

What are the reasons why you did not purchase goods or services the Internet for private use in the last 3 months?

(see categories above)

Disaggregation and classifications:

If data allow breakdown and disaggregation, the following can be considered:

• Breakdown by region, such as geographical areas, urban/rural.

• Classification by sex.

• Classification by age: countries can use the following age groups for reporting on individuals’ age: under 5; 5–9; 10–14; 15–24; 25–34; 35–44; 45–54; 55–64; 65–74 and 75 and over.

• Classification by highest education level attained: countries can use the ISCED 2011 classification for reporting on individuals’ level of education: primary education or lower (ISCED 0,1); lower secondary education (ISCED 2); upper secondary or post-secondary non-tertiary education (ISCED 3, 4); tertiary education (ISCED 5, 6); and post-tertiary education (ISCED 7, 8).

• Classification by labour force status: countries can use the following categories for reporting on individuals’ labour force status: employee; self-employed (includes employers, own account workers, members of producers’ cooperatives and contributing family workers); workers not classifiable by status, unemployed; and outside the labour force.

Table 6. Detailed information on core indicators on access to, and use of, ICT by households and individuals: HH1 to HH23 (continued)

Indicator HH23: Proportion of individuals who did not purchase goods or services online, by type of reason

• Classification by occupation: countries should use the ISCO 2008 categories where these are in use (if not, use ISCO-88 per Table 5 earlier in this chapter) for reporting on individuals’ occupation: managers; professionals; technicians and associate professionals; clerical support workers; service and sales workers; skilled agricultural, forestry and fishery workers; craft and related trades workers; plant and machine operators, and assemblers; elementary occupations; and armed forces occupations (noting that armed forces personnel may be out of scope).

Calculation:

The number of in-scope individuals not having purchased goods or services purchased over the Internet for a certain reason is calculated by aggregating the weighted responses (see Chapter 8).

The proportion of individuals not having purchased goods or services purchased over the Internet for a certain reason is expressed as a percentage and is calculated by dividing the total number of in-scope individuals not having purchased goods or services purchased over the Internet for a certain reason by the total number of in-scope individuals, and then multiplying the result by 100.

HH23reasonsl% = [(number of in-scope individuals not having purchased goods or services purchased over the Internet for a certain reason) / (total number of in-scope individuals)]*100

Policy relevance:

E-commerce is relevant for enterprises as it offers opportunities to reach new markets. For consumers, e-commerce offers convenience (accessing providers all over the world, comparing prices, buying from home or any other location, and at any time). Policies may target the promotion of e-commerce while preserving certain modes of traditional commerce (e.g. proximity commerce). Understanding the behavior of consumers helps companies design new goods or services, new forms of delivery or additional services.

Understanding the reasons that hamper e-commerce is important for suppliers of online goods and services, as well as for public authorities wishing to promote e-commerce. This may be used as evidence for cybersecurity and trust policies, planning delivery infrastructure (e.g. postal services) and protection of traditional trade modes.

Other measurement topics related to ICT household statistics

220. The core list of household indicators is a starting point for collection of data on ICT. Many countries will have information needs for policy purposes that are not satisfied by the Partnership indicators alone.

221. Besides the core list of indicators, countries may wish to collect other statistics which are relevant to their ICT policies. The inclusion of ICT-related questions in existing surveys (such as labour force or income and expenditure surveys) can provide interesting breakdowns and cross-tabulations. This is discussed in Chapter 5.

222. A number of other topics of interest for policy purposes are being discussed by the Expert Group on ICT Household Indicators (EGH), such as cybersecurity and trust, children and youth online protection, the Internet of Things (IoT), and gender-relevant ICT indicators.

223. A glance at the OECD and Eurostat model questionnaires (OECD, 2005, 2013; Eurostat 2013b), shows that there are many topics of interest beyond those covered by the core indicators (see Boxes 18 and 19). This section outlines additional topics that countries may wish to measure in addition to those required to construct the core indicators. The discussion in this section is based on work by the OECD’s Working Party on Indicators for the Information Society (WPIIS, now the Working Party on Measuring the Digital Economy (WPMADE)) the manuals for the Eurostat model surveys,[30] ITU’s Child Online Protection Statistical Framework and Indicators (ITU, 2010b) and discussions held by the EGH and the Partnership.

Box 18. Choice of topics in European ICT surveys

In the European Union, data on ICT usage in households and by individuals are based on an annual sample survey which is part of the European Statistical Programme, and therefore, compulsory for all its Member States. The methodology and the statistical tools are completely harmonized to Eurostat requirements by legal requirements (Regulation No.808/2004 of the European Parliament and the Council). However, countries have some degree of flexibility to include additional topics. For example, in Bulgaria, the topics included are:

• Access to, and use of, ICT systems by individuals and/or in households;

• use of Internet for different purposes by individuals and/or in households;

• ICT security;

• ICT competence;

• e-commerce;

• barriers to use of ICT and the Internet;

• perceived effects of ICT usage on individuals and/or in households.

In Ireland, the annual Information and Communications Technology (ICT) Household Survey 2017 included additional questions relating to the Shared Economy and e-skills.

Sources: http://www.nsi.bg/en/content/6097/%D0%BC%D0%B5%D1%82%D0%B0%D0%B4%D0%B0%D0%BD%D0%BD%D0%B8/ict-usage-households;

https://www.cso.ie/en/methods/surveybackgroundnotes/informationsocietystatistics-households/

Box 19. Brazil: measuring online cultural activities

The ICT Households Survey has been conducted annually in Brazil since 2005 by the Regional Center for the Development of the Information Society (Cetic.br). In its 13th edition, the 2017 ICT Households Survey for the first time allowed compiling results about online cultural activities carried out by Internet users, giving continuity to the efforts initiated by the Culture and Technologies in Brazil Qualitative Survey (Brazilian Internet Steering Committee – CGI.br, 2017a) and by the first edition of the ICT in Culture Survey (CGI.br, 2017b). Among the activities surveyed relating to enjoyment of content, the most common were watching videos, TV programs, films or series online and listening to music online.

Source: https://www.cetic.br/media/docs/publicacoes/2/tic_dom_2017_livro_eletronico.pdf

Cybersecurity: trust in the online environment and child online protection

224. The question of trust in the online environment is an important one from a policy point of view, as lack of trust potentially impedes the uptake of ICT by individuals and other entities, such as businesses. Both the OECD and Eurostat have made attempts to measure trust via their model surveys of household ICT access/use and the efforts of member countries (see Box 20 on the experience of Canada).[31] However, evidence indicates that the topic presents difficult measurement challenges. Problems include the difficulties of asking individuals about IT security in terms of: the incidents they have encountered, what action they take to protect themselves and whether the computer or similar device they use at home is protected. Anecdotal evidence indicates that respondents have difficulty responding to such technical questions.

Box 20. Canada: Questions on Cybersecurity

The following questions are about security, privacy and trust when using the Internet for personal use, from any location. Please exclude business-related use.

During the past 12 months, what methods did you use to verify your identity over the Internet? Select all that apply.

Did you use:

• A username and password e.g., to log into online accounts

• Answers to personalized security questions

• Partner login e.g., social networking or bank account verification to access or create accounts for other applications or services

• Two-step verification with your mobile phone e.g., after initial username and password entered, code sent to your cellular phone through text message to also be entered online to access your account

• Biometric security features for online functions e.g., fingerprint or facial recognition. Exclude unlocking home screens on devices.

• Password manager program e.g., saved passwords through browser

• Other

OR

• None

During the past 12 months, what cyber security incidents have you experienced over the Internet?

Select all that apply.

Did you experience:

• A virus or other computer infection e.g., worm or Trojan

• Identity theft

• Other abuse of personal information e.g., misuse of pictures, videos or personal data uploaded on websites

• Received fraudulent emails or other messages

• Hacked accounts or fraudulent messages sent from your accounts

• Getting redirected to fraudulent websites asking for personal information

• Fraudulent payment card use e.g., credit, debit

• Loyalty program points fraud

• Asked to pay a cyber-ransom e.g., extortion, ransomware

• Other

OR

• None

Box 20. Canada: Questions on Cybersecurity (continued)

What actions did you take following the cyber security incidents in the past 12 months?

Select all that apply.

Have you:

• Reported the incident to the company through which the incident occurred

• Reported the incident to your Internet service provider

• Reported the incident to a governmental authority e.g., police

• Changed your Internet service provider

• Installed, upgraded or subscribed to a protection software

• Started carefully reading terms and conditions related to subscriptions and applications

• Changed passwords more frequently

• Deleted accounts associated with the security incidents

• Changed credit or debit card number associated with the incidents

• Other

OR

• None

Have you done any of the following to protect your laptop or computer from cyber security incidents ?

Select all that apply.

Did you:

• Enable automatic updates on your operating systems

• Manually update your operating systems on a regular basis

• Use additional cyber security measures beyond the default software e.g., Norton, McAfee, Avast, Virtual Private Network

OR

• No

Have you done any of the following to protect your mobile devices, such as a smartphone and tablet, from cyber security incidents?

Select all that apply.

Did you:

• Enable automatic updates on your operating systems

• Manually update your operating systems on a regular basis

• Use additional cyber security measures beyond the default software e.g., Norton, McAfee, Avast, Virtual Private Network

OR

• No

Source: https://www.statcan.gc.ca/eng/statistical-programs/instrument/4432_Q2_V2.

225. Indicator HH15 on ICT skills of individuals includes response categories relevant for the measurement of cybersecurity- and trust-related behaviour, namely:

• Setting up effective security measures (e.g. strong passwords, log-in attempt notification) to protect devices and online accounts

• Changing privacy settings on your device, account or app to limit the sharing of personal data and information (e.g. name, contact information, photos)

• Verifying the reliability of information found online

226. Complementary sources on cybersecurity include statistics on security incidents and the protection measures put in place by enterprises.[32] These topics are not dealt with in this Manual as they are not collected by household surveys.

227. A particularly relevant issue regarding security is child online protection, which comprises topics such as awareness and attitudes, risk-prone behaviour of children, incidents and children’s responses and preventive actions. A statistical framework for measuring child online protection has been developed by ITU (ITU, 2010b). Some of the core indicators can be considered as part of this framework by considering the relevant age breakdowns (children under 15). These include:

• percentage of individuals under 15 who used the Internet during the past three months (HH7);

• risk-prone activities, indicated by the following categories of HH9 (Internet activities undertaken by individuals):

o Participating in social networks (creating user profile, posting messages or other contributions to Facebook, Twitter, Instagram, Snapchat, etc.)

o Accessing or posting opinions via any device on chat sites, blogs, newsgroups or online discussions (e.g. on civic or political issues, general interest topics) that may be created by any individual or organization;

o Purchasing or ordering goods or services;

o Watching web television (either paid or free of charge);

o Streaming or downloading images, movies, videos or music; playing or downloading games;

o Uploading self/user-created content to a website to be shared (text, images, photos, videos, music, software, etc.);

o Making calls (telephoning over the Internet/VoIP using Skype, WhatsApp, Viber, iTalk, etc.; includes video calls via webcam);

• location of individual use (HH8), especially those locations where parental control is more difficult (outside the home); and

• frequency of individual use (HH12), as it can provide a rough indication of time spent online and potential risk of addiction.

228. In order to be able to collect indicators about the online activities of children, the scope of relevant surveys should include individuals under 15. This should be taken into consideration when including ICT-related questions in an existing household survey vehicle that restricts its age scope to populations over 15 (such as labour force surveys in most countries).

229. Indicator HH14 (barriers to household Internet access) includes the response category “privacy or security concerns” as a potential reason for not having Internet. Similarly, indicators HH19 (reasons for not using the Internet for individuals) and HH23 (reasons for not purchasing online) include the same response category. These imply a conscious choice by the household or the individual based on trust in the online environment, rather than a limitation in terms of availability of infrastructure or service.

230. There are two potential new indicators that have been discussed by the EGH but no standards are set:

• children who have been subject to forms of victimization, such as cyberbullying, exposure to a medium that might foster harmful behaviour on the part of children, exposure to pornography, child solicitation or grooming, online encounters resulting in offline meetings and

• actions taken by parents about children’s use of Internet, such as agreeing on house rules about use of Internet and personal devices, installing an Internet content filter, supervising or monitoring children’s use of Internet, allowing the children to access the Internet only from a public area in the house, and educating children about safe and appropriate Internet use.

The social and economic impacts of ICT access and use by households and individuals

231. There has been relatively little data collection and analytical work done on the impacts of ICT access and use by households and individuals. However, the availability of ICT has obviously changed – and will continue to change – the types of jobs people do, how they work (for instance, teleworking or cloud computing), how they communicate with others, how they access commercial and government services, and what they do in their leisure time.

232. In respect of social statistics, the following ideas for impact measurement have been suggested (OECD, 2007):

• include perception questions in social surveys (e.g. asking people how the Internet has affected their lives);

• use of standard time use and household expenditure survey methodologies and classifications in the area of ICT measurement, in order to find out how much time people are spending using ICT (and how this is changing) and how much money they are spending on ICT compared with other goods and services;[33] and

• collection of statistics on ‘teleworking’ and other changes in work patterns that are driven by ICT; such data may be collected from specific ICT use surveys or via labour force surveys.

Internet of Things (IoT)

233. The EGH meeting held in September 2017 agreed to introduce a discussion on items relating to the Internet of Things (IoT) and how information on households’ and individuals’ access and use of IoT devices could be captured in household surveys. In the few existing cases, the survey questions are focused on the availability of devices in the household.

Box 21. Canada: measuring IoT in household surveys

In 2018, the Canadian Internet Use Survey used a very basic question to try and get a sense of which and how many IoT (smart) devices were located in a respondent's home. While not directly aligned, these may be used as an indicator of IoT infrastructure in a home. The question is worded as follows:

What Internet-connected smart home devices do you currently use in your primary residence?

Internet-connected smart home devices have the ability to be controlled or monitored remotely through an app or a website. Select all that apply.

Do you use:

• a smart speaker (e.g., Google Home, Amazon Echo)

• a video camera connected to the Internet (e.g., security camera, Nest Cam, baby monitor)

• a smart door or window lock

• a smart thermostat (e.g., Ecobee, Nest, Sensi)

• a smart switch or lights (e.g., Samsung Smart Switch, Phillips Hue Light)

• a smart large appliance (e.g., fridge, stove, dishwasher)

• a smart TV

• other (e.g., garage door opener, vacuum)

• none

Source: Contribution to the EGH Forum.

234. At this stage, there are no statistical standards on how to collect IoT-related indicators in household surveys.

Gender-sensitive indicators

235. While all indicators related to ICT use can be disaggregated by sex, there are other gender-sensitive indicators that could be considered. ITU, as a member of the Partnership is working together with UNCTAD and various partners in elaborating a list of core indicators to monitor gender equality and integration in the context of ICT[34]. There are a number of areas where further sex-disaggregated data and indicators are needed, such as on skills, content, employment, education, representation of women in ICT decision-making and impact of ICTs on women lives.

236. Gender‐related core indicators related to mobile phones are essential because mobile phones are the predominant ICT globally, far more numerous than computers. One indicator added to the core list, which has special relevance for gender-based analysis is mobile phone ownership (HH18), which has been selected as indicator to monitor SDG Target 5.b. This is an important indicator to examine gender differentials because of the intense interest in the mobile phone as an instrument for women’s empowerment. Mobile phone access through sharing or borrowing rather than owning often entails dependency relationships and obligations that may be uncomfortable for women while owning a phone confers privacy, convenience and greater security.

237. Regarding the classification of households, the consideration of female-headed households can provide information about the disadvantages in comparison to other types of households in ICT access and usage. Statistical evidence of disparities between female-headed households and other households could lead to policy and programmes that facilitate female-headed-households access to ICT, also likely increasing access and use by girls. Collecting data on household composition and the notion of a head of household can be difficult for enumerators and respondents, in a context of growing diversity of household form (such as same-sex households, with and without children, child-headed households, jointly headed households and households of unrelated individuals, among others).

Time-related issues

238. There are several aspects of a survey that have a time characteristic. They are:

• survey frequency (how often a survey is conducted);

• reference period/s (recall period/s) used in the questionnaire, typically when asking questions of an individual’s ICT use activities;

• reference date/s used in the questionnaire when asking about a given situation (e.g. whether the household has Internet access); and

• time series, that is a data series derived from surveys that are sufficiently compatible to allow comparison of data over time.

239. For a particular country, survey frequency will be determined by national priorities, available resources and the level of penetration of ICT infrastructure and use. A country’s multiyear statistical programme should be taken as a reference to establish the frequency of inclusion of ICT-related questions in a range of household surveys.

240. Where an annual collection is not feasible, countries should attempt to align their collection years with those of countries they wish to be compared with. This will commonly be countries in a region or countries with the same development level.

241. Most developed economies that conduct household ICT access/use surveys do so annually. Among developing economies, the situation is complicated by different frequencies for measuring household access and individual ICT use. A small number of countries with strong interest in ICT issues have conducted surveys more frequently than annually (e.g. the Republic of Korea and China). In Latin America, most countries include questions on household ICT access annually, for instance, in multipurpose and life conditions household surveys. However, the frequency of inclusion of questions on individual ICT use varies. Some countries have collected the information annually, while other countries have included ICT use questions every two or three years. Table 8 has more information on surveys used by Latin American and Caribbean countries to measure household access to, and individual use of, ICT.

242. Reference period is the time period referred to in the survey when asking about individual use of ICT. The core ICT indicators standards and this version of the Manual recommend a three month reference period instead of the 12-month period previously recommended. Country practices vary, but ideally, the reference period of three months should be aligned among countries in order to obtain comparable data. Although longer reference periods allow the capture of more events, both ‘memory’ and the ‘telescoping’ effects (described below) are likely to increase with a longer recall period.

243. When a respondent is asked about a particular event during a reference period, two type of difficulties can arise. The first is the ‘memory effect’ (or ‘recall effect’): respondents tend to forget events. The second problem is less obvious: the event is not forgotten but recall of the date of occurrence may be inaccurate. In particular, people may report events as occurring within the reference period when they actually occurred outside of it. This is called the ‘telescoping effect’. Empirical evidence from the field of psychology shows that there is a tendency to ‘telescope’ particularly when an event involves activities that reveal some kind of social or economic status.

244. As the core indicators on household access are at a reference date (‘point-in-time’ data), it is preferable to have alignment of reference dates across participating countries. Reference dates are typically the day of interview, with questions of the type “Does this household have a computer?”. However, they may be another date, for example, the last day of a calendar year.

245. Issues related to the reference period with respect to questionnaire design are also dealt with in Chapter 6.

246. Countries that are dependent on existing survey vehicles may be unable to adopt all these timing recommendations, because of constraints imposed by the timing of the vehicles used.

247. Other time-related issues include the importance of time series data in tracking changes over time and to improve survey processes and understanding. It is expected that the extra cost of conducting surveys on a regular basis will be offset, to some degree, by the availability of experienced staff and existing systems and procedures.

[1] Note that ITU collects the data as numbers, not proportions, and then prepares the indicators in the form of proportions (percentages).

[2] COICOP is now relatively out of date in respect of ICT equipment and services. While COICOP categories have been used in Table 4, some suggestions have been made regarding the inclusion of more recent examples, such as tablets and e-book readers.

[3] Reference period is the time period referred to in the survey when asking about individual use of ICT. The core ICT indicators standards and the Manual recommend a 3-month reference period, instead of 12 months as in a previous edition. Country practices vary, but ideally, reference periods should be aligned in order to obtain comparable data.

[4] Following the OECD definition, an e-commerce purchase occurs where an individual buys or orders a good or service online, irrespective of the method of payment (and whether the payment is made online or offline). A payment that occurs online without the ordering of a good or service (e.g. e-banking) is not regarded as e-commerce.

[5] In-app purchasing refers to the buying of goods and services from inside an application on a mobile device, such as a smartphone or tablet. In-app purchases allow developers to provide their application for free.

[6] See the UNCTAD Manual for the Production of Statistics on the Information Economy: (https://unctad.org/en/pages/PublicationWebflyer.aspx?publicationid=1079).

[7] Some NSOs such as Statistics New Zealand are collecting information about the related concepts of sex, gender and gender identity (see: https://www.stats.govt.nz/reports/sex-gender-and-sexual-orientation) but this will not be discussed in this Manual as it is at the pilot stage. Undoubtedly, ICT facilitates communication between members of specific communities especially in countries were sexual minorities suffer some type of discrimination.

[8] The ‘youth’ age standard as specified in 2005 (in Partnership, 2005) was 16-24 years per recommendations made by OECD and Eurostat for their model surveys. This was revised to conform to UN standards and the practices of a number of countries for the 2010 revision (Partnership, 2010). Countries which use 16 (or, less commonly, 18) as their lowest age should report on that basis but note it in survey metadata.

[9] See https://www.itu.int/dms_pub/itu-d/opb/ind/D-IND-COP.01-11-2010-PDF-E.pdf

[10] Principles and Recommendations for Population and Housing Censuses Revision 2 (UNSD, 2008a).

[11] See: http://uis.unesco.org/sites/default/files/documents/international-standard-classification-of-education-isced-2011-en.pdf (UNESCO, 2011) for details.

[12] ILO (1993). The ICSE-93 consists of the following six categories: employees; employers, own account workers, members of producers’ cooperatives, contributing family workers; and workers not classifiable by status. The ICSE categories refer to work for pay or profit, per decisions made by the 19th International Conference of Labour Statisticians in October 2013, http://www.ilo.org/global/statistics-and-databases/meetings-and-events/international-conference-of-labour-statisticians/19/lang--en/index.htm.

[13] https://www.ilo.org/wcmsp5/groups/public/---dgreports/---stat/documents/meetingdocument/wcms_636039.pdf

[14] For more information on ISCO, see: http://www.ilo.org/public/english/bureau/stat/isco/index.htm

[15] According to the ILO, the basic criteria used to define the system of major, sub-major, minor and unit groups are the skill level and skill specialization required to competently perform the tasks and duties of the occupations.

[16] Correspondence tables are available from ILO, http://www.ilo.org/public/english/bureau/stat/isco/isco08/index.htm.

[17] See: https://ec.europa.eu/jrc/en/publication/eur-scientific-and-technical-research-reports/digcomp-20-digital-competence-framework-citizens-update-phase-1-conceptual-reference-model.

[18] See: https://ec.europa.eu/digital-single-market/news/new-comprehensive-digital-skills-indicator.

[19] See for instance the Toolkit “ICT and disability” at: https://www.un.org/esa/socdev/documents/disability/Toolkit/ICTandDisability.pdf for a detailed discussion on how ICT can help PwD.

[20] Of particular interest is the use of such categories to identify non-users. According to ECLAC, variables like ethnicity and language spoken are relevant for ICT access discrimination in some Latin American countries. Countries with a very high proportion of non-national migrant workers (such as those in the Persian Gulf) distinguish between national and non-nationals in their questionnaires.

[21] See the UNSD Recommendations for Population and Housing Censuses (UNSD,2017), Part four, Chapter II, Section C, par. 4.92 to 4.100

[22] http://gsars.org/wp-content/uploads/2018/12/GS-GUIDELINES-RURAL-AREAS-EN-FINAL-2018.pdf

[23] For examples of such cross-classifications, see ITU (2008).

[24] ECLAC (2007).

[25] See OSILAC ICT Statistical Information System (http://www.cepal.org/tic/flash).

[26] See: http://www.itu.int/ITU-D/treg/related-links/links-docs/USOF-India.pdf

[27] See: http://www.fcc.gov/encyclopedia/universal-service. Since the 1997 USF, the FCC has made changes to the USF to include the financing of access to broadband.

[28] See, for instance, OECD (2010).

[29] Recognizing the potential of connecting more households with a computer and Internet access, a number of governments, including Brazil, Uruguay and Peru, have subsidized the purchase of a computer for low-income families. For example, Uruguay has a national ‘one laptop per child’ programme in public schools (100 per cent of students have their own laptop and over 95 per cent of public schools have Internet access). An impact of this program is a narrowing of the digital divide between high- and low-income households to only 6 per cent (in 2012). See: https://www.anep.edu.uy/sites/default/files/images/Archivos/publicaciones/plan-ceibal/plan%20ceibal%20in%20uruguay.pdf

[30] Available for various years in http://epp.eurostat.ec.europa.eu/portal/page/portal/information_society/methodology.

[31] Several questions (and parts of questions) in the OECD 2005 model questionnaire deal with the topic of trust in the online environment. The questions concern the issue of IT security (questions 8, 15 and 16) and privacy, security or trust as barriers (questions 5 and 23). See OECD (2011).

[32] Eurostat disseminates enterprise-level data on ICT security that have been collected by in the EU “Community survey on ICT Usage and e-commerce in Enterprises” (ICT-Entr) for 2019 and 2015, including up to 26 questions on security measures, incidents and cyberinsurance. Enterprises with 10 or more persons employed are covered, with stratification by economic sectors. No micro-data can be used for enterprise-level studies, but a comprehensive database is available for further research (see: https://ec.europa.eu/eurostat/web/digital-economy-and-society/data/comprehensive-database) . This database also includes other relevant indicators, such as security reasons as obstacle for adopting certain technological solutions. The OECD has worked on the harmonization of Computer Security Incident Response Teams (CSIRT) statistics (see: http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=DSTI/ICCP/REG(2013)9/FINAL&doclanguage=en). At the national level, CSIRTs act as trusted points of contact for computer security incident response. Their daily flow of activities generate data that can be translated into meaningful cybersecurity indicators. The European Union Agency for Cybersecurity (ENISA) compiles indicators in security incidents from national CSIRTs to produce the “Threat Landscape Report”.

[33] Note that indicator HH16 addresses this issue.

[34] See https://www.itu.int/en/ITU-D/Statistics/Documents/events/wtis2013/001_E_doc.pdf for the report of stocktaking and assessment on measuring ICT and Gender (2013).

Chapter 5. Data sources and collection techniques for ICT household statistics

248. This chapter considers the sources of data and the data collection methods for ICT household statistics that can be employed by statistical agencies. Sources and methods chosen will be influenced by:

• the resources and time available for the project;

• available survey vehicles, i.e. other household surveys to which questions on the access to, and use of, ICT can be added;

• requirements to maintain consistency over time;

• infrastructure and expertise available in the statistical agency; and

• practicalities such as geography, accessibility of households via postal, telephone or email communication and language.

249. This chapter examines the advantages and disadvantages of various options. It also looks at the practices of countries that collect ICT household data.

Data sources: surveys, administrative data and Big Data

250. The main data sources for social and demographic data are surveys and administrative data. The types of surveys considered in this Manual are surveys, including household sample surveys and population censuses.[1] Countries may use a combination of sources to investigate the access and usage of ICT (see Box 22).

Box 22. Nepal: surveys used to collect household ICT access data

The Central Bureau of Statistics of Nepal conducts several multipurpose surveys that collect ICT household data. They are:

Labour Force Survey

The Nepal Labour Force Survey is generally conducted every five years; the last survey was in respect of 2017-18. In that survey, the household head was asked about the presence in the household of radios, TVs, fixed telephones, mobile telephones and computers, and also how many of each type of item were present in the household (for instance, how many radios). The sample size for the 2017-18 survey was 18,000 households and data collection was by face-to-face personal interview using tablet computers with relevant individuals.

Living Standards Survey

The Nepal Living Standards Survey is also generally conducted every five years. The last survey was conducted during 2010-11 and the household head was asked about the availability in the dwelling unit of a telephone, mobile telephone, pager, cable TV, e-mail and the Internet. This survey is particularly interesting because it was designed as a rotating panel. Since this was the third round of the survey (the first one was in 1995-6), it is possible to study the growth in ICT access over time. It is important to emphasize that datasets are available for researchers. This is an example of good practice for the developing world.

Population Census, 2011

Nepal included most of the ICT access questions recommended by UNSD (2008a) in its last population census. These are: household access to radio, TV, fixed-line telephone, mobile cellular telephone and computer. The census also included a question on household access to the Internet. An advantage of using a census to collect this information is the potential for detailed tabulation, for instance by geographic area.

Sources: UNCTAD and ITU, unpublished reports and research.

https://nepalindata.com/media/resources/items/20/bNLFS-III_Final-Report.pdf

https://cbs.gov.np/poverty/

251. In all known cases, the ICT household indicators considered in this Manual are collected by surveys (including censuses). Administrative data sources are unlikely to generate indicators on household ICT access and individual ICT use, because administrative data are generally not classifiable according to the characteristics of interest. However, such sources are successfully used by ITU to produce statistics on available infrastructure, prices and number of subscriptions. It is important to note that subscription[2] data are not equivalent to survey data on households and individuals, but they can be very useful, especially for monitoring uptake of ICT in countries with low levels of ICT access and use. The use of administrative data is briefly described in paragraphs 261 to 265.

Box 23. Kenya: Inclusion of ICT Questions in the Population Census, 2019

The Kenya National Bureau of Statistics (KNBS), the national statistics organization in Kenya included some ICT indicators for the individuals and households in the Kenya population and housing census 2019. Indicators at individual level were: mobile phone ownership (HH3), use of mobile phones (HH10) (to capture those who don’t own a mobile phone but use it), use of the Internet (HH7), and use of computers (HH5), which targeted the population aged 3 years and above. In addition, an e-commerce question was also included targeting the population aged 15 years and above.

At the household level, the indicators provided were ownership of radios (HH1), computers/laptops/tablets (HH4), Internet accessed via mobile phones/modems and Internet accessed via fixed Internet at home (HH6 & H11), televisions (HH2 & HH13) (which was divided into 4 categories: (1) TV with Free to Air set-top box/digital TV, (2) TV with Pay TV decoder, (3) Internet Protocol TV (IPTV), (4) Analogue TV (TV with no connection/signal). Analogue TV was included, since after the switch over to Digital TV in Kenya, several households are now unable to view content on their television, so this category would capture those still excluded.

Source: Kenya National Bureau of Statistics

252. The so-called “Data Revolution” has vastly increased the amount of data collected and stored every day by automatic means, opening promising avenues for private and public services based on the advanced use of such data. Many NSOs and international organisations are exploring the possibility of using Big Data sources such as Call Record Data from mobile companies, satellite images, machine-to-machine transactions, etc. for the production of official statistics. This requires not only agreements for access to such sources, often produced by private service providers, but also advanced data science skills of NSO staff. In the field of ICT statistics, pilot projects[3] have been undertaken with the collaboration of volunteering mobile phone service providers. As the practice is in a preliminary phase and has not been routinely incorporated into the production process of NSOs, it will not be further described in this Manual.

253. This Manual will focus therefore on the range of surveys that can be used to collect ICT household data.

Types of surveys

254. There are various types of surveys used to collect information about households and household members. Two main types can be distinguished for the purposes of this Manual: stand-alone surveys focused on ICT, and other household surveys that may contain some questions on ICT topics.

Surveys not specific to ICT

255. Countries may decide to include ICT questions in different household surveys, in order to have a more complete picture of access and use for different population segments. Table 8 describes the range of surveys used by Latin American and Caribbean countries that include some questions on household access to, and individual use of, ICT.

256. Surveys not specific to ICT include:

• Multipurpose household surveys. Multipurpose household surveys collect data on more than one subject via a single household survey. The usefulness and the feasibility of multipurpose household surveys have been largely proven by the Living Standard Measurement Survey (LSMS) project of the World Bank[4] which has been applied in many countries since the 1980s. Despite some necessary compromises, this type of survey can be cost effective and produce timely results if the survey vehicle is ongoing. Once collected and computerized, data can be further edited and tabulated by separate subject matter teams. Other than the LSMS, specific surveys that are sometimes used for collecting ICT household data in developing economies include Demographic and Health Surveys (DHS), the Multiple Indicator Cluster Survey (MICS) funded by UNICEF, and other multi-topic surveys carried out by NSOs. Box 24 describes use of multipurpose surveys to collect ICT data in the Palestinian Authority and Costa Rica.

Box 24. Inclusion of ICT questions in the Palestinian Housing Conditions Survey 2015 and the Multipurpose Household Survey of Costa Rica

The Palestinian Central Bureau of Statistics has carried out several specific household surveys on the use of ICT by businesses (2007, 2009 and 2011). In 2015, questions relating to access to ICT at the household level were included in the questionnaire of the Housing Conditions Survey: access to a TV, computer, laptop, Ipad/Tablet, telephone line, smart phone, Internet, and mobile phone.

In Costa Rica, the National Institute for Statistics and Censuses (INEC) included in the National Household Survey (ENAHO) of 2014 the questions necessary for the compilation of ICT indicators. These include access to radio, colour TV, cable TV, fixed and mobile telephone, fax, computer, Internet, as well as electricity to provide a reference indicator.

Sources: Palestinien CBS, Survey Catalogue, (http://www.pcbs.gov.ps/PCBS-Metadata-ar-v4.3/index.php/catalog);

Costa Rica, (http://www.inec.go.cr/vivienda)

• Household budget surveys. Household expenditure (budget) surveys are designed to measure household expenditure and are also used by a number of countries to identify household access to ICT equipment and services.[5] Some countries include questions on household income in their household expenditure surveys, and this is the recommended source for indicator HH16 on ICT expenditure. As we have seen above, income is a useful classificatory variable for ICT access (and use) data.

• Population censuses. Population censuses can be used to collect a small number of ICT access and/or use data items. Although this is usually an expensive option and population censuses are infrequent (usually once in a decade), it remains a good alternative in countries that have never collected any ICT household data and are not planning to do so in the near future. In addition, population censuses provide very good detail about the variables collected and can provide a basis for the design of samples for future ICT-specific surveys. UNSD (2017) presents standards for the 2020 round of population and housing censuses. The standards include ICT access as a core topic. In particular, the following indicators are suggested for inclusion in censuses:

o Household having radio

o Household having television set

o Household having fixed-line telephone

o Household having mobile cellular telephone(s)

o Household having personal computer(s)

o Household accessing the Internet (landline and mobile connections)

o Household accessing the Internet from elsewhere other than home, and

o Household without access to the Internet.

257. In countries with low statistical capacity, other surveys have been used as vehicles for ICT-related questions, such as internationally-sponsored Demographic and Health Surveys (DHS), Multiple Indicator Cluster Surveys (MICS)[6], Living Standards Surveys (LSS) and Family Budget Surveys.

Stand-alone ICT surveys

258. Stand-alone household surveys dealing with ICT access and use allow for more details to be collected than is usually possible in an existing survey vehicle designed for investigating other topics. An ICT household survey can have a customized sample design, while the information gathered through other surveys will depend on the design of those surveys. The advantages and disadvantages of specific ICT surveys compared to the inclusion of ICT questions in existing survey vehicles are discussed below.

Box 25. Saudi Arabia: Households and Individuals ICT Access and Usage Survey

The General Authority for Statistics in Saudi Arabia (GASTAT) has established a survey programme aimed at creating a statistical database in all fields (including the so-called “knowledge statistics) to promote development of studies that tackle indicators on the ICT sector, including the measure of households’ access and individuals’ usage.

As part of these surveys, GASTAT implemented for the first time in 2017 the survey on Households and Individuals ICT Access and Usage Survey.

International recommendations and standards have been taken into account while preparing the survey methodology and designing the survey form. From 2017, this survey is conducted on an annual basis.

Households and Individuals ICT Access and Usage Survey questionnaire includes 7 main sections: Section I: Household identification detail; Section II: Housing characteristics; Section III: Household access to ICT; Section IV: Postal services provided to households, Section V: characteristics of household members; Section VI: individuals’ usage of cell phone and computer and Section VII: Individuals’ usage of Internet.

For the implementation of the survey,1600 enumeration areas were selected from the 3600 enumeration areas of the 2010 Census Framework, later updated in 2016. In each enumeration area, 15 households were selected, totalling 24,000 in the sample. Data collection was carried out by personal interview, using tablets.

Source: https://www.stats.gov.sa/en/survey/13502.

Box 26. Community Survey on the usage of ICT among households and individuals- Luxembourg

The ICT survey collects data on the usage of Information and Communication Technologies (ICT) among households and individuals aged between 16 and 74.

The data collected deal with:

• household access to ICT (computer, Internet and mobile phone);

• type(s) of Internet connections used at home and the reason(s) for not having access to ICTs;

• ICT equipment and frequency of use (for individuals);

• The use of e-government and e-commerce by individuals;

• E-skills and problems.

Every year, an ad-hoc module focusing on a specific topic is inserted into the main questionnaire:

• 2018: Trust, security and confidentiality, ICT at work

• 2017: No specific topic

• 2016: Protection of personal data

• 2015: Internet security

• 2014: Use of Cloud Services

• 2013: Use of e-Government

• 2012: Mobile use of the Internet and ubiquitous connectivity

• 2011: E-skills

• 2010: Internet security

• 2009: E-commerce

The results of the ICT survey are disseminated via STATEC publications (Regards, Bulletins…) as well as on the Statistics Portal.

Source: https://statistiques.public.lu/en/surveys/espace-households/TIC-survey/index.html

Advantages and disadvantages of using stand-alone ICT surveys

259. Given the cost of developing and running a household survey, it is likely that in statistical systems with little resources, statistical producers would prefer not to run stand-alone household surveys on ICT access and use (or at least only infrequently). Since the publication of earlier versions of this Manual, many countries have adopted ICT stand-alone surveys. Countries that have collected ICT statistics through other household surveys have included the relevant questions in multipurpose and living conditions surveys (see tables 14 and 15 for examples). In a few countries, they have been included in a different survey such as a labour force or budget survey (see Table 7).

260. Stand-alone household surveys on ICT have been implemented mostly in developed economies,[7] but also some developing economies have conducted at least one and some do so annually, such as Brazil and Qatar.

Table 7. Types of surveys where ICT questions have been included between 2014 and 2017

Type of survey

Number of surveys

2014

2015

2016

2017

Labour Force Survey (LFS)

7

8

8

6

Multipurpose Household Survey (MPHS)/Household survey

5

6

9

7

Living Standard/Conditions Survey or Budget survey

16

11

12

10

ICT stand-alone survey

48

57

59

70

Other types of household surveys

11

12

11

10

Population census

4

1

1

2

Other and unknown data sources (estimation, projection…)

4

5

7

2

TOTAL

95

100

107

107

Source: ITU database (2019) as per data reported by countries to ITU. The table refers to surveys rather than countries.

261. There are advantages to using an existing survey vehicle, apart from cost. The main one is that a number of other data items and classificatory variables are available and these can be cross-tabulated against ICT data to produce a richer dataset. For example, including an ICT-related module in a labour force survey would allow a breakdown of use of ICT by individuals according to their employment status and occupation. In a household budget survey, it would allow analysis of differences in ICT access by household income groups. It is also important for producing ICT statistics where use by sex can be cross-tabulated with variables such as income, labour force status and/or occupation.

262. However, there are disadvantages as well. The use of existing survey vehicles may impose constraints on applying some of the recommendations presented in this Manual. For instance, an existing labour force survey will have an established methodology and questions.[8] These may not be optimal for collecting ICT data, nor for producing disaggregated ICT indicators.

263. Another disadvantage of using an existing survey vehicle is that ICT topics will be competing for space and time, given a likely pressure to minimize interview time in order to reduce respondent burden and cost. This is particularly important in the case of ICT topics in a population census. Most countries will therefore need to determine a small set of important questions. The core list of ICT indicators developed by the Partnership is a recommended starting point, in conjunction with other information requirements of a country’s policy-makers.

264. Given the increasing importance of ICT statistics, it is recommended that countries explore the possibilities of implementing specific ICT surveys to households as a priority.

Administrative sources

265. The use of administrative registers for the production of social and economic statistics has been identified as one of the key directions for the modernization of official statistics worldwide. Records collected and managed by public institutions for the implementation of diverse sectoral policies (education, health, labour, social protection, housing, etc.) can be re-used for the compilation of social statistical data, thus decreasing the response burden and production cost. Their use requires, nonetheless, the application of sound statistical procedures to guarantee their relevance, coherence, comparability, precision and other statistical quality dimensions.

266. Administrative records about households and individuals can be used to reduce the response burden by pre-filling questionnaires with administrative information made available to the NSO. They can also be used to link individual records across data sets. For example, a survey on ICT use can eventually be linked to education registers of respondents, allowing for the calculation of breakdowns by education level. At the household level, administrative information such as tax records or socio-economic classifications can be used to break down ICT indicators by segments of households.

Table 8. Surveys used by countries of the Latin American and Caribbean region to measure household access to, and individual use of ICT

Country

Survey

Type of Survey

Year

Argentina

Access and use of information and communication technologies, National Survey-ENTIC 2015

Stand-alone ICT survey

2015

Module of Access and Use to Information and Communication Technologies- Permanent Household Survey

Multipurpose household survey

2016 and 2017

Bolivia

Household Survey

Multipurpose household survey

2014

Brazil

ICT Households Survey

Stand-alone ICT survey

2014- 2017

Chile

Survey of access, uses and users of Internet in Chile

Stand-alone ICT survey

2015- 2017

Colombia

National Quality of Life Survey - NQLS

Life conditions survey

2014- 2017

Costa Rica

National Household Survey

Multipurpose household survey

2014- 2018

Cuba

National Occupation Survey

Labour force survey

2014- 2017

Dominican Republic

National Household Survey of Multiple Purposes (ENHOGAR)

Multipurpose household survey

2015- 2018

El Salvador

Multiple Purpose Household Survey (EHPM)

Multipurpose household survey

2014–2017

Guatemala

National Life Conditions Survey

Life conditions survey

2014- 2016

Honduras

Permanent Multiple Purpose Household Survey

Multipurpose household survey

2015–2017

Mexico

Module on Information and Communication Technologies in households (ENDUTIH)

ICT module attached to the labour force survey

2014

National survey on the availability and use of ICT in households.

Stand-alone ICT survey

2015- 2018

Nicaragua

Labour Measurement Household Survey

Labour Force Survey

2006

National Life Conditions Survey

Life Conditions Survey

2001, 2005, 2009

Panamá

Multiple Purpose Household Survey

Multiple Purpose Household Survey

2014–2016

Paraguay

Permanent Household Survey

Life conditions survey

2014- 2018

Peru

National Household Survey (ENAHO)

Life conditions survey

2014- 2018

Table 8. Surveys used by countries of the Latin American and Caribbean region to measure household access to, and individual use of ICT (continued)

Country

Survey

Type of Survey

Year

Uruguay

Continuous Household Survey

Multipurpose household survey

2014 and 2015

Venezuela

Household Sample Survey

Multipurpose household survey

2003, 2005–2013

Source: ITU database (2019) as per data reported by countries to ITU.

267. In addition to records on persons and households owned by public entities, administrative data are also held by the providers of ICT services such as mobile phone companies and Internet providers, for management and commercial purposes (e.g. billing, advertising, customer service). The access to such sources is restricted by the commercial practices and the legal framework for protection of personal data. In order to use such data for the production of ICT indicators, the NSOs have to explore the possibilities for collaboration with such private entities. In most cases, existing experience of this collaboration is limited to pilot exercises to test the feasibility of using such data.

268. The use of ICT-related administrative records requires that definitions and concepts used by private data owners are harmonized or adapted to their use for statistical production. In the field of ICT statistics, a well-known case is the difference between Internet subscriptions and access and use of the Internet (a subscription can be used by more than one person in a household, and one person may have more than one subscription).

269. A specific type of administrative record is that related to geo-positioning of dwellings, such as postal addresses. By including geo-spatial information at the time of data collection, NSOs may be able to produce disaggregated ICT indicators, useful to explore the gaps in rural and urban areas, or within large cities. Combining statistical and geo-spatial information requires advanced mathematical modelling and the protection of confidentiality of personal and household data. These models will not be discussed in the Manual[9].

Data collection techniques

270. Households, and/or individuals within them, can be presented with questions in face-to-face interviews, by telephone interviews, by a self-enumerated questionnaire (posted or delivered) or by e-mail/website interaction. Information for some members of the household can be provided by proxy if another member of the household answers questions on their behalf.

271. Most countries use personal interview techniques for collecting ICT household data. Personal interviewing includes face-to-face interviewing (generally in the respondent’s home), telephone interviewing and a combination of the two (see Box 27 for examples). A personal interview, whether face-to-face or telephone, requires careful training and evaluation of interviewers to avoid bias (for instance, affecting the likelihood of a response option by their tone of voice).

Box 27. Use of a combination of techniques to collect ICT household data

Macao, China

In 2018, data collection for the Survey on Information Technology Usage in the Household Sector was conducted in the fourth quarter of 2018 through a supplementary questionnaire of the Employment Survey. In the Employment survey, data are collected by face-to-face and telephone interviews. For the first visit, the visiting enumerator collects information from the household through a face-to-face interview. With the approval of the household, the second visit (3 months later) is carried out using a CATI (Computer-Assisted Telephone Interviewing) system to verify/update respective information of the household. If not, the second visit is also takes the form of a face-to-face interview.

New Zealand

In New Zealand (NZ), the Household Use of ICT (HHICT) survey is a supplement to the Household Labour Force Survey (HLFS) and was conducted every three years from 2006–2012.

Personal telephone interviews using CATI were conducted for the majority of respondents, with face-to-face interviews using computer-aided personal interviewing (CAPI) for the minority. This follows the methodology used for New Zealand’s labour force surveys, where households remain in the survey for eight quarters. Newly selected households are initially interviewed face-to-face and thereafter by telephone.

The other sources used to compile HHICT by NZ statistics[1] are:

• 2013 Census – household cellphone and Internet access

• Household Economic Survey: Year ended June 2013 – household ICT spending

• New Zealand General Social Survey: 2014 – individuals’ perceptions of Internet safety for online transactions

Denmark

The 2018 ICT household survey was conducted with a combination of self-administered web survey and CATI. Respondents received a letter with an Internet link. They could choose to fill out the questionnaire on the web or wait for a phone call from Statistics Denmark, in which case CATI was used. If a person has filled out the questionnaire on the web, he/she was not called. At least 6 attempts were made to reach all possible respondents.

Luxembourg

Since 2018, the ICT survey has been conducted using a mixed-mode data collection approach combining telephone and web interviewing. A private company is responsible for the fieldwork on Statec’s behalf.

Source: ITU (2009b), Eurostat survey metadata.

https://www.dsec.gov.mo/Statistic.aspx?NodeGuid=0cd0907c-c23a-42b3-90aa-8f849413e70c

http://archive.stats.govt.nz/browse_for_stats/industry_sectors/information_technology_and_communications/hhold-use-of-ict.aspx

https://circabc.europa.eu/sd/a/b2eb96e3-8740-489f-a063-ad0303bbc949/isoc_sdds_hh_dk_2018.htm

https://statistiques.public.lu/en/surveys/espace-households/TIC-survey/index.html

[1] In 2015, Statistics NZ did not collect household use of information and communication technology (HHICT) data. As a result, HHICT statistics were not published in April 2016.

272. Each data collection method has advantages and disadvantages and these are summarized below. For a general discussion on the use of different data collection methods in household surveys, especially in developing economies, see UNSD (2005a); in particular, on the effect of the data collection mode on measurement errors, see Section C, Chapter IX.

Personal face-to-face interviews

273. Personal face-to-face interviewing is very common in household surveys in both developed and developing economies.

274. The main advantages of face-to-face interviews are facilitation of responses by the interviewer (and therefore a higher unit and item response rate) and that the technique does not rely on having an existing survey frame of households or individuals (Chapter 7 describes sampling techniques for household surveys that include cluster sampling based on geographic sampling).

275. In respect of ICT access and use data, face-to-face interviewing enables better explanation of technical terms by the interviewer. It may also allow the interviewer to check aspects of the respondent’s technical set-up, such as the type of Internet connection (for example, dial-up using analogue modem versus a broadband modem or the existence of a fixed line telephone). It is also the most obvious data collection mode when no telephone is available or when telephone listings are incomplete.

276. Where a number of languages or dialects are spoken, a face-to-face technique may be very effective, although it relies on being able to assign the most appropriate interviewer for a given respondent.[10] Cultural aspects should be taken into account when selecting interviewers according to language(s) spoken, sex, age, educational background, etc.

277. The main disadvantage of face-to-face interviewing as a method of data collection is that it is costly, requiring more time spent by the interviewer travelling and locating respondents (though this is offset where respondents are geographically clustered). A relatively recent problem for many countries is the difficulty of accessing some kinds of dwellings, for instance, apartment buildings with access secured by electronic or physical locks. While face-to-face interviews allow careful explanation of relevant topics and questions, interviewers need to be trained not to indicate (e.g. by their facial expression) judgements of responses.

278. Face-to-face interviewing can be facilitated by ICT in the form of direct entry of responses into computers (often portable computers, PDAs or tablets, carried in the field by the interviewer) and supporting software (Computer Assisted Personal Interview software – CAPI). Such software also provides automatic question sequencing (via a series of ‘skips’) and immediate editing of responses. See Box 28 for an example of use of PDAs in a household survey in Lebanon.

Box 28. Lebanon: use of PDAs in Labour Force and Households’ Living Conditions Survey

The Central Administration for Statistics (CAS) used personal digital assistants (PDAs) for the collection of data for the Labour Force and Household’s Living Conditions Survey 2017. The LFHLCS survey will be the first to produce estimates at the national, governorate (mohafaza) and subnational district (caza) levels. It will provide regional disparities and characteristics. Such indicators will inform policy-making and labour market information systems that are instrumental for human development planning. The use of PDAs also shortened the duration of the survey process, as there was no need to design and print paper questionnaires, nor to have a separate data entry process. Data coding was automatically provided for most fields. Note that the technology required a full and detailed training for enumerators and other staff on the use of the devices.

Sources: CAS, http://www.cas.gov.lb/

ILO, https://www.ilo.org/beirut/projects/WCMS_340472/lang--en/index.htm

Box 29. Malta: ICT Usage by Individuals and Households

The survey on ICT usage in Households and by Individuals is conducted on an annual basis, and a letter is sent to the selected individuals informing them that they have been chosen to take part in the ICT survey. Subsequently, an interviewer phones and sets an appointment with the respondent in order to visit and fill in the questionnaire. Interviews are only conducted by phone in cases when a face-to-face interview is refused.

Source: https://nso.gov.mt/en/nso/Sources_and_Methods/Unit_C4/Education_and_Information_Society_Statistics/Pages/ICT-Usage-by-Individuals-and-Households.aspx

Personal telephone interviews

279. Telephone interviewing is less common than face-to-face interviewing but is still reasonably common, especially for developed economies. Like face-to-face interviews, telephone interviewing can be supported by ICT. For instance, interviews can be conducted in a call centre environment and use supporting software (Computer-Assisted Telephone Interview (CATI) software). As with CAPI, CATI software also provides automatic filtering and immediate editing of responses. Telephone interviewing may be based on a list of telephone numbers or be implemented by randomly dialing a sequence of digits that comprise a valid telephone number (called random digit dialing or RDD).

280. The main advantage of telephone interviewing is that it is relatively inexpensive compared with face-to-face interviewing and there is greater flexibility in assigning the most appropriate interviewer (for instance, in a call centre environment, staff with skill in a particular language can be quickly chosen and assigned to deal with respondents who speak that language).

281. While response to telephone interviews is facilitated by interaction with a human interviewer, the relationship between the interviewer and respondent is likely to be less effective than in a face-to-face situation. In addition, the response rate may be lower as it is likely to be easier to reject a telephone caller than a person who is physically present at the respondent’s dwelling.

282. For many developing economies, there are insufficient households with a telephone connection, or a telephone directory listing, to make this method feasible. In particular, there would be problems creating a representative sample (and making contact with selected households) if telephone is the only option. This will apply, for instance, where telephone penetration is low or it is not possible to include mobile or unlisted subscribers. In many cases, there are no listings of cellular phone subscribers, which are increasingly used in developing economies. RDD may overcome the problem of lack of directory listings but may result in an unrepresentative sample. Frequently, quotas that broadly represent the population are set and the quotas are gradually filled as RDD telephone interviews proceed.

283. The telephone can be a useful complementary method of interview, subsequent to face-to-face interviewing, once contact has been made and a telephone number obtained.

284. Table 9 shows metadata on the mode of data collection, survey vehicle and response rate for a number of countries conducting the Eurostat community survey.

Self-enumeration

285. Logically, household surveys can also be conducted by mail, although this is rarely done (no instances are known for developing economies). While there are cost advantages in the use of postal surveys, there are obvious potential disadvantages as well. Some countries use mail to make initial contact and then conduct a personal interview.

286. Self-enumerated surveys are those where respondents complete a questionnaire themselves, usually paper-based. The questionnaire may be mailed (a postal survey) or delivered to the respondent, together with appropriate instructions.[11] Following completion, the questionnaire may be collected by field staff or posted back to the collection agency by the respondent. Postal surveys require a current and complete sampling frame of addresses for households or individuals. They do not involve personal interaction with the respondent, therefore technical questions on ICT use may be less well understood and questionnaire logic is likely to be more complex.

287. Where questionnaires are delivered directly to mail boxes, there may be no need for a sampling frame of addresses. Where questionnaires are collected by field staff, they may be able to check responses and assist with any problems the respondent has in completing the questionnaire.

288. One advantage of self-enumerated questionnaires is that bias due to interaction with interviewers is likely to be reduced or eliminated, especially for sensitive questions. However, unit and item response rates are likely to be lower than for personal interviewing, leading to higher sampling error and likely non-response bias. In situations where there are several languages or dialects spoken in a country, or where literacy is low, self-enumeration is likely to be ineffective, unless combined with other methods. In addition, some ICT questions can be complicated for non-experts, for example, questions on Internet services.

289. The need for good questionnaire design is likely to be greater for self-enumerated questionnaires as interviewers are not available to interpret the questions or manage the flow of questions. The principles of good questionnaire design are therefore particularly important for such surveys; this is explored further in Chapter 6.

Table 9. Mode of data collection, survey vehicle and response rate for countries conducting the Eurostat community survey, 2017

Country

Data collection method(s)

Survey vehicle

Response rate (individuals)

Response rate (households)

Belgium

Self-administered postal survey, via drop-off of paper questionnaire by LFS-interviewer, with possibility to answer via web application. 71.4% of the respondents answered by postal survey and 28,6% via web application.

Embedded in the Labour Force Survey

-

49.3%

Bulgaria

Face-to-face interviews

Stand-alone survey

86.0%

-

Czech Republic

Face-to-face - CAPI (90%) and telephone interview (10%)

Embedded in the Labour Force Survey

-

70.7%

Denmark

Telephone interviews CATI (26.2%) and web-survey (73.8%)

Stand-alone survey

54.3%

-

Germany

Self-administered mail survey with 2 separate questionnaires: household questionnaire and individual questionnaire

Stand-alone survey

-

n.a

Estonia

Telephone interviews CATI (77%) and web-survey computer-assisted web interviewing (CAWI) (23%)

Stand-alone survey

72.2%

-

Ireland

Face-to-face - CAPI (Blaise)

Embedded in the Labour Force Survey (QNHS)

84.2%

-

Greece

Telephone interviews

Stand-alone survey

72.7%

-

Spain

Web-survey CAWI (34%), face-to-face CAPI (25%) and telephone interviews CATI (41%)

Stand-alone survey

77.8%

-

France

Telephone interview CATI (19.5%) and self-administered questionnaire: CAWI (40%) and PAPI (40.5%)

Stand-alone survey

-

57.7%

Table 9. Mode of data collection, survey vehicle and response rate for countries conducting the Eurostat community survey, 2017 (continued)

Country

Data collection method(s)

Survey vehicle

Response rate (individuals)

Response rate (households)

Croatia

CAWI (6%), telephone interviews CATI (76%) and CAPI (18%)

Stand-alone survey

-

60.6%

Italy

Face-to-face, PAPI, based on self-administrated questionnaire (76%) and CAWI (24%)

Embedded in the multipurpose social survey

75.7%

-

Cyprus

Face-to-face - CAPI (Blaise)

Stand-alone survey

99.6%

-

Latvia

Mix of techniques: web interviews (21%), face-to-face interviews - CAPI (45.9%) and telephone interviews - CATI (33.1%)

Stand-alone survey

-

74.3%

Lithuania

Face-to-face interviews (55.8%), telephone interviews (27.6%), self-completed web interviews (15%) and proxy interviews (1.7%)

Stand-alone survey

72.8%

-

Luxembourg

Telephone interviews - CATI

Stand-alone survey

9.1%

-

Hungary

Face-to-face interviews

Stand-alone survey

-

73.8%

Malta

Face-to-face interviews (94%) and telephone interviews (6%)

Stand-alone survey

76.2%

-

Netherlands

Web survey CAWI (76%) and telephone interviews CATI (24%)

Stand-alone survey

46.5%

-

Austria

Telephone interviews - CATI (Blaise)

Embedded in the Labour Force Survey / Micro-census

46.6%

-

Poland

Face-to-face CAPI

Stand-alone survey

88.8%

-

Portugal

Face-to-face CAPI (35.8%) and telephone interviews CATI (64.2%)

Stand-alone survey

79.4%

-

Romania

Face-to-face interviews

Stand-alone survey

-

87.0%

Table 9. Mode of data collection, survey vehicle and response rate for countries conducting the Eurostat community survey, 2017 (continued)

Country

Data collection method(s)

Survey vehicle

Response rate (individuals)

Response rate (households)

Slovenia

Face-to-face (83%) and telephone interviews (17%)

Stand-alone survey

67.7%

-

Slovak Republic

Face-to-face interviews (PAPI)

Stand-alone survey

82.3%

-

Finland

Mixed mode of data collection: web questionnaire (38%) , phone interviews (62%)

Stand-alone survey

51.5%

-

Sweden

Mixed mode: web survey (33.5%) and telephone interviews (66.5%)

Stand-alone survey

45.7%

United Kingdom

Face-to-face interviews in Great Britain (93%) and telephone interviews in Northern Ireland (7%)

Embedded in the Opinions and Lifestyle Survey

45.0%

Iceland

Telephone interviews - CATI

Stand-alone survey

64.1%

Norway

Telephone interviews – CATI

Embedded in the Omnibus survey

55.7%

Switzerland

Mixed mode: web survey (CAWI 75%) - telephone interviews (CATI 25%)

Stand-alone survey

57.7%

North Macedonia

Face-to-face interviews

Stand-alone survey

90.0%

Montenegro

Face-to-face interviews

Stand-alone survey

95.0%

Serbia

Telephone interviews - CATI

Stand-alone survey

83.0%

Turkey

Face-to-face CAPI

Stand-alone survey

94.9%

Note: - = non-applicable

Source: Metadata reports for the Community survey on ICT usage in households and by individuals (2017), Eurostat, https://circabc.europa.eu/faces/jsp/extension/wai/navigation/container.jsp

290. Despite the potential problems outlined above, there are obvious advantages of self-enumerated surveys. These include lower costs and the ability to ask list-based questions. At least two OECD countries, Germany[12] and Japan, have used mailed self-enumeration questionnaires to collect ICT household data.

Internet-based surveys

291. The final method is to use the Internet to assist data collection. This would entail either an online questionnaire, which respondents fill out on a webpage, or a questionnaire that can be e-mailed back to the agency conducting the survey. While this form of data collection for household surveys is rare, there are some examples. Denmark uses both a web form and CATI for its ICT household survey (see Box 27) and the Australian population census in 2011 had both a paper and a web form available for use by households.

292. While this method of data collection may seem attractive, it is only practical as a supplementary form of data collection for household ICT access and use data. For a start, it can only be used by respondents with Internet access and a reasonable level of ICT proficiency. This clearly excludes a large proportion of the population of interest for ICT access/use measurement (that is, non-users). The costs of developing software and appropriate questionnaires might also be high and not cost-effective for a method that can only ever supplement other methods of data collection. Advantages include concurrent editing while the questionnaire is being completed (although methods such as CATI and CAPI also do this), reduced actual data collection costs and perhaps novelty and convenience value for respondents that might increase response rates.

293. In summary, while there are some cost advantages to such an approach, there are a number of reasons why this method will not be feasible as a primary means of data collection, especially for developing countries.

294. Table 10 summarizes the main advantages and disadvantages of the different data collection methods.

Table 10. Summary of data collection methods[13]

Method

Main advantages

Main disadvantages

Face-to-face personal interview

This is the most direct method of collecting information. It facilitates direct interaction between the interviewer and the interviewee, allowing checking and follow-up questions.

An interviewer can also assist respondents to answer complex questions and can clarify concepts such as definitions of particular ICTs. Because the interviewer is in view, he/she can use visual prompts such as prompt cards.

In addition, face-to-face interviews are especially useful for questions about opinions or impressions, and for surveys that take a long time to complete.

Where a number of languages or dialects are spoken, it may be very effective if the interviewers are selected with the adequate linguistic skills.

The technique usually produces lower non-response rates. Data collection can be managed efficiently with specific software (Computer Assisted Personal Interviewing – CAPI).

High personnel costs may be incurred (for hiring and training interviewers). However, this could be a minor issue in developing economies where salaries of interviewers are low, or agreements are reached with certain institutions to provide part-time interviewers (such as university students).

Interviewers are part of the measurement tool and they can induce important biases if they have not received suitable training.

In developing economies with poor quality transport infrastructure, reaching households located in some remote areas may prove difficult.

Telephone personal interview

Although to a lesser extent than face-to-face personal interview, telephone interviewing allows direct interaction between the interviewer and interviewee.

It is a fast and relatively inexpensive way to collect information, since a small number of interviewers from a single call centre can carry out a great number of interviews.

The data collection can be managed efficiently with specific software (Computer Assisted Telephoning Interviewing – CATI).

The telephone can be a useful complementary method of interview, subsequent to face-to-face interviewing, once contact has been made and a telephone number obtained

Correct and comprehensive telephone numbers may not be available, particularly in developing economies where mobile telephony may be more common than fixed telephone.

Interviews must be relatively short, since a long telephone conversation can be perceived as an annoyance. Some people also feel that it is intrusive to be interviewed by telephone.

Telephone interviews may not be suitable for collecting quantitative information, for which the interviewee may have to check records.

The non-response rate is usually larger than for face-to-face interviews (but lower than for mail-based surveys).

Table 10. Summary of data collection methods (continued)

Method

Main advantages

Main disadvantages

Self-enumeration

The method is likely to be relatively inexpensive and allows the respondent to complete the questionnaire at his/her convenience.

It eliminates the problem of interviewer bias, though note that interviewer follow-up (e.g. for non-response or inconsistent answers) can potentially introduce bias if not managed properly. Where questionnaires are delivered directly to mail boxes, there may be no need for a sampling frame of addresses. Where questionnaires are collected by field staff, they may be able to check responses and assist with any problems the respondent has completing the questionnaire

Postal surveys require a current and complete sampling frame of addresses for households or individuals.

If questionnaires are not properly designed and tested, they can introduce bias to survey results that may be difficult to detect. In particular, because they do not involve interaction with the respondent, technical questions on ICT use may be less well understood and questionnaire logic is likely to be more complex.

Requires separate data entry unless advanced imaging technology tools (such as Optical Character Recognition, OCR) are available.

Unit and item response rates are likely to be lower than for personal interviewing, leading to higher sampling error and likely non-response bias.

In situations where there are several languages or dialects spoken in a country, or where literacy is low, self-enumeration is likely to be ineffective.

Delays in mailing back questionnaires can introduce delays in survey processing. In developing economies with a low quality postal system, such delays may be prohibitive.

Interview assisted by computer (CAPI/CATI) (can be applied to face-to-face or telephone interviews respectively)

CAPI and CATI systems can eliminate errors of flow and data consistency, and can thus improve input data quality and reduce the time for data capture and validation.

Modern IT equipment such as PDAs or tablets may present a cheap and comfortable tool for data collection.

CAPI and CATI techniques require interviewers with some technical skills.

CAPI and CATI systems are usually based on commercial software that may be costly. Skilled staff are required to adapt the software to the questionnaire.

CAPI requires that interviewers carry IT equipment, which can be damaged or stolen during field operations.

In developing economies with poor road networks, there is a risk of damaging equipment.

Table 10. Summary of data collection methods (continued)

Method

Main advantages

Main disadvantages

Internet-based surveys

Data editing is done concurrently with data entry, reducing the time for processing.

Novelty and convenience for some respondents may increase response rates in particular population segments difficult to reach at home (e.g. young employed persons).

Internet-based surveys can only be used by respondents with Internet access and a reasonable level of ICT proficiency. This clearly excludes a large proportion of the population of interest for ICT access and use measurement (that is, non-users).

For this reason, they are only practical as a supplementary form of data collection for household ICT access and use data.

The costs of developing/implementing software and appropriate questionnaires might also be high and not cost-effective for a method that can only ever supplement other methods of data collection.

[1] The word ‘survey’ is sometimes taken to mean ‘sample survey’ although, in fact, it includes completely enumerated surveys, that is, censuses. Surveys include those conducted by personal interview or by other means, such as mail.

[2] Subscriptions may be attributable to organizations (business, government or non-profit) or individuals. There may be more than one subscription in a household and several individuals may use the same subscription. Subscription data do not usually include information about the subscriber, except for the location where the service is provided. Exceptions to this are surveys of Internet service providers where providers are asked to split subscriptions by type (e.g. household and other). Such surveys are conducted by few countries, among them Australia (ABS, the Internet activity survey, available at: www.abs.gov.au/ausstats/abs@.nsf/mf/8153.0) and Norway (Statistics Norway, the Internet survey, available at: http://www.ssb.no/inet_en/)

[3] The ITU has implemented a pilot project with several mobile operators, proving the feasibility of producing ICT indicators (different from those of the core list discussed in this Manual). See: https://www.itu.int/en/ITU-D/Statistics/Pages/bigdata/default.aspx

[4] The LSMS has a broad objective of improving the quality of household statistics in developing economies. A more specific goal is to develop methods to monitor progress in raising of living levels in developing economies. See: http://go.worldbank.org/IFS9WG7EO0

[5] The main objective of household budget surveys is to measure household consumption (expenditure) for national accounts purposes and calculation of consumer price indices. Chapter XXIV of UNSD (2005a) discusses design and measurement issues specific to household budget surveys.

[6] https://www.itu.int/en/ITU-D/LDCs/Pages/Publications/LDCs/D-LDC-ICTLDC-2018-PDF-E.pdf

[7] Defined here as an economy not belonging to OECD or not covered by Eurostat ICT household collections (that is the European Union countries plus a small number of other countries such as Norway and Turkey).

[8] An example might be where the age scope of the labour force survey is limited to those of working age, whereas the age scope of an ICT household survey is usually broader than this.

[9] The results of the Small Area Estimation (SAE) project of Eurostat, including methodological developments and software, are available at: https://ec.europa.eu/eurostat/cros/content/sae-finished_en.

[10] Chapter 6 discusses questionnaire issues where a number of languages are spoken in a country.

[11] These may be incorporated into the questionnaire or provided as a separate document.

[12] See: https://www.destatis.de/EN/Publications/Specialized/InformationSociety/informationtechnology.pdf?__blob=publicationFile.

[13] Adapted from various sources including UNCTAD (2009) and ITU (2009b).

Chapter 6. Question and questionnaire design for ICT household surveys

295. This chapter considers general issues of question and questionnaire design and specific issues relating to the model questions associated with the core list of ICT indicators. These were presented in Table 6 in Chapter 4.

296. Poor question and questionnaire design can be a significant source of survey error. It is therefore very important that design is undertaken carefully and that sufficient time is allowed for thorough testing.

297. There will never be a questionnaire design that works optimally for every household or individual in a survey, so it will be necessary to undertake careful research and testing to ensure an effective compromise. Once data requirements are determined, appropriate questions can be drafted, although it usually takes quite a lot of effort to finalize the wording, establish a logical order for the questions and construct a sequence guide to fit all situations.

298. As most countries collect ICT household statistics by personal interview, the assumption made in this chapter is that questionnaires will be administered by interviewers. However, much of the material presented applies equally to self-enumerated questionnaires.[1]

General principles of questionnaire design for household surveys

299. Household surveys are often conducted by personal interview (either face-to-face or by telephone). This enables terms to be explained and logic applied by trained interviewers. The method of asking questions will vary with the collection methodology used, for instance, telephone interviewers will ask questions differently from face-to-face interviewers.

300. In general, questionnaires should be designed to:

• Maintain respondents’ cooperation by being as short as possible; this will also generally improve the quality of responses by avoiding respondent fatigue. A measure of the time needed for completing the questionnaire should be recorded to evaluate the response burden[2].

• Maintain respondents’ interest and motivation to complete the form, for example, by clearly explaining the aims and methodology of the survey, starting with simple and interesting questions, and, as far as possible, avoiding complicated questions.

• Appear logical by having related questions grouped (possibly into modules) and with a logical flow; if there is more than one respondent in the household, group their questions so as to use their time most efficiently. In the paper version of a questionnaire, the layout of questions for the different household members should allow for a variable number of members.

• Where a question has a number of response categories (or items), these need to be handled carefully to avoid recall problems. For instance, if items are all read out to the respondent at once, he/she may have a clearer recollection of the last two or three items than the first few, or an impression that the order of the responses reflects their importance. Many of the response categories in the model questions are ‘multiple responses’, that is, all instances should be reported. This should minimize an order effect. The model question associated with HH9 (Internet activities) has a large number of response items. To avoid recall problems, an appropriate way to ask this question might be by showing a list (in the case of a face-to-face interview or paper questionnaire) or asking a series of ‘yes/no’ questions about each activity (in a telephone interview). The concept of “non-aware Internet users” is elaborated below.

• Ensure that question wording is clear, uses plain language, is unbiased and is unambiguous. It is particularly important to avoid bias in question wording and not to ask ‘leading questions’ (that is, a question where a particular answer is implied, thus resulting in a biased response).

• Avoid double-barreled questions (that is, questions that require a single answer but have actually several parts, for instance: “How often and how much time do you spend when you use the Internet?” and double-negative questions (that is, questions that include two negative terms, for instance “Do you think that using your mobile telephone while driving should not be forbidden?”).

• Build trust with the respondent by avoiding sensitive questions, as far as possible, and assuring confidentiality of responses. Questions that are sensitive could be asked at the end of the interview so that they do not affect responses to other questions.[3]

301. Interviewer-administered questionnaires include prompts and skips[4] to guide the interviewer through the questionnaire. These will specify the population for each question and ensure, as far as possible, consistent conduct of the interview. Figure 4 shows the populations that are asked each of the model ICT questions, for instance, only those who have used the Internet in the last three months are asked questions about the location, activity and frequency of Internet use. It should be noted that an interviewer or respondent may have an incentive to answer negatively when an affirmative answer implies numerous follow-up questions. For instance, the interviewer may be tempted to encourage a ‘no’ answer in the question on Internet use in order to avoid having to ask questions on location, activities and frequency.

302. Past experience can often be used to refine question wording and the logic aspects of form design.

303. Where more than one language is used in a country, UNSD (2005a) strongly recommends that questionnaires be translated into all major languages spoken in order to ensure preservation of meaning and uniformity of presentation by interviewers. They present evidence that the alternative, being an ‘on-the-spot’ translation by the interviewer, increases errors by a factor of between two and four. Interviewers who do not speak the local language should not be used, as they may not be understood by respondents or may need to use local interpreters – another potential source of bias. Decisions on how many languages should be represented will be a function of things like the number of people speaking only a minority language and the likelihood that their omission would bias survey results.[5]

304. Ensuring clear wording in the original version of a questionnaire does not guarantee that it will be as clear in translated versions, especially as it is common for local languages to be spoken but not written. An advisable practice when translating questionnaires is to translate from the original language and then back again, after which the two versions in the original language are compared. The back-translation should be done by someone who was not closely involved in the development of the questionnaire in order to avoid contaminating the translation with prior knowledge.

305. Questions and whole questionnaires should be thoroughly tested before use in a survey. They should always be tested with actual respondents to ascertain whether the questions can be understood and answered accurately and whether respondents have a common understanding of the meaning of the questions. Testing can be qualitative or quantitative. Qualitative tests include focus group tests[6] and cognitive research.[7] Quantitative testing includes pilot tests and dress rehearsals.[8]

306. Box 30 provides more information on qualitative testing and an example of its use from Brazil.

Box 30. Cognitive interviewing as a tool to evaluate questions: a practical case from Brazil

Cognitive interviewing seeks to evaluate survey questions by use of techniques that gauge respondents’ understanding of, and response to, questions. The techniques include:

• ‘think-aloud’ interviews: the respondent speaks their thoughts while answering questions, or recalls their thoughts directly afterwards;

• paraphrasing: the respondent is asked to rephrase the question in his or her own words;

• probing: the interviewer asks follow-up questions after each question or group of questions to probe respondents’ interpretation of the question/s; and

• definitions: the respondent is asked to explain key terms.

An important application of cognitive interviewing is evaluating the translation and adaptation of cross-national questionnaires. In 2012, CETIC.br conducted the Kids Online Brazil Survey for the first time in order to measure risks and opportunities related to Internet use among children aged 9 to 16. The questionnaires used in the survey were based on those developed for the EU Kids Online project and followed the framework designed by the London School of Economics. The European questionnaires were translated into Portuguese from the master questionnaires in English and then adapted to the Brazilian context. Cognitive interviews were carried out with the objectives of:

• learning how Brazilian respondents understood the critical concepts of the survey;

• testing the translation of the questionnaire;

• identifying possible sensitivity of specific issues; and

• checking that the questions were age appropriate.

The interviews were carried out in two different phases, allowing different aspects to be evaluated in each phase. The respondents recruited were of different socio-demographic profiles and varied in age, sex and socio-economic status. As a result of the cognitive interviewing process, a number of changes were made to the Brazilian version of the original questionnaires, including changes in wording of questions or response items.

Source: Cetic.br.

307. Testing of a set of draft questions at an early stage can greatly assist planning as well as questionnaire design. Such testing may be undertaken by statistical agency staff, including field supervisors. This would serve as a familiarization opportunity for those who will have roles in training household interviewers once the methodology has been determined. As well as firming up the final set of questions to be asked, testing will also help determine the best collection method, if not already known, and the time necessary to conduct interviews. On this point, it should be noted that interviews conducted within the testing phase tend to take longer than real interviews as responses may be probed and discussed more than in a ‘live interview’. Additionally, interviewers are not as familiar with the questionnaire as they will be for the actual survey.

308. Testing can be done in two stages[9] – pretesting parts of the questionnaire on a small number of respondents (this may occur a number of times) and a comprehensive field test (or pilot test) involving more respondents who are selected to be reasonably representative of the population. One reason for testing in stages is that the most mistakes are usually detected in the first few days. Once those are identified and corrected, the second stage can take place within a larger geographic area. Following UNSD,[10] it is recommended that questionnaire modules be tested on at least 50 respondents to that module (therefore, for individual ICT use, the questions should be pilot tested on 50 individuals who have used the Internet; for questions on lower prevalence, e.g. e-commerce or ICT use by PwD, the pilot sample may have to be larger to find an adequate number of test cases). A large survey may also involve a final ‘dress rehearsal’ prior to launch. This will test all aspects of the survey, including procedures and will also provide valuable information on costs, the adequacy of training and documentation, and the need for fine-tuning of timetables.

Table 11. Structure and logic of a model questionnaire/module for collecting ICT household data

Section 1: Household characteristics[11]

Household number of members

Population: all in-scope households; includes household members outside any individual age scope applied

Household composition (whether there are children under 15)

Population: all in-scope households; includes household members outside any individual age scope applied

Optional questions on topics such as household access to electricity, household income, location (e.g. urban/rural)[12]

Population: all in-scope households

Section 2: Household access to information and communication technology

Household with a radio (HH1)

Population: all in-scope households

Household with a television (HH2)

Population: all in-scope households

Household with multichannel television (HH13)

Population: all in-scope households with a TV

Household with a fixed telephone line (HH3)

Population: all in-scope households

Household with a mobile telephone (HH3)

Population: all in-scope households

Household with a smartphone (HH3)

Population: all in-scope households with a mobile phone

Household with a computer (HH4)

Population: all in-scope households

Household with Internet (HH6)

Population: all in-scope households

Types of Internet access services used by households at home (HH11)

Population: all in-scope households with Internet access at home

Barriers to household Internet access (HH14)

Population: all in-scope households without Internet access at home

Table 11. Structure and logic of a model questionnaire/module for collecting ICT household data (continued)

Section 3: Individual characteristics[13]

Age

Population: all selected in-scope individuals

Sex

Highest educational level attained

Labour force status

Occupation

Optional questions, for example, income, disability status, languages spoken/read

Section 4: Individual use of information and communication technology

Individual use of a mobile cellular telephone (HH10)

Population: all selected in-scope individuals

Individual use of a smartphone (HH10)

Population: all selected in-scope individuals having used a mobile cellular phone

Possession (ownership) of a mobile phone (HH18)

Population: all selected in-scope individuals

Possession (ownership) of a smartphone (HH18)

Population: all selected in-scope individuals who have a mobile phone

Individual use of a computer (any location, last three months) (HH5)

Population: all selected in-scope individuals

Individuals with ICT skills: activities carried out in the last three months at any location (HH15)

Population: all selected in-scope individuals

Individual use of the Internet (any location, last three months) (HH7)

Population: all selected in-scope individuals

Location of individual use of the Internet in the last three months (HH8)

Population: all selected in-scope individuals who used the Internet from any location in the last three months

Type of connection/portable device used to access the Internet, last three months) (HH17)

Population: all selected in-scope individuals who used the Internet from any location in the last three months

Frequency of individual use of the Internet in the last three months at any location (HH12)

Population: all selected in-scope individuals who used the Internet from any location in the last three months

Table 11. Structure and logic of a model questionnaire/module for collecting ICT household data (continued)

Section 4: Individual use of information and communication technology

Internet activities undertaken by individuals in the last three months at any location (HH9)

Population: all selected in-scope individuals who used the Internet from any location in the last three months

Type of goods or services purchased or ordered over the Internet for private use in the last three months (HH20)

Population: all selected in-scope individuals who used the Internet from any location in the last three months for purchasing or ordering services

Mode of payment for goods or services purchased or ordered over the Internet for private use in the last three months (HH21)

Population: all selected in-scope individuals who used the Internet from any location in the last three months for purchasing or ordering services

Mode of delivery for goods or services purchased or ordered over the Internet for private use in the last three months (HH22)

Population: all selected in-scope individuals who used the Internet from any location in the last three months for purchasing or ordering services

Reasons for not having purchased goods or services over the Internet for private use in the last three months (HH23)

Population: all selected in-scope individuals who did not use the Internet from any location in the last three months for purchasing or ordering services

Reasons for not having used the Internet in the three last months (HH19)

Population: all selected in-scope individuals who did not use the Internet from any location in the last three months

309. Non-question elements of a questionnaire will differ for different form types (paper based personal interview, CAPI or CATI, self-enumerated questionnaire) and may include:[14]

• an identifier for each variant of a questionnaire[15] and a unique identifier for each copy of the form (with a check digit[16] if the identifier is to be entered by keying);

• boxes of appropriate size to record responses (for instance, the number of household members can be expected to require space for two digits, while household income might require space for 7 or more digits, depending on the currency);

• ‘office use only’ spaces might be used by interviewers, or data entry and processing staff to record information;

• if Optical Character Recognition (OCR), Intelligent Character Recognition (ICR) or Optical Mark Reading (OMR) software are used to convert responses on a paper questionnaire to a computer record, there may be need for additional information on the form (such as page identifiers) or a particular style of layout; and

• for self-enumerated forms, information about the survey (e.g. its purpose, name and reference period, due date, how the questionnaire should be returned, how to obtain assistance with completing the form, contact details of the respondent and legal obligations) and general instructions to respondents (e.g. how to mark boxes, how to correct errors)

• information to geo-locate the household, with the purpose of fieldwork checking and further geo-spatial analysis of responses.

ICT model questions

310. Model questions for the ICT household indicators are shown in Table 6 in Chapter 4 and a model questionnaire is presented in Annex 2. It is important to note that the model questions and questionnaire need to be adapted by each country to suit:

• the survey vehicle and method of data collection (for instance, a questionnaire that is designed for a telephone survey will differ from a questionnaire that is used for face-to-face interviewing);

• cultural and linguistic conventions of the country; and

• ICT services available in the country, in particular in the case of Internet services (HH11) and multichannel TV services (HH13).

311. For comparability purposes, it is important that the meaning of the model questions is preserved and that the specified populations of households or individuals are asked each question.

312. A number of the model questions have response categories (for example, location of Internet use, Internet activities, frequency of Internet use, type of Internet access, multichannel TV services, barriers to household Internet access, ICT activities, mode of payment and mode of delivery of purchased/ordered goods or services and household ICT expenditure). Countries have some options about how they deal with these. For instance, countries can add or split categories, with some examples shown below. For international reporting, split categories need to be re-aggregated. This process is explained in Chapter 8.

313. Another variation is that countries can add an ‘other’ category to some of the category lists. The model questions for the response category indicators, as presented in the model questionnaire in Annex 2, often include ‘other’ categories. It is important to note that one of the aims of questionnaire testing is to eliminate or at least reduce the use of the ‘other’ category. Ideally, countries would ask respondents to specify details when they select an ‘other’ category. This enables countries to recode ‘other’ category responses to an existing category. Where there are significant responses in an ‘other’ category, countries should note the details for future questionnaire design. Options might be to better describe existing categories or add additional categories.

314. For HH8 (location of Internet use), some countries may wish to specify a particular location of importance for policy purposes, for example, Internet access provided through government supported public access centres (as a subcategory of ‘Community Internet access facility’). The addition of such locations as a separate category would assist policy-makers to evaluate government support for such centres.

315. For HH9 (individual Internet activities), countries might like to split the broader categories to obtain more detailed information. Countries could also ask the question in other ways, for example, rate each activity according to its frequency of use, or rank the activities by frequency.

316. For HH11 (household Internet access by type of service), categories should be chosen such that responses can be aggregated and compared at the global level. The suggested question in the model questionnaire (Annex 2) serves mostly as an example of the categories that could be used for comparison purposes, rather than the actual question that would be asked in a country survey. Examples on how some countries have asked this question are presented in Box 31.

Box 31. How to ask the question on Internet access by type of access

Mexico

The National Institute of Statistics and Geography of Mexico (INEGI) includes a module on access to, and use of, ICT in households in the national Labour Force Survey. In 2010, for example, the question on the type of access is asked as follows:

The main way of connection to the Internet… (check the one that applies)

1. Is through the telephone line, and while you are connected to the Internet it is not possible to talk over the phone? (dial-up)

2. Is through the telephone line, and you can talk over the phone when connected to the Internet? (dedicated telephone line)

3. Is through Cable TV (the connection is made via a cable TV network and you can watch TV while connected to the Internet)

4. Is through wireless mode? (connection by satellite, mobile broadband or 3G. It excludes any wireless technology that allows for movement by using additional peripheral devices)

Czech Republic

Is your household connected to the Internet at home by:

1. ADSL technology?

2. A cable TV line (e.g. Cable Internet from UPC)?

3. Optical fibre?

4. Fixed wireless access (e.g. Wi-Fi)?

5. Dial-up over normal telephone line or ISDN (e.g. Digital line ISDN2 from O2)?

6. Broadband 3G mobile phone network and computer (e.g. Mobile Connection for your Notebook or Tablet)?

7. Broadband 3G mobile phone network and mobile phone (e.g. Internet on your Mobile)?

Sources: INEGI, http://www.inegi.org.mx/prod_serv/contenidos/espanol/bvinegi/productos/encuestas/especiales/endutih/2010/endutih2010.pdf

Czech Statistical Office.

317. For HH12 (frequency of Internet use), countries can split a category; for example, ‘less than once a week’ could be split into ‘at least once a month but not every week’ and ‘less than once a month’.

318. For HH15, more detailed data on activities carried out could be collected where there is interest in knowing about additional or more specific skills.

319. For HH16, more detailed expenditure data could be collected where national classifications support more detail. Note that this indicator is not generally collected through ICT household surveys, but through household budget (income and expenditure) surveys.

Reference period

320. Reference period (also known as recall period) refers to the period in respect of which respondents are asked to report. For ICT use statistics, the length of the reference period has been much debated. The general argument is that a question asked in respect of a long period is less likely to yield an accurate response because of problems the respondent has remembering the required information (see Chapter 4 for a discussion of recall issues). On the other hand, a long reference period is more likely to capture a ‘rare event’.

321. For the model questions associated with the core indicators, a 3-month reference period has been agreed by the EGH. Note that before the 2014 edition of this Manual, the recommended reference period was 12 months.

ICT concepts that may be difficult to understand

322. There are a number of concepts used in the core list of ICT indicators that may be difficult for respondents to understand and respond to consistently. They include:

• definition of computer,

• definition of portable devices

• Internet access services,

• multichannel television services,

• Internet activities related to government organizations,

• mobile cellular telephones, and

• computer-related activities (to measure ICT skills).

323. It is important when designing questions for ICT household surveys that these concepts are explained clearly and placed in a country and cultural context.

Definition of computer

324. Definition of a computer is used for indicators HH4 and HH5 and refers to “a desktop computer, a laptop computer or a tablet or similar handheld computer. It does not include equipment with some embedded computing abilities, such as smart TV sets, and devices with telephony as a main function, such as mobile or smartphones.”

325. With rapid changes and convergence of devices, it is difficult to cover potential new devices that will appear in the near future. Already it is difficult to clearly differentiate between a smartphone and a tablet. However, during the latest revision of the household indicators, only tablets are considered to be computers, as they usually have more computing and processing capacity and also because mobile phones are covered in separate indicators.

Definition of portable devices

326. The definition of portable devices is relevant for indicators HH3, HH10, HH17 and HH18.

327. A portable device can be a mobile phone (including a smart phone), tablet or a portable computer (such as laptop, notebook, netbook). Portable devices can connect to the Internet via mobile connections. The network used to access the Internet can be either via a mobile cellular network or via other wireless networks (e.g. WiFi).

Internet access services

328. Household core indicator HH11 deals with the types of Internet access services used by households with Internet access. There are six response categories for the indicator, though in practice, countries may use a different number on national questionnaires, with wording adapted to the local situation. Whichever approach is chosen, the categories are likely to be quite technical and the types of services may change as technology evolves. Anecdotal evidence indicates that many respondents – and interviewers – will not understand the meaning of the Internet access service categories used. It is therefore desirable that questions on this topic use categories that are relevant to services existing within the country at the time of the survey and are therefore likely to be understood in a local context. Explanation of categories could use the product names or company brands of widely available broadband services, or alternatively, describe the technological aspects in an easy-to-understand way (see Box 31 for examples on how Mexico and the Czech Republic have asked the question). There may also be a policy interest in particular Internet access services. In this area, in particular, it is important that interviewers are familiar with technologies and their brand names.

Non-aware Internet users

329. Country experiences show that some Internet users may not be aware that they used the Internet, and answer negatively to the question on Internet use, while answering positively to certain activities requiring the Internet, such as posting comments or contents in social media. This is particularly the case when some Internet services are offered free of charge (“zero-rate services”). Box 32 presents the results of research highlighting this fact. This suggests that after answering positively to some categories of response of the question related to activities carried out (to measure indicator HH15), the interviewer should probe the answer to the question on use of the Internet. In particular, positive responses to the following categories would require the double-checking of answers on the use of Internet:

• Sending messages (e.g. e-mail, messaging service, SMS) with attached files (e.g. document, picture, video)

• Finding, downloading, installing and configuring software

Box 32. In some countries, many use the Internet without realizing

What is the Internet? And who is an Internet user? The questions may seem straightforward, but more than a decade of research in the United States and abroad suggests that some people who use the Internet may not be aware that they’re doing so. Results from recent Pew Research Center surveys in the U.S. and 11 emerging economies show that confusion about what the Internet is stems from two different – but related – sources.

First, many people who use smartphones are unaware that the apps and browsers on their devices involve using the Internet. In the Center’s survey of emerging economies, as many as 38% of those who say they do not use the Internet also indicate that they have a phone that connects to the Internet. Due to differences in Internet use across these countries, this group represents as much as 14% of the total adult population in South Africa, or as little as 3% in Venezuela.

These mismatches are often highest in developing countries and can even extend to people who use their smartphones to do things that necessitate using the Internet for tasks such as looking for or applying for jobs.

Across 11 developing countries surveyed in fall 2018, one of the defining factors in people’s awareness they are using the Internet is whether they have access to a home or office computer. Majorities of “unconscious Internet users” (that is, those who say they do not use the Internet, but do use social media, a smartphone or a feature phone) lack access to a home computer or tablet, meaning they likely visit the Internet primarily through a mobile phone. In three countries, those with lower levels of education are also somewhat more likely to be unconscious Internet users, though in most countries there is no relationship with educational attainment. But, while older people are somewhat less likely to use the Internet, smartphones or social media than younger people, they are not more likely to be unconscious users.

This phenomenon extends to advanced economies as well: Previous surveys by the Center have found that a small share of people in nearly every country surveyed underreport Internet use. Estimates that account for social media use and smartphone ownership tend to be somewhat larger than those that only include people’s self-reported Internet use. For example, 90% of South Koreans say they use the Internet, when asked, but 97% of South Koreans report using the Internet, owning a smartphone or using social media – a gap of 7 percentage points.

Second, apart from a lack of awareness that smartphones and feature phones connect to the Internet, many people who use social media and messaging apps appear unaware that the platforms themselves are part of the broader Internet. This is a relatively well-known phenomenon in the case of Facebook. And in countries like the Philippines, Facebook offers a free version that allows users to visit the site without incurring mobile data charges.

As was true with the distinction between smartphones and the Internet, this lack of understanding of the nature of social media is not confined to emerging economies. Among U.S. adults who say they do not use the Internet, some 14% indicate in each case that they use Facebook or the video sharing platform YouTube.

Taken together, these findings indicate that people can be unaware of what the Internet is in a variety of ways. Across all 11 countries surveyed, anywhere from 5% to 25% of the population fits this pattern of being an unconscious Internet user. The highest rates of this behavior occur in Kenya and the lowest rates occur in Lebanon and Vietnam.

Source: “In some countries, many use the Internet without realizing it”, Laura Silver and Aaron Smith, Pew Research Center, May 2019.

https://www.pewresearch.org/fact-tank/2019/05/02/in-some-countries-many-use-the-Internet-without-realizing-it/

330. The question on activities carried using the Internet is asked to in-scope individuals having answered positively to the use of Internet (in the last three months). However, the same effect has been observed for “non-aware Internet users”. Thus, countries may test the questionnaire to assess whether this question should be asked to all in-scope individuals, as some may not recognise that certain activities require access to the Internet.

Multichannel television services

331. Household core indicator HH13 deals with multichannel TV services, some of which may require some explanation (by interviewers or on questionnaires) as follows:

• Cable TV (CATV) is multichannel programming delivered over a coaxial cable for viewing on television sets;

• Direct-to-home (DTH) satellite services: TV services received via a satellite dish capable of receiving satellite television broadcasts.

o Note that DTH satellite services can be paid or not paid;

• Internet-protocol TV (IPTV) is multimedia services such as television/video/audio/text/graphics/data delivered over an IP-based network managed to support the required level of quality of service, quality of experience, security, interactivity and reliability; it does not include video accessed over the public Internet, for example, by streaming. IPTV services are also generally aimed at viewing over a television set rather than a personal computer.

o IPTV should not be confused with watching TV over the Internet. IPTV uses broadband networks to carry TV signals, while maintaining a guaranteed quality of service. It is generally aimed at viewing over a television set, making the quality of experience comparable with that of other TV platforms.

o IPTV should also not be confused with over-the-top (OTT) or online TV and video (e.g. YouTube, Netflix), which are delivered via the Internet.

• Digital terrestrial TV (DTT): the technological evolution from analogue terrestrial television, providing capability for significantly more channels

o DTT is by default not paid (for countries that have made the digital transition).

Internet activities related to government organizations

332. The measurement of e-government has been dealt with by the Partnership on Measuring ICT for Development, which has established a statistical framework, including a core list of indicators on e-government (Partnership and UNECA, 2014). The indicators considered are classified into four areas: use of ICT by employees of government, availability of ICT to government organizations, use of ICT by government organizations and supply of e-government services to citizens.

333. Household core indicator HH9 deals with the Internet activities undertaken by individuals. The response categories include ‘getting information from general government organizations’ and ‘interacting with general government organizations’. It is important to differentiate between these categories. Whereas the former refers to obtaining information (often from websites), the latter is more interactive and covers activities such as completing forms online and making online payments.

334. In both cases, the definition of what constitutes a general government organization may prove difficult for respondents to understand, especially in a consistent way. Statistically, it is recommended that the SNA93 (2008 revision) concept of government be used, as follows:

According to the SNA “… the principal functions of government are to assume responsibility for the provision of goods and services to the community or to individual households and to finance their provision out of taxation or other incomes; to redistribute income and wealth by means of transfers; and to engage in non-market production.” (UNSD, 2008b)

335. General government organizations include central, state and local government units, as well as all non-market non-profit institutions that are controlled by government units, and social security funds. It should be noted that general government organizations do not include public corporations (legal entities, predominantly owned and controlled by the government that are created for the purpose of producing goods and services for the market and may be a source of profit or other financial gain to their owner/s) (UNSD, 2008b).

336. Clearly, the concept of government is complex and needs to be interpreted within a country context. It could be useful to provide examples of websites of government organizations that are thought to be widely used (for instance, websites of government-run media organizations, transport services or national statistical office websites).

337. The response categories for HH9 (getting information from, and interacting with, general government organizations) may be accompanied by a list of e-government services. A more comprehensive list of Internet-based services is provided by the Partnership and UNECA (2012) in relation to indicator EG7: Selected Internet-based services available to citizens, by level of sophistication of service. This list may be used during the interview as a list of examples, although it is not comprehensive. Box 33 reproduces the question asked by the Census and Statistics Department of Hong Kong, China in 2008. The question assists the respondent by providing examples of each class of e-government services.

Box 33. Hong Kong, China: question on use of e-Government services, 2008

In the past 12 months, have you used any of the e-Government services for your personal matters? (e.g. to search for Government information, submit applications, book appointments for identity card applications, book leisure facilities, registration, pay taxes or other governmental charges, etc.)

If “yes”:

Show card

In the past 12 months, which e-Government services have you used for personal matters? (Allow multiple answers)

• Browsing and enquiring about information disseminated by the Government (e.g. enquiring about weather conditions, traffic conditions, statistical data, air pollution index, Government news, etc.)

• Online financial management (e.g. paying various Government bills, such as rates, Government rent, water charges, tax, purchasing tax reserve certificates, paying fixed penalty tickets for traffic offences or littering, etc.)

• Online appointment booking or license/certificate applications (e.g. appointment bookings for road tests and vehicle examinations, renewing driving and vehicle licenses, appointment bookings for registration of identity cards/giving of marriage notices, applying for copies of birth/death/marriage certificates)

• Online registration services (e.g. voter registration and volunteer scheme registration)

• Online change of personal details (e.g. change of address)

• Online job search and recruitment (e.g. searching for job vacancies, registering job vacancies and searching for suitable candidates)

• Online purchase of Government publications and Government statistical publications

• Online booking of Government venues or facilities (e.g. sports venues, training courses or leisure facilities)

• Online submission of information to Government (e.g. filing of tax return)

• Online library services (e.g. book reservation, book renewal)

• Download Government forms

• General browsing of government websites

• Others (Please specify): _______________

• Have used, but forget the types of service

Source: Questionnaire for the Thematic Household Survey on Information Technology Usage and Penetration in 2008 (Census and Statistics Department, Hong Kong, China, 2008).

Box 34. Luxembourg: Question on e-commerce

The following questions concern the purchase of goods and services over the Internet (e-commerce) for private use via any device (desktop, portable or handheld, including mobile or smart phone).

Purchases refer to ordering goods or services over the Internet for which payment is required; the payment does not have to be done online.

Orders via manually typed e-mails, SMS or MMS should be excluded. Orders via websites or apps should be considered.

When did you last buy or order goods or services for private use over the Internet? (excluding manually typed e-mails, SMS, MMS)

Please choose only one of the following options.

1. Within the last 3 months

2. Between 3 months and a year ago

3. More than 1 year ago

4. Never bought or ordered

What types of goods or services did you buy or order over the Internet for private use in the last 12 months? Please tick all that apply.

a. Food or groceries

b. Household goods (e.g. furniture, toys, etc; excluding consumer electronics)

c. Medicine

d. Clothes, sports goods

e. Computer hardware (including accessories such as printer, modem, mouse, scanner)

f. Electronic equipment (incl. cameras)

g. Telecommunication services (e.g. TV, broadband subscriptions, fixed line or mobile phone subscriptions, uploading money on prepaid phone cards, etc.)

h. Holiday accommodation (hotel etc.)

i. Other travel arrangements (transport tickets, car hire, etc.)

j. Tickets for events

k. Films, music

l. Books, magazines, newspapers

m. e-learning material

n. Video games software, other computer software and software upgrades

o. Other

From whom did you buy or order goods or services for private purpose over the Internet in the last 12 months?

Please tick all that apply

a. National sellers

b. Sellers from other EU countries

c. Sellers from the rest of the world

d. Country of origin of sellers is not known

Box 34. Luxembourg: Question on e-commerce (continued)

How many times did you order or buy goods or services over the Internet for private use in the last 3 months?

Please choose only one of the following options.

____ times

or

- 1-2 times

- 3-5 times

- 6-10 times

- > 10 times

How much as an estimate did you spend buying or ordering goods or services over the Internet (excluding shares or other financial services) for private use in the last 3 months?

Please choose only one of the following options.

___ euro

or

- less than 50 euro

- 50 to less than 100 euro

- 100 to less than 500 euro

- 500 to less than 1000 euro

- 1000 euro and more

- don’t know

- refuse to answer

Did you carry out any of the following financial activities over the Internet (excluding e-mail) for private purposes in the last 12 months?

Please tick all that apply.

a. Buying or selling shares, bonds, funds or other investment services

b. Buying or renewing existing insurance policies including those offered as a package together with another service (e.g. travel insurance offered together with a plane ticket)

c. Taking out a loan or mortgage or arranging credit from banks or other financial providers

d. none of the above.

Source: https://statistiques.public.lu/fr/enquetes/espace-menages/enquete-TIC/Questionnaire-TIC-ENG-2018.pdf.

338. More information on the conceptual challenges of e-government measurement can be found in the publication Framework for a set of e-government core indicators (Partnership and UNECA 2012).

Mobile cellular telephones (including smartphones)

339. There are both household and individual mobile telephone (including smartphones) indicators in the core list (HH3 for households and HH10 and HH18 for individuals) and corresponding model questions (see Table 6). The meaning in each context is different. In the household context, the interest is in whether the household has access to a mobile telephone that is an ‘asset’ of the household, as it has been traditionally understood for fixed telephones. In the individual context, the interest is either in the ownership of a mobile phone by an individual (HH18), or in the use of a mobile cellular telephone by an individual, irrespective of who owns or pays for it (HH10).

340. In the case of household-level access to a mobile phone (including smartphones), this phone should be available for all household members to use it. Obvious exceptions (such as little children in the household) who are prevented from using the phone should not be considered in the response.

341. The concept for individual mobile telephone (or smartphone) use is different from a mobile telephone subscription. The model question corresponding to HH10 specifies that “Use of a mobile telephone does not necessarily mean that the telephone is owned or paid for by the individual but should be reasonably available through work, a friend or family member, etc. It excludes occasional use, for instance, borrowing a mobile telephone to make a call.” Subscribers, on the other hand, subscribe to a mobile telephone service by a postpaid subscription or a prepaid account. They are therefore likely to be owners of a mobile telephone in a legal sense. They may also be organizations, such as businesses, rather than individuals.

Activities (to measure ICT skills)

342. The household core indicator HH15 deals with activities that reflect an individual’s level of ICT skills, independently of the device used. This is a difference with the previous versions of the Manual, where only activities carried out with a computer were considered. It is considered probable that if an individual does not understand the meaning of a particular task (e.g. creating electronic presentations with presentation software) then they are unlikely to have undertaken that task. Some countries may mention commonly used software to help respondents identify the kind of activities carried out (see Box 35 on the case of Canada).

Box 35. Canada- Question on digital skills

The following questions are about your digital skills.

During the past 12 months, what learning activities have you taken to improve your skills relating to the use of computers, software or applications?

Select all that apply.

Have you taken:

• Free online training or self-guided learning

e.g., How-to videos, language learning apps, blogs

• Instruction from friends or family

• Free training through community centres or senior centres

• Other free training provided by public programs or organizations, other than your employer

• Training paid for by yourself

• Training paid or provided by your employer

OR

• None

During the past 12 months, which of the following software-related activities have you carried out using any device?

Select all that apply

Have you:

• Copied or moved files or folders

• Used word processing software

e.g., Word, TextEdit, Google Docs

• Created presentations, or documents with text and pictures, tables or charts

e.g., PowerPoint, Prezi, Keynote, Google Slides

• Used spreadsheet software basic functions

e.g., Excel, Open Office, Google Sheets

• Used spreadsheet software advanced functions to organize and analyze data

e.g., SPSS, Stata, Minitab, Excel’s advanced functions

• Used software to edit photos, video or audio files

e.g., Adobe Photoshop, Pixlr, Paintshop

• Written code in a programming language

e.g., Notepad++, Atom, UltraEdi

• Uploaded files or photos to an online data storage space

e.g., iCloud, Google Drive, Dropbox

OR

• None

Box 35. Canada- Question on digital skills (continued)

During the past 12 months, which of the following Internet-related activities have you carried out?

Select all that apply.

Have you:

• Deleted your browser history

• Blocked emails, including junk mail and spam

• Blocked other types of messages

e.g., messages on instant messaging apps or social network accounts

• Downloaded files from the Internet to your computer or other devices

• Changed the privacy settings on accounts or apps to limit your profile or personal information

• Changed the privacy settings on accounts or apps to enable or disable your location

• Shared files using an online data storage space

e.g., iCloud, Google Drive, Dropbox

• Backed up files using an online data storage space

e.g., iCloud, Google Drive, Dropbox

OR

• None

During the past 12 months, have you carried out any of the following activities on any of your devices?

Select all that apply.

Have you:

• Connected a new device to a Wi-Fi network

• Used the Internet to transfer photos or videos from one device to another

• Changed the security settings on your router to limit or enable traffic

• Used your smartphone as a GPS device for directions

• Changed the privacy settings on your device to enable or disable your location

• Connected a new device via Bluetooth

e.g., smartphone, computer, speaker, car

• Enabled automatic updates for, or manually updated, operating systems on any of your devices

OR

• None

Source: https://www.statcan.gc.ca/eng/statistical-programs/instrument/4432_Q2_V2.

Questionnaire logic

343. The structure and logic for a set of core ICT questions were presented in Figure 4. It is assumed that the ICT access and use questions are included as a module in a larger household survey (which will usually be the case for developing economies) or define an independent, specific ICT survey to households. The structure, question wording and definitions suggested in this Manual would not necessarily be used unchanged (or literally translated). However, it is important to retain the suggested meanings and logic.

344. The structure should be used in conjunction with the model questions and associated definitions of terms and categories (Table 6).

345. Note that the term population refers to the units that are in scope for each indicator. For example, a question collecting ‘Types of Internet access services available at home’ (used for deriving HH11) is asked only of the (population of) households that have Internet access.

Box 36. Uganda- Questions on ICT skills

Which if any of the following ICT tasks can you perform with ease?

1. Using word processing software

2. Using basic arithmetic formulas in a spreadsheet

3. Using spreadsheet advanced functions to organize and analyze data, such as sorting, filtering, using formulas, creating charts

4. Using software for electronic presentations (slides)

5. Sending e-mails with attached files (document, picture, video)

6. Posting messages (e.g. to chat rooms, newsgroups or forums)

7. Transferring files (e.g. digital camera, mobile phone, mplayer)

8. Finding, downloading and installing software from the Internet

9. Modifying or verifying the configuration of software applications

10. Modifying the security settings of Internet browsers

11. Computer programming using a specialized language

12. Creating a web page

13. Installing or replacing an operating system

How did you gain these ICT skills or capabilities? (Circle all that apply)

1. Formal education;

2. Non-formal education;

3. Informal learning.

4. Others (specify)

Source: https://www.ucc.co.ug/wp-content/uploads/2017/09/Final-Report-on-Access-and-Usage-of-ICTs-by-PWDs_Public-Dissemination.pdf.

346. The data collection method used will influence the way questions are asked. A questionnaire delivered by personal interview will include instructions to the interviewer in the form of prompts and skips.[17] Prompts should reflect the definitions of terms (e.g. computer, the Internet) shown in Table 6.

347. Box 37 illustrates interviewer instructions through excerpts from the ICT household questionnaires of Canada and Hong Kong, China.

348. A model questionnaire covering the core list of ICT household indicators is presented in Annex 2.

Box 37. Interviewer instructions for ICT household surveys of Canada and Hong Kong, China

Canadian Internet Use Survey, 2005 (personal interview face-to-face or telephone)

Section: Ever Users (EV)

EV_BEG Begin Module

Coverage: All respondents

EV_Q01 Have you ever used the Internet (E-mail or World Wide Web) from home, work, school, or any other location for personal non-business use?

1 Yes

2 No..........................................(Go to EV_END)

DK, RF........................................(Go to EV_END)[1]

Coverage: All respondents

EV_Q02 How many years have you used the Internet?

INTERVIEWER: Read categories to respondent.

1 Less than 1 year

2 1 to 2 years (1 year or more but less than 2 years)

3 2 to 5 years (2 years or more but less than 5 years)

4 5 or more years

DK, RF

Coverage: Respondents who have ever used the Internet

EV_END End Module

Hong Kong, China, Thematic Household Survey on Information Technology Usage and Penetration in 2008 (personal interview face-to-face)

D14 Show card

Including all places, for what purposes did you usually use (desktop computer or laptop / notebook / tablet PC or desktop game console (e.g. Playstation II/III or Sony (PS2/PS3), Xbox/Xbox360 of Microsoft, Game Cube/Wii of Nintendo etc.)) via wired Internet connection? Any other purposes? Any others? (Allow multiple answers)

F4 Show card

In the past 12 months, which e-Government services have you used for personal matters? (Allow multiple answers)

Sources: Statistics Canada, 2005

Census and Statistics Department, Hong Kong, China, 2008.

[1] DK=don’t know; RF=refusal.

[1] That is, a questionnaire completed by a respondent, usually as a paper form but it could also be electronic. See previous chapter for details.

[2] The time to complete the survey can be automatically recorded in the case of a CAPI or CATI interview. In the case of paper questionnaires, it is recommended to add this information in the questionnaire as complementary information to evaluate the response burden.

[3] The United States 2003 Computer and Internet Use Supplement (to the Current Population Survey) asked two questions about concerns relating to the Internet. They were asked after the other ICT questions and were only asked of respondents on an outgoing rotation of the CPS.

[4] These are instructions to interviewers that guide them through a questionnaire. For instance, if a respondent answers ‘no’ to a question on whether they used the Internet, the enumerator would ‘skip’ to the next logical question and not ask about use of the Internet.

[5] See UNSD (2005a, Chapter III).

[6] Focus groups involve informal discussions of relevant survey issues or topics with small groups of people who are in scope of the survey.

[7] This involves research on how potential respondents interpret questions in a questionnaire. More information can be found in UNSD (2005a, Chapter IX).

[8] A dress rehearsal is a large scale pilot test.

[9] UNSD (2005a, Chapter III).

[10] UNSD (2005a, Chapter III).

[11] For a survey module, some or all of the household and individual characteristics information would be collected as part of a larger survey and therefore would not need to be included in a module on ICT access and use.

[12] Location will often be known already, in which case it will not need to be asked of respondents.

[13] For a survey module, some or all of the household and individual characteristics information would be collected as part of a larger survey and therefore would not need to be included in a module on ICT access and use.

[14] Adapted from UNSD (2005b, Chapter 9).

[15] Variants might include questionnaires in different languages.

[16] A check digit is a number or letter in a keyed sequence, the value of which is derived from a function involving the other digits in the sequence. If a data entry error is made, the derived check digit will differ from the actual check digit, thus signaling that a keying error has been made. Check digits are typically used for record identifiers and codes rather than quantitative data.

[17] These are instructions to interviewers that guide them through a questionnaire. For instance, if a respondent answers ‘no’ to a question on whether they used the Internet, the enumerator would ‘skip’ to the next logical question and not ask about use of the Internet.

Chapter 7. Sampling for ICT household surveys

349. This chapter focuses on the design of ICT household surveys, including survey scope and target populations, statistical units, and sample design and selection.

350. Many aspects of survey design are not specific to ICT household surveys. Indeed, as we saw in Chapter 5, about one-third of countries use existing household surveys to collect ICT access and use data. These may be multipurpose surveys, labour force surveys, household budget surveys or population censuses. Therefore, this chapter considers design of household surveys generally, with additional emphasis on the application to ICT measurement.

Scope and coverage for households and individuals

351. The scope of a survey refers to the statistical units (members of the target population) required to be represented by the survey and for which data are to be collected and tabulated. For household surveys, the scope may be all households, a subset of household types or geographic locations, or a set of certain individuals within households.

352. In respect of ICT household statistics, the Partnership has proposed statistical standards associated with the core list of ICT indicators, including recommendations on survey scope for households and individuals.

353. Youth and children with access to ICT are coming of age as digital natives, adopting ICTs early, as exemplified in Box 38. Although they are better positioned than their parents to harness the power of digital technologies in new and imaginative ways, it is also crucial to protect children from harmful uses of ICT. Therefore, the suggested age scope for individuals is 5 years old and above. There is no maximum age scope recommended.

Box 38. ICT gap between generation- Curaçao experience

The ICT & Media Survey conducted in 2017 by the Central Bureau of Statistics in Curaçao aims to understand the level of Internet penetration and usage in households and by persons aged 6 and older. With online presence intruding on all levels of society, such as social interaction, education and entertainment, it is imperative that we understand to which extent information technology has integrated our society.

The results show that younger generations are more involved with information technology than the older generations. It also depicts a gap between males and females. The reason for this gap might be generational as literature suggests however, the difference in generations will be explored in a Modus article “Technology and Media Use through Generations”.

Source: https://www.cbs.cw/website/ict-media-survey-2017_3403/item/integration-of-ict-in-private-households_2446.html.

354. Most countries will have individual scope limitations not related to age, for instance, the scope exclusion of individuals in institutions such as prisons and nursing homes, members of the armed forces, diplomats, short-term foreign visitors and those with no fixed address, such as nomads. In countries with a large proportion of temporary workers living in collective accommodation, it may be useful to include this subpopulation with a specific sampling scheme.

355. Ideally, the scope of surveys that collect household ICT access/use data should include both urban and rural areas.

356. Some countries, especially those with a large population of immigrant workers, may not cover the population living in collective housing (e.g. barracks in work locations). This should be highlighted in the metadata provided for international comparison. It is recommended that the whole resident population (including migrants staying for at least a year) is covered in statistical surveys on ICT use, especially because this population may include intensive users of ICT (e.g. for communication with their families, or for sending money transfers).

357. Coverage is the degree to which the in-scope units are actually on the sample frame (and therefore represented in the sample). Once the required scope is determined, alternative sample frames can be investigated to identify sources that provide the best (most up-to-date and/or complete) coverage of households and individuals that are within the scope of the survey.

358. There may be geographic coverage issues for some countries, for example, certain rural or remote populations may be in scope but poorly covered.

359. Other omissions will reflect coverage issues such as errors in the sample frame (discussed below). It is important that the omission of a significant proportion of the population, whether by a specific scope limitation or because of undercoverage, be mentioned in output metadata. One example of failure to do so is that comparisons are made between the whole population of one country and the urban population of another.

Target populations and sample frames

360. The target population is the population about which survey estimates will be produced, that is, the scope of the survey. The sample frame (alternatively known as a ‘survey frame’ or ‘population frame’) is the list from which the units in the sample are selected. Household survey samples are generally selected in two stages: first by area, then by household within area.

361. The frame most commonly used in the first sampling stage is a list of enumeration areas, frequently based on the most recent population census. In certain countries, alternative first-stage sample frames can be developed on the basis of the territorial division of the country into polling stations, or from property valuation files. Where a population census is used, enumeration areas are likely to consist of only a few hundred households. Areas sampled from the list of enumeration areas are often referred to as ‘primary sampling units’ (PSUs) and are selected either with equal probability or (much more often) with probability proportional to size (pps), using as a measure of size the number of households, the number of dwellings, or the population in the area, as reported by the census.[1] Using a proportional criterion for choosing the PSUs implies that a large PSU is more likely to be selected than a small one.[2]

362. The second-stage sample frame is usually the complete list of all households in each selected PSU. Within each PSU, a fixed number of households is generally selected from that list, with equal probability.

363. This two-stage strategy results in a sample composed of ‘clusters’ of households that are relatively closely located geographically.

364. The first sampling stage may occasionally need to be adapted to local conditions. One common alternative is the use of a ‘master sample frame’ – a common set of PSUs used by many different household surveys over several years.

365. Other forms of sample frames for ICT household surveys include registers of individuals (often known as central population registers). Such frames are used in some European countries (see Box 39 on the case of Luxembourg), where registers are maintained for administrative purposes. Some developing economies have registers of households or dwellings that may be based on population census records or maintained for administrative purposes, such as for land taxes, or some other listing, such as electricity connections.

Box 39. Luxembourg: Community Survey on the usage of ICT among households and individuals

A simple random sample of nearly 6,000 individuals aged between 16 and 74 is taken from the Registre National des Personnes Physiques (RNPP). The individuals who have a phone number registered in the phone book are then invited to reply to the questionnaire by telephone, while the others are requested to participate online. Furthermore, individuals still have the possibility to switch from one mode of data collection to another, should they wish so. Thus, the final sample should be more representative of the resident population in Luxembourg than when interviews are conducted only by telephone.

Source: https://statistiques.public.lu/en/surveys/espace-households/TIC-survey/index.html.

366. A frame may be quite suitable for one set of data requirements but hold unacceptable biases for other data.[3] This is clearly an issue to be considered when ICT questions are included in a multipurpose household survey. For example, a sample frame based on lists of mobile phone numbers is bound to introduce bias with regard to questions about mobile phone use, and quite possibly with regard to other indicators as well (such as Internet use, given that mobile phones increasingly allow Internet access).

367. Generally speaking, the desirable characteristics of sample frames are:

• completeness with respect to coverage of the in-scope population;

• timeliness: it should be as up to date as possible and have the potential to be updated in the future to permit further iterations of the household survey;

• accuracy of information on the records;

• availability of descriptive data that will assist in sample design and possibly classification of data, for example, location of units; and

• availability of household access information: usually address or telephone numbers.[4]

368. Major frame problems for household surveys include undercoverage, clusters of elements, blanks and duplicate listings. Undercoverage is a particular concern for household surveys in developing economies and may occur at the levels of geographic area, household and/or individual. A typical problem is the identification of households within an area. Clusters of elements refers to the situation where a single unit on the frame consists of multiple units in the target population, for example, a dwelling with several households. Blanks refer to units that do not contain a member of the target population, for example, an empty dwelling. Duplicate listings occur when a member of the target population appears more than once on the frame, for example, a nomad person moving from place to place may have a higher probability of selection.[5]

369. Field-testing of questionnaires and procedures may present an opportunity to test problems with the sample frame and to make adjustments if feasible.

370. UNSD[6] recommends that NSOs with significant household survey programmes invest resources in creating and maintaining a master frame of PSUs, based on the geographic areas defined and used in the preceding census. Ideally, the frame would be created as soon as possible after the completion of a population census, thereby reducing the amount of labour involved. The Population and Housing Census Round of 2020 may be an excellent occasion for countries to update their frames for social surveys.

371. It is often the case that only the NSO in a country has access, for confidentiality reasons, to an adequate household or individual survey frame. Other organizations that carry out ICT surveys (such as ministries for ICT, telecommunication regulatory authorities or private institutions) may not have a reliable frame. It is therefore important for them to collaborate with NSOs to avoid using poor frames that could produce biased estimates.

Statistical units

372. For ICT household measurement, there are typically two statistical units: households and individuals. The household unit is used to elicit information about the facilities in place in the household (e.g. whether there is a computer or Internet connection). Box 41 presents an example of modifications to the statistical unit at the household level. The individual unit is used to provide information on use of ICT (both at, and away from, home) and, most importantly, the nature of that use (for instance, frequency and range of activities undertaken). The core indicators require that both the household and the individual are statistical units. It is necessary to select both households and individuals, and to design questionnaires and other survey materials for both types of units.

Box 40. Definition of household in France

Since 2005, the definition of a household, in the sense of the household surveys conducted by INSEE, has been distinctly modified. A household is considered as being a set of (related or unrelated) people habitually sharing the same dwelling (whether it is their main residence or not) and who have a joint budget. The habitual residence is the dwelling in which they usually live.

The household is therefore composed of the people who share the same budget, that is :

1) who contribute resources towards the expenses incurred for the life of the household;

2) and/or who merely benefit from those expenses.

In surveys prior to 2005, people were required to share the same main residence to be considered as households (or “ordinary households”). It was not necessary for them to share a common budget. De facto, a household corresponded to a dwelling (main residence). Since 2005, however, a dwelling can comprise several households, referred to as “living units”.

Source: https://www.insee.fr/en/metadonnees/definition/c1106.

Box 41. Honduras: change in statistical units in household surveys

The National Statistical Institute of Honduras modified the statistical unit for several questions on access to ICT in the Living Conditions Surveys of 2005, 2006 and 2007. In particular, questions about access to a radio, TV set, fixed telephone and computer were recorded at dwelling level until 2006 but at household level in 2007, while access to a mobile telephone was asked at dwelling level until 2005 and at the individual level from 2006.

Source: Presentation by INIDE Honduras at the 4th workshop on the Measurement of Information Society in Latin America and the Caribbean (San Salvador, February 2008).

373. With respect to the definition of ‘household’, UNSD (2017) recommends that the household be used as the unit of enumeration. It is important to make the difference between household and family: a household can contain more than one family, or one or more families together with one or more non-related persons, or it can consist entirely of non-related persons. A family typically will not comprise more than one household[7]. Two components are thus important: the definition of a usual resident and the definition of a housing unit.[8] Of these, the definition of housing unit may be more problematic and it may not always be clear what constitutes a ‘housing unit’. UNSD considers that a housing unit definition takes into account whether the persons living there live and eat separately from others in the same structure.

374. The following definition is based on the ‘housekeeping concept’ described in the UNSD’s Principles and Recommendations for Population and Housing Censuses Revision 3 (UNSD, 2017):

“The concept of household is based on the arrangements made by persons, individually or in groups, for providing themselves with food and other essentials for living. A household may be either (a) a one-person household, that is to say, a person who makes provision for his or her own food and other essentials for living without combining with any other person to form a multi-person household or (b) a multi-person household, that is to say, a group of two or more persons living together who make common provision for food and other essentials for living. The persons in the group may pool their resources and have a common budget; they may be related or unrelated persons or a combination of persons both related and unrelated. This arrangement exemplifies the “housekeeping” concept.”[9]

375. This concept does not assume that the number of households and housing units are – or should be – equal. Further elaboration by UNSD makes it clear that the institutional population are not members of households, although they are in scope of population censuses. The institutional population consists of “… persons living in military installations, correctional and penal institutions, dormitories of schools and universities, religious institutions, hospitals and so forth.” However, households do include “…persons living in hotels or boarding houses …” who “… should be distinguished as members of one- or multi-person households, on the basis of the arrangements that they make for providing themselves with the essentials for living.”

376. For the purposes of the Manual, it is recommended that the definition of household be based on the UNSD housekeeping concept and be defined as follows. A household consists of one or more people, who:

• may or may not be related to each other,

• share accommodation, and

• make common provision for food.

377. With respect to the institutional population, it is suggested that it will generally be impractical to include them in scope of individuals.

378. Examples of household definitions are presented in Box 42.

Box 42. Definition of household used in Australia and Hong Kong, China

The Australian Bureau of Statistics (ABS), in its 2006–07 multipurpose survey, that included ICT topics, defined a household as “…a person living alone, or two or more related or unrelated persons who live and eat together in private residential accommodation.”

Hong Kong, China defined a domestic household as “a group of persons who live together and make common provision for essentials for living. These persons need not be related. If a person makes provision for essentials for living without sharing with other persons, he is also regarded as a household.”

Source: ABS (2007); Census and Statistics Department, Hong Kong, China (2008).

Sample design and selection

379. The main sample design and selection issue for ICT household statistics is the need to produce a representative sample of households and individuals (in order to measure household access to ICT and individual use of ICT). The issues applying more generally to household surveys also apply to those measuring ICT access and use.

380. Some general points related to sampling techniques, and calculation of sample sizes and sampling errors are presented below.[10]

• For household surveys using face-to-face interviews, where fine geographic tabulation is not required, the clustering of sampled units, that is, concentrating the sample into a few geographic areas, is very cost effective (with smaller transportation and listing costs). Potential losses of precision can usually be compensated by increases in sample size.

• Stratification refers to a grouping of population units into mutually exclusive groups of units referred to as ‘strata’, within each of which an independent sample is chosen. Stratification is usually done with one of two objectives: either to potentially improve the overall precision of the estimates by gaining control over the composition of the sample; or to produce estimates for subgroups of the population that could otherwise be poorly represented in the sample. These two objectives are not necessarily complementary, and it is the second that is generally pursued in household surveys to ensure adequate statistics for the relevant geographical units. An example of stratification for Greece is shown in Box 43. Another motivation for stratification can also be to increase the sampling fraction within those strata that are expected to have higher variance (perhaps due to being more heterogeneous in composition).

• Actual sample size, rather than the sampling fraction within strata, is the major determinant of the size of sampling error. Therefore, minimum sample sizes need to be maintained, even in strata where the population is low. Conversely, where total sample size is necessarily low because of costs, fine stratification should be avoided. Sample sizes will need to be higher where a higher degree of reliability or confidence is required.[11]

• In general, a greater level of detail in output requires a higher sample size for a constant degree of reliability (reflected by the magnitude of sampling error). This is relevant to ICT household statistics where data are disaggregated by some or all the classificatory variables described in Chapter 4. A particular case is sampling for topics where samples may be designed to provide good estimates, requiring some loss of efficiency (that is, the sample size is larger than needed for achieving the desired accuracy for estimates at the national level).[12]

• Individual selection . The number of household residents interviewed in ICT household surveys varies across countries, with some countries interviewing all members and others interviewing one selected person. Where all household members are selected for interview, in order to avoid selection bias, it is important to make contact with all members, rather than only those who are available at the time of interview. If only one member of the household is selected for interview, that member should be selected at random and in an unbiased manner. In the event that the selected person is not present when the interviewer visits or phones the dwelling, he/she should be contacted later by a follow up visit (or perhaps by a telephone call). Methods of random selection of a household member include: Kish grids,[13] selecting the person with the next or last (most recent) birthday, and selecting individuals according to random selection of pre-assigned identification codes.[14] Answers given on behalf of the absent individual (proxy answers) are undesirable in all kinds of surveys (but particularly in many of the questions related to ICT indicators measurement).[15]

Box 43. Greece: stratification of the sample of the ICT household survey

In Greece, the 2012 household survey on ICT was carried out as a stand-alone survey using a subsample of the Survey on Living Conditions (EU-SILC), which is harmonized in all European Union Member States. The sample design was three-stage stratified sampling, with primary sampling units defined as areas (one or more geographic areas) and the final unit the household. Collective households were excluded from the in-scope population. For the individual questionnaire, one member of the household was selected at random.

There were two levels of stratification:

(i) The first level was geographical stratification based on the partition of the country area into thirteen standard regions corresponding to the European NUTS II level. The two major city agglomerations of Greater Athens and Greater Thessalonica constituted separate major geographical strata.

(ii) The second level of stratification involved grouping of municipalities and communes within each NUTS II region by degree of urbanization (that is, according to their population size) into four categories. These categories were defined by the population size intervals 1–999, 1000–4999, 5000–29999 and 30000 and over. The number of final strata in the thirteen regions was 50. The two major city agglomerations, which account for 40 per cent of the population, were further partitioned into 31 and 9 substrata (administrative subsections), respectively, on the basis of the city blocks of the municipalities that constitute them. Thus, the total number of strata for this survey was 90.

Source: National Statistical Service of Greece, https://www.statistics.gr/en/statistics/-/publication/SFA20/2012

381. As introduced above, most countries have samples selected in a series of stages as follows:

• Area sampling. The first stage is a stratified sample of areas, known as ‘enumeration areas’ (or ‘primary sampling units’ (PSUs)) as a first step. The areas have known characteristics (commonly derived from the previous population census). The benefits of stratification are particularly evident at this stage, therefore a significant effort should be made to properly stratify geographic areas.[16]For most household surveys in developing and transition economies, PSUs are selected with probability proportional to population size (that is, a large area is more likely to be selected than a small one).

• This first stage of PSUs results in geographic ‘clusters’. As the name implies, these are sets of units (usually dwellings or households) that are closely located geographically (‘clustered’) in order to minimize the cost of collection. Clustering, while reducing costs, is also likely to increase sampling error because of higher homogeneity of units within clusters (the ‘clustering effect’). [17]

• There may be a second sampling stage at the subcluster level (for example, ‘segments’ or ‘blocks’).

• Household sampling. Households (or dwellings) within clusters (or subclusters) are typically listed in some fashion in order to create a sample frame. It is common to have a fixed size sample of households within each cluster, and to select it by systematic, equal probability sampling[18], though other sampling methods are also possible.

• If there is more than one household in a dwelling, there is a need to sample households within the dwelling. Typically, information about the household will be provided by an individual within that household (randomly chosen or a responsible person as defined by the statistical agency; some countries select the household head to provide the information, though this now tends to be discouraged).

• Sampling of individual/s within the household. As discussed above, some countries sample all individuals in a household and others sample a single individual (at random). The Manual recommends that all members be sampled where this is feasible.

382. Error associated with a sample is known as sampling error (or sample error). This is an element of data quality and is examined in Chapter 9.

383. Because the sample of households and individuals selected is unlikely to be representative of the population, it is important to weight responses according to independent estimated distributions of the population. This is further discussed in the next chapter.

Box 44. AfterAccess: Challenges of collecting robust data for policymaking in the Global South

AfterAccess is a series of ICT access and use surveys in 20+ Global South countries. A range of household and individual data was collected to provide insight into the demand-side barriers to digital equality, and thereby, comprehensive national and regional evidence-bases to inform policy and regulation.

The biggest challenge in collecting nationally-representative data, comparable across multiple countries, was sample design. Publicly available sample frames allowed random selection to the ward/village level, but random selection of households required that, in many of the survey countries, households be listed first. This was problematic in large wards (e.g., some wards in Mumbai have up to 100,000 households). Therefore, wards/villages exceeding a threshold size were further segmented using a pre-determined method with help from local authorities and local maps (printed and digital). A listing was thereafter conducted to randomly select 20-25 households per sampling point. Once a household was selected, survey respondents were selected using the Kish grid, maintaining randomness at all stages.

Enumerators received practical training in field procedures and on the survey content before being sent to the field. Listing and surveys were conducted using mobile devices (except in Sri Lanka where paper listing was complemented by GPS coordinates). This facilitated collection of GPS coordinates for real-time quality checking both on field and off field.

This listing process, though costly, ensured randomness in household selection, and allowed for similarly representative samples of other units to be drawn in a single field exercise. Thus, representative surveys of persons with disabilities in Nepal and Sri Lanka, and of small and medium enterprises in Sri Lanka were conducted more economically and simultaneously.

Source: LIRNEasia

Risks of other sample selection methods

384. The statistical inferences made from samples are based on the mathematical-probabilistic properties of estimators, which are used to extrapolate the sample results to the target population. (These properties include those related to unbiasedness, sample error, etc.). These methods are only valid if the selection method is probabilistic, in the sense that the ex-ante probability of inclusion in the sample of any unit is known, in accordance with the design procedures. However, many household surveys are still carried out with alternative methods chosen for the sake of convenience, such as facilitating location of respondents, or of cheaper cost. While NSOs generally avoid making inference from such samples, other agencies may turn to non-probabilistic methods.

385. The most frequent non-probabilistic methods are random walk and quota sampling, and random digit:

• In random walk sample selection[19], interviewers are instructed to begin the interview process at some geographic point in, say, a village, and follow a specified path of travel to select the households to interview. It may entail either selecting every nth household or screening each one along the path of travel to ascertain the presence of a special target population. In the latter instance, each qualifying household would be interviewed for the survey until a pre-determined quota has been reached. This methodology is often argued for as a way to avoid the costly and time-consuming expense of listing all the households in the sample area – village or cluster or segment – as a prior stage before selecting the ones to be interviewed. It is also argued for on the grounds that non-response is avoided since the interviewer continues beyond non-responding households until he/she obtains enough responding ones to fulfil the quota. In practice, however, it is dubious whether this method gives place to a set of ex-ante known probabilities of selection. This is attributable to (a) interviewer behaviour and (b) the treatment of non-response households, including those that are potentially non-response. It is not recommended for household surveys to provide official statistics.

• Random digit dialing is a method for selecting people for involvement in telephone-assisted statistical surveys, by generating telephone numbers at random. It has the advantage that it includes unlisted numbers that would be missed if the numbers were selected from a phone book. In populations where there is a high telephone-ownership rate, it can be a cost efficient way to get complete coverage of a geographic area. However, given that ICT surveys precisely want to measure, among other, access and use of telephones, random digit dialing will surely introduce a bias in the measurement, as it will only reach respondents who have access to telephone.

386. Non-probabilistic methods for sample selection are not recommended for ICT household surveys. Agencies other than NSOs which are responsible for carrying out ICT surveys should rely on the competences and statistical infrastructure (household sample frame) of the NSO to design the survey sample.

[1] Advice on the selection of Primary Sampling Units (PSUs) with pps can be found in Chapter XV of UNSD (2005a).

[2] PSUs that are exceptionally large may need to be divided into portions, one of which is selected per PSU in order to economize on household listings. This process, known as segmentation, only needs to be conducted in the large PSUs that happen to be selected in the sample. Segmentation is not a very desirable operation, because it is often conducted by unsupervised fieldworkers as a part of the household listing operation. This is almost impossible to supervise, thus entailing the serious risk of selection bias.

[3] An example is a frame consisting of households who live in homes that they own. While this may be a good frame for measuring characteristics of home ownership, it is likely to be a poor frame for measuring household ICT access because home owners could be more likely to have access to, and use, ICT.

[4] Though in some circumstances, survey questions may be posed, or subsequent household visits arranged, at a central location where householders or heads of households attend, for example, for voting.

[5] See UNSD (2005a, Chapter II) for more detail.

[6] See UNSD (2005a, Chapter V; 2005b, Chapter 4).

[7] However, the existence of polygamous families in some countries, as well as shared child custody and support arrangements in others, means that individual countries should decide how best to derive and report data on families.

[8] UNSD (2005a, Chapter VIII).

[9] UNSD (2017) also discusses the ‘household-dwelling’ concept of a household, whereby a household is associated with a single housing unit.

[10] Readers should refer to specific chapters of UNSD manuals (UNSD, 2005a, b) for more detailed information.

[11] Confidence in an estimate is often expressed as the 95% confidence interval around the estimate, that is the value of the estimate +/– two standard errors (this assumes a normal distribution of the variable being measured). It may also be expressed as the ratio of the estimate’s standard error to the estimate (referred to as the coefficient of variation or the relative standard error).

[12] UNSD (2005a, Chapter II).

[13] Kish, Leslie (September 1949), "A Procedure for Objective Respondent Selection within the Household", Journal of the American Statistical Association 44 (p247). The procedure involves the creation of a list individuals based on age and sex. Choices can be made based on rotation of combinations of age and sex.

[14] Ainsworth, Martha and Juan Muñoz (1986) “The Côte d'Ivoire Living Standards Survey: Design and Implementation”, LSMS Working Paper No. 26, The World Bank (pp15-16). The procedure involves assigning a code (1-20 in the original study) to household members and then making a random selection of the assigned codes.

[15] Collection of information about children’s use of ICT can be complicated by national laws preventing interviewing of minors. The alternative of asking another household member to provide information in respect of children’s activities can introduce bias (especially for older children whose activities may not be known to the respondent). However, a number of countries collect information about children in this way. Those that do should take measures to minimize response bias.

[16] UNSD (2005a, Chapter IV).

[17] See UNSD (2005b, Chapter 3) for more details.

[18] UNSD (2005b, Chapter 3).

[19] See Turner (2003) “Sampling strategies”: http://mdgs.un.org/unsd/demographic/meetings/egm/Sampling_1203/docs/no_2.pdf

Chapter 8. Data processing for ICT household statistics

387. This chapter discusses data processing for ICT household statistics, from the data entry phase of a survey cycle to calculation of output data. This corresponds to the GSBPM phases of “Collect”, “Process” and “Analyse”. Emphasis is placed on the importance of integrating computer-based quality controls to field operations for achieving the high data quality standards discussed in the following chapter. Tabulation of results, while an aspect of data processing, is discussed in Chapter 10, Dissemination.

388. Experience from the past three decades shows that data management can and should play a critical role beginning with the very earliest stages of the survey effort. Starting in the mid-1980s, the integration of computer-based quality controls to field operations has been identified as one of the keys to improving the quality and timeliness of household surveys. Under this strategy, data entry and consistency controls are applied on a household-by-household basis as a part of field operations, so that errors and inconsistencies are solved during the interview or by means of re-visits to the households.[1]

389. The most important and direct benefit of integration of these quality controls is that it significantly improves the quality of the information, because it permits correction of errors and inconsistencies while the interviewers are still in the field rather than by office ‘cleansing’ later. Besides being lengthy and time-consuming, office cleansing processes tend to produce databases that are internally consistent but do not necessarily reflect the realities observed in the field. The uncertainty stems from the myriad of decisions – generally undocumented – that need to be made far from where the data are collected, and possibly long after the data collection.

390. The integration of computer-based quality controls can also generate databases that are ready for tabulation and analysis in a timely fashion. Furthermore, databases may even be prepared as the survey is conducted, thus giving survey managers the ability to effectively monitor field operations. Integration also fosters the application of uniform criteria by all the interviewers and throughout the whole period of data collection.

391. Planning for the integration of computer-based quality controls should run in parallel with other survey planning and be well advanced prior to finalization of questionnaire design. Sequencing of questions and the inclusion of questions that facilitate edit checks need to be planned in conjunction with data processing preparations.

392. Many data processing tasks are not specific to ICT measurement. This chapter therefore pays particular attention to ICT-specific data processing issues, including editing and calculation of ICT indicators.

393. Even though the integration of computer-based quality controls should improve data quality, non-sampling errors may still occur in data processing. This will be discussed in the next chapter.

Data entry

394. As stated above, data should ideally be entered during fieldwork. When using CAPI or CATI programs this will result by default. When more traditional pen and paper interviews are in place, this can be done by organizing fieldworkers into teams that include a data entry operator, two to four interviewers and a supervisor. Where the data entry operator cannot join the team during its visits to each survey location (generally a primary sampling unit), the latter are visited at least twice, to give the operator time to enter and verify the consistency of the data in between visits so that interviewers can re-ask questions where errors, omissions or inconsistencies are detected by the data entry program. If notebook computers are available, the data entry operator can join the rest of the team in its visits to the survey locations or the interviewers can be asked to enter the data themselves. The whole team stays in the location until all the data are entered and certified as complete and correct by the data entry program.

395. Data entry may also occur as a separate process in a specialized data entry unit of the statistical agency (though, as discussed above, this is not the preferred approach). As with other aspects of survey implementation, it is important to have good training and procedures for data entry. These will help to minimize data entry errors as will techniques such as use of check digits[2] and other checks of keying accuracy.

396. As IT evolves, more applications for data entry are available, including for free, for NSOs to use. Many countries use tools developed by advanced statistical offices, such as CSPro (developed by the US Bureau of Census)[3] or Blaise (by Statistics Netherlands).[4] Before developing specific data entry applications for an ICT survey, countries (especially those with less resources) may wish to explore existing solutions[5].

Data editing

397. Data editing refers to the operations that are carried out to produce a final file ready for analysis. It includes checking the validity of individual records (i.e. at the individual and household levels) as well as that of aggregates. These forms of editing are commonly referred to as ‘microediting’ and ‘macroediting respectively.

Microedits

398. Microediting may also be called input editing and is applied to individual records. There are five different types of microedits: range edits, checks against reference data, skip checks, consistency checks and typographical checks. They can be described as follows:[6]

• Range edits check that data values are valid, for instance, categorical variables can only have a predefined value (e.g. sex can be coded only as 1 or 2).

• Checks against reference data are an example of a range check and involve a comparison of the reported value with external data (such as a reasonable household size range).

• Skip checks verify that the logic of the questionnaire has been followed, for instance, that the correct populations are asked each question; in a CAPI or CATI environment, the program will usually determine the skips, so errors should not occur if the programming has been done correctly. [7]

• Consistency checks determine whether the information in the questionnaire is internally consistent, for instance, reported age matches reported date of birth (see Box 45 for an example of a consistency check related to ICT data).

• Typographical checks (to find keying errors by the interviewer or data entry operator); these may be difficult to find and may be discovered as a result of other edits or through check-add (or ‘control’) totals or check digits.

Box 45. Example of a consistency check

An individual respondent belongs to a household that responded “No” to the question “Does any member of this household have Internet access at home regardless of whether it is used?”. If that individual responds “Yes” to the option “Home” as a response category to the question “Where did you use the Internet in the last three months?”, the response is not internally consistent and should be queried by the interviewer.

399. In addition to the five microedits types mentioned above, it can be useful to distinguish ‘fatal’ from ‘non-fatal’ edit failures. There is typically more tolerance of the latter, which are designed to identify values or conditions that are not impossible but are uncommon and worthy of investigation. Fatal errors indicate a situation that is logically impossible and will include things like data components not adding to a total or inconsistent information on age and date of birth. Where there are a large number of fatal errors, investigations into the reasons should be undertaken. They may indicate a problem with the editing program or systematic errors made by an individual interviewer. Fatal errors need to be resolved before compromised data records are incorporated into any final tabulation of results. Resolution could be to amend the values causing the failures or to omit the records in error, where they cannot be fixed.[8] It is important to note that both actions have implications for the final calculation of estimates.

400. Where data are collected by personal interview, data editing is often performed during the interview. As mentioned before, this can be facilitated by use of CAPI or CATI programs, which should automatically advise the interviewer of edit failures. However, on-the-spot editing during interview can also happen even when these computer assisted programs are not used. For instance, the interviewer may have prompts indicating a range of realistic responses. In the case of ICT statistics, an example of such a prompt is that where a household has access to the Internet at home, an individual who uses the Internet but does not select home as one of the locations of use should be probed by the interviewer. This is a ‘non-fatal’ error – a ‘no’ response to home use may be correct.

401. In cases where data are not edited at time of interview, good questionnaire design can be very helpful in minimizing respondent errors. Some checks may be used during data entry to uncover keying errors.

402. Table 12 suggests a number of microlevel edits for ICT data. They would ideally be applied during an interview – as prompts to the interviewer using a paper form, as part of data entry programs operated in integration with fieldwork. Or as part of CATI/CAPI programs. During an interview, microedits may result in probes (or prompts), for example, to confirm a ‘no’ response or to clarify an ‘other’ response. Examples of probes are also included in Table 12.

Macroedits

403. Macroediting might also be called output editing and consists of checks of aggregated data for coherence, including:

• consistency of estimates over time, for instance, use of the Internet by individuals is expected to grow over time;

• relationship with data external to the survey, for instance a survey estimate of the proportion of households with a fixed telephone line should be related to ITU’s infrastructure indicator, fixed telephone lines per 100 inhabitants; and

• logical rules are obeyed, for instance, components of a percentage distribution should add to 100. In cases where multiple responses are allowed (for instance HH9 – Internet activities) percentages are expected to add to more than 100.

404. If major errors are found during macroediting, they might be simply fixed by, for instance, correcting estimation programs if that is where the errors originated. However, if their source lies with original unit record data, it can be difficult to fix the problem once a survey is completed. It is therefore suggested that, where possible, interim tabulations and macroedits are performed during the course of the survey so that errors that cause failures may be addressed. An understanding of the macroedits to be applied might also be useful at the inception of the survey, for instance, in designing questionnaires or input edits so that failures are less likely to occur at the output stage.

405. Table 12 presents a range of possible macroedits applying to the core ICT indicators. Macroedits would be undertaken on tabulated data as explained above. Countries collecting ICT household statistics for the first time might use results from the surveys of similar countries to apply macroediting.

Table 12. Micro and macroedits for ICT household statistics

Indicator

Possible microedits and probes

(edits applied to individual records, preferably at time of interview)

Possible macroedits

(edits applied to aggregated data)

HH1

Proportion of households with a radio

If the interview takes place at the household’s residence, the interviewer could check for the presence of a radio (or a radio integrated in another device). The interviewer will need to ascertain whether at least one radio in working condition is generally available for use by all members of the household at any time, that is, that the radio is considered a household asset (if not, the correct response is that the household does not have a radio).

Historical trends, would expect a flat trend or slow growth when the definition is updated to include radios included in other devices.

HH2

Proportion of households with a television

If the interview takes place at the household’s residence, the interviewer could check for the presence of a TV aerial or a TV set (or a TV integrated in another device). The interviewer will need to ascertain whether at least one TV in working condition is generally available for use by all members of the household at any time, that is, that the TV is considered a household asset (if not, the correct response is that the household does not have a TV).

Historical trends, would expect a flat trend or slow growth.

Table 12. Micro and macroedits for ICT household statistics (continued)

Indicator

Possible microedits and probes

(edits applied to individual records, preferably at time of interview)

Possible macroedits

(edits applied to aggregated data)

HH3

Proportion of households with fixed telephone line

If the interview takes place at the household’s residence, the interviewer could check for the presence of a fixed telephone line if the respondent is not sure.

1. Historical trends, would expect a fairly flat trend or show slow decline.
2. Broad consistency with core indicator A1 (fixed telephone subscriptions per 100 inhabitants), absolute values and growth.

Proportion of households with mobile cellular telephone

As a mobile phone is a personal device, the interviewer will need to ascertain whether at least one mobile telephone in working condition is generally available for use by all members of the household at any time, that is, that the mobile phone is considered a household asset (if not, the correct response is that the household does not have a mobile telephone).

1. Historical trends, would expect medium to high growth (though once the revised concept of household access is adopted, countries may see a decline in the level of mobile telephone access).
2. Broad consistency with (but lower than) growth rate and values for core indicator A2 (mobile cellular telephone subscriptions per 100 inhabitants).

Proportion of households with a smartphone

As above, but with respect to smartphones.

As above.

HH4

Proportion of households with a computer

If the interview takes place at the household’s residence, the interviewer could check for the presence of a computer (noting the devices that are now defined as a computer). The interviewer will need to ascertain whether at least one computer in working condition is generally available for use by all members of the household at any time (if not, the correct response is that the household does not have a computer).

Historical trends, would expect medium to high growth (though once the revised concept of household access is adopted, countries may see a decline in the level of computer access).

HH5

Proportion of individuals using a computer

Where a household has access to a computer (HH4), it is quite likely that the interviewed individual/s (especially if young and educated) use a computer. Therefore probe a ‘no’ response.

Historical trends, would expect medium to high growth.

Table 12. Micro and macroedits for ICT household statistics (continued)

Indicator

Possible microedits and probes

(edits applied to individual records, preferably at time of interview)

Possible macroedits

(edits applied to aggregated data)

HH6

Proportion of households with Internet

If the interview takes place at the household’s residence, the interviewer could check for the presence of an Internet connection (e.g. modem connection, mobile network).

The interviewer will need to ascertain whether the Internet is generally available for use by all members of the household at any time (if not, the correct response is that the household does not have Internet).

Historical trends, would expect medium to high growth (though once the revised concept of household access is adopted, countries may see a decline in the level of Internet access).

HH7

Proportion of individuals using the Internet

Where a household has access to the Internet (HH6), it is quite likely that the selected individual/s use the Internet. Therefore probe a ‘no’ response.

Historical trends, would expect medium to high growth.

HH8

Proportion of individuals using the Internet, by location

If HH7 is ‘yes’ at least one of the response categories must be selected.

See suggestions for response items, below.

1. Historical trends may not change much unless there is a large change in locations of access, for instance, home Internet access increases or government run community access facilities are opened.
2. The values of these categories, as a percentage of individuals using the Internet, would add to more than 100 (per cent), because at least some individuals use the Internet at more than one location.

Home

Where a household has access to the Internet, Internet users who record a ‘no’ against this category should be probed. Where a household does not have access to the Internet, a ‘yes’ against this category is possible (according to the revised concept of household access), but could be probed.

Where household access to the Internet is high, this will generally be the largest output category.

Table 12. Micro and macroedits for ICT household statistics (continued)

Indicator

Possible microedits and probes

(edits applied to individual records, preferably at time of interview)

Possible macroedits

(edits applied to aggregated data)

HH8
cont.

Work

Those in the workforce who are Internet users and who record a ‘no’ against this category could be probed.

Place of education

Students who are Internet users and who record ‘no’ against this category could be probed. Individuals who are not students should not report ‘yes’ against this category; teachers and others who work at a place of education, would report ‘work’ as place of Internet use. Where a place of education is used as a location for general community Internet access, such use should be reported under Community Internet access facility.

Another person’s home

At facility open to the public

Interviewers may need to use examples relevant to the country to explain this category.

Community Internet access facility (typically free of charge)

Interviewers may need to use examples relevant to the country to explain this category.

The values in this category may increase with policy intervention.

While commuting, in transport or walking

It is likely that values will continue to increase as mobile and portable devices receiving mobile network signal spread.

Other locations (where used)[9]

A ‘yes’ response should be probed as it may need to be recoded.

The value of this category should be very low.

Table 12. Micro and macroedits for ICT household statistics (continued)

Indicator

Possible microedits and probes

(edits applied to individual records, preferably at time of interview)

Possible macroedits

(edits applied to aggregated data)

HH9

Proportion of individuals using the Internet, by type of activity

If HH7 is ‘yes’ at least one of the response categories must be selected.

The values of these categories, as a percentage of individuals using the Internet, should add to significantly more than 100 (per cent), as most individuals undertake more than one activity. Note that categories are not mutually exclusive (that is, there is overlap between some categories).

Getting information about goods or services

This is often the second highest output category, after Sending or receiving e-mail.

Seeking health- related information (on injury, disease, nutrition etc.).

Interviewers may need to use examples.

Getting information from government organizations

Interviewers may need to use examples to show which organizations conform to the definition of general government organizations.

There may be data from government organizations indicating the extent of usage of their websites.

Using services related to travel or travel-related accommodation

Interviewers may need to provide examples of major websites in their country.

Downloading software or applications (includes patches and upgrades, either paid or free of charge)

Interviewers may need to provide examples per the definition.

Reading or downloading online newspapers or magazines, electronic books

Interviewers may need to provide examples.

Sending or receiving e-mail

Interviewers may need to explain what e-mail is.

This is likely to be the largest output category.

Table 12. Micro and macroedits for ICT household statistics (continued)

Indicator

Possible microedits and probes

(edits applied to individual records, preferably at time of interview)

Possible macroedits

(edits applied to aggregated data)

HH9
cont.

Making calls (telephoning over the Internet/VoIP using Skype, WhatsApp, Viber, iTalk, etc.; includes video calls via webcam)

Interviewers may need to explain the technicalities involved in this category and provide examples, such as Skype.

It is likely that values will continue to increase over time for countries at earlier stages of Internet use. For other countries, flat trends are expected.

Participating in social networks

Interviewers may need to explain the technicalities involved in this category and provide examples, such as Facebook, Twitter.

Making an appointment with a health practitioner via a website

Interacting with general government organizations

Interviewers may need to use examples to show which organizations conform to the definition of general government organizations

There may be data from government organizations indicating the extent of interactive usage of their websites.

Taking part in consultations or voting via the Internet to define civic or political issues (urban planning, signing a petition etc.)

Accessing or posting opinions via any device on chat sites, blogs, newsgroups or online discussions (e.g. on civic or political issues, general interest topics) that may be created by any individual or organization

Interviewers may need to explain the technicalities involved in this category.

Table 12. Micro and macroedits for ICT household statistics (continued)

Indicator

Possible microedits and probes

(edits applied to individual records, preferably at time of interview)

Possible macroedits

(edits applied to aggregated data)

HH9
cont.

Purchasing or ordering goods or services (purchase orders placed via the Internet whether or not payment was made online; excludes orders that were cancelled or not completed; includes purchasing of products such as music, travel and accommodation via the Internet)

Interviewers may need to provide definitions so that payments are excluded.

Interviewers may need to provide examples of major selling websites such eBay, Mercado libre, Facebook, Amazon, Alibaba, etc.

The size of this category may be related to several factors, including the online security environment and the availability of Internet commerce sites.

The proportion of Internet users having purchased or ordered goods /services should match with that obtained for HH20, HH21 or HH22.

Selling goods or services

Interviewers may need to provide examples of major selling websites such eBay, Mercado libre, Facebook, Amazon, Alibaba, etc.

Internet banking

Interviewers may need to explain the activities per the definition and give examples available (such as M-pesa)

The size of this category should be related to the availability of Internet banking.

Doing an online course (in any subject)

Consulting wikis, online encyclopedias or other websites for formal learning purposes

Interviewers may need to provide examples of major websites such as Wikipedia (there may be national examples as well).

Looking for a job or sending/submitting a job application

Interviewers may need to provide examples.

Participating in professional networks

Interviewers may need to provide examples of major websites in their country (e.g. LinkedIn, Xing, Bark, Opportunity and Jobcase)

Listening to web radio (either paid or free of charge)

Interviewers may need to provide examples of major websites in their country.

Table 12. Micro and macroedits for ICT household statistics (continued)

Indicator

Possible microedits and probes

(edits applied to individual records, preferably at time of interview)

Possible macroedits

(edits applied to aggregated data)

HH9
cont.

Watching web television (either paid or free of charge)

Interviewers may need to provide examples of major websites in their country.

Streaming or downloading images, movies, videos or music; playing or downloading games (either paid or free of charge)

Interviewers may need to provide examples of major websites in their country.

Uploading self/user-created content to a website to be shared

Interviewers may need to provide examples of major social networking websites such as Facebook (there may be national examples as well).

Using storage space on the Internet to save documents, pictures, music, video or other files

Interviewers may need to provide examples of major applications (e.g. Google Drive, Dropbox, Windows Skydrive, iCloud, Amazon Cloud Drive)

This is a new category (introduced in 2013).

Using software run over the Internet for editing text documents, spreadsheets or presentations

HH10

Proportion of individuals using a mobile cellular telephone

The definition of use should be included in question wording and/or probes. It does not equate to a subscriber or owner.

1. Historical trends, would expect medium to high growth.
2. Broad consistency with core indicator A2 (mobile subscriptions/100 inhabitants), absolute values and growth.[10]
3. Check growth with published information from service providers.

Proportion of individuals using a smartphone

As above.

As above.

Table 12. Micro and macroedits for ICT household statistics (continued)

Indicator

Possible microedits and probes

(edits applied to individual records, preferably at time of interview)

Possible macroedits

(edits applied to aggregated data)

HH11

Proportion of households with Internet, by type of service:

If HH6 is ‘yes’ at least one of the response categories must be selected.

1. Historical trends, depending on services available, may see an increase in broadband and mobile broadband via a card or USB modem, as a proportion of households with Internet.

2. The values of these categories, expressed as a percentage of households with Internet access, should add to more than 100 (per cent), allowing for some households having more than one access method.

Fixed narrowband

Fixed broadband

Terrestrial fixed broadband

Satellite broadband

Mobile broadband via a handset

Mobile broadband via a card or USB modem

Interviewers may need to assist by providing specific examples of Internet service providers and products available in the country, in each category used in the questionnaire.

If an interview takes place at household residence, interviewers could ask to see an invoice and check the name of the operator and service, or physically check the type of connection.

Additionally, interviewers could check the type of devices and/or cable plug used for the Internet connection. It should be possible to differentiate:

• fixed Internet connections from mobile Internet connections

• dial-up connections from other fixed connections by the type of modem or by asking/checking whether the telephone line is available for calling while there is an open Internet connection

• mobile Internet connections via a mobile telephone or over USB modem/dongle.

Table 12. Micro and macroedits for ICT household statistics (continued)

Indicator

Possible microedits and probes

(edits applied to individual records, preferably at time of interview)

Possible macroedits

(edits applied to aggregated data)

HH12

Proportion of individuals using the Internet, by frequency

If HH7 is ‘yes’ at least one of the response categories must be selected.

1. Historical trends, may expect slight growth in frequent use and a decrease in infrequent use.
2. The values of the three categories, expressed as a percentage of Internet users, should add to 100 (per cent) with allowances for rounding (up to 2 percentage points).

At least once a day

At least once a week but not every day

Less than once a week

These categories refer to a typical period; therefore, respondents should ignore weekends (if they only use the Internet at work or school etc) and breaks from their usual routine, such as holidays.

Table 12. Micro and macroedits for ICT household statistics (continued)

Indicator

Possible microedits and probes

(edits applied to individual records, preferably at time of interview)

Possible macroedits

(edits applied to aggregated data)

HH13

Proportion of households with multichannel television, by type

There should be no response to this question if HH2 is ‘no’ (that is, no TV).

HH13 (households with any form of multichannel TV) should be less than or equal to HH2.

This is a new indicator. For some or all categories, there may not be historical series on which to compare.

Absolute values for Cable TV and Internet-protocol TV can be compared with the corresponding ITU subscription indicator data for these services because they are paid (therefore, require a subscription).

Cable TV

Direct-to-home satellite services

Internet-protocol TV

Digital terrestrial TV

Interviewers may need to assist by providing specific examples of popular TV programmes that are offered in each TV platform. A mapping of programmes onto TV platforms should allow interviewers to determine what type of TV connection(s) the household has. An interviewer could directly check for availability of given TV channels if respondent is not sure (if interview takes place at household residence).

An interviewer could check for the presence of a satellite dish or a cable connection if respondent is not sure (if interview takes place at household residence).

If the household subscribes to a pay-TV service, interviewers could ask to see an invoice and check the name of the operator and service, thereby identifying the type of TV platform.

If the household does not pay for the TV service, it will most likely receive: analogue terrestrial free-to-air TV (not counted as multichannel TV), digital terrestrial free-to-air TV or free-to-air satellite services. It should be possible to differentiate between each of these TV platforms based on the programmes received.

Table 12. Micro and macroedits for ICT household statistics (continued)

Indicator

Possible microedits and probes

(edits applied to individual records, preferably at time of interview)

Possible macroedits

(edits applied to aggregated data)

HH13
cont.

Since piracy may distort the difference between free and pay TV, in some cases a more practical approach would be just asking about the TV programmes received and then deriving from them the TV platform(s) available.

HH14

Barriers to household Internet access

Note that the population for this question is households without Internet access therefore if HH6 is ‘no’, at least one of the response categories of HH14 should be selected.

The values of these categories, expressed as a percentage of households without Internet access, should add to more than 100 (per cent), allowing for some households having more than one barrier to Internet access.

Do not need the Internet

Have access to the Internet elsewhere

Cost of the equipment is too high

Cost of the service is too high

Privacy or security concerns

Internet service is not available in the area

Internet service is available but it does not correspond to household needs

Cultural reasons

No electricity in the household

Other reason, specify

The reasons may need to be explained per the definitions in the indicator (Table 6).

Table 12. Micro and macroedits for ICT household statistics (continued)

Indicator

Possible microedits and probes

(edits applied to individual records, preferably at time of interview)

Possible macroedits

(edits applied to aggregated data)

HH15

Individuals with ICT skills, by type of skills

For each individual having answered ‘yes’ to use of computer (HH5), at least one of the response categories should be selected.

The values of these categories, expressed as a percentage of individuals having used computers, should add to much more than 100 (per cent), allowing for most individuals having done more than one activity.

Using copy and paste tools to duplicate or move data, information and content in digital environments (e.g. within a document, between devices, on the cloud)

Sending messages (e.g. e-mail, messaging service, SMS) with attached files (e.g. document, picture, video)

Using basic arithmetic formulae in a spreadsheet

Connecting and installing new devices

Finding, downloading, installing and configuring software

Creating electronic presentations with presentation software

Transferring files or applications between a computer and other devices

The activities may need to be explained per the definitions in the indicator (Table 6).However, it is considered probable that if an individual does not understand the meaning of a particular task, then they are unlikely to have undertaken that task.

The tasks are broadly ordered from less complex to more complex, although there is no requirement for a respondent to select simpler tasks before selecting a more complex task.

Table 12. Micro and macroedits for ICT household statistics (continued)

Indicator

Possible microedits and probes

(edits applied to individual records, preferably at time of interview)

Possible macroedits

(edits applied to aggregated data)

HH15
cont.

Setting up effective security measures to protect devices and online accounts

online

Programming or coding in digital environments

Changing privacy settings on your device, account or app to limit the sharing of personal data and information (e.g. name, contact information, photos)

Verifying the reliability of information found

HH16

Household expenditure on ICT

This indicator will usually be derived from a household budget survey. Where household access data are also obtained from the budget survey, comparisons may be possible. For example, if a household does not have a particular ICT, there should be low (or no) expenditure in the corresponding expenditure category.

1. Historical trends, may expect growth in proportion of ICT expenditure over time.

2. ICT expenditure as a proportion of total household expenditure could be compared across overall income or expenditure intervals (e.g. quartile or quintile).

HH17

Proportion of individuals using the Internet, by type of portable device and network used to access the Internet

If HH7 is ‘yes’ at least one of the response categories must be selected

The total number of individuals using the Internet should match with that used for calculating

If the category “Mobile” is “yes” for any type of network, then the answer to question for HH10 should be “yes”.

The values of these categories, expressed as a percentage may add to more than 100 (per cent), allowing for some individuals connecting via more than one device/network.

Table 12. Micro and macroedits for ICT household statistics (continued)

Indicator

Possible microedits and probes

(edits applied to individual records, preferably at time of interview)

Possible macroedits

(edits applied to aggregated data)

HH18

Proportion of individuals who own a mobile phone

Individuals who own a mobile phone (answering “yes”) should also normally be users of such device, therefore answering “yes” to the question for HH10.

HH18 should be normally lower than HH10, as some mobile users can have access to devices provided by other (e.g. the employer).

Proportion of individuals who own a smartphone

As above.

As above.

HH19

Proportion of individuals not using the Internet, by type of reason

At the individual response level, interviewers may probe the coherence of answers regarding the use of Internet (answering “no” to the question on use of Internet, HH7), and whether they perform any of the activities listed under HH9.

HH20

Proportion of individuals who purchased goods or services online, by type of good and service purchased

At the individual level, if any of the categories of HH20 is “yes” then the respondent should be classified as Internet user.

All breakdowns by type of good/service for HH20 should be lower than HH7.

HH21

Proportion of individuals who purchased goods or services online, by type of payment channel

At the individual level, if any of the categories of HH21 is “yes” then the respondent should be classified as Internet user.

All breakdowns by type of payment of HH21 should be lower than HH7.

HH22

Proportion of individuals who purchased goods or services online, by method of delivery

At the individual level, if any of the categories of HH22 is “yes” then the respondent should be classified as Internet user.

All breakdowns by type of payment of HH22 should be lower than HH7.

HH23

Proportion of individuals who did not purchase goods or services online, by type of reason

HH23 should be higher than the complementary of HH7 (100%-HH7)

Imputation for missing data (non-response)

406. Non-response can occur for the whole response – unit non-response, where the respondent refuses, or is unable, to take part in the survey. It can also occur at the primary sampling unit (e.g. district, village) if weather conditions, natural disasters, conflicts or other reasons prevent collection of information from households in that selected sampling area. Non-response can also apply to parts of a questionnaire and this is referred to as item non-response. An example is where a respondent refuses to answer a sensitive question.

407. In many cases, the agency conducting the survey will make estimates for non-response; this is usually referred to as imputation. Both non-response per se and non-response estimates can be a significant source of bias and therefore need to be handled carefully, especially if there is a high rate of non-response.

408. According to the experience in European countries, there are systematic patterns of non-response in older respondents or lower educated respondents as they are more at risk of not understanding the questions (Eurostat, 2013a).

409. Non-response rates can be distributed by type. The most commonly used categories for non-response include: no contact (for example, due to wrong address, temporary or permanent absence, language barrier or health impairment) and refusals. In general, survey organizations will record rates of response for each reason, to understand possible biases and improve operations of future surveys (for example, by launching communication campaigns to request cooperation, improving the address directory or preparing different language versions of the questionnaire).

410. Imputation for unit non-response may occur in household surveys though, commonly, it will be dealt with by substituting other respondents. Imputation, if performed, may take the form of adjusting weights so that responding units have higher weights and non-responding units have a zero weight.[11]

411. Item non-response (that is non-response to parts of a questionnaire) can be treated like unit non-response if a large proportion of the questionnaire is unanswered. Where this is not the case, the missing data may be imputed, that is, they are replaced with information derived from survey data (from the same or other respondents). In ICT household surveys, there are usually not sensitive questions that might provoke item non-response. However, some of the questions are technical and it is useful for interviewers to have access to technical information, such as definitions of particular ICTs, and information on ICT services and websites available in the country.

412. Imputation techniques for item non-response include the use of two large categories of methods: algorithmic procedures and model-based procedures. The former – usually classification and/or prediction methods from the machine learning domain, and used more by advanced NSOs – use an algorithm to produce the results, with an underlying model implied. On the other hand, in model-based procedures the predictive distributions have a formal statistical model, and the assumptions are explicit. The algorithmic techniques more widely applied are deck imputation, nearest neighbour imputation[12] and tree-based methods, while the model-based methods of regression (linear, logistic) are often used. Annex 3 includes examples of imputation for missing items[13].

413. It should be noted that imputation is not guaranteed to remove a significant potential problem with non-response, which is non-response bias – non-respondents cannot be assumed to be similar to respondents in their questionnaire responses. Even though it is not possible to completely eliminate non-response, there are many actions that can help to reduce it. For instance, interviewer selection, good interviewer training and survey material (including initial contact letters or telephone calls, questionnaires and publicity material, if used), workload assigned to each interviewer and supervision. Furthermore, when respondents are not available during an initial interview, they should be re-contacted, possibly by telephone in the case of a face-to-face interview methodology.[14]

414. The analysis of non-response, its treatment and impact should be reported as part of the quality assessment of the survey implementation and results. The quality assessment in relation to non-response may include:

• The presentation of unit non-response rates by types of non-response, and for different population segments (urban/rural, language groups, etc.).

• The presentation of item non-response rates for a selected number of questionnaire items. This may be accompanied by a statistical analysis of the distribution of item non-response for different population segments.

• For more detailed analysis, the organization may analyse the relationship between non-response and the conditions under which the survey was carried out, such as time and day of the interview, interviewer, place of the interview or number of attempts to contact the interviewee. These parameters are usually recorded and referred to as survey paradata.

• The calculation of different measures of bias and impact.[15]

Weighting of data

415. Data from a sample are weighted to represent the population. The initial design weight of a unit in a particular stratum is the inverse of its probability of selection. For example, if a stratum has a population of 100 and 20 units are sampled randomly with equal probability, then the probability of selection is 20/100 and the design weight is 100/20 (that is, 5). We saw in the last chapter that most household surveys have more than one stage of sampling. The design weights will incorporate the probability of selection at each stage, that is, weights are given to primary sampling units and to each household in them. If individuals are sampled within the household, they should also be given a weight. A simple example of data weighting is presented in Annex 3.

416. Design weights will often need to be adjusted to reflect non-response, unknown eligibility, out-of-scope units and/or frame problems, such as duplicate records and undercoverage.[16] It is also important to weight responses according to independent estimated distributions of the population. This form of weighting compensates for non-representativeness of the effective sample (that is, the population of respondents to the survey). It may be referred to as ‘post-stratification’ or ‘benchmarking’. Box 46 provides an example of post-stratification.

Box 46. Australia: example of post-stratification

The Australian multipurpose household survey (MPHS) for 2016–17 included a number of ICT household questions. The first step in calculating weights for each unit was to assign an initial weight, which is the inverse of the probability of being selected in the survey. For example, if the probability of a person being selected in the survey was 1 in 600, then the person would have an initial weight of 600 (i.e. they represent 600 people).The initial weights were then calibrated to align with independent estimates of the population of interest, referred to as ‘benchmarks’, in designated categories of age by sex by area of usual residence. Weights calibrated against population benchmarks ensure that the survey estimates conform to the independently estimated distribution of the population rather than the distribution within the sample itself. Calibration to population benchmarks helps to compensate for over or under-enumeration of particular categories of persons/households which may occur due to either the random nature of sampling or non-response.

For household estimates, the MPHS was benchmarked to independently calculated estimates of the total number of households in Australia. The MPHS estimates do not (and are not intended to) match estimates for the total Australian person/household populations obtained from other sources.

For person estimates, the survey was benchmarked to the Estimated Resident Population (ERP) in each state or territory at December 2016.

Survey estimates of counts of persons or households are obtained by summing the weights of persons or households with the characteristic of interest.

Source: http://www.abs.gov.au/AUSSTATS/abs@.nsf/Lookup/8146.0Explanatory%20Notes12016-17?OpenDocument

417. An additional tier of weighting, ‘trimming of weights’, consists of reducing the size of very large weights to reduce their contribution to the magnitude of estimates and their variance. However, its impact on bias should also be taken into account.[17]

418. For more information on weighting, readers are referred to UNSD (2005b, Chapter 6), which presents a comprehensive technical discussion of the subject.

Calculating and reporting ICT household indicators

419. Calculation of the ICT household indicators, while not complex, needs to be clear so will be described in some detail.

420. Indicators arising from ICT use surveys will generally be presented as proportions.[18] They include proportions of the whole population of households/individuals or of subpopulations, such as particular household types or age groups. Additionally, countries may present data as a proportion of households/individuals that have access to, or use, computers or the Internet, etc. Having two methods of calculation for some indicators can be potentially confusing to users, so it is important to be clear which denominator is used to construct a particular indicator and to have a common approach for reporting purposes. Formulae for the calculation of the core indicators are provided in Chapter 4.

421. Using statistical software for data processing facilitates the processes of production of ICT indicators. Increasingly, NSOs are adopting statistical software which is provided by other statistical offices (such as the above-mentioned tools for data entry, CSPro and Blaise), and free software such as R, which is currently the most widely used one in the academic statistical community. As more NSOs are starting to use R, the possibility of sharing libraries (i.e. computer-coded functions to perform tasks which are undertaken when statistics are produced by NSOs) can increase their productivity. Box 47 presents the use of R in Brazil for the production of ICT indicators.

Box 47. Using R to tabulate ICT survey data in Brazil

The Regional Centre for Studies on the Development of the Information Society (Cetic.br) produces indicators and statistics on the use of information and communication technologies in Brazil. One of its practices is to use tools based on the R software package to automate data processing and tabulation, generating outputs, such as sampling error and estimates.

In 2014, Cetic.br started the development of a tool based on R that can tabulate data from five comma-separated value (CSV) files, each containing some survey information: indicators, questions, question labels, desired cross-tabulations, and the database of encoded answers to the survey. Based on these five files, it is possible to tabulate any question and perform cross-tabulation analyses based on the variables available in the survey. The result is a set of tables: proportions, totals, error margins for proportions and totals, and coefficients of variation.

The tool has been used since 2015 and continuously improved ever since. It allows data to be tabulated in a matter of minutes, greatly accelerating and facilitating annual comparisons. Cetic.br is currently developing an R package to allow other organizations and researchers to use this tool, which will only require the creation of the CSV files for the desired survey.

Source: Cetic.br

422. When reporting data to ITU, countries should provide estimates of the total numbers of units (rather than proportions or percentages) with a particular ‘ICT characteristic’, for example, the estimated number of adult Internet users or the estimated number of adult male computer users. In addition, the estimated total number of units in the subpopulations (in this example, all in-scope adults and all in-scope adult males) also need to be provided so that proportions can be calculated. All data should be weighted estimates of the population rather than the number of units in the sample. An example of data reporting is provided in Table 13.

423. It can be seen that the presentation in Table 13 provides the data user with maximum flexibility. Examples of calculations that can be made include:

• the proportion of individuals using a computer,

• the proportion of males and females using the Internet,

• the proportion of individuals using the Internet at home,

• the proportion of female Internet users using the Internet at work,

• the proportion of Internet users aged 15 years or over using the Internet at another person’s home, and

• the proportion of young people aged 5–24 using computers.[19]

Table 13. Example of data reporting: partial table

Variable

Sex

Age

Male

Female

5–14

15–24

25+

Total estimated population (total in-scope population, not sample number), in thousands

1,214

1,121

370

507

1,458

HH5

Estimated number of individuals who used a computer (from any location) in the last three months, in thousands

595

605

183

403

614

HH7

Estimated number of individuals who used the Internet (from any location) in the last three months, in thousands

402

439

122

297

422

HH8

Estimated number of individuals who used the Internet at home in the last three months, in thousands

206

217

81

150

192

HH8

Estimated number of individuals who used the Internet at work in the last three months, in thousands

189

152

5

147

189

HH8

Estimated number of individuals who used the Internet at their place of education in the last three months, in thousands

130

134

107

119

38

HH8

Estimated number of individuals who used the Internet at another person’s home in the last three months, in thousands

53

68

46

38

37

424. Care should be taken when aggregating response categories into those required for international comparability. Some countries may construct indicators for the core indicators HH8, HH9, HH11, HH12 and HH13 based on more detailed response categories. For example, in the ‘location of use’ question in a country survey, the response category ‘community Internet access facility’ could be comprised of the subcategories, public libraries, digital community centres and other government agencies.

425. In this example, the proportion of Internet users using the Internet at community Internet access facilities is calculated by deriving the number of users who used the Internet at one or more of the locations, public libraries, digital community centres or other government agencies. Clearly, this aggregation has to be done at the unit record level rather than from aggregated data and gives a different answer than if the percentages or numbers accessing the Internet at each of the component locations are summed (this is because those individuals who use the Internet at more than one of these locations will be counted more than once).

426. More information on ITU’s data collection and dissemination for ICT household statistics can be found in Chapter 10 and Annex 4 (which shows an extract from the questionnaire ITU uses to collect ICT household statistics).

[1] Ideally while the interviewing team is still working in the area.

[2] A number or letter in a keyed sequence, the value of which is derived from a function involving the other digits in the sequence. If a data entry error is made, the derived check digit will differ from the actual check digit, thus signaling that a keying error has been made. Check digits are typically used for recording identifiers and codes rather than quantitative data.

[3] https://census.gov/data/software/cspro.html

[4] https://www.cbs.nl/en-gb/our-services/blaise-software

[5] A comparative survey of CAPI applications can be read at: http://siteresources.worldbank.org/INTSURAGRI/Resources/7420178-1294259038276/CAPI.Software.Assessment.Main.Report.pdf

[6] Adapted from UNSD (2005a, Chapter XV).

[7] However, this apparent advantage of CAPI/CATI systems can also make some errors undetectable: if the interviewer enters 2 instead of 22 as a person’s age, the program will obediently – and wrongly – omit the questions on ICT use, which should be asked of adults but not of very young children.

[8] Such records would be ‘flagged’ with an edit status indicating the existence of fatal errors. It is a simple matter to omit records with such a flag.

[9] Other locations is not a category in core indicator HH8. However, it can be useful to include ‘Other’ categories in questionnaires. Where the set of response categories (excluding ‘Other’) is considered to be comprehensive, then ‘Other’ responses should be queried and the response recoded if appropriate.

[10] The relationship between the indicators HH10 and A2 may be complex.

[11] Information on this and other imputation techniques for unit non-response can be found in UNSD (2005a, Chapter VIII).

[12] These are described in some detail in UNSD (2005a, Chapters VIII and XVI).

[13] A detailed comparison of methods used in official statistics can be read in Rey, P. (2012) “USE OF MACHINE LEARNING METHODS TO IMPUTE CATEGORICAL DATA “ at https://www.unece.org/fileadmin/DAM/stats/documents/ece/ces/ge.44/2012/37_Eurostat.pdf

[14] UNSD (2005a, Chapter VIII) recommends the use of more skilled interviewers to undertake such callback work.

[15] The description of advanced methods of treatment of non-response and analysis of its effect is outside the scope of this Manual. For a useful reference, consider de Leeuw et al. (2003), Journal of Official Statistics, Vol. 19 No. 2, 2003, pp 153-176.

[16] See UNSD (2005b, Chapter 6) for adjustment of design weights.

[17] UNSD (2005b, Chapter 6).

[18] In particular, all the core ICT household indicators are proportions data.

[19] The last two calculations involve aggregation of categories. This would not be possible if only percentages (expressed as the proportion within each age group) were provided.

Chapter 9. Data quality and evaluation for ICT household statistics

427. An underlying principle, applicable to all aspects of survey design and implementation, is that it is important to understand sources of error so that it can be minimized. This chapter outlines survey data quality issues, in particular, those specific to household ICT access and use statistics.

Statistical quality

428. Quality may be defined as “the degree to which a set of characteristics fulfils requirements” using the much-cited ISO standard 9000 (2005). In the case of official statistics, most countries are adopting the framework for statistical quality of the European Statistical System, which uses a multidimensional definition, based on the following characteristics: relevance, accuracy and reliability, timeliness and punctuality, coherence and comparability, accessibility and clarity.

429. Accuracy is generally considered to be a key measure of quality. Total survey error is a conceptual framework, describing errors that can occur in a sample survey and the error properties. It may be used as a tool in the design of the survey, working with accuracy, other quality characteristics, and costs. Error sources are considered one-by-one to estimate the uncertainty and also to obtain some indication of the importance of that source. The errors arise from: sampling, frame coverage, measurement, non-response, data processing, and model assumptions.

430. This chapter examines more in detail both sampling and non-sampling error, and provides information on quality assurance techniques and frameworks. It concludes with some general advice on data evaluation.

431. In general, countries should aim to reduce survey error as much as possible by:

• using well-designed samples that are of sufficient size to produce reliable data (that is having low standard errors for the required aggregates);

• careful design and testing of questions and question sequences;

• intensive training and supervision of interviewers and other staff;

• reducing non-response rates as far as possible;

• integrating computer-based quality controls to fieldwork; and

• minimizing data entry, editing and other processing errors.

432. In practice, there will often be trade-offs between sampling and non-sampling error. UNSD[1] cites the example of having smaller sample sizes and using more skilled interviewers. The latter allows for better management, which would be expected to reduce non-sampling error, including non-response bias. However, for a given sample design, this strategy will result in higher sampling error, as sampling error is related to sample size.

Sampling error

433. Sampling error is the component of survey error that occurs because only a fraction of the total population is sampled. It is said to measure the precision of an estimate. The sampling error of an estimate can be expressed in three ways, all of them a function of the standard error around the estimate,[2] noting that the quoted standard error of a survey estimate is itself an estimate based on observations from the sample.

434. Sampling error may be expressed in terms of the standard error of the estimate of interest, but more commonly it is expressed as a ratio of the standard error of the estimate to the value of the estimate and converted to a percentage (for instance, as 2 per cent of the value of the estimate). This provides scale in relation to the estimate and enables simple comparison of the sampling error around different estimates (see Box 48 for an illustration). The ratio is referred to as the ‘relative standard error’ (RSE) or ‘coefficient of variation’ (CV).

435. Sampling error can also be expressed as a confidence interval around an estimate, commonly, a 95 per cent confidence interval. This indicates that if the exercise were to be repeated 20 times with random sampling, then we would expect the estimate to fall within that interval on 19 of those times.

Box 48. Sampling error of the value of an ICT household indicator

If the estimated proportion of individuals who used the Internet in country A in the last year is 0.83 and the standard error of the estimate is 0.01, then the value for core indicator HH7 is 83 per cent and the relative standard error is 100*0.01/0.83 = 1.2 per cent of the estimate. If the estimated proportion of people aged 55–64 who used the Internet in country A in the last year is 0.49 and the SE of the estimate is 0.03, then the value for core indicator HH7 (individuals aged 55–64) is 49 per cent and the RSE is 100*0.03/0.49 = 6.1 per cent of the estimate. It can be seen that the reliability of the two estimates can be directly compared by using the value of the respective RSEs.

With a probability of 95 per cent, the value of the parameter to be estimated will lie within the 95 per cent confidence interval. The interval can be expressed as: the sample estimate +/– twice its standard error. In the first example, the 95 per cent confidence interval around the estimate is 0.83 +/– 0.02 (twice the SE). Therefore, with a confidence level of 95 per cent, the value of the parameter to be estimated (in percentage terms) will lie in the interval 83–2 to 83+2, that is, between 81 and 85 per cent. The 95 per cent confidence interval in the second example is 0.49 +/– 0.06, that is 43 to 55 per cent.

436. The core indicators for household ICT access and use are all proportions, for instance, the proportion of individuals who used the Internet in the last three months. Proportions may be of the total population (for instance, the proportion of households with Internet access) or a subpopulation, such as the proportion of households with Internet access that use broadband to access the Internet at home. In the first case, the estimate is of a proportion. In the second, it is a ratio calculated as an estimate of the number of households with broadband access to the estimate of the number with Internet access.

437. In the case of a simple random sample without replacement (which is rarely the case for household surveys), the standard error (SE) of a sample proportion, , of the population (the first example above) is estimated by:

(1) ,

where is the sample estimate of the true proportion, P.

438. Under the same sampling design, the relative standard error (or ‘coefficient of variation’) of is . Examples of the use of standard errors to show RSEs and confidence intervals of simple proportions are provided in Box 48.

439. For a complex proportion, both the numerator x and denominator y are estimated separately from the survey, forming a ratio estimate .

440. There is no unbiased estimate with a closed form for the RSE of a ratio, and generally approximations are used. As an example, the Australian Bureau of Statistics (ABS, 2007), used the following approximation to calculate the RSE of a ratio estimate as a function of the RSE of x and the RSE of y:

(2) .

441. ABS applied the formula to the estimate of the proportion of households with broadband access, where x was the estimate of the number of households with broadband access and y was the estimate of the number of households with Internet access.

442. Of particular interest for many statistics, including ICT statistics, is the calculation of the SE of an estimate of change over time (for instance, the number of computer users in 2000 compared with the number in 2007). If the samples on which the two estimates are based are independent, then the SE of the difference over time, xt+1 – xt, is as follows:

(3) .

443. There is a relationship between the SE and the sample size (n), with the SE decreasing as n increases (SE is inversely proportional to √n). Commonly, survey designers will determine in advance (and preferably in accordance with users’ needs) the required reliability for major aggregates (e.g. the estimated percentage of households with Internet access should have a RSE of no more than 2 per cent) and then use estimates of the SEs around those aggregates to determine sample size.[3] Other determinants of the size of the SE are the population size, the sampling method and the inherent variability in the population of the variable being estimated.

444. Determination of the SE of an estimate becomes much more complicated with a complex survey design, for example, a design involving stratification and two sampling stages. Consideration must be given to the degree of homogeneity of units within a cluster and other design effects.[4] In this case, SEs are usually calculated as approximations, using specialized software.[5]

445. It is beyond the scope of this Manual to do more than introduce the subject of sample design and sampling error. Readers are referred to UNSD (2005b), which deals primarily with the design of household surveys and UNSD (2005a), which covers all aspects of household sample survey design and implementation in respect of developing and transition economies. UNSD[6] also provides an overview of analysis software packages that may be used to estimate sampling error. A more complete review may be found on the Harvard’s Summary of Survey Analysis Software webpage.[7]

Non-sampling error

446. Non-sampling error (also referred to as ‘bias’) refers to errors in output that are not due to sampling. Non-sampling error (NSE) has diverse origins. The sources and prevention of NSE have been discussed throughout this Manual. NSEs include those related to:

• frame inadequacies, including undercoverage, duplicates and incorrect data;

• inappropriate or misunderstood question wording;

• poor quality or inconsistent interview performance;

• non-response;

• respondent effects (for instance, proxy reporting); and

• problems in data processing and tabulation.

447. While non-sampling error is controllable in theory, in practice some NSE will often persist and it is important to try to describe its extent and nature to data users, even though the size (and sometimes even the direction) of the error is usually difficult to measure.

448. As we saw in Chapter 3, careful planning and testing should reduce NSE. In particular, attention to interviewer training and questionnaire design, will be well rewarded. Furthermore, human supervision, as mentioned in Chapter 3, is crucial for reducing NSE.

449. More information on NSE can be found in UNSD,[8] which discusses ‘non-observation’ errors. These are errors arising from non-response or non-coverage (which is a frame problem, referred to as undercoverage in this Manual). UNSD[9] also discusses NSE arising from the questionnaire, method of data collection, the interviewer and the respondent. It provides some techniques that will assist in understanding bias from these sources and broadly quantifying it.

450. Based on the experiences of countries in the collection of ICT data, the level of response to questions referring to specific technologies (e.g. type of device or connection) may vary, as some may appear more difficult to respond to.

Assessment of data quality

451. Within the constraints of survey resources, a high level of data quality should be the goal of every survey statistician.[10] Good statistical agencies have a strong data quality culture that not only enables them to produce high quality data, but also lends credibility to those data. Data quality considerations must be paramount at each stage of the survey process.

452. Data quality can be considered in terms of several dimensions or criteria (e.g. relevance, accuracy,[11] timeliness and punctuality, accessibility and clarity, comparability, and coherence). It is recommended that countries use at least these dimensions to ensure the quality of ICT household statistics. More specific guidance may be found from the experiences of those statistical organizations that have quality assurance (QA) frameworks based on these dimensions. They include Eurostat’s 2014 ESS Handbook for Quality Reports[12] and the IMF’s Data Quality Assessment Framework.[13] The World Bank, in the context of the International Household Survey Network (IHSN), has developed a survey quality assurance framework (SQAF) adapted to the specific conditions of developing economies.[14]

453. Detailed data quality reports are typically produced for internal approval processes and should address the quality dimensions referred to above. They will also include detailed information about the survey’s results including the results of macro (or output) editing (see Chapter 8) and an explanation of divergences from expected findings.[15] Table 19 in Chapter 10 presents metadata associated with the core ICT indicators. The metadata include topics relating to data quality.

Evaluation

454. An element of both data quality and evaluation is careful documentation of survey processes and procedures, corresponding to the GSBPM phase “Evaluation”. These may include a priori descriptions of survey plans, including costs, methodologies and procedures. They should also include documentation of costs and procedures as they occur during survey implementation. An example of the latter is the importance of documenting any changes in sample design during the field phase, reflecting changes or adaptations.[16] This is necessary for later adjustments and is particularly useful for designing future surveys.

455. After the survey is completed, further documentation should describe the collection and present survey data and metadata. A subset of such information should be disseminated as part of the survey’s output. This is addressed in the next chapter.

456. Staff contributions can also be important to survey evaluation. The experiences of all staff, including interviewers, supervisors and data entry operators, can be very useful input to evaluation of a survey and should be obtained shortly after completion of the survey.[17]

457. The references on quality assessment mentioned above, as well as those on planning mentioned in Chapter 3 can be used to prepare evaluation reports. In particular, the Eurostat suite of documents on quality management[18] include tools for documenting the quality of statistical output as well as of processes and can be applied to all types of surveys, including ICT household surveys.

[1] UNSD (2005a, Chapter II).

[2] The standard error of an estimate is the square root of the estimate’s variance.

[3] Such estimates might come from previous data. Where the survey is run for the first time, other information could be used, for instance, data on sampling error from similar countries.

[4] Described by UNSD (2005a) as follows: “The design effect represents the factor by which the variance of an estimate based on a simple random sample of the same size must be multiplied to take account of the complexities of the actual sample design due to stratification, clustering and weighting.” The square root of the design effect (which UNSD calls the ‘design factor’) is the multiplier applied to a standard error. The design effect is specific to each estimate in a given survey and is generally greater than one.

[5] See: UNSD (2005a, Chapter XXI) for details. A free R software library is designed for calculation of sampling errors for complex surveys. See: https://cran.r-project.org/web/packages/sampling/sampling.pdf

[6] UNSD (2005b, Chapter XXI).

[7] See: http://www.hcp.med.harvard.edu/statistics/survey-soft/

[8] UNSD (2005a, Chapter VIII).

[9] UNSD (2005a, Chapter IX).

[10] Data quality, like everything else in a system of limited resources, is not boundless. For instance, sample sizes may be smaller than ideal or interviewers may be relatively inexperienced. A good survey manager will maximize the quality of the output and may need to suppress data which he/she considers to be unreliable.

[11] The word “accuracy” is used by UNSD (2005b) to refer only to non-sampling error. The Manual uses the term to refer to both sampling and non-sampling error. This is consistent with usage of Eurostat and the ABS. See the Glossary of terms and abbreviations for more information.

[12] See: https://ec.europa.eu/eurostat/documents/3859598/6651706/KS-GQ-15-003-EN-N.pdf

[13] Even though the IMF primarily deals with economic statistics, a DQAF module on household income in a poverty context has been developed in collaboration with the World Bank. See: https://dsbb.imf.org/dqrs/DQAF/

[14] See: https://ihsn.org/projects/survey-quality-assessment-framework-SQAF

[15] UNSD (2005a, Chapter X) discusses quality assurance in some detail. It suggests the need for quality standards, QA procedures and evaluation of QA. It describes the World Health Survey quality standards and assurance procedures devised by the World Health Organization. The standards are presented in stages, representing the stages of a survey cycle and include useful checklists of QA procedures and issues.

[16] UNSD (2005b, Chapter 5).

[17] See UNSD (2005a, Chapter IV). Their input should be encouraged at other stages as well, given their specialized knowledge of the operations of the survey.

[18] See: https://ec.europa.eu/eurostat/web/quality/quality-reporting

Chapter 10. Dissemination of ICT household data and metadata

458. This chapter deals with dissemination of ICT household statistical data and metadata. It includes example tabulations and visualizations of core indicator data for household ICT access and individual ICT use.

459. The chapter also considers the international data collection and dissemination work carried out by ITU. An extract from the questionnaire used by ITU to collect the core ICT household data from countries is presented in Annex 4.

Data dissemination

460. Data may be released in different formats,[1] including:

• hardcopy publications;

• electronic publications (for instance pdf files of hardcopy publications, web publications in html form);

• tables on websites, spreadsheets;

• visualization of statistical results and

• microdata files, containing individual confidentialized records from surveys.

461. Data may be free or charged, or a combination. For instance, pdf files might be free to download but printed versions may be charged, reflecting their cost of production. Hardcopy releases may be provided free to some users, for instance, government departments, but provided at cost to other users. Some agencies provide some free data on their website, but may charge for more detailed data.

462. Some countries may provide microdata from household surveys. These are data at unit record level that have been confidentialized. Such information can be very useful for detailed analysis. For a comprehensive set of tools for disseminating and documenting microdata, see the Accelerated Data Program,[2] an initiative in the frame of the International Household Survey Network.

463. All released data should be based on a single approved version of the dataset, unless revisions are later made (in which case, all released data are subject to revision).

464. Where data are deemed unreliable (usually because of high standard errors), they should be suppressed and that should be indicated by a symbol in the cell, for instance, n.a. (for ‘not available’).

465. Where categories are combined for purposes of national reporting, the advice on aggregating response categories should be followed (see Chapter 8).[3]

Box 49. Use of disseminated microdata on ICT e-commerce in Spain

The National Statistical Institute of Spain (INE) disseminates microdata of the “Use of ICT in households” annual survey for researchers and the general public, in a modern data dissemination format. This encourages more advanced use of the data. The raw data are available along with a variety of documents related to the design and implementation of the survey. It is a rotating survey, which includes 15,000–20,000 dwellings each year. The same dwelling is interviewed for a maximum of four (consecutive) years, and around 30% of the dwellings are replaced every year. 60% of the interviews are conducted by phone (CATI) and 40% in person (CAPI).

By using anonymized microdata of several surveys from 2008 to 2016 and the dwelling identifier, it is possible to create panels of individuals given that the same household is interviewed for a maximum of four consecutive years. Using the sociodemographic information available for each member of a dwelling, filters can be applied (like gender and date of birth) to identify whether the respondent for a particular dwelling was always the same individual or not. Issues like determinants of the adoption of digital services are possible to be investigated both with pooled data and with panel data models.

One example of a model derived from this data is logistic regression, using a binary response variable for e-commerce (yes/no), and explanatory variables such as gender, age, habitat, nationality, education, employment situation and digital skills, as well as lagged values of e-banking and e-government activities. The model allows exploring the significance of each explanatory variable in the distribution of the response variable, thus providing insights on the causes of digital divide in the practice of e-commerce in Spain.

Source: Pérez-Amaral, T. et al. E-commerce by individuals in Spain using panel data 2008–2016. Telecommunications Policy,

https://doi.org/10.1016/j.telpol.2019.101888

Tabulation plans for ICT indicators

466. We saw in Chapter 8 how the ICT household indicators should be calculated and how they should be provided to collection agencies for international compilation purposes.

467. The situation is different for national presentation purposes. In a national context, ICT household data are most likely to be presented as percentages, rather than numbers of individuals or households. For indicators with multiple response categories, it may be more relevant to present data using the population that was asked the question as the denominator, for example, in an indicator referring to location of Internet use, the proportion could be calculated using as denominator the number of individuals using the Internet rather than the whole population of individuals. In addition, national presentation may be more (or less) detailed and could include additional classificatory variables of national interest (such as an urban/rural split or by administrative divisions of the country).

468. Tables 14 and 15 below are examples of how ICT household core indicator access data might be tabulated for dissemination purposes. Table 16 presents an example of tabulation for individual ‘whole population’ ICT use indicators. Table 17 presents a tabulation for the location of Internet use. Table 18 presents an example tabulation for ICT skills of individuals. The examples include all the classificatory variables recommended for use with the core indicators.

Table 14. Example tabulation for household ICT access core indicators

Household characteristics

Households with:

Radio

TVi

Telephone

Com-
puter

Internet access at home

any

fixed
only

mobile
only

both fixed and mobile

any

fixed
narrow-
band

sate-
llite
broad-
band

fixed broad-
band

mobile
broad-
band

Percentage of households

Household composition

has children under 15

does not have children under 15

Household size (number of members)

1

2

3–5

6–10

more than 10

Household incomeii

Province or stateiii

Zone

Urban

Rural

All households

i. Countries able to collect data for HH13 (Households with multichannel TV) could split this into the following categories: any TV (HH2), cable TV, direct-to-home (DTH) satellite services, Internet-protocol TV (IPTV) and digital terrestrial TV (DTT).

ii. Household income. While this is not a recommended core indicator classificatory variable, it can be a very useful household characteristic. No categories or presentation are suggested, although a quartile presentation might be the simplest.

iii. Province or state, and Zone. These are not recommended core indicator classificatory variables but can also be very useful household characteristics for policy purposes, especially where information is required in respect of different areas of a country, including information about any rural/urban divide.

Table 15. Example tabulation for barriers to household Internet access

Household characteristics

Households with barriers to Internet access:

do not need Inter
net

have access else-where

lack confid-
ence, skills

high equip-ment cost

high service cost

privacy or secu-
rity con-
cerns

Internet service not avail-
able

service
avail-
able but not sui-
table

Cul-
tural
rea-
sons

Percentage of households

Household composition

has children under 15

does not have children under 15

Household size (number of members)

1

2

3–5

6–10

more than 10

Household income

Province or state

Zone

Urban

Rural

All households

Table 16. Example tabulation for ‘whole population’ individual ICT use indicators[4]

Individual characteristics^5[5]

Individuals who used (from any location, last three months):

a computer

the Internet

a mobile telephone

Percentage of individuals

Age

under 5

5–9

10–14

15–24

25–34

35–44

45–54

55–64

65–74

75 and over

Sex

male

female

Highest education level[6]

primary education or lower

lower secondary education

upper secondary or post-secondary non-tertiary

tertiary or post-tertiary education

Labour force status[7]

employees

self-employed

workers not classifiable by status

Table 16. Example tabulation for ‘whole population’ individual ICT use indicators (continued)

Individual characteristics

Individuals who used (from any location, last three months):

a computer

the Internet

a mobile telephone

Percentage of individuals

unemployed

outside the labour force

Occupation[8]

legislators, senior officials and managers

professionals

technicians and associate professionals

clerks

service workers and shop and market sales workers

skilled agricultural and fishery workers

craft and related trades workers

plant and machine operators and assemblers

elementary occupations

armed forces[9]

not applicable (outside the labour force)

Languages read/spoken[10]

Disability status[11]

All individuals

Table 17. Example tabulation for the location of Internet use

Individual
characteristicsi

Location of Internet use (in the last three months):

home

work

place of education

another person’s home

at facility open to the public

community Internet access facility

while commuting, in transport or walking

Percentage of Internet users

Age

under 5

5–9

10–14

15–24

25–34

35–44

45–54

55–64

65–74

75 and over

Sex

male

female

Highest education level

primary education or lower

lower secondary education

upper secondary or post-secondary non-tertiary

tertiary or post-tertiary education

Labour force status

employees

self-employed

workers not classifiable by status

unemployed

outside the labour force

Occupation

legislators, senior officials and managers

professionals

technicians and associate professionals

clerks

service workers and shop and market sales workers

skilled agricultural and fishery workers

craft and related trades workers

plant and machine operators and assemblers

elementary occupations

armed forces

not applicable (outside the labour force)

All individuals

i. For notes on the classifications, including other possible classifications, see Chapter 4.

Table 18. Example tabulation for ICT skills of individuals

Individual characteristics

using copy and paste tools

sending e-mails with attached files

using basic arithmetic formulae in spread-sheet

connect-ing, installing new devices

finding, down-
loading, installing, configu-
ring software

creating electronic presen-
tations with presen-
tation software

Transfer-
ring files between a computer and other devices

setting up effective security measures to protect devices and online accounts

changing privacy settings on your device, account or app to limit the sharing of personal data and infor-
mation

verifying the reliability of infor-
mation found online

Program-
ming or coding in digital environ-
ments

Age

under 5

5–9

10–14

15–24

25–34

35–44

45–54

55–64

65–74

75 and over

Sex

Male

Female

Highest education level

primary education or lower

lower secondary education

upper secondary or post-secondary non-tertiary

tertiary or post-tertiary education

Labour force status

Employees

self-employed

workers not classifiable by status

Unemployed

outside the labour force

Occupation

legislators, senior officials and managers

Professionals

technicians and associate professionals

Clerks

service workers and shop and market sales workers

skilled agricultural and fishery workers

craft and related trades workers

plant and machine operators and assemblers

elementary occupations

armed forces

not applicable (outside the labour force)

All individuals

Data visualization

469. Data visualization aims to aid users in exploring, understanding, and analysing data through visual exploration. With the development of user-friendly and powerful IT tools for data visualization, data visualization is spreading in a variety of applications, including scientific applications, etc. Although official statistics is not an exception to this trend, data visualization has not already developed its full potentiality in this domain. Many developing NSOs still disseminate data only in the format of numerical tables, thus limiting the comprehension of the results to less “statistically literate” audience.

470. Good practices in official statistical visualization are available from international organizations (such as the UNECE suite on “Making Data Meaningful”). [12] ITU’s publication “Facts and Figures” provides examples for visualization of ICT indicators which can be adapted by countries (see Figure 5), including maps, simple graphs and infographics.

Figure 5. Example of data visualization

Source: ITU Facts and Figures 2019, https://www.itu.int/en/ITU-D/Statistics/Documents/facts/FactsFigures2019.pdf

Metadata reporting and dissemination

471. It is important that countries include metadata about the survey as a whole and individual observations in published statistical output. This includes information on data quality, which may be considered in terms of several dimensions or criteria (e.g. relevance, accuracy, timeliness and punctuality, accessibility and clarity, comparability, and coherence). A number of statistical organizations have quality assurance (QA) frameworks based on these dimensions; these were discussed in the previous chapter. Eurostat provides guidance to member country statisticians on how to report data quality according to Eurostat’s quality criteria.[13] The reports for Eurostat’s information society statistics consist of web pages containing metadata on the community business and household surveys of ICT access and use.[14]

472. Limitations of survey data should be emphasized and would cover the likely existence of non-sampling error (with steps taken to reduce it) and, where relevant, the fact that output data were based on a sample of units. Any time series incompatibility should also be described, for instance, that results should not be compared with those of earlier surveys because of differences in scope or methodology.

473. Standard errors for at least the major aggregates from a survey should be disseminated to data users as they provide a vital indication of the reliability of the estimates. Where relative standard errors are high, users should be alerted to this fact. UNSD suggests that a RSE of 7.5 per cent of the estimate is the maximum that should be allowable for major aggregates.[15] Where RSEs are higher than this level, consideration should be given to suppressing the affected data because of its poor reliability and potential to mislead rather than inform. Some countries flag data with high standard error in statistical tables to inform users of the accuracy of figures.

474. Metadata can relate to a whole survey or to particular survey results (e.g. a particular indicator).[16] Table 19 recommends the broad classes of metadata that should be disseminated with ICT survey data and provides examples.

Data collection and dissemination of ICT statistics by ITU

475. ITU collects telecommunication data annually for over 200 economies worldwide, with some data series going back as far as 1960. These series traditionally refer to telecommunication and ICT infrastructure data, such us fixed-telephone subscriptions, mobile-cellular subscriptions and Internet subscriptions. Some of them are included in the Partnership’s core list of indicators (namely, the ICT infrastructure and access indicators, see Annex 1 for details). The main sources for these data are national regulatory authorities and sector ministries in charge of telecommunication/ICT that collect administrative data directly from operators and service providers. Data are collected using an online questionnaire available at the ITU ICT Eye[17] website.

476. ITU has expanded its statistical work since 2003 by collecting official household and individual ICT data from national statistical offices. From 2005, ITU has sent an annual questionnaire to all NSOs, requesting data on the core indicators on access to, and use of, ICT by households and individuals.

477. Calculation and reporting of data to ITU for international comparability purposes was described in Chapter 8. As discussed, the preferred approach is for countries to provide data on numbers of units (households, individuals) with a particular characteristic rather than proportions or percentages. Numbers for total populations and subpopulations also need to be provided to enable calculation of proportions for various population groups. The point was also made that data on the number of units with a particular ‘ICT characteristic’ (e.g. the number of adult Internet users) and number of units in the population of reference (e.g. the number of in-scope adults) should represent weighted estimates rather than the number of units in the sample.

478. ITU statistics are disseminated in various ways, including publications, and electronic download. In particular, data are disseminated through the World Telecommunication/ICT Indicators (WTI) Database, which is available by electronic download. The WTI Database includes annual time series for over 140 indicators. ITU also produces a number of global and regional reports to analyse regional market trends and developments. Some key telecommunication/ICT data are provided for free, through the ITU statistics website (www.itu.int), and via the Global Sustainable Development Goals Indicators Database (https://unstats.un.org/sdgs/indicators/database/).

479. Household and individual ICT data collected by ITU are disseminated via regional and global reports. Relevant results from the questionnaire have also been used to prepare other ITU statistical publications such as the Use of Information and Communication Technology by the World’s Children and Youth (ITU, 2008) as well as publications such as the Measuring the Information Society Report series (for example, ITU, 2013a), and the Measuring digital development series, which includes ICT Facts and Figures[18]. Publications of interest are available from the ITU statistics website. ICT household data collected by ITU will figure prominently in future ITU publications that analyse trends in ICT access and use.

[1] It is recommended that electronic formats for dissemination are open, such as text files (.TXT, .CSV) or software that does not require costly investment by the users.

[2] See: http://adp.ihsn.org/.

[3] Even though this advice applies to international reporting in the current context, it is true for any situation where response categories are combined.

[4] That is, those indicators, where the whole population (in each population category, e.g. males) is the only possible denominator.

[5] For notes on the classifications, see Chapter 4.

[6] Based on ISCED2011.

[7] Self-employed includes: employers, own account workers members of producers' cooperatives and contributing family workers.

[8] Based on ISCO-88; categories would change slightly with the implementation of ISCO-08 in 2008.

[9] Armed forces personnel may be out of scope, in which case the category would not be included.

[10] This is not a recommended core indicator classificatory variable but can be very useful in a country where a number of languages are read/spoken (especially where this might be limiting in terms of employment, education, ICT use or other activities).

[11] This is not a recommended core indicator classificatory variable but will be of policy interest in some countries. It is important to establish the existence of disability carefully and this might require several questions.

[12] See: https://www.unece.org/stats/documents/writing/

[13] Information to be reported for each quality criterion and indicators can be found here in the ESS Handbook for Quality Reports, https://ec.europa.eu/eurostat/web/quality/quality-reporting.

[14] The webpage https://ec.europa.eu/eurostat/web/digital-economy-and-society/methodology is a very useful reference for information society statisticians and includes links to questionnaires and other methodological information (including the methodological manuals for information society statistics). See also the ‘Rolling review of the Information Society (IS) statistics’ carried out by Eurostat in 2011: https://ec.europa.eu/eurostat/documents/64157/4375784/28-Information-society-RR-2011.pdf/271f6e95-1ccb-405c-97cc-a59154ea3fd3

[15] Based on the recommendation in UNSD (2005b, Chapter 3). Note that this is higher than the levels stated by Eurostat, which recommends for ICT household surveys that “The estimated standard error (standard deviation), expressed by the square root of the estimate of the sampling variance, shall not exceed 2 percentage points of the overall proportions and shall not exceed 5 percentage points for the proportions relating to the different subgroups of the population, where these subgroups constitute at least 10% of the total population in the scope of the survey.” (Eurostat, 2013a).

[16] An example of survey result (or survey estimate) metadata is information that the estimated proportion of households with a computer is understated because computers were restricted to desktop computers.

[17] See: http://www.itu.int/net4/ITU-D/icteye/Login.aspx

[18] See for example the Facts and Figures 2019: https://www.itu.int/en/ITU-D/Statistics/Documents/facts/FactsFigures2019.pdf

Table 19. Metadata associated with ICT household surveys

Metadata class

Examples of metadata

Reference period

Survey name and organization conducting the survey (if relevant)

General information about the survey

Survey history.

Main changes over time to survey scope, methodology, definitions, etc.

Frequency of survey (quarterly, annual etc.).

Whether a stand-alone survey or not; a stand-alone survey only collects ICT use data and any other information required to tabulate results (e.g. age, sex). If not a stand-alone survey, indicate the survey vehicle used to include ICT questions.

Whether a mandatory or voluntary survey (legal basis for the survey).

Main statistical standards used

Major underlying statistical frameworks used (e.g. Partnership core ICT indicators concepts and definitions).

Divergence from international or national standards (for instance, scope differences or differences in concepts or definitions used for individual ICT indicators).

Changes in standards over time and their likely impact (for instance, a change in the definition of ‘computer’ between one survey and the next).

Might include a glossary of terms used (e.g. the Internet).

Main classifications used (e.g. ISCED).

Scope (target population), survey frame and coverage, statistical units

Description of household and individual scope where relevant.

Any significant limitations on the scope or coverage of the survey (e.g. the exclusion of non-urban populations).

Survey frame/s used including any inaccuracies such as undercoverage and duplication.

Statistical units (household and individual).

Survey methodology

Collection technique (e.g. face-to-face interview using CAPI).

Sample size.

Sample design (stratification, sampling stages).

Changes in survey methodology and impact on estimates.

Weighting methods, including final benchmarking against independent estimates.

Response rate

For each unit type, households and individuals, and possibly for major disaggregations (e.g. for males and females, areas, linguistic groups).

Non-response treatment

Imputation rules for missing units.

Imputation rules for missing items.

Relative standard errors (coefficients of variation) or confidence intervals

The best presentation for RSEs is probably as a table corresponding to data tables; alternatively, RSEs may be presented in a range for each type of aggregate (e.g. total level, province level).

Known non-sampling error

Bias (e.g. non-response, frame errors, questionnaire bias) and attempts made to minimize it.

Reference to further information about the survey (usually a website link)

Might include links to more detailed methodological information, questionnaires, how to obtain more detailed data, future plans etc.

Contact information for further information about the survey or survey data

An e-mail address is very useful. For privacy purposes, this could be generic but it is recommended to specify the national focal point for communication with ITU on ICT indicators.

Annex 1. Core list of ICT Indicators (as of 2016)

Core indicators on ICT infrastructure and access

This list is revised by the Expert Group on Telecommunication/ICT Indicators (EGTI).

A1

Fixed-telephone subscriptions per 100 inhabitants

A2

Mobile cellular telephone subscriptions per 100 inhabitants

A3

Fixed broadband Internet subscriptions per 100 inhabitants, broken down by speed

A4

Active mobile-broadband subscriptions per 100 inhabitants

A5

International Internet bandwidth per inhabitant (bits/second/inhabitant)

A6

Percentage of the population covered by a at least a 3G mobile network

A7

Fixed broadband Internet prices per month

A8

Mobile cellular telephone prepaid prices per month

A9

Mobile broadband Internet prices per month

A10

TV broadcasting subscriptions per 100 inhabitants

Core indicators on access to, and use of, ICT by households and individuals

This list is revised by the Expert Group on Household (EGH).

HH1

Proportion of households with a radio

HH2

Proportion of households with a TV

HH3

Proportion of households with telephone

HH4

Proportion of households with a computer

HH5

Proportion of individuals using a computer

HH6

Proportion of households with Internet

HH7

Proportion of individuals using the Internet

HH8

Proportion of individuals using the Internet, by location

HH9

Proportion of individuals using the Internet, by type of activity

HH10

Proportion of individuals using a mobile cellular telephone

HH11

Proportion of households with Internet, by type of service

HH12

Proportion of individuals using the Internet, by frequency

HH13

Proportion of households with multichannel television, by type

HH14

Barriers to household Internet access

HH15

Proportion of individuals with ICT skills, by type of skills

HH16

Household expenditure on ICT

HH17

Proportion of individuals using the Internet, by type of portable device and network used to access the Internet

HH18

Proportion of individuals who own a mobile phone

HH19

Proportion of individuals not using the Internet, by type of reason

HH20*

Proportion of individuals who purchased goods or services online, by type of good and service purchased

HH21*

Proportion of individuals who purchased goods or services online, by type of payment channel

HH22*

Proportion of individuals who purchased goods or services online, by method of delivery

HH23*

Proportion of individuals who did not purchase goods or services online, by type of reason

*These are more recent additional indicators, that have not yet been endorsed by the 2016 version of this core list.

Core indicators on use of ICT by enterprises

B1

Proportion of businesses using computers

B2

Proportion of persons employed routinely using computers

B3

Proportion of businesses using the Internet

B4

Proportion of persons employed routinely using the Internet

B5

Proportion of businesses with a web presence

B6

Proportion of businesses with an intranet

B7

Proportion of businesses receiving orders over the Internet

B8

Proportion of businesses placing orders over the Internet

B9

Proportion of businesses using the Internet by type of access

B10

Proportion of businesses with a Local Area Network

B11

Proportion of businesses with an extranet

B12

Proportion of businesses using the Internet by type of activity

Core indicators on the ICT sector and trade in ICT goods

ICT1

Proportion of total business sector workforce involved in the ICT sector

ICT2

ICT sector share of gross value added

ICT3

ICT goods imports as a percentage of total imports

ICT4

ICT goods exports as a percentage of total export

Core indicators on ICT in education

ED1

Proportion of schools with a radio used for educational purposes

ED2

Proportion of schools with a television used for educational purposes

ED3

Proportion of schools with a telephone communication facility

ED4

Learners-to-computer ratio in schools with computer-assisted instruction

ED5

Proportion of schools with Internet access by type of access

ED6

Proportion of learners who have access to the Internet at school

ED7

Proportion of learners enrolled at the post-secondary level in ICT-related fields

ED8

Proportion of ICT-qualified teachers in schools

EDR1

Proportion of schools with electricity

Core indicators on e-government

EG1

Proportion of persons employed in central government organizations routinely using computers

EG2

Proportion of persons employed in central government organizations routinely using the Internet

EG3

Proportion of central government organizations with a local area network

EG4

Proportion of central government organizations with an intranet

EG5

Proportion of central government organizations with Internet access, by type of access

EG6

Proportion of central government organizations with a web presence

EG7

Selected Internet-based online services available to citizens, by level of sophistication of service

Annex 2. Model questionnaire for measuring ICT access and use by households and individuals

The model questionnaire can be used in a stand-alone ICT household survey or as a module in an existing survey vehicle, for example, a multipurpose household survey. If used as a module in an existing survey, it is likely that sections 1 and 3 (Household characteristics and Individual characteristics, respectively) would not be needed (or could be reduced) as such information is likely to be included elsewhere in the survey vehicle.

The model questionnaire is not an operational questionnaire that can be used directly in countries’ household surveys. This is because countries conduct household ICT access and use surveys in different ways, each requiring their own types of survey instruments. For instance, a questionnaire that is self-enumerated will look quite different from one designed for a telephone survey, which in turn will differ from a questionnaire used in face-to-face interviewing.

It is not expected that the structure, question wording or definitions that comprise the model questionnaire would be used unchanged (or literally translated) in national surveys. However, it is important for comparability purposes that:

• Where questions are used, their meanings are preserved, and

• The logic is preserved to the extent that the same populations of households or individuals are asked each question.

Notes and instructions to questionnaire designers

• Where there is no ‘Go to’ direction in the model questionnaire, the respondent is asked the next question.

• For questions 11, 12, 23, 25 and 28, country variations are: remove categories where items are not feasible; and add or split categories corresponding to country data requirements. Care should be taken when adding or splitting categories that statistical bias is not introduced. This could occur if the provision of alternative categories affects response. Care should be taken when aggregating detailed subcategories of these questions to avoid double counting individuals who respond to more than one of the subcategories.

• For multiple response questions (except for Q27), countries may ask about response categories as a series of yes/no questions, rather than a single ‘list’ question. The method chosen will reflect the method of data collection e.g. a telephone interview is more likely to use a series of yes/no questions, especially for the questions with a large number of response items.

• For question 27, countries are able to add additional frequency categories if they wish to obtain finer level information.

Model questionnaire for measuring ICT access and use by households and individuals

Section 1: Household characteristics

1

Number of household members

-This question is asked of all in-scope households.
-It includes members who are older or younger than any individual age scope used for the survey.

2

Are there any children aged under 15 years living in this household?

Yes

No

-This question is asked of all in-scope households.

Section 2: Household access to information and communication technology

3

Does this household have a radio?

Yes

No

-This question is asked of all in-scope households.
-A radio is defined as a device capable of receiving broadcast radio signals, using common frequencies, such as FM, AM, LW and SW. A radio may be a stand-alone device, or it may be integrated with another device, such as an alarm clock, an audio player, a mobile telephone or a computer.
-The equipment should be in working condition at the time of the survey.

4

Does this household have a television?

Yes

No

Go to Q6

-This question is asked of all in-scope households.
-A television (TV) is a device capable of receiving broadcast television signals, using popular access means such as over-the-air, cable and satellite. A television set is typically a stand-alone device, but it may also be integrated with another device, such as a computer or a mobile telephone.
-The equipment should be in working condition at the time of the survey.

5

Does this household have any of the following television services? Please tick all that apply.

Cable TV (CATV)
-Multichannel programming delivered over a coaxial cable for viewing on television sets.

Internet-protocol TV (IPTV)
-Multimedia services such as television/video/audio/text/graphics/data delivered over an IP-based network managed to support the required level of quality of service, quality of experience, security, interactivity and reliability; it does not include video accessed over the public Internet, for example, by streaming. IPTV services are also generally aimed at viewing over a television set rather than a personal computer.

Direct-to-home (DTH) satellite services
-TV services received via a satellite dish capable of receiving satellite television broadcasts.

Digital terrestrial TV (DTT)
-The technological evolution from analogue terrestrial television, providing capability for significantly more channels.

Additional question notes
-This question is asked of all in-scope households with a television.
-Record all multichannel services used by the household (that is, allow multiple responses).
-The TV service(s) selected should be working at the time of the survey.

6

Does this household have a fixed telephone line?

Yes

No

-This question is asked of all in-scope households.
-A fixed telephone line refers to a telephone line connecting a customer's terminal equipment (e.g. telephone set, facsimile machine) to the public switched telephone network (PSTN) and which has a dedicated port on a telephone exchange. This term is synonymous with the terms main station or Direct Exchange Line (DEL) that are commonly used in telecommunication documents. It may not be the same as an access line or a subscription.
-The equipment should be in working condition at the time of the survey.

7

Does this household have a mobile telephone?

Yes

No

Go to Q9

-This question is asked of all in-scope households.
-A mobile (cellular) telephone refers to a portable telephone subscribing to a public mobile telephone service using cellular technology, which provides access to the PSTN. This includes analogue and digital cellular systems and technologies such as IMT-2000 (3G) and IMT-Advanced. Users of both post-paid subscriptions and pre-paid accounts are included.
-The equipment should be in working condition at the time of the survey.

8

Does this household have a smartphone?

Yes

No

-This question is asked of all in-scope households that answered yes to question 7.
-A smart telephone (or smartphone) refers to a mobile handset that is used as the person’s primary phone device which has smart capabilities, including Internet-based services, and performs many of the functions of a computer, including having an operating system capable of downloading and running applications, also those created by third-party developers. Users of both postpaid subscriptions and prepaid accounts are included.
-The equipment should be in working condition at the time of the survey.

9

Does this household have a computer (desktop, laptop, tablet or similar)? Please tick all that apply.

Desktop
-Desktop: a computer that usually remains fixed in one place; normally the user is placed in front of it, behind the keyboard.

Laptop
-Laptop (portable) computer: a computer that is small enough to carry and usually enables the same tasks as a desktop computer; it includes notebooks and netbooks but does not include tablets and similar handheld computers.

Tablet
-Tablet (or similar handheld computer): a tablet is a computer that is integrated into a flat touch screen, operated by touching the screen rather than (or as well as) using a physical keyboard.

-This question is asked of all in-scope households.
-The equipment should be in working condition at the time of the survey.
-A computer refers to a desktop computer, a laptop (portable) computer or a tablet (or similar handheld computer). It does not include equipment with some embedded computing abilities, such as smart TV sets, and devices with telephony as their primary function, such as smartphones.

10

Does this household have Internet?

Yes

No

Go to Q12

-This question is asked of all in-scope households.
-The Internet is a worldwide public computer network. It provides access to a number of communication services including the World Wide Web and carries e-mail, news, entertainment and data files, irrespective of the device used (not assumed to be only via a computer - it may also be by mobile telephone, tablet, PDA, games machine, digital TV etc.).
-Access can be via a fixed or mobile network.
-An Internet connection in the household should be working at the time of the survey.

11

What type/s of Internet services are used for Internet access at home? Please tick all that apply.

Fixed narrowband network, at advertised download speeds below 256 kbit/s:
-Analogue modem (dial-up via standard telephone line)
-ISDN (Integrated Services Digital Network)
-DSL (Digital Subscriber Line) at advertised download speeds below 256 kbit/s
-Other fixed narrowband with an advertised download speed of less than 256 kbit/s

Fixed broadband network, at advertised download speeds of at least 256 kbit/s:
-DSL (Digital Subscriber Line) at advertised download speeds of at least 256 kbit/s
-Cable modem
-High speed leased lines
-Fibre-to-the-home/building
-Powerline
-Other fixed broadband

Terrestrial fixed broadband network, at advertised download speeds of at least 256 kbit/s:
-WiMAX
-Fixed CDMA

Satellite broadband network (via a satellite connection), at advertised download speeds of at least 256 kbit/s

Mobile broadband network (at least 3G, e.g. UMTS) via a handset

Mobile broadband network (at least 3G, e.g. UMTS) via a card:
-Integrated SIM card in a computer
-USB modem

Additional question notes
-This question is asked of all in-scope households with the Internet at home.
-Record all Internet services used by the household (that is, allow multiple responses).
-The response categories need to be adapted by countries according to plans and services offered by operators, and terminologies that are more familiar to users.
-According to local context, additional information could be collected to better identify the correct type of access.

Go to Q13

12

Why does this household not have Internet access? Please tick all that apply.

Do not need the Internet
-Not useful, not interesting

Have access to the Internet elsewhere

Cost of the equipment is too high

Cost of the service is too high

Privacy or security concerns

Internet service is not available in the area

Internet service is available but it does not correspond to household needs
-For example, quality, speed.

Cultural reasons
-For example, exposure to harmful content.

Lack of local content

No electricity in the household

Other reason (please specify)

Additional question notes
-This question is asked of all in-scope households which did not have the Internet at home.
-Record all reasons (that is, allow multiple responses).

Section 3: Individual characteristics

13

Age (years)

-This question is asked of all in-scope individuals.

14

Sex

Male

Female

-This question is asked of all in-scope individuals.

15

Highest educational level. Please tick one.

Primary education or lower

Lower secondary education

Upper secondary education or post-secondary non-tertiary education

Tertiary

Post-tertiary

Additional question notes
-This question is asked of all in-scope individuals.
-Only one educational level can be selected.
-Categories are based on UNESCO's International Standard Classification of Education (ISCED-A 2011), or the equivalent national classification.

16

Labour force status. Please tick one.

Employee

Self-employed
-Includes employers, own-account workers, members of producers' cooperatives and contributing family workers.

Workers not classifiable by status
-For whom insufficient relevant information is available, and/or who cannot be included in either of the preceding categories.

Unemployed

Outside the labour force
-Individuals who are not economically active: usually students (not in the workforce), people undertaking home duties only, and those who are retired or infirm.

Additional question notes
-This question is asked of all in-scope individuals.
-Only one labour force status category can be selected. Respondents should select the option that best describes their labour force status.
-If respondents tick either of the last two categories, they should go to Q18.
-Categories are based on the International Labour Organization's (ILO) International Classification of Status in Employment (ICSE-93), or the equivalent national classification, with additional categories for unemployed and outside the labour force.

17

Main occupation (please describe)

-This question is asked of all in-scope individuals who are employed (employees, self-employed and workers not classifiable by status).-Responses should be coded according to the 1-digit categories of the International Labour Organization's International Standard Classification of Occupations (ISCO 1988 or 2008), or the equivalent national classification.

Section 4: Individual use of information and communication technology

18

Have you used a mobile telephone in the last three months?

Yes

No

Go to Q20

-This question is asked of all in-scope individuals.
-Mobile telephone is defined in Q7.
-Use of a mobile telephone does not necessarily mean that the telephone is owned or paid for by the individual but should be reasonably available through work, a friend or family member, etc. It excludes occasional use, for instance, borrowing a mobile telephone to make a call.

19

Have you used a smartphone in the last three months?

Yes

No

-This question is asked of all individuals having used a mobile phone.
-Smartphone is defined in Q8.
-Use of a smartphone does not necessarily mean that the telephone is owned or paid for by the individual but should be reasonably available through work, a friend or family member, etc. It excludes occasional use, for instance, borrowing a mobile telephone to make a call.

20

Do you own a mobile phone?

Yes

No

Go to Q22

-This question is asked of all individuals having used a mobile phone.
- Mobile telephone is defined in Q7.
- An individual owns a mobile cellular telephone if he/she has a mobile cellular phone device with at least one active SIM card for personal use. It includes mobile cellular phones supplied by employers that can be used for personal reasons (to make personal calls, access the Internet, etc.) and those who have a mobile phone for personal use that is not registered under his/her name. It excludes individuals who have only active SIM card(s) and not a mobile phone device.

21

Do you own a smartphone?

Yes

No

-This question is asked of all individuals owning a mobile phone.
- Smartphone is defined in Q8.
- An individual owns a smartphone if he/she has a smartphone with at least one active SIM card for personal use. It includes smartphones supplied by employers that can be used for personal reasons (to make personal calls, access the Internet, etc.) and those who have a smartphone for personal use that is not registered under his/her name. It excludes individuals who have only active SIM card(s) and not a smartphone.

22

Have you used a computer (desktop, laptop, tablet or similar) from any location in the last three months? Please tick all that apply.

Desktop
-Desktop: a computer that usually remains fixed in one place; normally the user is placed in front of it, behind the keyboard.

Laptop
-Laptop (portable) computer: a computer that is small enough to carry and usually enables the same tasks as a desktop computer; it includes notebooks and netbooks but does not include tablets and similar handheld computers.

Tablet
-Tablet (or similar handheld computer): a tablet is a computer that is integrated into a flat touch screen, operated by touching the screen rather than (or as well as) using a physical keyboard.

-This question is asked of all in-scope individuals.
-Computer is defined in Q9.

23

Which of the following activities have you carried out in the last three months (independent of the device used)? Please tick all that apply.

Using copy and paste tools to duplicate or move data, information and content in digital environments (e.g. within a document, between devices, on the cloud)

Sending messages (e.g. e-mail, messaging service, SMS) with attached files (e.g. document, picture, video)

Using basic arithmetic formulas in a spreadsheet

Connecting and installing new devices
-For example, a modem, camera, printer.

Finding, downloading, installing and configuring software

Creating electronic presentations with presentation software
-Including text, images, sound, video or charts

Transferring files or applications between a computer and other devices

Setting up effective security measures (e.g. strong passwords, log-in attempt notification) to protect devices and online accounts

Changing privacy settings on your device, account or app to limit the sharing of personal data and information (e.g. name, contact information, photos)

Verifying the reliability of information found online

Writing a computer program using a specialized programming language (Programming or coding in digital environments e.g. computer software, app development)

Additional question notes
-Record all activities (that is, allow multiple responses).

24

Have you used the Internet from any location in the last three months?

Yes

No

Go to Q33

-This question is asked of all in-scope individuals (not only those who have used a computer).
-Internet is defined in Q9.
-Access can be via a fixed or mobile network.

25

Where did you use the Internet in the last three months? Please tick all that apply.

Home

Work
-Where a person’s workplace is located at his/her home, then he/she would answer yes to the home category only.

Place of education
-applies only to students – teachers and others who work at a place of education would report ‘work’ as the place of Internet use; where a place of education is also made available as a location for general community Internet use, such use should be reported in the Community Internet access facility category.

Another person’s home
-The home of a friend, relative or neighbour.

Facility open to the public
-use at a facility open to the public regardless of payment, type of connection or nature of the facility, such as libraries, telecenters, cafes, restaurants, and shopping malls.

Of which, at a Community Internet access facility
-Typically free of charge; includes Internet use at community facilities such as public libraries, publicly provided Internet kiosks, non-commercial telecentres, digital community centres, post offices, other government agencies; access is typically free and available to the general public.

While commuting, in transport or walking
- While moving between places, commuting or on the streen, independently of the device being used.

Other locations (please specify)

Additional question notes
-This question is only asked of individuals who used the Internet in the last three months.
-Access via a mobile device should be classified to the appropriate location or to ‘while commuting, in transport or walking’, that is, while mobile.
-Record all locations where individuals used the Internet (that is, allow multiple responses).

26

Have you used the Internet in the last three months using…? Please tick all that apply.

A mobile phone

A mobile phone via the cellular network

A mobile phone via other wireless networks (e.g. WiFi)

A tablet

A tablet via the cellular network, using USB key or SIM card

A tablet via other wireless networks (e.g. WiFi)

A portable computer (laptop, notebook, netbook)

A portable computer via the cellular network, using USB key or SIM card

A portable computer via other wireless networks (e.g. WiFi)

Other portable devices (e.g. portable games consoles, watches, e-book readers etc.)

27

How often did you typically use the Internet during the last three months (from any location)? Please tick one.

At least once a day
-Once a working day for respondents who only (or most frequently) use the Internet from work or school etc.

At least once a week but not every day

Less than once a week

Additional question notes
-This question is only asked of individuals who used the Internet in the last three months.
-Only one frequency can be selected.
-The question refers to a typical period; therefore, respondents should ignore weekends (if they only use the Internet from work or school) and breaks from their usual routine, such as holidays.

28

For which of the following activities did you use the Internet for private purposes in the last three months (from any location)? Please tick all that apply.

Access to information:

Getting information about goods or services

Seeking health information
-On injury, disease, nutrition etc.

Getting information from general government organizations

General government organizations should be consistent with the SNA93 (2008 revision) concept of general government. According to the SNA “… the principal functions of government are to assume responsibility for the provision of goods and services to the community or to individual households and to finance their provision out of taxation or other incomes; to redistribute income and wealth by means of transfers; and to engage in non-market production.” (General) government organizations include central, state and local government units.

Using services related to travel or travel-related accommodation

Downloading software or applications
-Includes patches and upgrades, either paid or free of charge.

Reading or downloading online newspapers or magazines, electronic books
-Includes accessing news websites, either paid or free of charge; includes subscriptions to online news services.

Communication, civic participation and collaboration:

Sending or receiving e-mail

Making calls (telephoning over the Internet/VoIP, using Skype, iTalk, etc.; includes video calls via webcam).

Participating in social networks
-Creating user profile, posting messages or other contributions to Facebook, Twitter etc.

Making an appointment with a health practitioner via a website

Interacting with general government organizations
-Downloading/requesting forms, completing/lodging forms online, making online payments and purchasing from government organizations etc.

Taking part in online consultations or voting to define civic or political issues
-Urban planning, signing a petition etc.

Accessing or posting opinions on chat sites, blogs, newsgroups or online discussions

Electronic commerce, trade and transactions

Purchasing or ordering goods or services
-Purchase orders placed via the Internet whether or not payment was made online; excludes orders that were cancelled or not completed; includes purchasing of products such as music, travel and accommodation via the Internet.

If NO, skip Q29,Q30 and Q31

Selling goods or services
-Via eBay, Mercado libre, Facebook etc.

Internet banking
Includes electronic transactions with a bank for payment, transfers, etc. or for looking up account information; excludes electronic transactions via the Internet for other types of financial services such as share purchases, financial services and insurance.

Learning

Doing a formal online course (in any subject)

Consulting wikis (Wikipedia etc.), online encyclopaedias or other websites for formal learning purposes

Professional life

Looking for a job or sending/submitting a job application
-Includes searching specific web sites for a job; sending/submitting an application online.

Participating in professional networks
-These are also seen in the broader context of social networking and have the same requirement of profile creation, contributing through messaging or chat, or uploading text or audio-visual content files.
-Examples of professional or business networks are LinkedIn and Xing.

Entertainment, digital content consumption

Listening to web radio
-Either paid or free of charge.

Watching web television
-Either paid or free of charge.

Streaming or downloading images, movies, videos or music; playing or downloading games
-Either paid or free of charge.

Digital content creation

Uploading self/user-created content to a website to be shared
-Text, images, photos, videos, music, software, etc.

Using storage space on the Internet to save documents, pictures, music, video or other files
-For example, Google Drive, Dropbox, Windows Skydrive, iCloud, Amazon Cloud Drive.

Using software run over the Internet for editing text documents, spreadsheets or presentations

Other activities (please specify)

Additional question notes
-This question is only asked of individuals who used the Internet in the last three months.
-Record all Internet activities undertaken (that is, allow multiple responses). Activities are not mutually exclusive, that is, there is overlap between some categories.
-"Private purposes" means not as part of one's current job or business.

29

What types of goods or services did you buy or order over the Internet for private use in the last 3 months? Please tick all that apply.

Books, magazines or newspapers

Clothing, footwear, sporting goods or accessories

Computer equipment or parts (including peripheral equipment)

Computer or video games

Computer software (includes upgrades and paid apps; not games)

Cosmetics

Financial products (including shares and insurance)

Food, groceries, alcohol or tobacco

Household goods (e.g. furniture, toys, etc.; excluding consumer electronics)

ICT services (excluding software)

Medicine

Movies, short films or images

Music products

Photographic, telecommunications or optical equipment

Tickets or bookings for entertainment events (sports, theatre, concerts, etc.)

Travel products (travel tickets, accommodation, vehicle hire, transport services etc.)

30

How did you pay for the goods or services you bought over the Internet for private use in the last 3 months? Please tick all that apply.

Cash on delivery

Credit card online

Debit card or electronic bank transfer online

Mobile money account (account connected to the mobile number)

Online payment service (e.g. PayPal, Google Checkout)

Prepaid gift card or online voucher

Points from rewards or redemption program (e.g. Air Miles)

Other (e.g. bank check by post, etc.)

31

How did you receive the goods or services you bought over the Internet for private use in the last 3 months? Please tick all that apply.

Delivery directly to the buyer using regular postal services or other forms of delivery

Picked up from point of sale or service point

Online / electronic delivery by downloading from a website or through an application, software or other device (e.g. in-app purchases, streaming services etc.).

32

What are the reasons why you did not purchase goods or services the Internet for private use in the last 3 months? Please tick all that apply.

Not interested

Prefer to shop in person

Security concerns

Privacy concerns

Technical concerns

Trust concerns

Lack of confidence, knowledge or skills

33

What are the reasons for not having used the Internet?

Do not need the Internet (not useful, not interesting)

Do not know how to use it

Cost of Internet use is too high (service charges, etc.)

Privacy or security concerns

Internet service is not available in the area

Cultural reasons (e.g. exposure to harmful content)

Don’t know what Internet is

Not allowed to use the Internet

Lack of local content

Other reason, specify

Annex 3. Examples of imputation and weighting

Imputation for missing data

The following table shows raw data from a survey on ICT use by households. Each row represents data from one survey record. ‘Missing’ refers to item non-response. The statistician is interested in imputing a value for the missing data ‘use of Internet’ for records #1, #4 and #6.

Record ID

Access to the Internet

(household)

Education level

(individual)

Use of Internet

(individual)

#1

No

Primary

Missing

#2

No

Primary

No

#3

Yes

Tertiary

Yes

#4

Yes

Secondary

Missing

#5

Yes

Tertiary

Yes

#6

Yes

Tertiary

Missing

#7

No

Secondary

Yes

#8

No

Primary

No

The statistician can select different rules for imputation as follows:

• Rule 1: A deterministic rule that imputes ‘Yes’ to all missing values for the item ‘Use of Internet’ if the household has access to the Internet and ‘No’ otherwise.

• Rule 2: A deterministic rule that imputes ‘Yes’ to all missing values for the item ‘Use of Internet’ if the household has access to the Internet and the level of education of the individual is ‘Tertiary’, and ‘No’ otherwise.

• Rule 3: A rule that imputes the most frequent value of the item ‘Use of Internet’ in the set of records with the same value for ‘Access to the Internet’.

• Rule 4: A rule that imputes the most frequent value of the item ‘Use of Internet’ in the set of records with the same value for ‘Level of education’.

The application of the above rules is shown below.

Rule

Imputed value for ‘use of Internet’

Record #1

Record #4

Record #6

1

No

Yes

Yes

2

No

No

Yes

3

No

(there are 2 ‘No’ and 1 ‘Yes’ responses in the set of records with no access to the Internet)

Yes

(there are 2 ‘Yes’ and 0 ‘No’ responses in the set of records with access to the Internet)

Yes

(there are 2 ‘Yes’ and 0 ‘No’ responses in the set of records with access to the Internet)

4

No

(there are 2 ‘No’ and 0 ‘Yes’ responses in the set of records with Primary education)

Yes

(there is 1 ‘Yes’ and 0 ‘No’ responses in the set of records with Secondary education)

Yes

(there are 2 ‘Yes’ and 0 ‘No’ responses in the set of records with Tertiary education)

It should be noted that the selection of one or other rule will produce biases in the final estimates. In general, methods based on the replacement of a missing value by a modal, median or average value of the same variable in a set of ‘similar’ records reduces the bias, but also artificially reduces the variance of the population.

Tree-based algorithms

Trees are simple representations for decision rules that can be used for estimation, classification or imputation of missing values. It is a statistical technique based on finding groups of observations (“terminal nodes”) as homogeneous as possible (thus, minimizing measures of variance such as sum-of-squares). The statistical basis of this technique involves advanced mathematical and statistical properties of data spaces. Each “node” is defined by values of an explanatory variable (which can be dichotomic, categorical or quantitative). By using different explanatory variables, one can obtain different trees. By combining a large number of trees calculated by selecting at random the explanatory variables to be used in the decision model, the predictions are usually better. This is the principle of Random Forests, which aggregate the results of the individual trees by averaging the predicted values (in the case of quantitative variables) or by taking the modal (or “majority”) value (in the case of qualitative variables)[1].

The examples below include 3 different trees, used to model the response dichotomic variable “Use of Internet by an individual”, obtained using the explanatory variables: “Access to the Internet” and “Education level”.

Random tree #1: two explanatory variables

Random tree #2 : only one explanatory variable

Random tree #3

Imputation with Random Forest and “majority vote” rule:

Imputed value for “Use of Internet”

Individual ID

Access to the Internet

(household)

Education level

(individual)

Use of Internet

(individual)

Prediction with Random Tree #1

Prediction with Random Tree #2

Prediction with Random Tree #3

Prediction with Random Forest (majority vote)

Training set

#2

No

Primary

No

No (prediction without error)

No (prediction without error)

No (prediction without error)

No (prediction without error)

#3

Yes

Higher education

Yes

Yes (prediction without error)

Yes (prediction without error))

Yes (prediction without error)

Yes (prediction without error)

#5

Yes

Higher education

Yes

Yes (prediction without error)

Yes (prediction without error)

Yes (prediction without error)

Yes (prediction without error)

#7

No

Secondary

Yes

No (prediction error)

Yes (prediction without error)

No (prediction error)

No (prediction error)

#8

No

Primary

No

No (prediction without error)

No (prediction without error)

No (prediction without error)

No (prediction without error)

Test set

#1

No

Primary

Missing

No

No

No

No

#4

Yes

Secondary

Missing

Yes

Yes

Yes

Yes

#6

Yes

Higher education

Missing

Yes

Yes

Yes

Yes

Rate of prediction error

20%

0%

20%

20%

Weighting records

Suppose that a population of households has been sampled, producing a stratified random sample of 9 units from strata A and B as follows:

Stratum

Population size

Sample size

A

3,000

5

B

1,000

4

The results of the survey provide the following records:

Household ID

Stratum

Access to computer

(household)

Access to the Internet

(household)

#1

A

No

No

#2

A

No

No

#3

B

Yes

No

#4

A

Yes

Yes

#5

B

Yes

Yes

#6

A

Yes

Yes

#7

B

No

No

#8

A

Yes

No

#9

B

Yes

No

The sampling weight of each household in A is equal to 3,000 /5 = 600, while for one in B, it is 1,000/4 = 250. Population estimates for the proportion of households with access to a computer are calculated by weighting each household in A by 600 and each one in B by 250. This gives the following statistics:

Stratum

Number of households with access to computer (unweighted)

Number of households with access to the Internet (unweighted)

Number of households with access to computer (weighted)

Number of households with access to the Internet (weighted)

A

3

2

1,800

1,200

B

3

1

750

250

The weighted proportions of households with a computer and access to the Internet are:

Stratum

Total number of households

Number of households with access to computer (weighted)

Number of households with access to the Internet (weighted)

Proportion of households with access to computer (weighted)

Proportion of households with access to the Internet (weighted)

A

3,000

1,800

1,200

1,800/3,000 = 60%

1,200/3,000 = 40%

B

1,000

750

250

750/1,000 = 75%

250/1,000 = 25%

While, in each stratum, the weighted estimate is equivalent to the estimate based on the sample proportion, for the total population, the estimates are:

Population

Total number of households

Number of households with access to computer (weighted)

Number of households with access to the Internet (weighted)

Proportion of households with access to computer

Proportion of households with access to the Internet (weighted)

A+B

4,000

1,800+750

= 2,550

1,200+250

= 1,450

2,550/4,000

= 63.75%

1,450/4,000

= 36.25%

Use of modern software

Many NSOs are considering the use of open software for data processing, especially, the R software (https://www.r-project.org/). The advantage of using R is the large number of libraries, freely accessible and documented, allowing most calculations needed by statisticians, such as data imputation and weighting. There are many repositories of libraries, but the most used is CRAN (https://cran.r-project.org/web/packages/available_packages_by_name.html). For analysing data from sampling surveys with different sampling schemes, the package “survey” (https://cran.r-project.org/web/packages/survey/survey.pdf) can be used.

It is recommended that NSOs continuously train their staff in survey statistical techniques and the use of modern software.

[1] A complete introduction to the mathematical and statistical methods can be found in Hastie, T., R. Tibshirani, J. Friedman (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer Verlag. Freely available at: https://web.stanford.edu/~hastie/ElemStatLearn/.

Annex 4. ITU Questionnaire on Information and Communication Technology (ICT) Access and Use by Households and Individuals

Annex 4 shows the aggregated data that ITU collects from countries. The country questionnaire is based on the core indicators presented in this Manual.

Note that there may be a confusion between the ITU Questionnaire on ICT Access and Use by Households and Individuals, and the ITU Model Questionnaire: while the first collects aggregated data from countries (usually their NSOs), the second can be adapted by the NSOs to collect household-level data.

ICT Access by urban/rural and household composition (Table 1a of the questionnaire)

No.

Indicator

All house-
holds

Urban/Rural

Household composition

Urban

Rural

has children under 15

does not have children under 15

Total

Urban

Rural

Total

Urban

Rural

HH1

Number of households with a radio

HH2

Number of households with a television

HH3

Number of households with telephone (fixed or mobile)

Number of households with a fixed telephone (regardless of whether they have a mobile telephone)

Number of households with a mobile phone (regardless of whether they have a fixed telephone)

Number of households with fixed telephone only

Number of households with mobile cellular telephone only

Number of households with both fixed and mobile telephone

Number of households with a smart phone

HH4

Number of households with a computer (all types of computer)

Desktop

Laptop (portable) computer

Tablet (or similar handheld computer)

HH6

Number of households with Internet access

HH11

Number of households with access to Internet,
by type of service

Fixed narrowband network

Total broadband (fixed and/or mobile) network

Fixed broadband network only (regardless of the type of fixed broadband connection)

Mobile broadband network only (regardless of the type of mobile broadband connection)

Both fixed broadband and mobile broadband network (regardless of the type of connection)

Fixed broadband network

Terrestrial fixed broadband network

Satellite broadband network

Mobile broadband network via a handset

Mobile broadband network via a card or USB modem

HH13

Number of households with multichannel television by type

Cable TV (CATV)

Direct-to-home (DTH) satellite services

Internet-protocol TV (IPTV)

Digital terrestrial TV (DTT)

HH14

Reasons for not having Internet access (i.e. Barriers)

Do not need the Internet (not useful, not interesting)

Have access to the Internet elsewhere

Cost of the equipment too high

Cost of the service too high

Privacy or security concerns

Internet service is not available in the area

Internet service is available but it does not correspond to household needs (e.g. quality, speed)

Cultural reasons (e.g. exposure to harmful content)

Lack of local content

No electricity in the household

Other reason

HH16

Total household expenditure on ICT

Information and communication equipment (COICOP 08.1)

Software excluding games computer software packages, such as operating systems, applications, programming languages, etc. (COICOP 08.2)

Information and communication services (COICOP 08.3)

Games toys and hobbies (COICOP 09.2.1)

TOTAL HOUSEHOLD EXPENDITURE

HHR1

Number of households with electricity

ICT Usage by sex and urban/rural (Table 2a of the questionnaire)

All indivi-
duals

Sex

Urban

Rural

No.

Indicators

Male

Female

Total

Male

Female

Total

Male

Female

HH5

Number of individuals who used a computer (from any location) in the last three months

Desktop

Laptop (portable) computer

Tablet (or similar handheld computer)

HH10

Number of individuals who used a mobile cellular telephone in the last three months

Number of individuals who used a smart phone in the last three months

HH18

Number of individuals who own a mobile cellular telephone

Number of individuals who own a smart phone

HH15

Number of individuals with ICT skills, by type of skills

Copying or moving a file or folder

Using copy and paste tools to duplicate or move information within a document

Sending e-mails with attached files (e.g. document, picture, video)

Using basic arithmetic formulas in a spreadsheet

Connecting and installing new devices (e.g. a modem, camera, printer)

Finding, downloading, installing and configuring software

Creating electronic presentations with presentation software (including images, sound, video or charts)

Transferring files between a computer and other devices

Writing a computer program using a specialized programming language

HH7

Number of individuals who used the Internet (from any location) in the last three months

HH8

Number of individuals who used the Internet in the last three months, by location of use

at home

at work

at place of education

at another person's home

at facility open to the public

at community Internet access facility

While commuting, in transport or walking

HH12

Number of individuals who used the Internet (from any location) in the last three months, by frequency

at least once a day

at least once a week but not every day

less than once a week

HH19

Number of individuals not using the Internet, by type of reason

Do not need the Internet

Do not know how to use it

Cost of Internet use is too high (service charges, etc.)

Privacy or security concerns

Internet service is not available in the area

Cultural reasons (e.g. exposure to harmful content)

Don’t know what Internet is

Not allowed to use the Internet

Lack of local content

Other reason

HH9

Number of individuals using the Internet in the last three months, by type of activity

Sending or receiving e-mail

Making calls (Telephoning over the Internet/VoIP, using Skype, iTalk, etc.; includes video calls via webcam)

Accessing or posting opinions on chat sites, blogs, newsgroups or online discussions

Participating in social networks

Uploading self/user-created content to a website to be shared

Participating in professional networks

Looking for a job or sending/submitting a job application

Doing a formal online course

Consulting wikis, online encyclopaedias or other websites for formal learning purposes

Seeking health information (on injury, disease, nutrition, etc.)

Making an appointment with a health practitioner via a website

Reading or downloading on-line newspapers or magazines, electronic books

Getting information about goods or services

Purchasing or ordering goods or services

Selling goods or services

Internet banking

Using services related to travel or travel-related accommodation

Getting information from general government organizations

Interacting with general government organizations

Taking part in on-line consultations or voting to define civic or political issues

Streaming or downloading images, movies, videos or music, playing or downloading games

Listening to web radio

Watching web television

Downloading software or applications

Using storage space on the Internet to save documents, pictures, music, video or other files

Using software run over the Internet for editing text documents, spreadsheets or presentations

HH17

Number of individuals using the Internet in the last three months, by type of portable device and network used to access the Internet

a. Mobile phone

a1) via mobile cellular network

a2) via other wireless networks (e.g. WiFi)

b. Tablet

b1) via mobile cellular network, using USB key/dongle or integrated data SIM card

b2) via other wireless networks (e.g. WiFi)

c. Portable computer (laptop, notebook, netbook)

c1) via mobile cellular network, using USB key/dongle or integrated data SIM card or mobile cellular telephone as modem

c2) via other wireless networks (e.g. WiFi)

d. Other portable devices (e.g. portable games consoles, watches, e-book readers etc.)

HH20

Number of individuals who purchased goods or servies online in the last three months, by type of goods and services purchased online

Books, magazines or newspapers

Clothing, footwear, sporting goods or accessories

Computer equipment or parts (including peripheral equipment)

Computer or video games

Computer software (includes upgrades and paid apps; not games)

Cosmetics

Financial products (including shares and insurance)

Food, groceries, alcohol or tobacco

Household goods (e.g. furniture, toys, etc.; excluding consumer electronics)

ICT services (excluding software)

Medicine

Movies, short films or images

Music products

Photographic, telecommunications or optical equipment

Tickets or bookings for entertainment events (sports, theatre, concerts, etc.)

Travel products (travel tickets, accommodation, vehicle hire, transport services etc.)

HH21

Number of individuals who purchased goods or services online in the last three months, by type of payment channel

Cash on delivery

Credit card online

Debit card or electronic bank transfer online

Mobile money account (account connected to the mobile number)

Online payment service (e.g. PayPal, Google Checkout)

Prepaid gift card or online voucher

Points from rewards or redemption program (e.g. Air Miles)

Other (e.g. bank check by post, etc.)

HH22

Number of individuals who purchased goods or services online in the last three months, by method of delivery

Delivery directly to the buyer using regular postal services or other forms of delivery

Picked up from point of sale or service point

Online / electronic delivery by downloading from a website or through an application, software or other device (e.g. in-app purchases, streaming services etc.)

HH23

Number of individuals who did not purchase goods or services online in the last three months, by reason

Not interested

Prefer to shop in person

Security concerns (e.g. about giving debit or credit card details)

Privacy concerns (e.g. about giving personal details)

Technical concerns (e.g. about websites, payment or delivery)

Trust concerns (e.g. about warranties, receiving or returning products)

Lack of confidence, knowledge or skills

ICT Usage by age and sex (Table 2b of the questionnaire)

Age less than 15

Age 15-24

Age 25-74

Age 75 and over

No.

Indicators

Total

Male

Female

Total

Male

Female

Total

Male

Female

Total

Male

Female

HH5

Number of individuals who used a computer (from any location) in the last three months

Desktop

Laptop (portable) computer

Tablet (or similar handheld computer)

HH10

Number of individuals who used a mobile cellular telephone in the last three months

Number of individuals who used a smart phone in the last three months

HH18

Number of individuals who own a mobile cellular telephone

Number of individuals who own a smart phone

HH15

Number of individuals with ICT skills, by type of skills

Copying or moving a file or folder

Using copy and paste tools to duplicate or move information within a document

Sending e-mails with attached files (e.g. document, picture, video)

Using basic arithmetic formulas in a spreadsheet

Connecting and installing new devices (e.g. a modem, camera, printer)

Finding, downloading, installing and configuring software

Creating electronic presentations with presentation software (including images, sound, video or charts)

Transferring files between a computer and other devices

Writing a computer program using a specialized programming language

HH7

Number of individuals who used the Internet (from any location) in the last three months

HH8

Number of individuals who used the Internet in the last three months, by location of use

at home

at work

at place of education

at another person's home

at facility open to the public

at community Internet access facility

While commuting, in transport or walking

HH12

Number of individuals who used the Internet (from any location) in the last three months, by frequency

at least once a day

at least once a week but not every day

less than once a week

HH19

Number of individuals not using the Internet, by type of reason

Do not need the Internet

Do not know how to use it

Cost of Internet use is too high (service charges, etc.)

Privacy or security concerns

Internet service is not available in the area

Cultural reasons (e.g. exposure to harmful content)

Don’t know what Internet is

Not allowed to use the Internet

Lack of local content

Other reason

HH9

Number of individuals using the Internet in the last three months, by type of activity

Sending or receiving e-mail

Making calls (Telephoning over the Internet/VoIP, using Skype, iTalk, etc.; includes video calls via webcam)

Accessing or posting opinions on chat sites, blogs, newsgroups or online discussions

Participating in social networks

Uploading self/user-created content to a website to be shared

Participating in professional networks

Looking for a job or sending/submitting a job application

Doing a formal online course

Consulting wikis, online encyclopaedias or other websites for formal learning purposes

Seeking health information (on injury, disease, nutrition, etc.)

Making an appointment with a health practitioner via a website

Reading or downloading on-line newspapers or magazines, electronic books

Getting information about goods or services

Purchasing or ordering goods or services

Selling goods or services

Internet banking

Using services related to travel or travel-related accommodation

Getting information from general government organizations

Interacting with general government organizations

Taking part in on-line consultations or voting to define civic or political issues

Streaming or downloading images, movies, videos or music, playing or downloading games

Listening to web radio

Watching web television

Downloading software or applications

Using storage space on the Internet to save documents, pictures, music, video or other files

Using software run over the Internet for editing text documents, spreadsheets or presentations

HH17

Number of individuals using the Internet in the last three months, by type of portable device and network used to access the Internet

a. Mobile phone

a1) via mobile cellular network

a2) via other wireless networks (e.g. WiFi)

b. Tablet

b1) via mobile cellular network, using USB key/dongle or integrated data SIM card

b2) via other wireless networks (e.g. WiFi)

c. Portable computer (laptop, notebook, netbook)

c1) via mobile cellular network, using USB key/dongle or integrated data SIM card or mobile cellular telephone as modem

c2) via other wireless networks (e.g. WiFi)

d. Other portable devices (e.g. portable games consoles, watches, e-book readers etc.)

HH20

Number of individuals who purchased goods or servies online in the last three months, by type of goods and services purchased online

Books, magazines or newspapers

Clothing, footwear, sporting goods or accessories

Computer equipment or parts (including peripheral equipment)

Computer or video games

Computer software (includes upgrades and paid apps; not games)

Cosmetics

Financial products (including shares and insurance)

Food, groceries, alcohol or tobacco

Household goods (e.g. furniture, toys, etc.; excluding consumer electronics)

ICT services (excluding software)

Medicine

Movies, short films or images

Music products

Photographic, telecommunications or optical equipment

Tickets or bookings for entertainment events (sports, theatre, concerts, etc.)

Travel products (travel tickets, accommodation, vehicle hire, transport services etc.)

HH21

Number of individuals who purchased goods or services online in the last three months, by type of payment channel

Cash on delivery

Credit card online

Debit card or electronic bank transfer online

Mobile money account (account connected to the mobile number)

Online payment service (e.g. PayPal, Google Checkout)

Prepaid gift card or online voucher

Points from rewards or redemption program (e.g. Air Miles)

Other (e.g. bank check by post, etc.)

HH22

Number of individuals who purchased goods or services online in the last three months, by method of delivery

Delivery directly to the buyer using regular postal services or other forms of delivery

Picked up from point of sale or service point

Online / electronic delivery by downloading from a website or through an application, software or other device (e.g. in-app purchases, streaming services etc.)

HH23

Number of individuals who did not purchase goods or services online in the last three months, by reason

Not interested

Prefer to shop in person

Security concerns (e.g. about giving debit or credit card details)

Privacy concerns (e.g. about giving personal details)

Technical concerns (e.g. about websites, payment or delivery)

Trust concerns (e.g. about warranties, receiving or returning products)

Lack of confidence, knowledge or skills

ICT Usage by highest education level attained/received and sex (Table 2c of the questionnaire)

Primary education or lower
(ISCED 0,1)

Lower secondary education
(ISCED 2)

Upper secondary or post-secondary non-tertiary
(ISCED 3,4)

Tertiary and post-tertiary education
(ISCED 5,6,7,8)

No.

Indicators

Total

Male

Female

Total

Male

Female

Total

Male

Female

Total

Male

Female

HH5

Number of individuals who used a computer (from any location) in the last three months

Desktop

Laptop (portable) computer

Tablet (or similar handheld computer)

HH10

Number of individuals who used a mobile cellular telephone in the last three months

Number of individuals who used a smart phone in the last three months

HH18

Number of individuals who own a mobile cellular telephone

Number of individuals who own a smart phone

HH15

Number of individuals with ICT skills, by type of skills

Copying or moving a file or folder

Using copy and paste tools to duplicate or move information within a document

Sending e-mails with attached files (e.g. document, picture, video)

Using basic arithmetic formulas in a spreadsheet

Connecting and installing new devices (e.g. a modem, camera, printer)

Finding, downloading, installing and configuring software

Creating electronic presentations with presentation software (including images, sound, video or charts)

Transferring files between a computer and other devices

Writing a computer program using a specialized programming language

HH7

Number of individuals who used the Internet (from any location) in the last three months

HH8

Number of individuals who used the Internet in the last three months, by location of use

at home

at work

at place of education

at another person's home

at facility open to the public

at community Internet access facility

While commuting, in transport or walking

HH12

Number of individuals who used the Internet (from any location) in the last three months, by frequency

at least once a day

at least once a week but not every day

less than once a week

HH19

Number of individuals not using the Internet, by type of reason

Do not need the Internet

Do not know how to use it

Cost of Internet use is too high (service charges, etc.)

Privacy or security concerns

Internet service is not available in the area

Cultural reasons (e.g. exposure to harmful content)

Don’t know what Internet is

Not allowed to use the Internet

Lack of local content

Other reason

HH9

Number of individuals using the Internet in the last three months, by type of activity

Sending or receiving e-mail

Making calls (Telephoning over the Internet/VoIP, using Skype, iTalk, etc.; includes video calls via webcam)

Accessing or posting opinions on chat sites, blogs, newsgroups or online discussions

Participating in social networks

Uploading self/user-created content to a website to be shared

Participating in professional networks

Looking for a job or sending/submitting a job application

Doing a formal online course

Consulting wikis, online encyclopaedias or other websites for formal learning purposes

Seeking health information (on injury, disease, nutrition, etc.)

Making an appointment with a health practitioner via a website

Reading or downloading on-line newspapers or magazines, electronic books

Getting information about goods or services

Purchasing or ordering goods or services

Selling goods or services

Internet banking

Using services related to travel or travel-related accommodation

Getting information from general government organizations

Interacting with general government organizations

Taking part in on-line consultations or voting to define civic or political issues

Streaming or downloading images, movies, videos or music, playing or downloading games

Listening to web radio

Watching web television

Downloading software or applications

Using storage space on the Internet to save documents, pictures, music, video or other files

Using software run over the Internet for editing text documents, spreadsheets or presentations

HH17

Number of individuals using the Internet in the last three months, by type of portable device and network used to access the Internet

a. Mobile phone

a1) via mobile cellular network

a2) via other wireless networks (e.g. WiFi)

b. Tablet

b1) via mobile cellular network, using USB key/dongle or integrated data SIM card

b2) via other wireless networks (e.g. WiFi)

c. Portable computer (laptop, notebook, netbook)

c1) via mobile cellular network, using USB key/dongle or integrated data SIM card or mobile cellular telephone as modem

c2) via other wireless networks (e.g. WiFi)

d. Other portable devices (e.g. portable games consoles, watches, e-book readers etc.)

HH20

Number of individuals who purchased goods or servies online in the last three months, by type of goods and services purchased online

Books, magazines or newspapers

Clothing, footwear, sporting goods or accessories

Computer equipment or parts (including peripheral equipment)

Computer or video games

Computer software (includes upgrades and paid apps; not games)

Cosmetics

Financial products (including shares and insurance)

Food, groceries, alcohol or tobacco

Household goods (e.g. furniture, toys, etc.; excluding consumer electronics)

ICT services (excluding software)

Medicine

Movies, short films or images

Music products

Photographic, telecommunications or optical equipment

Tickets or bookings for entertainment events (sports, theatre, concerts, etc.)

Travel products (travel tickets, accommodation, vehicle hire, transport services etc.)

HH21

Number of individuals who purchased goods or services online in the last three months, by type of payment channel

Cash on delivery

Credit card online

Debit card or electronic bank transfer online

Mobile money account (account connected to the mobile number)

Online payment service (e.g. PayPal, Google Checkout)

Prepaid gift card or online voucher

Points from rewards or redemption program (e.g. Air Miles)

Other (e.g. bank check by post, etc.)

HH22

Number of individuals who purchased goods or services online in the last three months, by method of delivery

Delivery directly to the buyer using regular postal services or other forms of delivery

Picked up from point of sale or service point

Online / electronic delivery by downloading from a website or through an application, software or other device (e.g. in-app purchases, streaming services etc.)

HH23

Number of individuals who did not purchase goods or services online in the last three months, by reason

Not interested

Prefer to shop in person

Security concerns (e.g. about giving debit or credit card details)

Privacy concerns (e.g. about giving personal details)

Technical concerns (e.g. about websites, payment or delivery)

Trust concerns (e.g. about warranties, receiving or returning products)

Lack of confidence, knowledge or skills

ICT Usage by labour force status and sex (Table 2d of the questionnaire)

Paid employee

Self-employed

Unemployed

Not in the labour force

Not classifiable by status

No.

Indicators

Total

Male

Female

Total

Male

Female

Total

Male

Female

Total

Male

Female

Total

Male

Female

HH5

Number of individuals who used a computer (from any location) in the last three months

Desktop

Laptop (portable) computer

Tablet (or similar handheld computer)

HH10

Number of individuals who used a mobile cellular telephone in the last three months

Number of individuals who used a smart phone in the last three months

HH18

Number of individuals who own a mobile cellular telephone

Number of individuals who own a smart phone

HH15

Number of individuals with ICT skills, by type of skills

Copying or moving a file or folder

Using copy and paste tools to duplicate or move information within a document

Sending e-mails with attached files (e.g. document, picture, video)

Using basic arithmetic formulas in a spreadsheet

Connecting and installing new devices (e.g. a modem, camera, printer)

Finding, downloading, installing and configuring software

Creating electronic presentations with presentation software (including images, sound, video or charts)

Transferring files between a computer and other devices

Writing a computer program using a specialized programming language

HH7

Number of individuals who used the Internet (from any location) in the last three months

HH8

Number of individuals who used the Internet in the last three months, by location of use

at home

at work

at place of education

at another person's home

at facility open to the public

at community Internet access facility

While commuting, in transport or walking

HH12

Number of individuals who used the Internet (from any location) in the last three months, by frequency

at least once a day

at least once a week but not every day

less than once a week

HH19

Number of individuals not using the Internet, by type of reason

Do not need the Internet

Do not know how to use it

Cost of Internet use is too high (service charges, etc.)

Privacy or security concerns

Internet service is not available in the area

Cultural reasons (e.g. exposure to harmful content)

Don’t know what Internet is

Not allowed to use the Internet

Lack of local content

Other reason

HH9

Number of individuals using the Internet in the last three months, by type of activity

Sending or receiving e-mail

Making calls (Telephoning over the Internet/VoIP, using Skype, iTalk, etc.; includes video calls via webcam)

Accessing or posting opinions on chat sites, blogs, newsgroups or online discussions

Participating in social networks

Uploading self/user-created content to a website to be shared

Participating in professional networks

Looking for a job or sending/submitting a job application

Doing a formal online course

Consulting wikis, online encyclopaedias or other websites for formal learning purposes

Seeking health information (on injury, disease, nutrition, etc.)

Making an appointment with a health practitioner via a website

Reading or downloading on-line newspapers or magazines, electronic books

Getting information about goods or services

Purchasing or ordering goods or services

Selling goods or services

Internet banking

Using services related to travel or travel-related accommodation

Getting information from general government organizations

Interacting with general government organizations

Taking part in on-line consultations or voting to define civic or political issues

Streaming or downloading images, movies, videos or music, playing or downloading games

Listening to web radio

Watching web television

Downloading software or applications

Using storage space on the Internet to save documents, pictures, music, video or other files

Using software run over the Internet for editing text documents, spreadsheets or presentations

HH17

Number of individuals using the Internet in the last three months, by type of portable device and network used to access the Internet

a. Mobile phone

a1) via mobile cellular network

a2) via other wireless networks (e.g. WiFi)

b. Tablet

b1) via mobile cellular network, using USB key/dongle or integrated data SIM card

b2) via other wireless networks (e.g. WiFi)

c. Portable computer (laptop, notebook, netbook)

c1) via mobile cellular network, using USB key/dongle or integrated data SIM card or mobile cellular telephone as modem

c2) via other wireless networks (e.g. WiFi)

d. Other portable devices (e.g. portable games consoles, watches, e-book readers etc.)

HH20

Number of individuals who purchased goods or servies online in the last three months, by type of goods and services purchased online

Books, magazines or newspapers

Clothing, footwear, sporting goods or accessories

Computer equipment or parts (including peripheral equipment)

Computer or video games

Computer software (includes upgrades and paid apps; not games)

Cosmetics

Financial products (including shares and insurance)

Food, groceries, alcohol or tobacco

Household goods (e.g. furniture, toys, etc.; excluding consumer electronics)

ICT services (excluding software)

Medicine

Movies, short films or images

Music products

Photographic, telecommunications or optical equipment

Tickets or bookings for entertainment events (sports, theatre, concerts, etc.)

Travel products (travel tickets, accommodation, vehicle hire, transport services etc.)

HH21

Number of individuals who purchased goods or services online in the last three months, by type of payment channel

Cash on delivery

Credit card online

Debit card or electronic bank transfer online

Mobile money account (account connected to the mobile number)

Online payment service (e.g. PayPal, Google Checkout)

Prepaid gift card or online voucher

Points from rewards or redemption program (e.g. Air Miles)

Other (e.g. bank check by post, etc.)

HH22

Number of individuals who purchased goods or services online in the last three months, by method of delivery

Delivery directly to the buyer using regular postal services or other forms of delivery

Picked up from point of sale or service point

Online / electronic delivery by downloading from a website or through an application, software or other device (e.g. in-app purchases, streaming services etc.)

HH23

Number of individuals who did not purchase goods or services online in the last three months, by reason

Not interested

Prefer to shop in person

Security concerns (e.g. about giving debit or credit card details)

Privacy concerns (e.g. about giving personal details)

Technical concerns (e.g. about websites, payment or delivery)

Trust concerns (e.g. about warranties, receiving or returning products)

Lack of confidence, knowledge or skills

ICT Usage by occupation (Table 2e of the questionnaire)

legislators, senior officials and managers

professionals

technicians and associate professionals

Etc.

No.

Indicators

Total

Male

Female

Total

Male

Female

Total

Male

Female

Total

HH5

Number of individuals who used a computer (from any location) in the last three months

Desktop

Laptop (portable) computer

Tablet (or similar handheld computer)

HH10

Number of individuals who used a mobile cellular telephone in the last three months

Number of individuals who used a smart phone in the last three months

HH18

Number of individuals who own a mobile cellular telephone

Number of individuals who own a smart phone

HH15

Number of individuals with ICT skills, by type of skills

Copying or moving a file or folder

Using copy and paste tools to duplicate or move information within a document

Sending e-mails with attached files (e.g. document, picture, video)

Using basic arithmetic formulas in a spreadsheet

Connecting and installing new devices (e.g. a modem, camera, printer)

Finding, downloading, installing and configuring software

Creating electronic presentations with presentation software (including images, sound, video or charts)

Transferring files between a computer and other devices

Writing a computer program using a specialized programming language

HH7

Number of individuals who used the Internet (from any location) in the last three months

HH8

Number of individuals who used the Internet in the last three months, by location of use

at home

at work

at place of education

at another person's home

at facility open to the public

at community Internet access facility

While commuting, in transport or walking

HH12

Number of individuals who used the Internet (from any location) in the last three months, by frequency

at least once a day

at least once a week but not every day

less than once a week

HH19

Number of individuals not using the Internet, by type of reason

Do not need the Internet

Do not know how to use it

Cost of Internet use is too high (service charges, etc.)

Privacy or security concerns

Internet service is not available in the area

Cultural reasons (e.g. exposure to harmful content)

Don’t know what Internet is

Not allowed to use the Internet

Lack of local content

Other reason

HH9

Number of individuals using the Internet in the last three months, by type of activity

Sending or receiving e-mail

Making calls (Telephoning over the Internet/VoIP, using Skype, iTalk, etc.; includes video calls via webcam)

Accessing or posting opinions on chat sites, blogs, newsgroups or online discussions

Participating in social networks

Uploading self/user-created content to a website to be shared

Participating in professional networks

Looking for a job or sending/submitting a job application

Doing a formal online course

Consulting wikis, online encyclopaedias or other websites for formal learning purposes

Seeking health information (on injury, disease, nutrition, etc.)

Making an appointment with a health practitioner via a website

Reading or downloading on-line newspapers or magazines, electronic books

Getting information about goods or services

Purchasing or ordering goods or services

Selling goods or services

Internet banking

Using services related to travel or travel-related accommodation

Getting information from general government organizations

Interacting with general government organizations

Taking part in on-line consultations or voting to define civic or political issues

Streaming or downloading images, movies, videos or music, playing or downloading games

Listening to web radio

Watching web television

Downloading software or applications

Using storage space on the Internet to save documents, pictures, music, video or other files

Using software run over the Internet for editing text documents, spreadsheets or presentations

HH17

Number of individuals using the Internet in the last three months, by type of portable device and network used to access the Internet

a. Mobile phone

a1) via mobile cellular network

a2) via other wireless networks (e.g. WiFi)

b. Tablet

b1) via mobile cellular network, using USB key/dongle or integrated data SIM card

b2) via other wireless networks (e.g. WiFi)

c. Portable computer (laptop, notebook, netbook)

c1) via mobile cellular network, using USB key/dongle or integrated data SIM card or mobile cellular telephone as modem

c2) via other wireless networks (e.g. WiFi)

d. Other portable devices (e.g. portable games consoles, watches, e-book readers etc.)

HH20

Number of individuals who purchased goods or servies online in the last three months, by type of goods and services purchased online

Books, magazines or newspapers

Clothing, footwear, sporting goods or accessories

Computer equipment or parts (including peripheral equipment)

Computer or video games

Computer software (includes upgrades and paid apps; not games)

Cosmetics

Financial products (including shares and insurance)

Food, groceries, alcohol or tobacco

Household goods (e.g. furniture, toys, etc.; excluding consumer electronics)

ICT services (excluding software)

Medicine

Movies, short films or images

Music products

Photographic, telecommunications or optical equipment

Tickets or bookings for entertainment events (sports, theatre, concerts, etc.)

Travel products (travel tickets, accommodation, vehicle hire, transport services etc.)

HH21

Number of individuals who purchased goods or services online in the last three months, by type of payment channel

Cash on delivery

Credit card online

Debit card or electronic bank transfer online

Mobile money account (account connected to the mobile number)

Online payment service (e.g. PayPal, Google Checkout)

Prepaid gift card or online voucher

Points from rewards or redemption program (e.g. Air Miles)

Other (e.g. bank check by post, etc.)

HH22

Number of individuals who purchased goods or services online in the last three months, by method of delivery

Delivery directly to the buyer using regular postal services or other forms of delivery

Picked up from point of sale or service point

Online / electronic delivery by downloading from a website or through an application, software or other device (e.g. in-app purchases, streaming services etc.)

HH23

Number of individuals who did not purchase goods or services online in the last three months, by reason

Not interested

Prefer to shop in person

Security concerns (e.g. about giving debit or credit card details)

Privacy concerns (e.g. about giving personal details)

Technical concerns (e.g. about websites, payment or delivery)

Trust concerns (e.g. about warranties, receiving or returning products)

Lack of confidence, knowledge or skills

Annex 5. Glossary of terms and abbreviations

Term or abbreviation

Notes

Source

3G mobile cellular network

Third generation of mobile communications technology, a group of mobile technologies that have been approved by ITU as IMT-2000. These technologies allow voice, data and video communications. Currently, five standards have been specified as IMT-2000 based on various combinations of mobile technologies: CDMA Direct Spread (WCDMA), CDMA Multi-Carrier (CDMA2000), CDMA Time division (TD-CDMA), TDMA Single-Carrier and FDMA/TDMA and OFDMA TDD WMAN (IEEE 802.16)

ITU (2011)

Accuracy

Denotes the closeness of computations or estimates to the exact or true values. Statistics are not equal with the true values because of variability (the statistics change from implementation to implementation of the survey due to random effects) and bias (the average of the possible values of the statistics from implementation to implementation is not equal to the true value due to systematic effects).

Eurostat Standard Quality Report (2003)

ADSL

Asymmetric digital subscriber line is a modem technology that converts twisted-pair telephone lines into access paths for multimedia and high-speed data communications. The bit rates transmitted in both directions are different.

ITU (2011)

Area sampling

Selection of geographical area units that comprise sampling frame (may include selection of area segments, defined as mapped subdivisions of administrative area)

UNSD (2005b)

Blog (short for Web log)

A blog (a truncation of the expression web log) is a discussion or informational site published on the World Wide Web and consisting of discrete entries ("posts") typically displayed in reverse chronological order (the most recent post appears first).

Wikipedia

Broadband

A general term meaning a telecommunications signal or device of greater bandwidth, in some sense, than another standard or usual signal or device; the broader the band, the greater the capacity for traffic). In data communications, the term refers to a data transmission rate of at least 256 kbit/s.

ITU (2011)

Cable modem

A cable modem is a layer two termination device that terminates the customer end of the J.112 (or J.122) connection.

ITU (2011)

Cable TV (CATV)

Multichannel programming delivered over a coaxial cable for viewing on television sets

This Manual, HH13

CAPI

Computer-assisted personal interviewing

CATI

Computer-assisted telephoning interviewing

CAWI

Computer-assisted web interviewing

Cluster sampling

Sampling in which next-to-last stage is geographically-defined unit such as census enumeration area (EA)

UNSD (2005b)

Clustering; clustered

Refers to tendency of sample units – persons or households – to have similar characteristics

UNSD (2005b)

Commercial Internet access facility

Enables Internet use at publicly available commercial facilities such as Internet or cyber cafés, hotels, airports etc., where access is typically paid (i.e. not free of charge)

This Manual, HH8

Community Internet access facility

Enables Internet use at community facilities such as public libraries, publicly provided Internet kiosks, non-commercial telecentres, digital community centres, post offices, other government agencies; access is typically free and is available to the general public

This Manual, HH8

Complex sample design

Refers to use of multiple stages, clustering and stratification in household survey samples, as opposed to simple random sampling

UNSD (2005b)

Confidence level

Describes degree of statistical confidence with which precision or margin of error around the survey estimate is obtained, 95 per cent generally being regarded as the standard

UNSD (2005b)

Design effect – deff

Ratio of variance from complex sample design to simple random sample of same sample size; deft is ratio of standard errors; sometimes referred to as clustering effect though deff includes effects of stratification as well as clustering

UNSD (2005b)

Desktop computer

A computer that usually remains fixed in one place. Normally the user is placed in front of it, behind the keyboard.

This Manual, HH4

Dial-up Internet access

Uses an (analogue) modem and fixed telephone line to connect to the Internet; it requires that the modem dial a telephone number when Internet access is needed

ITU (2011)

Digital terrestrial TV (DTT)

The technological evolution from analogue terrestrial television, providing capability for significantly more channels.

This Manual, HH13

Direct-to-home (DTH) satellite services

Television services received via a satellite dish capable of receiving satellite television broadcasts.

This Manual, HH13

DQAF

Data Quality Assessment Framework (IMF)

DSL

Digital subscriber line/s, a technology for bringing high-bandwidth information to homes and small businesses over ordinary copper telephone lines

ITU (2011)

EU

European Union

Fixed broadband network

Refers to technologies at advertised download speeds of at least 256 kbit/s, such as DSL, cable modem, high speed leased lines, fibre-to-the-home/building, powerline and other fixed broadband

This Manual, HH11

Fixed narrowband network

Includes analogue modem (dial-up via standard telephone line), ISDN (Integrated Services Digital Network), DSL (Digital Subscriber Line) at advertised download speeds below 256 kbit/s, and other forms of access with an advertised download speed of less than 256 kbit/s

This Manual, HH11

Fixed telephone line

A telephone line connecting a customer's terminal equipment (e.g. telephone set, facsimile machine) to the public switched telephone network (PSTN) and which has a dedicated port on a telephone exchange. This term is synonymous with the terms main station or Direct Exchange Line (DEL) that are commonly used in telecommunication documents. It may not be the same as an access line or a subscription.

This Manual, HH3

General government organizations

Are defined per the SNA93 (2008 revision) concept of general government. According to the SNA "… the principal functions of government are to assume responsibility for the provision of goods and services to the community or to individual households and to finance their provision out of taxation or other incomes; to redistribute income and wealth by means of transfers; and to engage in non-market production." (General) government organizations include central, state and local government units.)

This Manual, HH9

Handheld computer

A small computer including a personal digital assistants (PDA), also known as a palmtop computer.

This Manual, HH4

Homepage

A home page, index page, or main page is a page on a website.

Wikipedia

Household

For the purposes of the Manual, a household consists of one or more people, who may or may not be related to each other; share accommodation; and make common provision for food.

This Manual, Chapter 7

HTML

Hypertext markup language

ICT

Information and communication technology

ILO

International Labour Organization

Internet

A worldwide public computer network. It provides access to a number of communication services including the World Wide Web and carries e-mail, news, entertainment and data files, irrespective of the device used (not assumed to be only via a computer – it may also be by mobile phone, PDA, games machine, digital TV etc.). Access can be via a fixed or mobile network.

This Manual, HH6

Internet-protocol TV (IPTV)

Multimedia services such as television/video/audio/text/graphics/data delivered over an IP-based network managed to support the required level of quality of service, quality of experience, security, interactivity and reliability; it does not include video accessed over the public Internet, for example, by streaming. IPTV services are also generally aimed at viewing over a television set rather than a personal computer.

This Manual, HH13

IP

Internet protocol

ISDN

Integrated services digital network, a network that provides digital connections between user-network interfaces

ITU (2011)

ISP

Internet service provider

IT

Information technology

ITU

International Telecommunication Union

kbit/s (or Kbit/s or kbps)

Kilobits per second (1 kilobit per second=one thousand bits per second). A kilobit is 1,024 bits. A bit expresses a 1 or a 0 in a binary numeral, or a true or false logical condition.

ITU (2011) and ABS (2007)

Laptop (portable) computer

A computer that is small enough to carry and usually enables the same tasks as a desktop computer. It includes notebooks and netbooks but does not include tablets and similar handheld computers.

This Manual, HH4

LTE

Long-term evolution: A 4G wireless-broadband technology developed by the Third Generation Partnership Project (3GPP), an industry trade group

ITU (2011)

Master sample

A supersample intended to be used for multiple surveys and/or multiple rounds of the same survey, usually over 10-year time frame

UNSD (2005b)

Mobile (cellular) telephone

A portable telephone subscribing to a public mobile telephone service using cellular technology, which provides access to the PSTN. This includes analogue and digital cellular systems and technologies such as IMT-2000 (3G) and IMT-Advanced. Users of both post-paid subscriptions and pre-paid accounts are included.

This Manual, HH3, HH10

Mobile broadband network via a card or USB modem

Mobile broadband network (at least 3G, e.g. UMTS) via a card (e.g. integrated SIM card in a computer) or USB modem

This Manual, HH11

Mobile broadband network via a handset

Mobile broadband network (at least 3G, e.g. UMTS) via a handset

This Manual, HH11

Modem

Modulator-demodulator, a device or program that enables a computer to transmit data over, for example, telephone or cable lines

This Manual, HH11

Non-sampling error

Bias in survey estimate arising from errors in design and implementation; refers to accuracy or validity of an estimate as opposed to its reliability or precision

UNSD (2005b)

NSDS

National strategy for the development of statistics (PARIS21)

NSO

National statistical office

OCR

Optical character recognition

OECD

Organisation for Economic Co-operation and Development

OSILAC

Observatory for the Information Society in Latin America and the Caribbean

Partnership

Partnership on Measuring ICT for Development

PDA

Personal digital assistant

Primary sampling unit, PSU

Geographically-defined administrative unit selected at first stage of sampling

UNSD (2005b)

Probability sampling

Selection methodology whereby each population unit (person, household, etc.) has known, non-zero chance of inclusion in the sample

UNSD (2005b)

Radio

A device capable of receiving broadcast radio signals, using common frequencies, such as FM, AM, LW and SW. A radio may be a stand-alone device, or it may be integrated with another device, such as an alarm clock, an audio player, a mobile telephone or a computer.

This Manual, HH1

Reliability (precision, margin of error)

Refers to degree of sampling error associated with a given survey estimate

UNSD (2005b)

RSE

Relative standard error (coefficient of variation), standard error as percentage of survey estimate, i.e. standard error divided by estimate

UNSD (2005b)

Sample frame(s)

Set of materials from which sample is actually selected, such as a list or set of areas

UNSD (2005b)

Sample size

Number of units (households or persons) selected

UNSD (2005b)

Sampling error (standard error)

Random error in survey estimate due to the fact that a sample rather than entire population is surveyed; square root of sampling variance

UNSD (2005b)

Sampling in phases; also known as double sampling or post-stratified sampling

Selecting sample in (generally) two time periods, with second phase typically a subsample of first-phase sample; not to be confused with trend sampling (see below)

UNSD (2005b)

Sampling in stages

Means by which sample of administrative areas and households/persons is chosen in successive stages to pinpoint geographic locations where survey is conducted

UNSD (2005b)

Sampling variance

Square of standard error or sampling error

UNSD (2005b)

Satellite broadband network

Satellite broadband network (via a satellite connection), at advertised download speeds of at least 256 kbit/s

This Manual, HH11

Segment

A delineated, mapped subdivision of a larger cluster

UNSD (2005b)

Smartphone

A smart telephone (or smartphone) refers to a mobile handset that is used as the person’s primary phone device which has smart capabilities, including Internet-based services, and performs many of the functions of a computer, including having an operating system capable of downloading and running applications, also those created by third-party developers. Users of both postpaid subscriptions and prepaid accounts are included.

This Manual, HH3, HH10, HH18

SNA

System of National Accounts

Social network/networking

Social networking can be distinguished from other communication and content activities by the aspect of creating a profile on certain websites.

Eurostat (2013)

Stratified sampling

Technique of organizing sample frame into subgroupings that are internally homogeneous and externally heterogeneous to ensure sample selection is spread properly across important population subgroups

UNSD (2005b)

Systematic sampling

Selection from a list, using a random start and predetermined selection interval, successively applied

UNSD (2005b)

Tablet

A computer that is integrated into a flat touch screen, operated by touching the screen rather than (or as well as) using a physical keyboard.

This Manual, HH4

Target population

Definition of population intended to be covered by survey; also known as coverage universe

UNSD (2005b)

Television

A stand-alone device capable of receiving broadcast television signals, using popular access means such as over-the-air, cable and satellite. A television set is typically a stand-alone device, but it may also be integrated with another device, such as a computer or a mobile telephone.

This Manual, HH2

Terrestrial fixed broadband network

Refers to technologies at advertised download speeds of at least 256 kbit/s, such as WiMAX, fixed CDMA

This Manual, HH11

UIS

UNESCO Institute for Statistics

UMTS

Universal mobile telecommunications system, the telecommunications system, incorporating mobile cellular and other functionality that is the subject of standards produced by 3GPP (Third Generation Partnership Project)

ITU (2011)

UNCTAD

United Nations Conference on Trade and Development

UNECA

United Nations Economic Commission for Africa

UNECLAC

United Nations Economic Commission for Latin America and the Caribbean

UNESCAP

United Nations Economic and Social Commission for Asia and the Pacific

UNESCO

United Nations Educational, Scientific and Cultural Organization

UNESCWA

United Nations Economic and Social Commission for Western Asia

UNSC

United Nations Statistical Commission

UNSD

United Nations Statistics Division

URL

Uniform resource locator

USB modem

Universal serial bus, an external bus standard that supports data transfer rates of 12 Mbit/s

ITU (2011)

User-created content

Can be uploaded by anyone, includes texts, photos, music files and video clips, which often act as the centre for interaction within a network (e.g. YouTube, MySpace).

Eurostat (2013)

VoIP

Voice over Internet protocol, refers to managed VoIP and is the same as IP telephony.

ITU (2011)

Web presence

Includes a website, homepage or presence on another entity's website. It excludes inclusion in an online directory and any other web pages where the entity does not have control over the content of the page. A web presence includes social media pages and accounts (for example, Facebook, YouTube and Twitter) if the entity has control over content.

Partner-
ship and ECA (2012)

Website

Location on the World Wide Web identified by a web address. Collection of web files on a particular subject that includes a beginning file called a home page. Information is encoded with specific languages (Hypertext mark-up language (HTML), XML, Java) readable with a web browser, like Netscape's Navigator or Microsoft's Internet Explorer.

ITU (2009)

WiFi

Wireless fidelity, a wireless local area network based on the Institute of IEEE 802.11 standard

ITU (2011)

Wiki

Usually refers to a web application which allows people to add, modify, or delete content in a collaboration with others. Text is usually written using a simplified markup language or a rich-text editor.

Wikipedia

WiMAX

Wireless interoperability for microwave access/Worldwide Interoperability for Microwave Access, a family of telecommunications protocols that provide fixed and mobile Internet access, based on the IEEE 802.16 standard

ITU (2011)

WPIIS

Working Party on Indicators for the Information Society (OECD)

WSIS

World Summit on the Information Society

WWW

World Wide Web

xDSL

Any of the various types of digital subscriber lines technologies, e.g. ADSL

ITU (2011)

Bibliography

ABS (Australian Bureau of Statistics) (2007), Household Use of Information Technology, Australia, 2006-07, cat. no. 8146.0, https://www.abs.gov.au/AUSSTATS/abs@.nsf/allprimarymainfeatures/D65ED90A15BC6A1BCA2575220013B1EF?opendocument.

Broadband Commission for Sustainable Development (2018). The State of Broadband: Broadband catalyzing sustainable development. https://www.itu.int/dms_pub/itu-s/opb/pol/S-POL-BROADBAND.19-2018-PDF-E.pdf,

Census and Statistics Department, Hong Kong, China (2008), Questionnaire for the Thematic Household Survey on Information Technology Usage and Penetration in 2008.

Eurostat (2007), Methodological Manual for statistics on the Information Society, https://ec.europa.eu/eurostat/ramon/statmanuals/files/isoc_meth_manual_2007_EN.pdf

Eurostat (2008), Final Report, Information Society: ICT impact assessment by linking data from different sources, https://ec.europa.eu/eurostat/documents/341889/725524/2006-2008-ICT-IMPACTS-FINAL-REPORT-V2.pdf.

Eurostat (2018), Methodological Manual for statistics on the Information Society 2006-2018 and model questionnaires, https://circabc.europa.eu/faces/jsp/extension/wai/navigation/container.jsp

Global Strategy for Agriculture and Rural Statistics (2018), Guidelines on defining rural areas and compiling indicators for development policy, http://gsars.org/wp-content/uploads/2018/12/GS-GUIDELINES-RURAL-AREAS-EN-FINAL-2018.pdf

Hastie, T., R. Tibshirani, J. Friedman (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer Verlag. Freely available at https://web.stanford.edu/~hastie/ElemStatLearn/.

IHSN (International Household Survey Network) (2013), Home page, https://ihsn.org/.

ILO (International Labour Organization) (1993), International Classification of Status in Employment, https://ilostat.ilo.org/resources/methods/classification-status-at-work/.

ILO (2013), International Standard Classification of Occupations, ISCO website page, http://www.ilo.org/public/english/bureau/stat/isco/index.htm.

ITU (International Telecommunication Union) (2005), WSIS Outcome Documents: Geneva 2003 – Tunis 2005, http://www.itu.int/wsis/outcome/booklet.pdf.

ITU (2008), Use of Information and Communication Technology by the World’s Children and Youth: a Statistical Compilation, http://www.itu.int/ITU-D/ict/material/Youth_2008.pdf.

ITU (2009a), Measuring the Information Society: The ICT Development Index, http://www.itu.int/ITU-D/ict/publications/idi/material/2009/MIS2009_w5.pdf.

ITU (2009b), Manual for Measuring ICT Access and Use by Households and Individuals, http://www.itu.int/ITU-D/ict/publications/hhmanual/2009/material/HHManual2009.pdf.

ITU (2010a), World Telecommunication Development Conference (WTDC-10): Final Report, https://www.itu.int/publ/D-TDC-WTDC-2010/en.

ITU (2010b), Child Online Protection - Statistical Framework and Indicators, https://www.itu.int/dms_pub/itu-d/opb/ind/D-IND-COP.01-11-2010-PDF-E.pdf.

ITU (2011), Handbook for the collection of administrative data on telecommunications/ICT, http://www.itu.int/ITU-D/ict/publications/hb/2011/.

ITU (2014), Manual for Measuring ICT Access and Use by Households and Individuals,, 2014 edition, https://www.itu.int/dms_pub/itu-d/opb/ind/D-IND-ITCMEAS-2014-PDF-E.pdf.

ITU (2016), Measuring the information Society 2016, https://www.itu.int/en/ITU-D/Statistics/Documents/publications/misr2016/MISR2016-w4.pdf.

ITU (2018), Measuring the Information Society 2018, https://www.itu.int/en/ITU-D/Statistics/Pages/publications/misr2018.aspx

ITU (2019), World Telecommunication/ICT Indicators database 2019, 23rd edition, https://www.itu.int/pub/D-IND-WTID.OL-2019.

ITU (2020), Handbook for the Collection of Administrative Data on Telecommunications/ICT.

OECD (Organisation for Economic Co-operation and Development) (2005), “ICT Use by Households and Individuals: Proposal for Revising the OECD Model Survey”, Working Party on Indicators for the Information Society, DSTI/ICCP/IIS(2005)3/FINAL.

OECD (2007), “Measuring the impacts of ICT using official statistics”, Working Party on Indicators for the Information Society, DSTI/ICCP/IIS(2007)1.

OECD (2010), Educational Research and Innovation: Are the New Millennium Learners Making the Grade?: Technology Use and Educational Performance in PISA 2006, https://doi.org/10.1787/9789264076044-en.

OECD (2011), Guide to Measuring the Information Society, https://read.oecd-ilibrary.org/science-and-technology/oecd-guide-to-measuring-the-information-society-2011_9789264113541-en.

OECD (2013), “The OECD model survey on ICT usage by Households and Individuals: proposal for the 2nd revision”, DSTI/ICCP/IIS(2013)1.

OECD (2015a), “The OECD Model Survey on ICT Usage by Businesses, 2nd Revision”, https://www.oecd.org/sti/ieconomy/ICT-Model-Survey-Usage-Businesses.pdf

OECD (2015b), “The OECD Model Survey on ICT Access and Usage by Households and Individuals, 2nd Revision”, https://www.oecd.org/sti/ieconomy/ICT-Model-Survey-Access-Usage-Households-Individuals.pdf

OECD (2019a), “Measuring the Digital Transformation: A Roadmap for the Future”, https://doi.org/10.1787/9789264311992-en

OECD (2019b) “Guidelines for Supply-Use tables for the Digital Economy” SDD/CSSP/WPNA(2019)1/REV1, http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=SDD/CSSP/WPNA(2019)1/REV1&docLanguage=En

OECD (2019c) “Handbook on Measuring Digital Trade, Version 1” http://www.oecd.org/sdd/its/handbook-on-measuring-digital-trade.htm

OECD (2020), "Going Digital integrated policy framework", OECD Digital Economy Papers, No. 292, https://doi.org/10.1787/dc930adc-en

PARIS21 (2013), Home page, http://paris21.org.

Partnership on Measuring ICT for Development (2005), Core ICT Indicators, http://www.itu.int/ITU-D/ict/partnership/material/CoreICTIndicators.pdf.

Partnership on Measuring ICT for Development (2007), “Report of the Partnership on Measuring Information and Communication Technologies for Development: information and communication technology statistics”, Report to UN Statistical Commission, Thirty-eighth session, http://unstats.un.org/unsd/statcom/doc07/2007-5e-ICT.pdf.

Partnership on Measuring ICT for Development (2016), Core ICT Indicators, https://www.itu.int/en/ITU-D/Statistics/Documents/coreindicators/Core-List-of-Indicators_March2016.pdf.

Partnership on Measuring ICT for Development (2012), “Report of the Partnership on Measuring Information and Communication Technology for Development”, Report to UN Statistical Commission, Forty-third session, http://unstats.un.org/unsd/statcom/doc12/2012-12-ICT-E.pdf.

Partnership on Measuring ICT for Development (2014a), Final WSIS Targets Review: Achievements, Challenges and the Way Forward, https://www.itu.int/en/ITU-D/Statistics/Documents/publications/wsisreview2014/WSIS2014_review.pdf.

Partnership on Measuring ICT for Development (2014b), “Report of the Partnership on Measuring Information and Communication Technology for Development”, Report to UN Statistical Commission, Forty-fifth session, https://unstats.un.org/unsd/statcom/45th-session/documents/doc14/2014-8-ICT-E.pdf

Partnership on Measuring ICT for Development (2016), “Report of the Partnership on Measuring Information and Communication Technology for Development”, Report to UN Statistical Commission, Forty-seventh session, https://unstats.un.org/unsd/statcom/47th-session/documents/2016-13-Partnership-on-measuring-ICT-for-development-E.pdf.

Partnership on Measuring ICT for Development (2018), “Report of the Partnership on Measuring Information and Communication Technology for Development”, Report to UN Statistical Commission, Forty-ninth session, https://unstats.un.org/unsd/statcom/49th-session/documents/2018-26-ICT-E.pdf.

Partnership on Measuring ICT for Development and the United Nations Economic Commission for Africa (2012), Framework for a set of e-government core indicators, https://www.uneca.org/sites/default/files/PublicationFiles/framework-for-a-set-of-e-government-core-indicators_eng.pdf.

Partnership on Measuring ICT for Development and the United Nations Economic Commission for Africa (2014), Manual for measuring e-government (draft, unpublished). https://www.itu.int/en/ITU-D/Statistics/Documents/partnership/eGovernment_Manual_Final_2014.pdf

UNCTAD (United Nations Conference on Trade and Development) (2007), Manual for the Production of Statistics on the Information Economy.

UNCTAD (2008), Final report, “UNCTAD Mission to assist the Nepalese Government on ICT measurement” (unpublished).

UNCTAD (2009), Manual for the Production of Statistics on the Information Economy, revised 2009 edition, https://unctad.org/en/docs/sdteecb20072rev1_en.pdf.

UNCTAD (2015). International Trade in ICT Services and ICT-enabled Services: Proposed Indicators from the Partnership on Measuring ICT for Development, https://unctad.org/en/PublicationsLibrary/tn_unctad_ict4d03_en.pdf

UNCTAD (2019). Digital Economy Report 2019 - Value Creation and Capture: Implications for Developing Countries, https://unctad.org/en/PublicationsLibrary/der2019_en.pdf.

UNDESA (United Nations Department of Economic and Social Affairs) (2018), United Nations E-Government Survey 2018: Gearing e-government to support transformation towards sustainable and resilient societies, https://publicadministration.un.org/egovkb/Portals/egovkb/Documents/un/2018-Survey/E-Government%20Survey%202018_FINAL%20for%20web.pdf.

UNECLAC (United Nations Economic Commission for Latin America and the Caribbean) (2007), Compendium of Practices on the implementation of ICT questions in households and businesses surveys in Latin America and the Caribbean, UNECLAC-OSILAC, http://repositorio.cepal.org/bitstream/handle/11362/3806/1/S1000734_en.pdf.

UNESCAP (2018). ICT Statistics for Evidence-Based Policymaking in Pacific Island Countries. https://www.unescap.org/sites/default/files/ICT_Statistics_Guideline_FINAL_0.pdf

UNESCO (United Nations Educational, Scientific and Cultural Organization) (2011), International Standard Classification of Education, http://uis.unesco.org/sites/default/files/documents/international-standard-classification-of-education-isced-2011-en.pdf.

UIS (UNESCO Institute for Statistics) (2009), Guide to measuring information and communication technologies (ICT) in education, http://unesdoc.unesco.org/images/0018/001865/186547e.PDF.

UNSC (United Nations Statistical Commission) (1994), “United Nations Fundamental Principles of Official Statistics”, https://unstats.un.org/unsd/dnss/hb/E-fundamental%20principles_A4-WEB.pdf.

UNSC (2007), “Report on the Thirty-Eighth Session (27 February to 2 March 2007)”, E/2007/24 and E/CN.3/2007/30, http://unstats.un.org/unsd/statcom/doc07/FinalReport-Unedited.pdf.

UNSC (2012), “Report on the forty-third session (28 February-2 March 2012)”, E/2012/24 and E/CN.3/2012/34, http://unstats.un.org/unsd/statcom/doc12/2012-Report-E.pdf.

UNSD (United Nations Statistics Division)(2005a), Household Sample Surveys in Developing and Transition Countries, https://unstats.un.org/unsd/HHsurveys/pdf/Household_surveys.pdf.

UNSD (2005b), Designing Household Survey Samples: Practical Guidelines, https://unstats.un.org/unsd/demographic/sources/surveys/Handbook23June05.pdf.

UNSD (2008a), Principles and Recommendations for Population and Housing Censuses Revision 2.

UNSD (2008b), Updated System of National Accounts 1993 (1993 SNA): Volume 1: Chapters 1 – 17, https://unstats.un.org/unsd/nationalaccount/sna1993.asp.

UNSD (2017), Principles and Recommendations for Population and Housing Censuses Revision 3, https://unstats.un.org/unsd/demographic-social/Standards-and-Methods/files/Principles_and_Recommendations/Population-and-Housing-Censuses/Series_M67rev3-E.pdf.

UNSD (2018), Classification of Individual Consumption According to Purpose (COICOP) 2018. https://unstats.un.org/unsd/classifications/business-trade/desc/COICOP_english/COICOP_2018_-_pre-edited_white_cover_version_-_2018-12-26.pdf

World Bank (2013), Living Standards Measurement Study, Home page, http://www.worldbank.org/LSMS/.