Committed to connecting the world

ICTs for a Sustainable World #ICT4SDG

The ICT Development Index (IDI): conceptual framework and methodology

The ICT Development Index (IDI), which has been published annually since 2009, is a composite index that combines 11 indicators into one benchmark measure. It is used to monitor and compare developments in information and communication technology (ICT) between countries and over time.

The main objectives of the IDI are to measure:

The Index is designed to be global and reflect changes taking place in countries at different levels of ICT development. It therefore relies on a limited set of data which can be established with reasonable confidence in countries at all levels of development.

Recognizing that ICTs can be development enablers is central to the IDI’s conceptual framework. The ICT development process, and a country’s evolution towards becoming an information society, can be depicted using the three-stage model illustrated in Figure 1: 

Figure 1: Three stages in the evolution towards an information society

Source: ITU.

Based on this conceptual framework, the IDI is divided into the following three sub-indices, and a total of 11 indicators (Figure 2).

Figure 2: ICT Development Index: indicators, reference values and weights

Note: *This corresponds to a log value of 6.33, which was used in the normalization step.
Source: ITU.

The following provides more details on various steps involved, such as the indicators included in the index and their definition, the imputation of missing data, the normalization procedure, the weights applied to the indicators and sub-indices, and the results of the sensitivity analysis.

1. Indicators included in the IDI

The selection of indicators was based on certain criteria, including relevance for the Index objectives, data availability and the results of various statistical analyses such as the principal component analysis (PCA).1 The following 11 indicators are included in the IDI (grouped by the three sub-indices: access, usage and skills).

a) ICT infrastructure and access indicators

Indicators included in this group provide an indication of the available ICT infrastructure and individuals’ access to basic ICTs. Data for all these indicators are collected by ITU.2

1. Fixed-telephone subscriptions per 100 inhabitants

The term “fixed-telephone subscriptions” refers to the sum of active analogue fixed-telephone lines, voice-over-Internet Protocol (VoIP) subscriptions, fixed wireless local loop subscriptions, Integrated Services Digital Network voice-channel equivalents and fixed public payphones. It includes all accesses over fixed infrastructure supporting voice telephony using copper wire, voice services using Internet Protocol (IP) delivered over fixed (wired)-broadband infrastructure (e.g. digital subscriber line (DSL), fibre optic), and voice services provided over coaxial-cable television networks (cable modem). It also includes fixed wireless local loop connections, defined as services provided by licensed fixed-line telephone operators that provide last-mile access to the subscriber using radio technology, where the call is then routed over a fixed-line telephone network (not a mobile-cellular network). VoIP refers to subscriptions that offer the ability to place and receive calls at any time and do not require a computer. VoIP is also known as voice-over-broadband (VoB), and includes subscriptions through fixed-wireless, DSL, cable, fibre optic and other fixed-broadband platforms that provide fixed telephony using IP.

2. Mobile-cellular telephone subscriptions per 100 inhabitants

The term “mobile-cellular telephone subscriptions” refers to the number of subscriptions to a public mobile telephone service providing access to the public switched telephone network using cellular technology. It includes both the number of postpaid subscriptions and the number of active prepaid accounts (i.e. accounts that have been active during the previous three months). It includes all mobile-cellular subscriptions that offer voice communications. It excludes subscriptions via data cards or USB modems, subscriptions to public mobile data services, private trunked mobile radio, telepoint, radio paging, machine-to-machine (M2M) and telemetry services.

3. International Internet bandwidth (bit/s) per Internet user

The term “international Internet bandwidth” refers to the total used capacity of international Internet bandwidth, in megabits per second (Mbit/s). Used international Internet bandwidth refers to the average usage of all international links, including fibre optic cables, radio links and traffic processed by satellite ground stations and teleports to orbital satellites (expressed in Mbit/s). All international links used by all types of operators – namely fixed, mobile and satellite operators are taken into account. The average is calculated over the 12-month period of the reference year. For each individual international link, if the traffic is asymmetric, i.e. incoming traffic is not equal to outgoing traffic, then the higher value of the two is provided. The combined average usage of all international links can be reported as the sum of the average usage of each individual link. International Internet bandwidth (bit/s) per Internet user is calculated by converting to bits per second and dividing by the total number of Internet users.

4. Percentage of households with a computer

“Computer” refers to a desktop computer, laptop (portable) computer, tablet or similar handheld computer. It does not include equipment with some embedded computing abilities, such as smart television sets, or devices with telephony as a main function, such as mobile phones or smartphones.
Household with a computer means that the computer is available for use by all members of the household at any time. The computer may or may not be owned by the household, but should be considered a household asset.

Data are obtained by countries through national household surveys and are either provided directly to ITU by national statistical offices (NSOs) or obtained by ITU through its own research, for example, from NSO websites. There are certain data-related limits to this indicator, insofar as estimates have to be calculated for many developing countries that do not yet collect ICT household statistics. Over time, as more data become available, the quality of the indicator will improve.

5. Percentage of households with Internet access

The Internet is a worldwide public computer network. It provides access to a number of communication services, including the World Wide Web, and carries e-mail, news, entertainment and data files, irrespective of the device used (not assumed to be only a computer; it may also be a mobile telephone, tablet, PDA, games machine, digital television, etc.). Access can be via a fixed or mobile network. Household with Internet access means that the Internet is available for use by all members of the household at any time.

Data are obtained by countries through national household surveys and are either provided directly to ITU by NSOs or obtained by ITU through its own research, for example from NSO websites. There are certain data-related limits to this indicator, insofar as estimates have to be calculated for many developing countries which do not yet collect ICT household statistics. Over time, as more data become available, the quality of the indicator will improve.
  

b) ICT usage indicators

The indicators included in this group capture ICT intensity and usage. Data for all these indicators are collected by ITU.3

1. Percentage of individuals using the Internet

The term “individuals using the Internet” refers to people who used the Internet from any location and for any purpose, irrespective of the device and network used, in the previous three months. Usage can be via a computer (i.e. desktop computer, laptop computer, tablet or similar handheld computer), mobile phone, games machine, digital television, etc.). Access can be via a fixed or mobile network.

Data are obtained by countries through national household surveys and are either provided directly to ITU by NSOs or obtained by ITU through its own research, for example, from NSO websites. There are certain data-related limits to this indicator, insofar as estimates have to be calculated for many developing countries which do not yet collect ICT household statistics. Over time, as more data become available, the quality of the indicator will improve.

2. Fixed-broadband subscriptions per 100 inhabitants

The term “fixed-broadband subscriptions” refers to fixed subscriptions for high-speed access to the public Internet (a Transmission Control Protocol (TCP)/IP connection) at downstream speeds equal to or higher than 256 kbit/s. This includes cable modem, DSL, fibre-to-the-home/building, other fixed (wired)-broadband subscriptions, satellite broadband and terrestrial fixed wireless broadband. The total is measured irrespective of the method of payment. It excludes subscriptions that have access to data communications (including the Internet) via mobile-cellular networks. It includes fixed WiMAX and any other fixed wireless technologies, and both residential subscriptions and subscriptions for organizations.

3. Active mobile-broadband subscriptions per 100 inhabitants

The term “active mobile-broadband subscriptions” refers to the sum of data and voice mobile-broadband subscriptions and data-only mobile-broadband subscriptions to the public Internet. It covers subscriptions actually used to access the Internet at broadband speeds, not subscriptions with potential access, even though the latter may have broadband-enabled handsets. Subscriptions must include a recurring subscription fee to access the Internet or pass a usage requirement – users must have accessed the Internet in the previous three months. It includes subscriptions to mobile-broadband networks that provide download speeds of at least 256 kbit/s (e.g. WCDMA, HSPA, CDMA2000 1x EV-DO, WiMAX IEEE 802.16e and LTE), and excludes subscriptions that only have access to GPRS, EDGE and CDMA 1xRTT. 

c) ICT skills indicators

Data on mean years of schooling and gross secondary and tertiary enrolment ratios are collected by the United Nations Educational, Scientific and Cultural Organization Institute for Statistics.

1. Mean years of schooling rate

The term “mean years of schooling” is the average number of completed years of education of a country’s population, excluding years spent repeating individual grades. It is estimated using the distribution of the population by age group and the highest level of education attained in a given year, and time series data on the official duration of each level of education.4

2. Gross enrolment ratio (secondary and tertiary level)

According to the United Nations Educational, Scientific and Cultural Organization Institute for Statistics, the gross enrolment ratio is “the total enrolment in a specific level of education, regardless of age, expressed as a percentage of the eligible official school-age population corresponding to the same level of education in a given school-year.”

2. Imputation of missing data

A critical step in the construction of the Index is to create a complete data set, without missing values. A number of imputation techniques can be applied to estimate missing data.5

Each of the imputation techniques, like any other method employed in the process, has its own strengths and weaknesses. The most important consideration is to ensure that the imputed data will reflect a country’s actual level of ICT access, usage and skills.

Imputation of missing data for access and use of ICTs by households and individuals were made by applying multiple imputation models based on multiple regression techniques using actual data from more than 100 countries. The approach took into consideration many explanatory variables of ICT development, such as national income, fixed and mobile-broadband penetration, geographic affiliation, population distribution and the proportion of the population living in urban areas, and the level of 3G and 4G/LTE population coverage. However, given that ICT access and usage are both highly correlated with national income, gross national income per capita was the most determinant variable for imputing access and use of ICTs by households and individuals. The availability of relevant data from other sources was also taken into account during the imputation process.   

For example, the proportion of individuals using the Internet was estimated based on all available information on Internet use in the country from household surveys, and if no historical information was available, explanatory variables to estimate the level of Internet use were used. The estimates obtained from the regression analysis were benchmarked against estimates from other models, and against other countries in the region with similar characteristics, as well as against other key variables and data sources.

3. Normalization of data

Data normalization must be effected before any aggregation can take place, to ensure that the data set uses the same unit of measurement. Regarding the indicators selected to construct the IDI, the values must be converted into the same unit of measurement, since some values are expressed as a percentage of the population/total households, where the maximum value is 100, while other indicators can have values exceeding 100, such as mobile-cellular and active mobile-broadband penetration or international Internet bandwidth (expressed as bit/s per user).

Certain particularities need to be taken into consideration in selecting the normalization method for the IDI. For example, to identify the digital divide, it is important to measure the relative performance of countries (i.e. the divide among countries). Secondly, the normalization procedure should produce Index results that allow countries to track progress in their evolution towards an information society over time.

A further important criterion in selecting the normalization method is replicability by countries, as some countries have shown a strong interest in applying the Index methodology at the national or regional level. Certain methods therefore cannot be applied, for example, those that rely on the values of other countries, which might not be available to users.

For the IDI, the distance to a reference measure was used as the normalization method. The reference measure is the ideal value that could be reached for each variable (similar to a “goalpost”). For all the indicators chosen, this will be 100, except in regard to the following five indicators:

After normalizing the data, the individual series were all rescaled to identical ranges, from 1 to 10. This was necessary to compare the values of the indicators and the sub-indices.


4. Weighting and aggregation

The indicators and sub-indices included in the IDI were weighted on the basis of the PCA results obtained when the Index was first computed.7   

Weights
(indicators)​
Weights
(sub indices)
ICT access ​0.40
Fixed-telephone subscriptions per 100 inhabitants ​0.20
Mobile-cellular telephone subscriptions per 100 inhabitants ​0.20
International Internet bandwidth per Internet user ​0.20
Percentage of households with a computer ​0.20
Percentage of households with Internet access ​0.20
ICT usage ​0.40
Percentage of individuals using the Internet ​0.33
Fixed-broadband Internet subscriptions per 100 inhabitants ​0.33
Active mobile-broadband subscriptions per 100 inhabitants ​0.33
ICT skills ​0.20
Mean years of schooling ​0.33
Secondary gross enrolment ratio ​0.33
Tertiary gross enrolment ratio ​0.33

Source: ITU.

5. Calculating the IDI

Sub-indices were computed by summing the weighted values of the indicators included in the respective subgroup.

The values of the sub-indices were calculated first by normalizing the indicators included in each sub-index in order to obtain the same unit of measurement. The reference values applied in the normalization process were discussed above. The sub-index value was calculated by taking the simple average (using equal weighting) of the normalized indicator values.

For computation of the final Index, the ICT access and ICT usage sub-indices were each given a 40 per cent weighting, and the skills sub-index (because it is based on proxy indicators) a 20 per cent weighting. The final Index value was then computed by summation of the weighted sub-indices. 
       

6. Sensitivity analysis

Sensitivity analysis was carried out to investigate the robustness of the Index results in terms of the relative position in the overall ranking, using different combinations of methods and techniques to compute the Index.

Potential sources of variation or uncertainty can be attributed to different processes employed in the computation of the Index, including the selection of individual indicators, the imputation of missing values and the normalization, weighting and aggregation of the data.

Each of the processes or combination of processes affects the IDI value. A number of tests were carried out to examine the robustness of the IDI results (rather than the actual values). The tests computed the possible Index values and country rankings for different combinations of the processes mentioned above. Results show that, while the computed Index values change, the message remains the same. The IDI was found to be extremely robust with regard to different methodologies, with the exception of certain countries, including in particular those in the “high” group.

The relative position of countries included in the “high” group can change depending on the methodology used. Caution should therefore be exercised in drawing conclusions based on these countries’ rankings. However, the relative position of countries included in the “low” group is in no way affected by the methods or techniques used, and the countries in this group ranked low in all Index computations using different methodologies. This confirms the results conveyed by the IDI. 
 

-------------------
Notes:

1 PCA was used to examine the underlying nature of the data. A more detailed description of the analysis is available in Annex 1 to the 2009 report Measuring the Information Society – The ICT Development Index (ITU, 2009).

2 More information about the indicators is available in the ITU Handbook for the collection of administrative data on telecommunications/ICT (ITU, 2011) and the ITU Manual for Measuring ICT Access and Use by Households and Individuals (ITU, 2014).

3 See footnote 2.

4 See http://www.uis.unesco.org/Education/Documents/Mean-years-schooling-indicator-methodology-en.pdf. Data used in the calculation of the Index were based from the United Nations Development Programme Human Development Index 2016, available from http://hdr.undp.org/en/2016-report.

5 See OECD and European Commission (2008). 

6 See Human Development Report 2015, Technical Notes, available at http://hdr.undp.org/sites/default/files/hdr2015_technical_
notes.pdf
.

7  For more details, see Annex 1 to ITU (2009).