DDDM not only benefits businesses but also enables governments to make better policy decisions. For instance, DDDM can be utilized to uncover hidden patterns, unexpected relationships, and market trends or reveal preferences that may have been difficult to discover previously. Armed with this information, government entities can make better decisions about healthcare, infrastructure, and finances than they could before. Read this article from the Executive Summary through Chapter 2 to explore data-driven decision models, how data is changing development, and how data can fill the holes in policymaking.
Supply: Data Connectivity and Capacity
Goodbye Data Carriers, Hello Data Creators
The rise of the internet has altered communications network value chains. In the past, little value was perceived in the content of traffic carried over communications networks. In the telecommunication world, this traffic was mainly telephone calls.
While some of the calls may have triggered wealth, the direct income accrued to telecommunication carriers that transmitted the calls and billed them. In the case of broadcast and private networks, content was more financially significant but intra-industry
(such as broadcast transmissions to satellite and cable television companies, banking transactions).
The development of the internet in the 1960s, the World Wide Web in the 1990s, and its iteration in Web 2.0 more recently has modified the way content is obtained and created.
Traditional content providers such as the media and audiovisual companies have moved online either with their own websites or by licensing content to streaming platforms. Take the BBC, which has 98 million global internet users viewing 1.5 billion pages
a month. Or O Globo, one of Brazil's largest newspapers, whose online readers (23 million) outnumber print readers (300,000) by 75 times. A big difference is that not only can anyone access content anywhere on the public internet, they can also create
it. Users become creators by sharing their own content with others through blogs, videos, social networking posts, and product and service reviews. Attention has shifted from the carrier of the data to the creator; from "the medium is the message"
to the messages delivered over the medium. A company's telephone number is arguably no longer as important as its website, and individuals increasingly exchange their email or social networking links. Similarly, in video entertainment, power is shifting
from the company broadcasting the content to the creator. This is reflected in the rise of companies offering internet-delivered video such as Amazon and Netflix and television content creators now embracing the internet (HBO NOW streaming service).
Many of the world's top websites (ranked by a combination of users and page views) are platforms for usergenerated content such as social network posts, video sharing, blogs, and collaboration (for example, Wikipedia) (table 2.1). All of the top sites are headquartered in either the United States or China, the two countries with the most internet users (more than 900 million combined, or just over a quarter of all internet users). While the US sites are mostly global, Chinese ones are mainly local. The concentration of so much data on so few sites is concerning, particularly as the giant internet companies behind most of them branch into other domains. Many aspire to be the single window to communications, news, and shopping.
Table 2.1 top 10 global websites
Site |
Description |
Daily time on site (minutes: seconds) |
Daily page views per visitor |
Percent of traffic from search |
Total sites linking in |
Users (millions), 2016 |
|
---|---|---|---|---|---|---|---|
1 | Internet portal |
8:45 | 8.63 | 2.3 | 3,011,003 | ~1,000 | |
2 | YouTube | Video sharing | 9:23 | 5.40 | 8.6 | 2,347,245 | ~1,000 |
3 | Social network | 13:56 | 5.32 | 4.4 | 7,278,321 | 1,860 | |
4 | Baidu | Search engine | 7:43 | 6.68 | 4.5 | 118,000 | 657 (2015) |
5 | Wikipedia | Encyclopedia | 4:26 | 3.31 | 36.9 | 1,287,362 | 374 (2015) |
6 | Yahoo | Internet portal | 4:28 | 3.90 | 5.3 | 529,800 | 650a |
7 | Instant messaging | 5:05 | 4.52 | 3.7 | 211,248 | 877 | |
8 | Taobao | E-commerce | 8:33 | 4.48 | 3.8 | 48,973 | 407 (2015)b |
9 | News links | 13:31 | 9.28 | 12.3 | 416,267 | 234 | |
10 | Tmall | E-commerce | 5:51 | 3.45 | 1.0 | 8,642 | 407 (2015)b |
Data centers: Greener and further away, or closer to home?
The growth of internet content is driving the need for
places to store it. A data center is a location with networked
computers providing remote storage, processing, and distribution of data. They are mainly operated by global IT
companies, governments, and enterprises that host other companies' data (that is, colocation). Data centers vary in
size, capability, security, and redundancy. A so-called tier
1 data center provides basic nonredundant connections
between computer equipment and may be prone to electrical outages, and a tier 4 center has redundant components, multiple connections between computers, continues to
operate during maintenance, and is protected against most
physical events.
Statistics vary widely about the number of data centers in
the world. One challenge is that most centers are "small racks
in computer rooms in smaller companies".
Although the number of data centers has grown rapidly,
growth is forecast to slow because of the trend toward larger
spaces. More information is available about giant data centers,
referred to as "hyperscale" because of their size and ability to
add servers and storage as needed. They are operated by about
two dozen global IT companies, including heavyweights such
as Amazon, Microsoft, and IBM, as well as enterprises providing cloud-computing services. The 259 hyperscale data centers
in 2015 are projected to grow to 485 by 2020 (figure 2.3).
Figure 2.3 Hyperscale data centers
Although the majority of hyperscale data centers are in developed nations, data center growth in emerging markets has ticked up. As more users are connected to the internet in lower-income nations, demand for data is rising. IP traffic is forecast to grow fastest in developing regions during 2015–20. Some countries are concerned about data sovereignty, insisting that government data be stored in the country, driving demand for national data centers. Software parks have also become popular in developing nations as a way to grow their digital economies, and data centers are essential for these facilities. Although the operation of a data center does not create many jobs, they are an essential platform for companies using them to generate revenue and employment.
Telecommunications carriers are particularly keen on data centers as a way to offset declining revenue from traditional voice services. Japan's biggest carrier, NTT, is one of the largest data center operators in the world, with more than 140 across the
globe. Leading carriers have formed a working group of the Open Compute Project for the adoption of common standards for data centers. Operators in developing nations from Paraguay to the Philippines are busy constructing state-of-the-art data centers.
Mobile group Millicom has been launching data centers in African countries, and its Paraguayan data center won an award in 2016 for its modular design. The Philippines Long Distance Telephone Company has constructed eight data centers across the country
to be close to IT parks and support its cloud-based service offerings.
But data centers require significant electricity to power and keep equipment cool. According to a study, data centers in the United States accounted for 2 percent of that country's electricity consumption in 2014.
The data center industry is therefore constantly looking for ways to reduce reliance on fossil fuels, particularly given the possibility of data rationing due to shortages of electricity:
If governments and companies decide to rely upon increased energy generation, they will not be able to keep up with the demands of big data without significantly contributing to environmental pollution levels. In this future, how would the world look? Would governments step in to regulate Facebook usage, only in daylight hours? Would citizens have the right to only 12 Google searches per day? Should we tax companies on their levels of data usage? This might seem laughable now, but data rationing is a likely outcome if we do not tackle data growth and the underlying demands placed on power consumption.
This has made geographies with cool climates and abundant hydro or geothermal energy attractive locations for data centers. Google's Finnish data center is built in a restored machine hall designed by renowned architect Alvar Aalto and draws on water
from the Bay of Finland for cooling. Its data center in West Dublin does not need air conditioning units because of Ireland's cool climate. Some developing nations have similar environments, making them ideal for data centers. The Data Center Services
data center in the Thimpu TechPark draws on mountainous Bhutan's abundant hydropower and year-round cool climate and, in 2017, the government launched its first data center.
Many developing countries face a challenge competing with hyperscale data centers abroad given that electricity costs tend to be relatively high. In Rwanda, the government is considering subsidizing data center electricity costs to attract more digital
companies to the country and for local firms to transition their websites to local hosting enterprises. Small island developing states generally have high electricity costs due to the absence of local energy sources: 8 of the 10 most expensive countries
for electricity are such states. However, they are surrounded by a useful resource: cool seawater. As noted, Google uses seawater to cool its Finnish data center and Microsoft is testing underwater data centers. Mauritius has also experimented with
ocean water to cool data centers. Some Pacific small island developing states and other coastal economies have taken advantage of new submarine cable connections to bundle data centers into the landing station, lowering construction costs. Samoa,
which has among the highest electricity costs in the world (figure 2.4), installed an energy efficient prefabricated data center in its new cable landing station. Many developing nations also have abundant sunshine with great potential for solar energy
to lower costs. This is the thinking of mobile group MTN, which deployed Africa's first solar data center at its head office in Johannesburg.
Figure 2.4 Price of electricity (US cents per kilowatt-hour)
Reliability is critical for data centers. Developing countries, particularly in Africa, will need to improve the quality
of the electricity supply to create the proper environment to
attract investment in data centers (box 2.1). This will require
electricity sector reform and prioritizing reliability for firms.
Box 2.1 Sub-Saharan Africa: Reliable electricity and the digital economy
Many countries in Sub-Saharan Africa seek to diversify their economies with information and communication technologies (ICT), including expanding ICT as a sector and increasing its use in enterprises. The data center is a core element of ICT infrastructure. These facilities are a vital engine of the digital economy, storing data, hosting websites, and enabling cloud-based applications. Data centers are virtual data factories that make productive use of electricity, with measurable economic impact on gross domestic product, employment, and government tax revenue.
Data centers consume lots of electricity to power computer equipment and keep it cool. In 2011, Google reported that it used 260 megawatts of electric power for its data centers, which is greater than the 2014 installed capacity in 19 Sub-Saharan African countries. Data centers require high reliability to ensure seamless, nonstop data flow. Reliability is defined by industry standards, ranging from 99.670 percent availability with no more than 29 hours of interruption per year for tier 1 data centers, to 99.995 percent reliability with just 0.8 hour of interruption per year for the highest, tier 4 centers. Most Sub-Saharan African nations would find it difficult to meet even tier 1 reliability. The standards also call for a guaranteed source of electrical backup that can power the center for at least half a day.
Lack of enterprise-grade reliability requirements for industry certification generally rules out the feasibility of large data centers in many Sub-Saharan African countries. Although virtually every country in the region has a data center, the centers are small, serving a narrow set of business and government users. Because of the region's challenging environment for reliable and inexpensive electricity, most businesses host their data outside the region. This results in a large volume of data transmitted to overseas data centers, requiring significant amounts of international internet bandwidth. Along with connectivity and storage costs, it takes a longer time to access overseas data centers, raising latency. Security is also an issue, as increasing amounts of government, business, and personal information are transmitted abroad, with vague data protection.
To build up its national data center industry and improve latency, Rwanda launched an initiative to repatriate 1,000 websites hosted abroad. An analysis of the program found that quality was improved for domestic users because of faster access to the sites. Visitor engagement was high, with more page views and return visits due to the enhanced performance. The skills of web-hosting employees increased, due to technical requirements to manage additional websites. Although latency improved, it is still difficult to convince local businesses to place their websites in Rwanda because of the lower price of hosting overseas. This is primarily because of the high cost of electricity for data centers in the country. The government is contemplating subsidizing the cost of electricity for local data centers to make local hosting more attractive, improve latency, and strengthen data sovereignty.
Despite concerns about reliability, interest is growing in installing large data centers in the region to achieve better latency and reduce the cost of international bandwidth. In 2017, Microsoft, one of the world's largest owners of data centers, announced it would build two in South Africa to support its cloud-based services. Notably, South Africa's electricity supply is considered the second most reliable in the region after Mauritius. The new data centers will be faster than accessing cloud services in Europe or the United States, international connectivity costs will be lower, and trust higher, as the centers will have to comply with South Africa's data protection law. Electricity reliability is critical for other countries in the region that want to develop their digital economies.
IXPs and caches: Closer to the edge
Although trends suggest a move toward larger data centers,
the tendency is toward pushing data closer to the user or
the "edge" to reduce latency and lower costs.
Having data close to end users is critical, particularly in the
financial sector, in which a few milliseconds advantage has a
huge potential impact. This is raising traffic
on internet exchange points (IXPs), places where telecom
carriers and content providers come together to exchange
their traffic (peering). This is cheaper, particularly for developing countries, since internetwork traffic does not need
to be sent over costly international links only to return. In
addition, ISPs do not need to make peering agreements with
each potential partner. IXPs also improve quality since they
are situated closer to the user and hence have less latency.
"Soft" benefits are also associated with IXPs, such as developing technical skills and fostering a culture of cooperation, helping to sustain the internet. As the volume of data
transmitted over the internet accelerates, IXPs have become
even more relevant for ensuring that it is quickly exchanged
among different parties.
The largest IXPs (measured by traffic or members) are
mainly in Europe, with its long tradition of multistakeholder
internet cooperation. The biggest is the German Internet
Exchange (DE-CIX) founded in 1995, with locations in
Dusseldorf, Frankfurt, Hamburg, and Munich. DE-CIX
Frankfurt is the world's leading internet exchange, with peak
traffic of 5.6 terabits per second in March 2017. More than
700 networks are connected, and access is available from
20 data centers across the city. The networks connected to
DE-CIX are a smorgasbord of giant telecommunication
carriers (such as Deutsche Telekom, China Telecom, Verizon, and NTT) and emerging country operators (such as
Sri Lanka Telecom, Telkom South Africa, and Telkom Indonesia), big IT firms (such as Apple, Google, and Microsoft),
and content and service providers (such as eBay and Facebook). DE-CIX began expanding abroad in 2012 and now
operates IXPs in Dallas, Dubai, Istanbul, Madrid, Marseille,
New York, and Palermo.
Although IXPs are burgeoning in most developed markets,
growth has been uneven in developing nations. According to
one source, 78 economies are still without an IXP (map 2.1).
Map 2.1 Internet exchange points around the world, 2018
The establishment of an IXP is often hampered by small markets, vested interests, and limited or unbalanced competition. Powerful incumbents with a high level of control over international gateways prefer that ISPs use their overseas links for IP transit. Nevertheless, developing regions, such as Latin America and Africa, have been adding ISPs at relatively high levels (table 2.2). Where no powerful incumbent exists, IXPs can thrive. This is the case of the Rwanda IXP, where the historical operator no longer exists. The IXP has 13 members, including all of the country's infrastructure-based ISPs. Peak traffic load was over 1 gigabit per second in March 2017, up more than 50 percent from the previous year. An IXP is particularly relevant in landlocked countries like Rwanda, which is far from undersea fiber-optic cables.
Table 2.2 Internet exchange points by region
Internet exchange points | Domestic bandwidth production | |||||||
Region | February 2016 |
February 2017 |
Net change |
Percent change |
February 2016 |
February 2017 |
Net change |
Percent change |
Europe | 136 | 175 | +39 | +29 | 35.9T | 41.8T | +5.84T | +16 |
North America | 81 | 94 | +13 | +16 | 3.41T | 4.55T | +1.14T | +33 |
Asia and Pacific | 75 | 92 | +17 | +23 | 2.56T | 3.51T | +953G | +37 |
Latin America | 45 | 72 | +27 | +60 | 2.22T | 2.75T | +524G | +24 |
Africa | 30 | 42 | +12 | +40 | 325G | 417G | +92.6G | +29 |