Supply: Data Connectivity and Capacity

Goodbye Data Carriers, Hello Data Creators

The rise of the internet has altered communications network value chains. In the past, little value was perceived in the content of traffic carried over communications networks. In the telecommunication world, this traffic was mainly telephone calls. While some of the calls may have triggered wealth, the direct income accrued to telecommunication carriers that transmitted the calls and billed them. In the case of broadcast and private networks, content was more financially significant but intra-industry (such as broadcast transmissions to satellite and cable television companies, banking transactions).

The development of the internet in the 1960s, the World Wide Web in the 1990s, and its iteration in Web 2.0 more recently has modified the way content is obtained and created.

Traditional content providers such as the media and audiovisual companies have moved online either with their own websites or by licensing content to streaming platforms. Take the BBC, which has 98 million global internet users viewing 1.5 billion pages a month. Or O Globo, one of Brazil's largest newspapers, whose online readers (23 million) outnumber print readers (300,000) by 75 times. A big difference is that not only can anyone access content anywhere on the public internet, they can also create it. Users become creators by sharing their own content with others through blogs, videos, social networking posts, and product and service reviews. Attention has shifted from the carrier of the data to the creator; from "the medium is the message" to the messages delivered over the medium. A company's telephone number is arguably no longer as important as its website, and individuals increasingly exchange their email or social networking links. Similarly, in video entertainment, power is shifting from the company broadcasting the content to the creator. This is reflected in the rise of companies offering internet-delivered video such as Amazon and Netflix and television content creators now embracing the internet (HBO NOW streaming service).

Many of the world's top websites (ranked by a combination of users and page views) are platforms for usergenerated content such as social network posts, video sharing, blogs, and collaboration (for example, Wikipedia) (table 2.1). All of the top sites are headquartered in either the United States or China, the two countries with the most internet users (more than 900 million combined, or just over a quarter of all internet users). While the US sites are mostly global, Chinese ones are mainly local. The concentration of so much data on so few sites is concerning, particularly as the giant internet companies behind most of them branch into other domains. Many aspire to be the single window to communications, news, and shopping.

Table 2.1 top 10 global websites


Site
Description
Daily time on
 site (minutes:
seconds)
Daily page
views per
visitor
Percent of
traffic from
search
Total sites
linking in
Users
(millions), 2016
1 Google Internet portal
8:45 8.63 2.3 3,011,003 ~1,000
2 YouTube Video sharing 9:23 5.40 8.6 2,347,245 ~1,000
3 Facebook Social network 13:56 5.32 4.4 7,278,321 1,860
4 Baidu Search engine 7:43 6.68 4.5 118,000 657 (2015)
5 Wikipedia Encyclopedia 4:26 3.31 36.9 1,287,362 374 (2015)
6 Yahoo Internet portal 4:28 3.90 5.3 529,800 650a
7 Qq Instant messaging 5:05 4.52 3.7 211,248 877
8 Taobao E-commerce 8:33 4.48 3.8 48,973 407 (2015)b
9 Reddit News links 13:31 9.28 12.3 416,267 234
10 Tmall E-commerce 5:51 3.45 1.0 8,642 407 (2015)b


Data centers: Greener and further away, or closer to home?

The growth of internet content is driving the need for places to store it. A data center is a location with networked computers providing remote storage, processing, and distribution of data. They are mainly operated by global IT companies, governments, and enterprises that host other companies' data (that is, colocation). Data centers vary in size, capability, security, and redundancy. A so-called tier 1 data center provides basic nonredundant connections between computer equipment and may be prone to electrical outages, and a tier 4 center has redundant components, multiple connections between computers, continues to operate during maintenance, and is protected against most physical events.

Statistics vary widely about the number of data centers in the world. One challenge is that most centers are "small racks in computer rooms in smaller companies". Although the number of data centers has grown rapidly, growth is forecast to slow because of the trend toward larger spaces. More information is available about giant data centers, referred to as "hyperscale" because of their size and ability to add servers and storage as needed. They are operated by about two dozen global IT companies, including heavyweights such as Amazon, Microsoft, and IBM, as well as enterprises providing cloud-computing services. The 259 hyperscale data centers in 2015 are projected to grow to 485 by 2020 (figure 2.3).

Figure 2.3 Hyperscale data centers


Although the majority of hyperscale data centers are in developed nations, data center growth in emerging markets has ticked up. As more users are connected to the internet in lower-income nations, demand for data is rising. IP traffic is forecast to grow fastest in developing regions during 2015–20. Some countries are concerned about data sovereignty, insisting that government data be stored in the country, driving demand for national data centers. Software parks have also become popular in developing nations as a way to grow their digital economies, and data centers are essential for these facilities. Although the operation of a data center does not create many jobs, they are an essential platform for companies using them to generate revenue and employment.

Telecommunications carriers are particularly keen on data centers as a way to offset declining revenue from traditional voice services. Japan's biggest carrier, NTT, is one of the largest data center operators in the world, with more than 140 across the globe. Leading carriers have formed a working group of the Open Compute Project for the adoption of common standards for data centers. Operators in developing nations from Paraguay to the Philippines are busy constructing state-of-the-art data centers. Mobile group Millicom has been launching data centers in African countries, and its Paraguayan data center won an award in 2016 for its modular design. The Philippines Long Distance Telephone Company has constructed eight data centers across the country to be close to IT parks and support its cloud-based service offerings.

But data centers require significant electricity to power and keep equipment cool. According to a study, data centers in the United States accounted for 2 percent of that country's electricity consumption in 2014.

The data center industry is therefore constantly looking for ways to reduce reliance on fossil fuels, particularly given the possibility of data rationing due to shortages of electricity:

If governments and companies decide to rely upon increased energy generation, they will not be able to keep up with the demands of big data without significantly contributing to environmental pollution levels. In this future, how would the world look? Would governments step in to regulate Facebook usage, only in daylight hours? Would citizens have the right to only 12 Google searches per day? Should we tax companies on their levels of data usage? This might seem laughable now, but data rationing is a likely outcome if we do not tackle data growth and the underlying demands placed on power consumption.

This has made geographies with cool climates and abundant hydro or geothermal energy attractive locations for data centers. Google's Finnish data center is built in a restored machine hall designed by renowned architect Alvar Aalto and draws on water from the Bay of Finland for cooling. Its data center in West Dublin does not need air conditioning units because of Ireland's cool climate. Some developing nations have similar environments, making them ideal for data centers. The Data Center Services data center in the Thimpu TechPark draws on mountainous Bhutan's abundant hydropower and year-round cool climate and, in 2017, the government launched its first data center.

Many developing countries face a challenge competing with hyperscale data centers abroad given that electricity costs tend to be relatively high. In Rwanda, the government is considering subsidizing data center electricity costs to attract more digital companies to the country and for local firms to transition their websites to local hosting enterprises. Small island developing states generally have high electricity costs due to the absence of local energy sources: 8 of the 10 most expensive countries for electricity are such states. However, they are surrounded by a useful resource: cool seawater. As noted, Google uses seawater to cool its Finnish data center and Microsoft is testing underwater data centers. Mauritius has also experimented with ocean water to cool data centers. Some Pacific small island developing states and other coastal economies have taken advantage of new submarine cable connections to bundle data centers into the landing station, lowering construction costs. Samoa, which has among the highest electricity costs in the world (figure 2.4), installed an energy efficient prefabricated data center in its new cable landing station. Many developing nations also have abundant sunshine with great potential for solar energy to lower costs. This is the thinking of mobile group MTN, which deployed Africa's first solar data center at its head office in Johannesburg.

Figure 2.4 Price of electricity (US cents per kilowatt-hour)


Reliability is critical for data centers. Developing countries, particularly in Africa, will need to improve the quality of the electricity supply to create the proper environment to attract investment in data centers (box 2.1). This will require electricity sector reform and prioritizing reliability for firms.

Box 2.1 Sub-Saharan Africa: Reliable electricity and the digital economy

Many countries in Sub-Saharan Africa seek to diversify their economies with information and communication technologies (ICT), including expanding ICT as a sector and increasing its use in enterprises. The data center is a core element of ICT infrastructure. These facilities are a vital engine of the digital economy, storing data, hosting websites, and enabling cloud-based applications. Data centers are virtual data factories that make productive use of electricity, with measurable economic impact on gross domestic product, employment, and government tax revenue.

Data centers consume lots of electricity to power computer equipment and keep it cool. In 2011, Google reported that it used 260 megawatts of electric power for its data centers, which is greater than the 2014 installed capacity in 19 Sub-Saharan African countries. Data centers require high reliability to ensure seamless, nonstop data flow. Reliability is defined by industry standards, ranging from 99.670 percent availability with no more than 29 hours of interruption per year for tier 1 data centers, to 99.995 percent reliability with just 0.8 hour of interruption per year for the highest, tier 4 centers. Most Sub-Saharan African nations would find it difficult to meet even tier 1 reliability. The standards also call for a guaranteed source of electrical backup that can power the center for at least half a day.

Lack of enterprise-grade reliability requirements for industry certification generally rules out the feasibility of large data centers in many Sub-Saharan African countries. Although virtually every country in the region has a data center, the centers are small, serving a narrow set of business and government users. Because of the region's challenging environment for reliable and inexpensive electricity, most businesses host their data outside the region. This results in a large volume of data transmitted to overseas data centers, requiring significant amounts of international internet bandwidth. Along with connectivity and storage costs, it takes a longer time to access overseas data centers, raising latency. Security is also an issue, as increasing amounts of government, business, and personal information are transmitted abroad, with vague data protection.

To build up its national data center industry and improve latency, Rwanda launched an initiative to repatriate 1,000 websites hosted abroad. An analysis of the program found that quality was improved for domestic users because of faster access to the sites. Visitor engagement was high, with more page views and return visits due to the enhanced performance. The skills of web-hosting employees increased, due to technical requirements to manage additional websites. Although latency improved, it is still difficult to convince local businesses to place their websites in Rwanda because of the lower price of hosting overseas. This is primarily because of the high cost of electricity for data centers in the country. The government is contemplating subsidizing the cost of electricity for local data centers to make local hosting more attractive, improve latency, and strengthen data sovereignty.

Despite concerns about reliability, interest is growing in installing large data centers in the region to achieve better latency and reduce the cost of international bandwidth. In 2017, Microsoft, one of the world's largest owners of data centers, announced it would build two in South Africa to support its cloud-based services. Notably, South Africa's electricity supply is considered the second most reliable in the region after Mauritius. The new data centers will be faster than accessing cloud services in Europe or the United States, international connectivity costs will be lower, and trust higher, as the centers will have to comply with South Africa's data protection law. Electricity reliability is critical for other countries in the region that want to develop their digital economies.


IXPs and caches: Closer to the edge

Although trends suggest a move toward larger data centers, the tendency is toward pushing data closer to the user or the "edge" to reduce latency and lower costs. Having data close to end users is critical, particularly in the financial sector, in which a few milliseconds advantage has a huge potential impact. This is raising traffic on internet exchange points (IXPs), places where telecom carriers and content providers come together to exchange their traffic (peering). This is cheaper, particularly for developing countries, since internetwork traffic does not need to be sent over costly international links only to return. In addition, ISPs do not need to make peering agreements with each potential partner. IXPs also improve quality since they are situated closer to the user and hence have less latency. "Soft" benefits are also associated with IXPs, such as developing technical skills and fostering a culture of cooperation, helping to sustain the internet. As the volume of data transmitted over the internet accelerates, IXPs have become even more relevant for ensuring that it is quickly exchanged among different parties.

The largest IXPs (measured by traffic or members) are mainly in Europe, with its long tradition of multistakeholder internet cooperation. The biggest is the German Internet Exchange (DE-CIX) founded in 1995, with locations in Dusseldorf, Frankfurt, Hamburg, and Munich. DE-CIX Frankfurt is the world's leading internet exchange, with peak traffic of 5.6 terabits per second in March 2017. More than 700 networks are connected, and access is available from 20 data centers across the city. The networks connected to DE-CIX are a smorgasbord of giant telecommunication carriers (such as Deutsche Telekom, China Telecom, Verizon, and NTT) and emerging country operators (such as Sri Lanka Telecom, Telkom South Africa, and Telkom Indonesia), big IT firms (such as Apple, Google, and Microsoft), and content and service providers (such as eBay and Facebook). DE-CIX began expanding abroad in 2012 and now operates IXPs in Dallas, Dubai, Istanbul, Madrid, Marseille, New York, and Palermo.

Although IXPs are burgeoning in most developed markets, growth has been uneven in developing nations. According to one source, 78 economies are still without an IXP (map 2.1).

Map 2.1 Internet exchange points around the world, 2018


The establishment of an IXP is often hampered by small markets, vested interests, and limited or unbalanced competition. Powerful incumbents with a high level of control over international gateways prefer that ISPs use their overseas links for IP transit. Nevertheless, developing regions, such as Latin America and Africa, have been adding ISPs at relatively high levels (table 2.2). Where no powerful incumbent exists, IXPs can thrive. This is the case of the Rwanda IXP, where the historical operator no longer exists. The IXP has 13 members, including all of the country's infrastructure-based ISPs. Peak traffic load was over 1 gigabit per second in March 2017, up more than 50 percent from the previous year. An IXP is particularly relevant in landlocked countries like Rwanda, which is far from undersea fiber-optic cables.

Table 2.2 Internet exchange points by region

Internet exchange points Domestic bandwidth production
Region February
2016
February
2017
Net
change
Percent
change
February
2016
February
2017
Net
change
Percent
change
Europe 136 175 +39 +29 35.9T 41.8T +5.84T +16
North America 81 94 +13 +16 3.41T 4.55T +1.14T +33
Asia and Pacific 75 92 +17 +23 2.56T 3.51T +953G +37
Latin America 45 72 +27 +60 2.22T 2.75T +524G +24
Africa 30 42 +12 +40 325G 417G +92.6G +29