CS301: Computer Architecture

The TOP500 project ranks and details the 500 most powerful non-distributed computer systems in the world. The project was started in 1993 and publishes an updated list of the supercomputers twice a year. The first of these updates always coincides with the International Supercomputing Conference in June, and the second is presented at the ACM/IEEE Supercomputing Conference in November. The project aims to provide a reliable basis for tracking and detecting trends in high-performance computing and bases rankings on HPL, a portable implementation of the high-performance LINPACK benchmark written in Fortran for distributed-memory computers.

China currently dominates the list with 229 supercomputers, leading the second place (United States) by a record margin of 121. Since June 2020, the Japanese Fugaku is the world's most powerful supercomputer, reaching 415.53 petaFLOPS on the LINPACK benchmarks.

The TOP500 list is compiled by Jack Dongarra of the University of Tennessee,  Knoxville, Erich Strohmaier and Horst Simon of the National Energy Research Scientific Computing Center (NERSC) and Lawrence Berkeley National Laboratory (LBNL), and until his death in 2014, Hans Meuer of the University of Mannheim, Germany.

The TOP500 project lists also Green500 and HPCG benchmark list.

In the early 1990s, a new definition of supercomputer was needed to produce meaningful statistics. After experimenting with metrics based on processor count in 1992, the idea arose at the University of Mannheim to use a detailed listing of installed systems as the basis. In early 1993, Jack Dongarra was persuaded to join the project with his LINPACK benchmarks. A first test version was produced in May 1993, partly based on data available on the Internet, including the following sources:

• "List of the World's Most Powerful Computing Sites" maintained by Gunter Ahrendt
• David Kahaner, the director of the Asian Technology Information Program (ATIP); published a report in 1992, titled "Kahaner Report on Supercomputer in Japan" which had an immense amount of data.

The information from those sources was used for the first two lists. Since June 1993, the TOP500 is produced bi-annually based on site and vendor submissions only.

Since 1993, performance of the No. 1 ranked position has grown steadily in accordance with Moore's law, doubling roughly every 14 months. As of June 2018, Summit was fastest with an Rpeak of 187.6593 PFLOPS. For comparison, this is over 1,432,513 times faster than the Connection Machine CM-5/1024 (1,024 cores), which was the fastest system in November 1993 (twenty-five years prior) with an Rpeak of 131.0 G FLOPS.

Architecture and operating systems

As of November 2019, all supercomputers on TOP500 are 64-bit, mostly based on CPUs using the x86-64 instruction set architecture (of which 474 are  Intel EMT64-based and 6 are AMD AMD64-based). The few exceptions are all based on RISC architectures). Thirteen supercomputers, including the top two, are based on the Power ISA used by IBM POWER microprocessors, three on Fujitsu-designed SPARC64 chips, two on ARM architecture, and one on the Chinese Sunway SW26010 design. One computer uses another non-US design, the Japanese PEZY-SC (based on the British ARM) as an accelerator paired with Intel's Xeon.

In recent years heterogeneous computing, mostly using Nvidia's graphics processing units (GPU) or Intel's x86-based Xeon Phi as coprocessors, has dominated the TOP500 because of better performance per watt ratios and higher absolute performance; it is almost required to make the top 10 or the top spot; the only major recent exception being the aforementioned K computer and Sunway TaihuLight.  Tianhe-2 is also an interesting exception, as while it did use accelerators (just not GPUs), i.e. Xeon Phi, US sanctions blocked the upgrade, but still the upgraded Tianhe-2A is faster with non-US-based Matrix-2000, accelerators which where exploited ahead of schedule. Frontera supercomputer, ranked 5th, based on 28-core (56-thread) Intel Xeon Platinum is also an exception, as it was measured without help of GPUs which were later added, but it has two subsystems, both with Nvidia GPUs, and one of them additionally with POWER9 CPUs, and the other liquid immersion cooling.

Two computers which first appeared on the list in 2018 are based on architectures never before seen on the Top500. One was a new x86-64 microarchitecture from Chinese vendor Sugon, using Hygon Dhyana CPUs (these result from a collaboration with AMD, and are a minor variant of Zen-based  AMD EPYC) and is ranked 38th, and the other was the first ever ARM-based computer on the list (then upgraded for June 2019) – using Cavium ThunderX2 CPUs. Before the ascendancy of 32-bit x86 and later 64-bit  x86-64 in the early 2000s, a variety of RISC processor families made up most TOP500 supercomputers, including RISC architectures such as SPARC, MIPS, PA-RISC, and Alpha.

All the fastest supercomputers in the decade since the Earth Simulator supercomputer have used operating systems based on Linux. Since November 2017, all the listed supercomputers use an operating system based on the Linux kernel.

Since November 2015, no computer on the list runs Windows. In November 2014, Windows Azure cloud computer was no longer on the list of fastest supercomputers (its best rank was 165 in 2012), leaving the Shanghai Supercomputer Center's Magic Cube as the only Windows-based supercomputer on the list, until it was also dropped off from the list. It was ranked 436 in its last appearance on the list released in June 2015, while its best rank was 11 in 2008. There are no longer any Mac OS computers on the list. It had at most five such systems at a time, one more than the Windows systems that came later, while the total performance share for Windows was higher. The relative performance share of the whole list was however similar, and never high for either.

It has been well over a decade since MIPS systems dropped entirely off the list but the  Gyoukou supercomputer that jumped to 4th place in November 2017 (after a huge upgrade) has MIPS as a small part of the coprocessors. Use of 2,048-core coprocessors (plus 8× 6-core MIPS, for each, that "no longer require to rely on an external Intel Xeon E5 host processor") make the supercomputer much more energy efficient than the other top 10 (i.e. it is 5th on Green500 and other such ZettaScaler-2.2-based systems take first three spots). At 19.86 million cores, it is by far the biggest system: almost double that of the best manycore system in the TOP500, the Chinese Sunway TaihuLight, ranked 3rd.

TOP 500

From the 52nd list (November 2018) to the 53rd list (June 2019), the Xeon Platinum-based Frontera is the only new supercomputer in the top 10 (at number 5) and the upgraded  POWER9-based Lassen moved from 11th to 10th. Titan and  Sequoia became the last Blue Gene/Q models to drop out of the top10; they had been ranked 9th and 10th in the 52nd list (and 1st and 2nd in the November 2012, 40th list) and are now 12th and 13th.

"For the first time, all 500 systems deliver a petaflop or more on the High Performance Linpack (HPL) benchmark, with the entry level to the list now at 1.022 petaflops." However, for a different benchmark "Summit and Sierra remain the only two systems to exceed a petaflop on the HPCG benchmark, delivering 2.9 petaflops and 1.8 petaflops, respectively. The average HPCG result on the current list is 213.3 teraflops, a marginal increase from 211.2 six months ago."

Of the top 10 computers in the 54th Top500 list, four are in the top 10 of the November 2019 Green500 list (most energy-efficient supercomputers):

Summit is #5 in the Green500 and #1 in the Top500

AI Bridging Cloud (ABCI) is #6 in the Green500 and #8 in the Top500

PANGEA III is #9 in the Green500 and #11 in the Top500

Sierra is #10 in the Green500 and #2 in the Top500

Top 10 positions of the 55th TOP500 in June 2020

Rank	Rmax Rpeak (PFLOPS)	Name	Model	Processor	Interconnect	Vendor	Site country, year	Operating system
1	415.530 513.855	Fugaku	Supercomputer Fugaku	A64FX	Tofu interconnect D	Fujitsu	RIKEN Center for Computational Science  Japan, 2020	Linux (RHEL)
2	148.600 200.795	Summit	IBM Power System AC922	POWER9, Tesla V100	InfiniBand EDR	IBM	Oak Ridge National Laboratory  United States, 2018	Linux (RHEL)
3	94.640 125.712	Sierra	IBM Power System S922LC	POWER9, Tesla V100	InfiniBand EDR	IBM	Lawrence Livermore National Laboratory  United States, 2018	Linux (RHEL)
4	93.015 125.436	Sunway TaihuLight	Sunway MPP	SW26010	Sunway	NRCPC	National Supercomputing Center in Wuxi  China, 2016	Linux (Raise)
5	61.445 100.679	Tianhe-2A	TH-IVB-FEP	Xeon E5–2692 v2, Matrix-2000	TH Express-2	NUDT	National Supercomputing Center in Guangzhou  China, 2013	Linux (Kylin)
6	35.450 51.721	HPC5	Dell	Xeon Gold 6252, Tesla V100	Mellanox HDR Infiniband	Dell EMC	Eni  Italy, 2020	Linux (CentOS)
7	27.580 34.569	Selene	Nvidia	Epyc 7742, Ampere A100	Mellanox HDR Infiniband	Nvidia	Nvidia  United States, 2020	Linux (Ubuntu)
8	23.516 38.746	Frontera	Dell C6420	Xeon Platinum 8280 (subsystems with e.g. POWER9 CPUs and Nvidia GPUs were added after official benchmarking)	InfiniBand HDR	Dell EMC	Texas Advanced Computing Center  United States, 2019	Linux (CentOS)
9	21.640 29.354	Marconi-100	IBM Power System AC922	POWER9, Volta V100	Dual-rail Mellanox EDR Infiniband	IBM	CINECA  Italy, 2020	Linux (RHEL)
10	21.230 27.154	Piz Daint	Cray XC50	Xeon E5-2690 v3, Tesla P100	Aries	Cray	Swiss National Supercomputing Centre Switzerland, 2016	Linux (CLE)

Legend:

• Rank – Position within the TOP500 ranking. In the TOP500 list table, the computers are ordered first by their Rmax value. In the case of equal performances (Rmax value) for different computers, the order is by Rpeak. For sites that have the same computer, the order is by memory size and then alphabetically.
• Rmax – The highest score measured using the LINPACK benchmarks suite. This is the number that is used to rank the computers. Measured in  quadrillions of 64 bit floating point operations per second, i.e., peta FLOPS.
• Rpeak – This is the theoretical peak performance of the system. Computed in petaFLOPS.
• Name – Some supercomputers are unique, at least on its location, and are thus named by their owner.
• Model – The computing platform as it is marketed.
• Processor – The instruction set architecture or processor microarchitecture, alongside GPU and accelerators when available.
• Interconnect – The interconnect between computing nodes. InfiniBand is most used (38%) by performance share, while Gigabit Ethernet is most used (54%) by number of computers.
• Vendor – The manufacturer of the platform and hardware.
• Site – The name of the facility operating the supercomputer.
• Country – The country in which the computer is located.
• Year – The year of installation or last major update.
• Operating system – The operating system that the computer uses.

Other rankings

Top countries

Numbers below represent the number of computers in the TOP500 that are in each of the listed countries or territories.

Distribution of supercomputers in the TOP500 list by country (June 2020)

Country or Territory	Systems
China	226
United States	114
Japan	29
France	19
Germany	16
Netherlands	15
Ireland	14
Canada	12
United Kingdom	10
Italy	7
Singapore	4
Brazil	4
Korea, South	3
Saudi Arabia	3
Norway	3
Australia	2
Russia	2
United Arab Emirates	2
Taiwan	2
Switzerland	2
Sweden	2
India	2
Finland	2
Spain	1
Czechia	1
Hong Kong (China)	1
Poland	1
Austria	1

• Supercomputer Fugaku (Riken Center for Computational Science   Japan, June 2020 – Present) 
• IBM Summit (Oak Ridge National Laboratory   United States, June 2018 – June2020)
• NRCPC Sunway TaihuLight (National Supercomputing Center in Wuxi  China, June 2016 – November 2017)
• NUDT Tianhe-2A (National Supercomputing Center of Guangzhou  China, June 2013 – June 2016)
• Cray Titan (Oak Ridge National Laboratory   United States, November 2012 – June 2013)
• IBM Sequoia Blue Gene/Q (Lawrence Livermore National Laboratory   United States, June 2012 – November 2012)
• Fujitsu K computer (Riken Advanced Institute for Computational Science  Japan, June 2011 – June 2012)
• NUDT Tianhe-1A (National Supercomputing Center of Tianjin   China, November 2010 – June 2011)
• Cray Jaguar (Oak Ridge National Laboratory   United States, November 2009 – November 2010)
• IBM Roadrunner (Los Alamos National Laboratory   United States, June 2008 – November 2009)
• IBM Blue Gene/L (Lawrence Livermore National Laboratory   United States, November 2004 – June 2008)
• NEC Earth Simulator (Earth Simulator Center  Japan, June 2002 – November 2004)
• IBM ASCI White (Lawrence Livermore National Laboratory   United States, November 2000 – June 2002)
• Intel ASCI Red (Sandia National Laboratories  United States, June 1997 – November 2000)
• Hitachi CP-PACS (University of Tsukuba  Japan, November 1996 – June 1997)
• Hitachi SR2201 (University of Tokyo  Japan, June 1996 – November 1996)
• Fujitsu Numerical Wind Tunnel (National Aerospace Laboratory of Japan   Japan, November 1994 – June 1996)
• Intel Paragon XP/S140 (Sandia National Laboratories   United States, June 1994 – November 1994)
• Fujitsu Numerical Wind Tunnel (National Aerospace Laboratory of Japan   Japan, November 1993 – June 1994)
• TMC CM-5 (Los Alamos National Laboratory   United States, June 1993 – November 1993)
• NEC SX-3/44 ( Japan, 1992–1993)
• Fujitsu VP2600/10 ( Japan, 1990–1991)
• Cray Y-MP/832 ( United States, 1988–1989)
• Cray-2 ( United States, 1985–1987)
• Cray X-MP ( United States, 1983–1985)
• Cray-1 ( United States, 1976–1982)

Number of systems

By number of systems as of November 2018:

New developments in supercomputing

In November 2014, it was announced that the United States was developing two new supercomputers to exceed China's Tianhe-2 in its place as world's fastest supercomputer. The two computers, Sierra and  Summit, will each exceed Tianhe-2's 55 peak petaflops. Summit, the more powerful of the two, will deliver 150–300 peak petaflops. On 10 April 2015, US government agencies banned selling chips, from Nvidia to supercomputing centers in China as "acting contrary to the national security ... interests of the United States";  and Intel Corporation from providing Xeon chips to China due to their use, according to the US, in researching nuclear weapons – research to which US export control law bans US companies from contributing – "The Department of Commerce refused, saying it was concerned about nuclear research being done with the machine."

On 29 July 2015, President Obama signed an executive order creating a National Strategic Computing Initiative calling for the accelerated development of an exascale (1000 petaflop) system and funding research into post-semiconductor computing.

In June 2016, Japanese firm Fujitsu announced at the International Supercomputing Conference that its future  exascale supercomputer will feature processors of its own design that implement the ARMv8 architecture. The Flagship2020 program, by Fujitsu for RIKEN plans to break the exaflops barrier by 2020 through the Fugaku supercomputer, (and "it looks like China and France have a chance to do so and that the United States is content – for the moment at least – to wait until 2023 to break through the exaflops barrier.") These processors will also implement extensions to the ARMv8 architecture equivalent to HPC-ACE2 that Fujitsu is developing with ARM Holdings.

In June 2016, Sunway TaihuLight became the No. 1 system with 93 petaflop/s (PFLOP/s) on the Linpack benchmark.

In November 2016, Piz Daint was upgraded, moving it from 8th to 3rd, leaving the US with no systems under the TOP3 for only the 2nd time ever.

Inspur has been one of the largest HPC system manufacturer based out of Jinan, China. As of May 2017,  Inspur has become the third manufacturer to have manufactured 64-way system – a record which has been previously mastered by IBM and HP. The company has registered over $10B in revenues and have successfully provided a number of HPC systems to countries outside China such as Sudan, Zimbabwe, Saudi Arabia, Venezuela. Inspur was also a major technology partner behind both the supercomputers from China, namely Tianhe-2 and Taihu which lead the top 2 positions of Top500 supercomputer list up to November 2017. Inspur and Supermicro released a few platforms aimed at HPC using GPU such as SR-AI and AGX-2 in May 2017.

In November 2017, for the second time in a row there were no system from the USA under the TOP3. #1 and #2 were installed in China, a system in Switzerland at #3, and a new system in Japan was #4 pushing the top US system to #5.

In June 2018, Summit, an IBM-built system at the Oak Ridge National Laboratory (ORNL) in Tennessee, USA, took the #1 spot with a performance of 122.3 petaflop/s (PFLOP/s), and Sierra, a very similar system at the Lawrence Livermore National Laboratory, CA, USA took #3. These two system took also the first two spots on the HPCG benchmark. Due to Summit and Sierra, the USA took back the lead as consumer of HPC performance with 38.2% of the overall installed performance while China was second with 29.1% of the overall installed performance. For the first time ever, the leading HPC manufacturer is not a US company. Lenovo took the lead with 23.8 percent of systems installed. It is followed by HPE with 15.8 percent, Inspur with 13.6 percent, Cray with 11.2 percent, and Sugon with 11 percent. 

On 18 March 2019, the United States Department of Energy and Intel announced the first exaFLOP supercomputer would be operational at Argonne National Laboratory by the end of 2021. The computer, named "Aurora", is to be delivered to Argonne by Intel and Cray.

On 7 May 2019, The U.S. Department of Energy announced a contract with Cray to build the "Frontier" supercomputer at Oak Ridge National Laboratory. Frontier is anticipated to be operational in 2021 and, with a performance of greater than 1.5 exaflops, should then be the world's most powerful computer.

As of June 2019, all TOP500 systems deliver a petaflop or more on the High Performance Linpack (HPL) benchmark, with the entry level to the list now at 1.022 petaflops.

Large machines not on the list

Some major systems are not on the list. The largest example is the NCSA's Blue Waters which publicly announced the decision not to participate in the list because they do not feel it accurately indicates the ability of any system to do useful work. Other organizations decide not to list systems for security and/or commercial competitiveness reasons. Additional purpose-built machines that are not capable or do not run the benchmark were not included, such as RIKEN MDGRAPE-3 and  MDGRAPE-4.

Computers and architectures that have dropped off the list

IBM Roadrunner is no longer on the list (nor is any other using the Cell coprocessor, or PowerXCell).

Although Itanium-based systems reached second rank in 2004, none now remain.

Similarly (non-SIMD-style) vector processors (NEC-based such as the Earth simulator that was fastest in 2002) have also fallen off the list. Also the Sun Starfire computers that occupied many spots in the past now no longer appear.

The last non-Linux computers on the list – the two AIX ones – running on POWER7 (in July 2017 ranked 494th and 495th originally 86th and 85th), dropped off the list in November 2017.

Source: Wikipedia, https://en.wikipedia.org/wiki/TOP500
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.