A few months ago, the founder and CEO of a meta-company (you probably still remember it as a Facebook company), Mark Zuckerberg, spoke about his ambitious vision for the meta-wars: In the next decade, about a billion users will be inside the meta-wars, the emerging virtual worlds. With these significant user experiences and for other important purposes – Meta has been working on an advanced supercomputer for the past two years, to improve its performance.
In a post on the company’s blog, the company unveils the AI Research SuperCluster (RSC) – a supercomputer that began construction in 2020, and its construction will be completed during the year, which according to the company is now among the fastest computers in the world. “. According to Meta, the computer will be able to run calculations on a huge scale, process trillions of parameters in seconds and train models of artificial intelligence on the order of “unprecedented in the industry.”
Meta AI Research SuperCluster (RSC) / Photo: From the Meta AI YouTube channel
The computer is already in use today. According to the announcement, Meta says their researchers have already started using RSC to train large models in physiological linguistic routing (NLP) processing and computer vision, with the goal that in the future they will do so with trillions of parameters.
Meta says that the computer will be able to run quintillions of tasks (that is, 18 zeros), be able to train artificial intelligence in huge amounts of data and form the basis for computer vision systems, natural language processing and speech recognition. Of course all of these will help in various developments like relevant AR experiences to breakthroughs in the fields of scientific research and more. As mentioned, this development will affect the construction of technologies for the meta-wars, because the vision rests on advanced systems of artificial intelligence that need to be developed.
The computer in question will allow meta-artificial intelligence researchers to build improved and more accurate models in these areas. According to Meta, “Meta’s artificial intelligence teams have already begun training huge models for supercomputer research, and they continue to invest in developing the field of self-supervised learning in an effort to advance the entire artificial intelligence industry,” the statement said.
Meta AI Research SuperCluster (RSC) / Photo: From the Meta AI YouTube channel
A real leap forward
To understand the significant leap that Meta is going through here, one needs to understand the process. In 2017 Meta built the first generation of high-performance computing infrastructures. So at the time it was a system of 22 thousand NVIDIA V100 Tensor Core GPUs – in fact, the system knew how to perform 35 thousand training jobs a day. So she realized that the more efficient way to train the models is on graphics cards (GPUs), rather than a standard CPU (CPU), because it’s faster. In 2020 Meta decided that the best way to move forward from this point is to design a new computing infrastructure and take advantage of GPU technology and more.
Meta explains that a supercomputer is built from a server that integrates multiple GPUs, which are connected via a high-performance network to enable their communication. So RSC is supposed to be a real leap forward: RSC currently includes a total of 760 NVIDIA DGX A100 systems, which is basically the basic server of Anvidia with 8 GPUs on the same board – that is, 6,080 graphics processors in total.
Meta AI Research SuperCluster (RSC) / Photo: From the Meta AI YouTube channel
In order for the data processing to be particularly efficient and fast, and the connection between the graphics processors to be efficient, Meta says that it uses the NVIDIA Quantum InfiniBand network – which is basically an optical fiber that transmits the data at a speed of 200GB per second. This way it stays fast and efficient and enables data traffic.
This technology, InfiniBand (the communication acceleration technology of the supercomputer), was developed at the Israeli development center of Anvidia.
According to Meta, storage includes 175 petabytes of Pure Storage FlashArray including 46 petabytes of cache memory and 10 petabytes of Pure Storage FlashBlade. When the construction process is completed, according to the company, the InfiniBand network will connect 16,000 cards, making it one of the largest networks deployed to date.
Research SuperCluster / Photo: Meta PR
To all this joins the corona that has greatly influenced it. The corona plague caused it to become a “remote control” project, and the crew had to work from the houses until it came to a physical matter. Alongside this, the Corona has also led to supply chain problems, making it difficult for the company to obtain chips or other components.