Huawei has unveiled a new computer architecture called Peerium, designed to link up to 1 million artificial intelligence (AI) chips as a single computer. This architecture, unveiled at the company's annual CONNECT conference, aims to improve the efficiency of AI training by allowing chips to communicate quickly and efficiently, even as the number of chips scales into the hundreds of thousands. The technology powering Peerium relies on three key elements: nested parallelism, unified memory addressing, and peer-to-peer interconnects. Nested parallelism allows workloads to be broken down into smaller tasks and organized across hierarchical groups of chips that work in parallel. This is similar to how a company breaks down work into departments and then smaller teams, all of which coordinate their efforts toward a common goal. Huawei representatives claim that computers powered by Peerium could train models with up to 40 trillion parameters, a significant improvement over current hardware. For comparison, competing hardware has reported training at the trillion-parameter scale. The development of Peerium is part of a larger trend in China to establish frontier AI labs that can train large language models. Huawei's chief scientist, Liao Heng, stated that there will be around 6 or 7 such labs in China within the next two years, all aiming to train models with parameters ranging from 10 to 40 trillion. The creation of Peerium is a significant step towards achieving this goal, and could potentially revolutionize the field of AI training.