Beijing Academy of Artificial Intelligence (BAAI) developed the largest language model, WuDao 2.0, with 1.75 trillion parameters supporting English and Chinese. Google had earlier claimed the top spot for the largest language model with its Switch Transformer, a 1.6-trillion parameter model, when it introduced its supremacy in January 2021.
Since the release of NVIDIA’s Megatron in 2019 (8.2 billion parameter model), the race to obtain the largest language model has skyrocketed. Microsoft released Turing-NLG or T-NLG (17 billion parameter model) in early 2020, and OpenAI further pushed the bar with GPT-3, a 175 billion parameter model, in June 2020.
WuDao 2.0 has been trained with 4.9 terabytes of texts and images, including 1.2 terabytes of Chinese and English texts.
According to the researchers, WuDao 2.0 is able to perform tasks like writing poems, comprehending conversations, and understanding images. Just like GPT-3, WuDao 2.0 has the potential to revolutionize how humans work, especially with the Chinese language. Developed by a team of 100 researchers from different organizations, WuDao 2.0 demonstrates how close China with the U.S in terms of research.
While some researchers do not consider building a large language model as a breakthrough, many believe such advancements are crucial as they can open up new doors for other research. Gary Marcus and Yann LeCun have been critical of GPT-3 and consider it as the hype.
However, OpenAI’s model is on its way into the business world with the integration of GPT-3 in Microsoft’s Power App. Microsoft recently announced that the company is exploring ways to equip more products with GPT-3 to enhance productivity.
WuDao 2.0 has over 22 partners, including Xiaomi and Kuaishou, that can integrate the largest language model in the products and services in the coming month.
