A press conference was held on December 8 to announce PCL-BAIDU Wenxin (version Ernie 3.0 Titan), the world’s largest Chinese-language monolithic model and the world’s first knowledge-enhanced 100-billion-scale pre-trained language model. PCL-BAIDU Wenxin significantly improves more than 30 benchmarks in zero-shot and few-shot learning in natural language processing (NLP).
Using Peng Cheng’s Industry-Leading Cloud Brain II and Baidu’s PaddlePaddle deep learning platform, PCL-BAIDU Wenxin can have 260 billion different parameters than GPT-3. Besides addressing common model bottlenecks, PCL-BAIDU Wenxin also reduces artificial intelligence (AI) application development costs by eliminating the need for manual data labeling and reducing the weight of high-cost simulated data.
Wen Gao, PCL’s director and academician of the Chinese Academy of Engineering, said at the press conference that the large AI model is vital for continuing technological advancement and innovation and will enable more sectors to take advantage of AI. The development team at PCL-BAIDU Wenxin has introduced online distillation technology to accelerate the industrialization process, which boosts the model’s parameter compression rate up to 99.98 percent.
Read More: IDC and Baidu Whitepaper: Artificial Intelligence to Reduce Carbon Emissions by 70% by 2060
During the press conference, Baidu’s Chief Technology Officer and Director of the National Engineering Laboratory for Deep Learning Technology and Applications Wang Haifeng explained PCL-BAIDU Wenxin contains primary, general, and large models as a supportive toolkit and platform designed to support technological innovation and industrial development.
By combining Wenxin’s full-featured AI platform with Baidu’s artificial intelligence platform, a financial contract term analysis model can intelligently classify and handle hundreds of clauses in about one minute, which is hundreds of times more efficient than manual processing. China Unicom and Shanghai Pudong Development Bank have implemented Baidu AI Cloud’s intelligent customer service using Wenxin to enhance accuracy.
The creators of PCL-BAIDU Wenxin believe that it can solve significant language problems and lower real-world deployment thresholds, which will accelerate scientific and technological innovation and lead to the implementation of AI technologies on a large scale in many different industries.
