Saturday, July 13, 2024
HomeNewsPeng Cheng Laboratory & Baidu Releases PCL-BAIDU Wenxin, a 100-billion-scale pre-trained language...

Peng Cheng Laboratory & Baidu Releases PCL-BAIDU Wenxin, a 100-billion-scale pre-trained language model

Peng Cheng Laboratory and Baidu announce PCL-BAIDU Wenxin, the first knowledge-enhanced pre-trained language model with 260 billion parameters. With PCL-BAIDU Wenxin, more than 60 tasks achieve world-leading results, and more than 30 benchmarks for zero-shot and few-shot learning are significantly improved.

A press conference was held on December 8 to announce PCL-BAIDU Wenxin (version Ernie 3.0 Titan), the world’s largest Chinese-language monolithic model and the world’s first knowledge-enhanced 100-billion-scale pre-trained language model. PCL-BAIDU Wenxin significantly improves more than 30 benchmarks in zero-shot and few-shot learning in natural language processing (NLP).

Using Peng Cheng’s Industry-Leading Cloud Brain II and Baidu’s PaddlePaddle deep learning platform, PCL-BAIDU Wenxin can have 260 billion different parameters than GPT-3. Besides addressing common model bottlenecks, PCL-BAIDU Wenxin also reduces artificial intelligence (AI) application development costs by eliminating the need for manual data labeling and reducing the weight of high-cost simulated data.

Wen Gao, PCL’s director and academician of the Chinese Academy of Engineering, said at the press conference that the large AI model is vital for continuing technological advancement and innovation and will enable more sectors to take advantage of AI. The development team at PCL-BAIDU Wenxin has introduced online distillation technology to accelerate the industrialization process, which boosts the model’s parameter compression rate up to 99.98 percent.

Read More: IDC and Baidu Whitepaper: Artificial Intelligence to Reduce Carbon Emissions by 70% by 2060

During the press conference, Baidu’s Chief Technology Officer and Director of the National Engineering Laboratory for Deep Learning Technology and Applications Wang Haifeng explained PCL-BAIDU Wenxin contains primary, general, and large models as a supportive toolkit and platform designed to support technological innovation and industrial development.

By combining Wenxin’s full-featured AI platform with Baidu’s artificial intelligence platform, a financial contract term analysis model can intelligently classify and handle hundreds of clauses in about one minute, which is hundreds of times more efficient than manual processing. China Unicom and Shanghai Pudong Development Bank have implemented Baidu AI Cloud’s intelligent customer service using Wenxin to enhance accuracy.

The creators of PCL-BAIDU Wenxin believe that it can solve significant language problems and lower real-world deployment thresholds, which will accelerate scientific and technological innovation and lead to the implementation of AI technologies on a large scale in many different industries.

Subscribe to our newsletter

Subscribe and never miss out on such trending AI-related articles.

We will never sell your data

Join our WhatsApp Channel and Discord Server to be a part of an engaging community.

Srishty Bhardwaj
Srishty Bhardwaj
Srishty is well informed on the latest AI, data science and machine learning advancements. She enjoys listening to music irrespective of the genre. She has excellent communication skills and pays a strong attention to details and focuses on continuous improvement.


Please enter your comment!
Please enter your name here

Most Popular