Huawei trained the Chinese-language equivalent of GPT-3

Community
1

This week, a research team at Chinese company Huawei quietly detailed what might be the Chinese-language equivalent of GPT-3. Called PanGu-Alpha (stylized PanGu-α), the 750-gigabyte model contains up to 200 billion parameters — 25 million more than GPT-3 — and was trained on 1.1 terabytes of Chinese-language ebooks, encyclopedias, news, social media, and web pages. [Source]

up to 200 billion parameters — 25 million more than GPT-3 —

Wait, did GPT-3 write this article or do humans make errors too! :wink:

In all seriousness, an interesting development. I wonder about the CCP’s thoughts on potentially dangerous output. They’ll likely not make it available to their citizens…

1 Like
2

Another take…

Experts believe that the most important feature of PanGu Alpha is its availability in the Chinese language. However, it seems like that in terms of model architecture; this project doesn’t offer anything new. Further, the team itself has made no claims of overcoming major blockers, such as answering math problems correctly or responding to questions without paraphrasing training data. It does not even offer a solution to the shortcomings of the GPT-3 model (such as bias) on which it is modelled. In addition, it carries over the same challenge as that of other large models — the carbon footprint. [Source]

1 Like
4

sure they will. but the problem is lack of reliable documents in chinese language.
#parameters is important, volume of data is a key factor too.