Get the App
SLTechnology News&Howtos  ›  IT Information  › 

The effect can reach 96% of the OpenAI equivalent model, and the domestic open source AI language model TigerBot has been released.

Shulou Source: shulou.com Published: 2023-11-24 17:21:17 11月18日 Update

IT House June 8 news, the domestic multimodal large language model TigerBot has been officially released recently, including 7 billion parameters and 180 billion parameters of two versions, has been open source in GitHub.

On the GitHub page of ▲ source TigerBot, it is reported that the main innovations brought about by TigerBot are:

An innovative algorithm of instruction completion supervised fine tuning is proposed to improve the learnability of the model.

Using ensemble and probabilistic modeling methods to achieve controllable factuality and creativity

Break through the memory and communication problems in mainstream frameworks such as deep-speed in parallel training

In addition, the model also makes a more suitable optimization from tokenizer to training algorithm for the more irregular distribution of Chinese language.

Researcher Chen Ye said on the official website of Hubo Science and Technology: "this model can quickly understand what kind of problems human beings are asking with only a small number of parameters. According to the automatic evaluation of OpenAI InstructGPT papers on public NLP data sets, TigerBot-7B has reached 96% of the overall performance of OpenAI models of the same size."

The GitHub page of the ▲ map source TigerBot is said to be "better than the OpenAI equivalent comparable model". The open source code includes basic training and inference code, quantification and inference code of the double-card reasoning 180B model. The data includes 100G pre-training data and supervised fine-tuning 1G or 1 million pieces of data.

CTOnews.com friends can find GitHub open source projects here.

Tags: Model data training code parameters reasoning language algorithm problem page supervision domestic irregular ten thousand two mainstream home fact fact personnel Apple Docker Huawei Linux macOS MariaDB Microsoft MySQL NVidia OPPO Reno vpn Shulou Tech Info Apple MySQL macOS