Analysts say ChatGPT costs $700000 a day to run, and Microsoft tries to reduce costs by developing its own chips.

Beijing time, April 21 morning news, it is reported that chip industry research company SemiAnalysis chief analyst Dylan Patel (Dylan Patel) said that because ChatGPT runs on expensive computing infrastructure, OpenAI's daily cost of running ChatGPT may be as high as $700000 (currently about 4.816 million yuan).

Patel points out that ChatGPT needs a lot of math to provide feedback based on users' input, including writing cover letters, generating teaching plans, and helping users optimize their profiles. "most of the cost comes from expensive servers," he says. "

In addition, Patel's initial estimate is based on OpenAI's GPT-3 model, while ChatGPT may now be more expensive to run after adopting the latest GPT-4 model.

OpenAI has yet to respond to this.

Afzal Afzal Ahmad, another analyst at Patel and SemiAnalysis, said it had been noted that training the big language model behind ChatGPT could cost hundreds of millions of dollars, but the operating costs, or the cost of artificial intelligence reasoning, far exceeded the training costs at any reasonable deployment scale. "in fact, on a weekly basis, the reasoning cost of ChatGPT exceeds the cost of training," they point out. "

Over the past few years, companies that use the OpenAI language model have also been paying high prices. Startup Latitude has developed an artificial intelligence dungeon game that generates storylines based on user input. Nick Walton, the company's chief executive, said the cost of running the model and the corresponding purchase of Amazon AWS cloud servers would reach $200000 a month in 2021 (currently about 1.376 million yuan). As a result, Walton finally decided to switch to a language software provider supported by AI21 Labs. That helped him halve the company's artificial intelligence costs to $100000 a month (currently about 688000 yuan).

"We would joke that we have human and artificial intelligence employees who spend roughly the same cost on both types of employees," Walton said in an interview. we spend hundreds of thousands of dollars a month on artificial intelligence, and we're not a big start-up, so it's a huge expense. "

It has been reported recently that Microsoft is developing an artificial intelligence chip codenamed "Athena" in order to reduce the operating cost of the generative artificial intelligence model. The project was launched in 2019. A few years ago, Microsoft and OpenAI reached an investment agreement of $1 billion, requiring OpenAI to run its model only on Microsoft's Azure cloud server.

There are two aspects of thinking behind Microsoft's launch of the chip project. Microsoft executives are aware that they are lagging behind Google and Amazon in developing their own chips, according to people familiar with the matter. At the same time, Microsoft is looking for cheaper alternatives to Nvidia's GPU chips.

Currently, Microsoft has more than 300 employees working on the chip. Sources said the chip could be released as early as next year for internal use by Microsoft and OpenAI. Microsoft declined to comment on the news.