Shulou(Shulou.com)12/24 Report--

Xin Zhiyuan reports

Editors: run

[guide to Xin Zhiyuan] Ilya Sutskever, who was named as the "Top Ten Scientific figures in 2023", has repeatedly stressed that as long as he can predict the next token very well, he can help human beings achieve AGI.

Nature recently named Ilya Sutskever, the former chief scientist of OpenAI, as one of the "Top 10 Scientific people of 2023".

The "AI technology beacon", which has just moved away from the spotlight because of the OpenAI upheaval, does not seem ready to return to public view.

In a long introduction to him, Nature said, "Ilya rejected Nature's interview with him after OpenAI's upheaval." And his own Twitter didn't retweet that he was named Nature person of the year.

But Nature still spoke highly of Ilya's contribution to ChatGPT, calling him an AI prophet.

In Ilya's view, artificial intelligence has the ability to change the way human civilization exists, rather than just helping humans solve minor problems, as they did before OpenAI.

"as long as you can predict the next token very well, you can help humans achieve AGI. "

This is the view he has expressed on different occasions.

In a podcast, Ilya explains in detail why he thinks big language models are essentially just tools to predict what the next character is, capable of producing more intelligence than human intelligence combined.

Ilya explained, "many people think that large models only mimic human existing knowledge and abilities in the same way as statistics, and there is no way to surpass human beings." "

"but if your basic neural network is smart enough, you just have to ask it-what will a person with great insight, wisdom and ability do? Perhaps such people do not exist, but neural networks are likely to be able to infer the behavior of such people.

Then AGI's task becomes to predict how such a person might behave.

What does it mean to predict the next character well enough? This is actually a deeper problem than the literal meaning of the question.

Predicting the next token well means that you understand the underlying reality that led to the creation of this token.

Just like statistics, in order to understand these statistics and compress them, you need to know what is the world in which these statistics are created.

And if AGI is to predict human behavior very accurately, what determines people's behavior? Everyone has their own thoughts and feelings, and do things in a specific way.

All of this can be inferred from the next token forecast.

I think as long as you can well predict the next token,AI, you can guess what a person with this great insight, wisdom and ability will do, even if such a person does not exist. "

How to become a scientist like you who has made such a big breakthrough in your field of research?

"I worked really hard, I gave everything I had, and so far, my efforts have paid off. I think that's the whole reason. "

How much economic value can AI produce by 2030?

"it's hard to answer this question. I think there will be a lot. But there is no way to give an exact number.

But if AI does not generate much economic value in 2030, what is the most likely reason? I think it's reliability. "

How far are we from AGI?

This is a difficult question to answer. I'm not sure if I can give a specific number.

Because researchers who are optimistic about the technology tend to underestimate the time it takes to achieve their goals.

The way I keep my feet on the ground and think about this problem is to observe the development of autopilot. For example, if we look at Tesla's progress in autopilot, we can see that he can do almost any behavior required by autopilot.

However, it is also clear that Tesla still has a long way to go in terms of reliability.

Our model may be at a similar stage: it seems to be able to handle all problems, but until we solve all the challenges and ensure its reliability, stability, and good performance, it is difficult to say that we have reached AGI.

Do you think we need a breakthrough like Transformer before we reach AGI? Or does the existing technology already allow us to reach AGI?

Technological development may be a gradual process, and the reason why Transformer is considered a breakthrough is because it is not obvious to almost everyone.

So people think things. Let's consider the most fundamental progress in deep learning: a large neural network can do a lot of things after back propagation training. What is the novelty?

It's not about the neural network, it's not about the back propagation. But it is undoubtedly a huge conceptual breakthrough, because people have not been aware of it for a long time.

But now, now that everyone sees it, people will say-- of course, it's very obvious.

But it is also a very important breakthrough.

Now that different companies develop their own models separately, will different models and technologies be independent of each other or will they reach a common point in the future?

I expect a lot of research and work to develop in a similar direction.

Subsequently, there will be some differences in the long-term work, which means that different research groups or projects will choose different paths and methods.

However, once these long-term work begins to bear fruit, the field will again tend to converge, that is, multiple research paths may once again converge to similar results or theories.

The author also mentioned that there has been a decrease in the number of articles published, which may mean that it will take longer to rediscover and explore promising directions in this area.

Why did OpenAI give up the direction of the robot?

"in the past, the difficulty in robotics was that there was too little data, which limited the development.

In the past, to enter this field, you had to join a specialized robotics company, and a large team was needed to build and maintain robots.

Even with hundreds of robots, it is difficult to get enough data. Because the progress of robot technology depends largely on the combination of computing power and data, the lack of data has become a major obstacle.

The situation is different now, and there is already the possibility of opening up a new path.

But this requires people to really devote themselves to the research and development of robot technology.

This means building thousands of robots, collecting data from them, and finding a way to gradually improve so that robots can perform some basic useful tasks.

With the accumulation of data, more efficient models can be trained to enable robots to perform more complex tasks.

This is a gradual improvement process that requires more robots to be built and more data to be collected.

In order to achieve the development of robot technology, we must devote ourselves to it and be willing to solve all related physical and logistical problems.

This is completely different from pure software development. With enough effort and enthusiasm, it is possible to make great progress in robotics, and some companies have already made efforts in this regard. "

Reference:

Https://www.youtube.com/watch?v=YEUclZdj_Sc

This article comes from the official account of Wechat: Xin Zhiyuan (ID:AI_era)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.