Teacher Xiao Jie has not finished listening to the podcast of DeepMind AI boss Yao Shunyu for nearly 4 hours. There are a few impressive places: the original two yao shunyu are friends for many years, nothing to say, Tencent that yao recently want to dig this yao, this yao has a sense of humor, the character of the Buddhist system and a little hard (a little contradictory complex, hh), from quantum physics to the current direction of AI. He believes that the AI gap between China and the United States is really narrowing, and the density of AI talents in China is greater than that of Lao Mei. The current distillation level of large models in China is unique in the world. He believes that the distillate is hard steaming and soft steaming, hard steaming is a brain without a strategy to distill it, indicating that the company does not understand and has no direction, and he does not recommend this method. Soft steaming is to enhance your model with strategy and purpose by taking some good things along your predetermined route. This is reasonable. He believes that China may really be able to train a model that supports the multi-agent architecture in the real sense, because when steaming, more than one model should be steamed, and it is necessary to balance multiple models at the beginning, hhh. This view is too interesting and justified. Yao thinks that people who currently think of scaling law as the basis for large model pre-training may be divided into three categories:
- The scope of application of scaling law is at an end;
- Some conditions of scaling law cannot be met (such as data);
- There are bugs in the training method and it has not been repaired; According to his understanding, the third category is the majority. At present, the pace of model evolution has not slowed down at all. He observed that at least the people engaged in training in Silicon Valley have this feeling. He feels that model evolution continues to occur for at least 4 months, but he cannot be sure after 6 months, because now more than 4 months later, no one can say (pre-training), including many best practices. There are also a few words that are very rock: Don't waste your life for Olden. There are no individual heroes in the age of AI. "I don't have any mentors and old friends in this industry, of course I want to spray who spray who"