Reuters reported on 17 November that its review of more than 300 Chinese documents found AI distillation moving from academic experiments into industrial use. The technique uses answers from a powerful model to train a smaller one. The documents describe researchers obtaining answers, code and solutions from American models, companies building software to automate collection, and specialist workers preparing training material.
Eight Chinese government agencies published an AI+Manufacturing policy package in January that supports model training, fine-tuning and distillation for industrial applications. The government documents reviewed do not instruct companies to distil American models. In a November 2024 paper, Shanghai Jiao Tong University researchers described using about 5,000 answers from OpenAI’s o1 model to assemble 10,750 bilingual examples for further training of Alibaba’s Qwen2.5-72B.
Three people familiar with July meetings in Chengdu said US Commerce Department officials raised concerns with Chinese regulators about the collection of American model output, Reuters reported. A person familiar with separate discussions said Chinese developers have asked Beijing’s industry regulator for guidance on the boundary between ordinary use of foreign models and organised collection of their output for training.