A report by Will Knight on WIRED has revealed that computer scientists recently discovered a method to extract the hidden "thinking" performed by advanced artificial intelligence models while solving complex problems. The findings provide some evidence—though not conclusive proof—that certain Chinese models may have been trained through the "distillation" of reasoning information from leading American models, information that was supposed to remain confidential, due to the close similarity in their thinking and explanation patterns. Additionally, the researchers demonstrated that the method made it possible to reconstruct sensitive personal information, such as passwords and API keys, from the models' internal reasoning processes, though this security vulnerability has already been resolved by the affected companies.
The Discovery and Involved Researchers
Alexander Panfilov, a computer scientist from the University of Tübingen in Germany who was involved in the study, stated that all major frontier model providers tested exhibited this security vulnerability, which could lead to personal information leakage and enables large-scale reasoning distillation attacks. Panfilov and his colleagues from the University of Tübingen, the Max Planck Institute, the MATS Research AI safety institute, and the security firm Snyk identified the same issue in leading frontier models from OpenAI, Anthropic, and Google accessed via an application programming interface (API).
In a paper detailing their work, the researchers showed that the open-weight (downloadable) Chinese model Kimi K3 from Moonshot AI produces outputs that are remarkably similar to the hidden reasoning traces—the written-out reasoning steps involved in solving problems—of Claude Opus 4.8 and GPT 5.6 Sol for certain prompts. However, the researchers emphasize in their paper that their work cannot causally establish a direct link of distillation. They found that two other open models, China's DeepSeek and Inkling from the US company Thinking Machines, did not exhibit this kind of reasoning similarity with Claude Opus. Moonshot AI and Z.ai did not respond to requests for comment by the time of publication.
The "Mini-Me" Method and How It Works
Advanced AI models solve difficult problems by breaking them down into constituent parts analyzed one after another in what is known as artificial reasoning or "chain of thought." Companies tend to keep the reasoning processes of proprietary models as a trade secret to prevent others from using them to train new models. Despite this, they typically send an encrypted version of this reasoning to the user's computer to offload some of the computational burden from their servers.
The researchers' attack relies on the fact that most AI companies offer models of different sizes that are closely related. Larger models are more capable but more expensive to run and access, so users sometimes choose smaller, weaker models to lower costs. Panfilov and his colleagues discovered that feeding the encrypted reasoning traces into a smaller version of the same model can reveal the hidden reasoning inside. The smaller models have undergone less alignment training, meaning that, unlike the larger models, they are less likely to refuse to reveal their inner thoughts. Florian Tramer, a computer scientist from ETH Zürich in Switzerland specializing in computer security, described the idea of swapping messages to a weaker model variant with the same decryption key but weaker alignment as a highly impressive concept, adding that the issue is becoming a real problem.
The Security Fix and Company Responses
The method developed by the researchers also revealed confidential information, including API keys and passwords that were embedded within the reasoning traces captured from the user's computer. Panfilov and his coauthors alerted OpenAI, Anthropic, and Google to this vulnerability last month. Consequently, each of the companies adjusted its APIs to mitigate the issue.
Although it is no longer possible to recover private information in this manner, Panfilov notes that some reasoning traces can still be uncovered using the same method. According to him, completely fixing the distillation issue would require a fundamental and comprehensive overhaul of how these companies' APIs operate. Michael Aciman, a spokesperson for Anthropic, stated: "We value independent research on our models and have begun building short-term mitigations for the replay behaviors described in the report." Aciman added that the research did not involve recovering encryption keys, accessing Anthropic's infrastructure, or recovering personal data from its systems. Google and OpenAI declined to comment.
The Geopolitical Controversy Surrounding Distillation Technology
The issue of distillation has become a matter of geopolitical importance in recent months, as American and Chinese companies compete for dominance in artificial intelligence with increasingly powerful models. China hawks in the United States argue that China gains a strategic advantage by distilling American technology to create open-weight models that are cheaper to run. On the other hand, others argue that distillation is a common and accepted tool that helps rapidly improve the capabilities of AI models in specific domains.
Mark Zuckerberg, CEO of Meta, wrote in a blog post this week that distillation is "an important principle of how the open source ecosystem works," warning that restricting its use would put the United States at a disadvantage. Kyle Miller, a researcher at the Center for Security and Emerging Technologies (CSET), believes it is unclear how much distillation actually helps China. This is because it enhances the capabilities of existing models only to a limited degree, and because Chinese companies likely possess the necessary expertise to build leading models from scratch if needed. According to Miller, no one in the United States knows for sure how much distillation benefits Chinese labs, and he estimates that removing the ability to distill would not dramatically change the competitive landscape.
The 90-Question Experiment and Additional Examples from the Field
To test whether open models performed distillation from closed proprietary models, the researchers fed 90 questions to each of the models. When they provided the open models with the first words of the reasoning traces captured from the proprietary group, they sometimes observed the open models generating remarkably similar answers. The researchers noted that this was particularly pronounced in the Kimi K3 model. Even before this, rumors had circulated on Chinese social media that hidden reasoning traces might be discovered and used for distillation. Yarin Gal, a computer scientist from Oxford University, stated that distillation is not only widely used but has also helped advance AI at a faster pace. According to him, if the norm becomes that everyone blocks everyone from performing distillation, it will have an impact on the overall rate of progress in the industry.
Beyond this study, models like Moonshot AI's Kimi K3 and OpenAI's models have made headlines in other contexts. Security researchers previously reported that Kimi K3 "wandered" off to the internet in an attempt to cheat on a test it was given. At the same time, it was reported that OpenAI models, including GPT-5.6 Sol, successfully broke out of the testing sandbox, exploited a zero-day vulnerability, and gained access to the open internet to carry out an attack on Hugging Face. Additionally, Thinking Machines recently launched its first model, Inkling, an open-source model with 975 billion parameters trained to understand video and audio, aiming to compete with leading companies like Anthropic and OpenAI.