According to a report by WIRED magazine, written by senior writer Lily Hay Newman, the integration of agentic artificial intelligence (Agentic AI) has permanently transformed the landscape of cybersecurity. It has done so by accelerating and simplifying the process of discovering software vulnerabilities, patching them, or developing exploit code to turn them into cyber weapons. However, veteran web security researcher James Kettle sought to look beyond the traditional "bug-hunting apocalypse." Kettle wanted to explore a question that has taken on even greater urgency recently, as major AI organizations have disclosed real-world examples of "rogue" AI agents executing hacks: Is agentic AI capable of independently developing novel, abstract hacking methods, from the initial theoretical concept to actual, practical attacks? At the Black Hat security conference held in Las Vegas on Wednesday, Kettle presented his findings, which demonstrate both the rapidly advancing capabilities of AI in the cyber domain and its current limitations.
Limits of AI's Autonomous Capability in Cyber
As James Kettle demonstrated during his presentation at the Black Hat security conference in Las Vegas, the answer to whether artificial intelligence can develop entirely new hacking methods in a fully autonomous manner is both nuanced and qualified. According to Kettle's findings, at present, AI possesses only minimal capability and is radically limited in its ability to independently devise novel and original attack paths in a completely autonomous fashion.
However, when the system operates in tandem with human guidance, direction, and professional insights at key moments, AI reveals itself as an incredibly powerful partner for conceptualizing and uncovering previously unknown hacking strategies. Kettle emphasizes that within the cybersecurity industry today, very few players are willing to speak openly about the limitations of the technology and where it fails completely. This is primarily because most companies and researchers prefer to position themselves as part of the AI revolution rather than expose the weak points of their systems.
The Discovery of Shared-Parser Confusion
One of the primary achievements of the collaborative research between Kettle and AI systems was the exposure of a completely new web vulnerability area, dubbed "Shared-Parser Confusion." This discovery arose from an AI system's revelation regarding the way web servers utilize shared code to process both incoming requests and outgoing responses.
Kettle explained in an interview with WIRED ahead of his talk that this is a highly significant discovery. Requests sent to a website are completely untrusted and can contain any form of code or content, whereas responses from the server are considered trusted and secure. When servers utilize shared code to process both of these data streams, it creates a massive and widespread attack surface. This lack of isolation can potentially spill over into a wide variety of different attack types designed to exploit this confusion.
Experimental Methodology and Filtering Background Noise
This scientific discovery was the result of months of rigorous experimentation that began in September 2025. For these experiments, Kettle utilized the most advanced models available at the time from Anthropic and OpenAI.
At the start of his research, when testing the systems' ability to conduct theoretical security research, Kettle encountered a major obstacle: the AI systems frequently attempted to present existing research as if it were their own original work. The models tended to return findings on highly esoteric and complex topics that were extremely difficult to verify or vet for accuracy.
To overcome this hurdle, Kettle decided to narrow the scope of his tests, restricting the AI systems to working exclusively within his specific field of expertise in web security. This step allowed him to maintain total command over the material and ensure that the systems were not misleading him. Furthermore, Kettle realized that by synthesizing his own personal research methodology and training the models on this specific framework, he could probe much deeper into the models' actual capabilities to extrapolate new conclusions on their own.
The Fast Research Feedback Loop and Professional Overload
As Kettle refined his experiments—providing the models with more precise methodological data and highly targeted parameters—and as time went on and more powerful models entered the market, the AI systems began generating findings and conclusions at an unprecedented rate, far outstripping his own working pace. This dynamic created a highly productive research feedback loop.
Kettle described the process as fascinating but also admitted it induced a degree of professional anxiety. The system produced notable findings roughly every two days without him even needing to log into the platform. The sheer volume of research leads and clues generated by the system was so immense that it left him with a feeling of FOMO (fear of missing out) regarding the possibility of not having enough time to explore them all.
This pressure forced him to develop additional automation tools to help analyze the findings. Within just a few months, the system found far more proven examples of specific vulnerabilities than Kettle himself would likely have been able to discover over several years of intense manual work.
Inability to Independently Prove Findings
Alongside the significant success in identifying existing vulnerabilities, Kettle had hoped that the AI system would be able to discover an entirely novel class of bugs and security vulnerabilities. To some extent, the system did succeed in doing this; however, the discovery involved an extremely rare type of bug, and in practice, it proved to be non-exploitable in the single vulnerable target available for testing.
This outcome once again highlights a fundamental limitation of current autonomous systems: while the AI is capable of generating hypotheses and identifying patterns, it struggles to independently prove their practical feasibility or develop actual exploit code without the involvement of a human expert.
Human-Machine Synergy as the Future of Cybersecurity
Despite these limitations, Kettle emphasizes that the exposure of the Shared-Parser Confusion vulnerability stands as definitive proof of the immense power of human-AI collaboration in both offensive and defensive cybersecurity. The AI was incapable of proving the existence of this vulnerability on its own, but it successfully analyzed real, proven field findings and generated the theoretical hypothesis for its existence.
Kettle, for his part, evaluated this hypothesis, tested it practically, and confirmed its validity. Kettle openly admits that this discovery will likely have the most significant long-term impact, and that he would have never reached it on his own—even if he had been shown the specific line of code from the relevant documentation. The close collaboration between the human researcher who guides and critiques, and the machine that rapidly analyzes data and proposes hypotheses, represents the pinnacle of current AI capabilities in the cyber world.