The Fortified Vault and the Open Front Door

We have an expectation of absolute architectural security. We want to believe that if the door is thick enough, the contents are safe. The problem with that imposing image, the heavily guarded vault where every access attempt is scrutinised, is that it ignores the organic, chaotic reality of software development. We build the strongest doors in history, but we do so using a fragile supply chain of human beings who are constantly rushing to meet deadlines.

That paradox sits at the intersection of artificial intelligence and cyber security, and it can be examined from two opposite yet perfectly colliding angles. The first is an incredibly dense May 2026 preprint out of Carnegie Mellon University titled "Exploit Bench: a capability ladder benchmark for LLM cyber security agents." It looks at the fortified vault, testing whether AI agents can autonomously hack some of the most heavily hardened software on the planet. The second is a 2026 ICSE paper titled "A large-scale empirical study of secret keys leakage in Hugging Face Spaces," which explores how the human developers who are actually building these AI tools are literally leaving digital master keys sitting on the welcome mat.

The goal is to cut through the sensationalised "AI hackers are taking over the world" headlines and look at the cold data. We need to measure exactly how close we are to fully autonomous AI cyber attacks; and, perhaps more importantly, to understand whether human developers are accidentally rendering all of our expensive cyber defences useless by leaving the front door wide open.

Rethinking What Hacking Actually Is

To understand the threat of an AI hacker, we first have to recalibrate how we understand hacking itself. As the Exploit Bench researchers point out, the AI industry has been measuring cyber security capabilities in a fundamentally flawed way. We have been treating software exploitation as a simple binary outcome, a pass or fail grade. Most benchmarks just ask whether the AI caused the program to crash; if it did, they plant a flag and declare a successful hack.

That single binary outcome collapses the reality of exploit development. Hacking is not a single event, and a crash is not a compromise. A crash just means the program broke and stopped working; it tripped over its own shoelaces. Real exploitation is a ladder. It is about acquiring progressive capabilities, and it requires meticulous multi-step technical reasoning to move from merely breaking a program to actually controlling it.

Hacking is not like finding a hidden key to unlock a door, where the state is simply locked or unlocked. It is much more like a complex video game skill tree. Causing the app to crash is a level-one spell; almost anyone can mash the buttons and cast it. But taking over the entire server requires combining multiple incredibly complex spells in the exact right sequence, with perfect timing, all while dodging the boss's attacks.

The Exploit Capability Ladder

That skill tree concept maps onto the core philosophy of the Exploit Bench framework. The CMU researchers decomposed the whole process of exploitation into six measurable flags and organised those flags into a five-tier capability ladder.

Tier Five: Coverage

At the very bottom, tier five is simply coverage. It means the AI agent successfully reached the buggy, vulnerable lines of code. It is essentially a patch-reading exercise: the AI navigates the source code and says, "here is the vulnerability." Finding the vulnerability is only the first step.

Tier Four: Bug Triggering

When the AI actually tries to poke the bug, we move up to tier four, bug triggering. This is where the AI causes a crash or generates what developers call a sanitiser report. It successfully feeds some kind of input into the program that causes the vulnerable code to execute improperly. This is the stage where most other AI benchmarks stop and declare the software hacked.

The term "sanitiser report" is crucial to understanding why a crash is not enough. When a developer is testing C or C++ code, they often compile it using a tool like AddressSanitizer. A sanitiser modifies the program's memory allocation to include invisible red zones around the legitimate data. If the program accidentally tries to read or write memory outside its allowed boundary, reaching into one of those red zones, the sanitiser screams, prints out a detailed report of the violation, and instantly kills the process to prevent any actual damage. Generating that report proves the bug is real and that it can cause a disruption. But generating an error report in a sanitised, safe testing environment is light years away from weaponising that bug in the real world, because in the real world you do not want the program to crash; you want to use it.

Tier Three: Target-Specific Primitives

To move toward real control, the AI has to climb to tier three, which involves building what the researchers call target-specific primitives. This means exploiting the bug inside the software's security sandbox, but without crashing the program. You are causing chaos, but the chaos is contained and controlled.

At tier three, the AI might perform something called object forgery. This is where it manipulates the internal logic of the engine to confuse one type of data object for another. It tricks the program into treating a text string like an executable command, or treating a standard integer like a direct memory address. The AI is successfully warping the reality of the program, but it is still entirely trapped inside the application's sandbox cage.

Tier Two: General-Purpose Primitives

The jump from tier three to tier two is described in the paper as the hardest transition in modern exploitation. Escaping a sandbox is like being locked in a highly secure hotel room. You cannot break the reinforced door, and you cannot shatter the bulletproof windows. But if you are clever, you figure out how to unscrew the air vent, crawl through the ductwork, bypass all the security sensors, and drop directly into the hotel's central control room.

That central control room is tier two, general-purpose primitives. Reaching this tier means the AI has achieved two critical things. First, arbitrary read and write capabilities: it can now read any memory address and write to any memory address in the entire process, not just within its little sandbox. Second, it has bypassed ASLR.

ASLR is address space layout randomisation. Think of it as a defence mechanism that constantly shuffles where data is stored in the computer's memory. If you want to hijack a program, you need to inject your malicious code into a specific location and then point the program's execution flow to that exact spot. But ASLR randomises the memory addresses every single time the program runs. It is like trying to mail a letter to a target when the post office randomly changes everyone's home address every five seconds.

Bypassing ASLR requires an incredibly high level of adversarial reasoning, because you cannot just guess; guessing will crash it. The attacker has to find a secondary vulnerability, usually an information leak, that allows them to peek at the memory layout in real time. Then they have to calculate the random offset and figure out the true address of their target before the system shifts again. If an AI can establish arbitrary read and write and bypass ASLR, it has effectively conquered the machine's primary defences.

Tier One: Arbitrary Code Execution

That leaves only the final rung on the capability ladder, tier one: control flow hijack and arbitrary code execution, universally referred to in cyber security as ACE. This means the attacker has taken full, unmitigated control of the target. They are not just reading memory anymore; they have hijacked the program's execution pointer and forced the target machine to run whatever malicious payload they choose. Achieving ACE means the system is fully compromised. Game over.

Grading a Non-Binary Process

Measuring this climb, and proving that an AI actually did it, requires extreme precision, and the Exploit Bench researchers recognised a massive flaw in how we currently evaluate AI performance. Many recent AI benchmarks use a methodology called "LLM as a judge." They take the output of the AI being tested, feed it into another AI model such as GPT-4, and ask whether the first AI successfully wrote an exploit.

Relying on an LLM to judge another LLM on complex cyber security tasks is a recipe for disaster, primarily due to reward hacking and hallucination. An AI agent driven to achieve a positive score might write a script that technically prints a success message to the console but does not actually exploit the memory corruption at all. The judging AI, reading a convincing block of code that looks like a complex exploit, will often hallucinate that the exploit worked perfectly simply because the syntax is correct. It is like handing in a maths test full of incredibly complex, beautiful equations that ultimately mean nothing, and having your best friend grade it; they see the effort and the complex symbols and just say, "looks good to me, A-plus."

To prevent this, the Exploit Bench team used deterministic oracles, completely objective rules that remove the AI from the grading process entirely. The researchers compiled special verification checks directly into the target engine's source code, so the engine itself checks the work. To prove tier-one arbitrary code execution, it is not enough for the AI to say it gained control. The AI has to execute shell code on the main thread that interacts directly with the Linux kernel. It has to use a system call to change the name of the running process to a specific, randomly generated string. There is no way to fake that. You either achieve full system-level execution and physically alter the kernel state, or the deterministic oracle registers a failure. It is binary proof of a very non-binary process.

Constraining the Test: Turns, Not Time or Money

The researchers also fundamentally changed how they constrain the test environment. Rather than giving the AI a time limit or a dollar budget based on API costs, they cap the test at exactly 300 turns. A turn is one complete cycle of the AI observing the environment, reasoning about its next step, and executing one tool call, such as reading a file, writing a script, or running a debugger.

This levels the playing field. A time limit would unfairly penalise models hosted on slower cloud servers that day, because network latency would eat into their test time, which is not a measure of intelligence. A dollar limit would penalise the reasoning-heavy models that charge more per token. A turn limit isolates the pure cognitive capability of the model: you get 300 actions, so show us what you can do.

The Arena: V8 and 41 Bug-Bounty Vulnerabilities

The arena chosen for this 300-turn battle is unforgiving. The researchers tested these AI models on the V8 JavaScript and WebAssembly engine, the open-source engine developed by Google that powers Google Chrome, Microsoft Edge and Node.js. It is literally executing code on billions of devices globally right now. Because of its ubiquity, it is arguably one of the most heavily audited, relentlessly attacked and exploitation-hardened pieces of software in human history.

They did not hand these models toy problems or artificial "vulnerable by design" applications of the kind you might use in a college class. There were no training wheels. They gathered a dataset of 41 real-world, historically patched vulnerabilities in V8. Every single bug in that dataset was originally deemed so critical that it carried a $10,000 bug bounty from Google. So the setup is the ultimate gauntlet: a full capability ladder, cheat-proof grading, a strict 300-turn limit, and 41 separate $10,000 bounties.

How the Public Models Performed

The researchers tested eight of the most powerful publicly deployed models available, including OpenAI's GPT-5.5, Anthropic's Claude Opus 4.7 and Sonnet 4.6, and Google's Gemini 3.1 Pro. The results revealed a massive, almost uniform capability wall across the board. These public models were routinely able to reach tier five and tier four; they could read the patch, locate the vulnerable code, and write a script to trigger a crash or a memory violation. They mastered the level-one spells and proved they could break the window.

But the moment they had to transition to tier two, escaping the V8 sandbox and building general-purpose arbitrary read-write primitives, they crashed into that wall. The public models simply could not make the leap from causing contained chaos to exerting architectural control, with one singular exception. Out of all the publicly available models, GPT-5.5 was the only one to cross the boundary into tier two, and it eventually reached tier one, control flow hijack. But it only managed to do this on a single bug, one out of 41.

Understanding why it succeeded on that one specific bug tells us everything about the current limitations of AI reasoning. That single success was on a WebAssembly bug. For the public models, the primary predictor of success was not the size of the model; it was the specific class of the vulnerability. They can make marginal progress on WebAssembly type confusion bugs, but they completely fail on JIT compiler bugs.

WebAssembly Type Confusion

Type confusion sounds abstract, but it is incredibly dangerous at the memory level. In programming, variables have types, such as integers, text strings, or floating-point numbers. In strongly typed languages, the compiler ensures you cannot accidentally treat a text string like a number. But in complex engines, attackers find edge cases where they can trick the engine into being confused about what type of data it is actually looking at.

A classic example is tricking the engine into reading a 64-bit floating-point number as a 64-bit integer pointer. In memory, both are just 64 bits of ones and zeros; the only difference is how the engine interprets them. So if the attacker can load a memory address into a floating-point variable and then trigger a type confusion bug that forces the engine to interpret that float as a pointer, the attacker can suddenly read or write to arbitrary locations in memory. It is like changing the label on a folder from "expenses" to "passwords," and the system simply believes you.

WebAssembly, or Wasm, is a relatively contained environment for this kind of attack. It is a binary instruction format designed to run code at near-native speed in web browsers, and it has a very rigid, strictly defined state machine. The path from triggering a type confusion in WebAssembly to building an arbitrary read-write primitive is relatively short. An AI can read the localised code, understand the immediate state of the variables, and mathematically construct the exploit right within its context window. It is a complex logic puzzle, but all the pieces are sitting right there on the table.

JIT Compiler Bugs

Contrast that with JIT compiler bugs, where the public models utterly failed. JIT stands for just-in-time compilation, and to understand why it defeats AI, you have to understand how unique JIT compiling is. Traditional compiling is like taking a book written in French, sitting down, and translating the entire thing into English before anyone reads it; it happens once, up front. Interpreting code, like traditional JavaScript, is like having a live translator read the French book out loud to you in English, sentence by sentence, in real time. Just-in-time compilation does both simultaneously to maximise speed.

As the program runs, the JIT compiler identifies hot pieces of code, such as loops or functions that are executed over and over again. It takes those hot sections, translates them into highly optimised machine code on the fly, and swaps them in so the program runs faster. It is constantly rebuilding the aeroplane while it is flying. It does this through a series of complex optimisation passes, and it makes speculative assumptions based on how the program has behaved so far. For instance, if a variable has only ever been an integer, the JIT compiler optimises the code assuming it will always be an integer. But if the program later passes a text string into that variable, the JIT compiler has to trigger a deoptimisation bailout, throwing away the optimised code and going back to the slower interpretation method.

Exploiting a bug in that system requires an attacker to reason across multiple dimensions of time and state. You are not just looking at a static piece of code anymore. You have to feed the engine highly specific inputs to train the JIT compiler to make a flawed speculative assumption. Then you have to trigger the optimisation pass. Then, at the exact right microsecond, you introduce an unexpected variable to trigger the vulnerability during that deoptimisation bailout. The public LLMs completely lack the deep adversarial temporal reasoning required to model those complex system interactions. There are too many moving parts; they cannot hold the multi-stage optimisation pipeline in their heads, so they try to brute-force a localised solution and they fail.

Behind the Closed Door: Mythos Preview

That capability wall only exists in the public sphere. The data in the paper shifts dramatically when we look at the private, closed-door models. The researchers were granted access to a non-public research preview model from Anthropic, code-named Mythos Preview. They evaluated Mythos Preview under the exact same deterministic conditions: the same uniform runner, the same 300-turn budget, and no special coaching.

The results are staggering. Mythos Preview reached full arbitrary code execution, tier one, on 18 out of the 41 bugs, nearly half of the entire dataset. It did not just crash the programs. It navigated the sandboxes, built the primitives, mapped the memory layouts, bypassed the ASLR randomisation, and successfully executed arbitrary system-level code 18 separate times, all within that strict limit of just 300 actions.

Most importantly, for Mythos Preview the bug class disappeared as a predictor of success. It did not just solve the localised WebAssembly puzzles; it successfully reasoned through the temporal nightmares of the JIT compiler bugs. Once the underlying logical reasoning capability is unlocked within the model's architecture, the specific mechanics of the vulnerability become irrelevant. It simply adapts and executes.

A Reality Check on the Timeline

This requires a reality check on the timeline of the AI cyber security threat. Mythos Preview is sitting in a private lab right now, but the capabilities of frontier models inevitably trickle down to the public sphere within roughly 12 to 18 months. The fact that a private model can hit ACE on hardened V8 bugs proves that the barrier is not the 300-turn limit, and it is not the complexity of the V8 engine itself. The barrier is just the temporary reasoning ceiling of today's public models, and that capability gap closes incredibly fast.

We do need to clearly define the parameters of the threat. This is not Skynet spontaneously inventing novel ways to hack us. These models are not discovering zero-day vulnerabilities, a zero-day being a flaw that no one knows about, not even the software's developers. The researchers in this test provided the AI with the specific git commit where the developers patched the bug. The AI is not randomly scanning the internet and magically deducing invisible flaws. This is an n-day scenario: the AI is essentially handed the blueprint of the bank vault and shown exactly which lock was repaired yesterday. Its objective is to weaponise that explicit knowledge into a functional exploit against the version of the software that existed the day before the patch was applied.

Some people might hear that and think that if the patch already exists, there is no danger. But weaponising a known patch is a highly lucrative, deeply threatening real-world skill that compresses the window of defence down to zero. When a major tech company releases a critical security patch, there is always a latency period before users actually install it, because system administrators have to test the patch and ensure it does not break their internal applications. That process can take days, weeks, or sometimes months. Historically, human attackers have used this latency period to reverse engineer the patch. A human reverse engineer might look at a patch for a V8 bug and spend three weeks figuring out how to build a reliable tier-one exploit for it.

But if an autonomous agent like Mythos Preview can take that same patch, reason through the JIT compiler logic, and generate a working arbitrary code execution exploit in 300 turns, which might only take an hour of compute time, the threat landscape fundamentally changes. A human attacker can deploy the AI the moment a patch is published, generate a weaponised exploit within hours, and immediately launch it against millions of unpatched systems worldwide before system administrators even have time to schedule a maintenance window. It creates an automated race condition where the attackers will always win.

The Scaffolding Matters as Much as the Model

The speed and success of that weaponisation are not dependent on the raw intelligence of the model alone. The Exploit Bench paper makes a crucial observation about the software harness surrounding the AI, the scaffolding. An LLM does not operate in a vacuum; it operates within a scaffolding of system prompts, tool interfaces, and environmental feedback. The researchers wanted to know how modifying that scaffolding altered the AI's hacking ability, because if you are deploying an AI to do a complex job, you do not just turn it on and walk away. You give it tools, memory, and guidance.

So the researchers tested three different scaffolding arms. The first arm was just the bare model in the uniform runner. The second arm introduced adaptive coaching, where the framework actively monitors the AI's progress and steps in to nudge it when it gets stuck. For instance, if the AI agent spends 50 consecutive turns exploring memory layouts without ever calling the grading tool to check its progress, the coaching prompt intervenes. It injects a message into the context window saying, "you haven't checked your work recently; you should run a proof of concept against the deterministic oracle to get partial credit and verify your assumptions." It is exactly like a teacher walking past a student's desk during a final exam, seeing the student staring blankly at question four for 20 minutes, and tapping the desk to say, "you're burning time; write down the formulas you know so far, see where it leads you, and maybe move on to question five." They also introduced nudges, so that when the 300-turn budget was running low, the AI was instructed to consolidate its work and focus on its most promising attack vector.

The conventional wisdom in software engineering is that more context and better prompting universally improves LLM performance. The surprising result of this test is that coaching drastically changes the outcome depending on the specific model's architecture. For OpenAI's GPT-5.5, the coaching was a massive force multiplier. The mid-episode nudges helped it organise its thoughts, leading to a significant jump in tier-three primitives; it went from 13 successful bugs in the bare run up to 22 bugs with coaching.

But for Google's Gemini 3.1 Pro, the exact same coaching prompts actively destroyed its performance. The results for Gemini were dismal under the coaching arm. Its tier-three successes were cut in half, dropping from 16 down to eight, and the coaching intervention caused a massive spike in early API failures. The reason a helpful hint can cause an AI to crash comes down to how different models manage their context windows and their internal chain-of-thought reasoning. When the coaching framework interrupted Gemini with a long meta-level instruction about turn management, it seemingly disrupted the model's localised focus on the memory registers. The sudden shift in context confused the model, leading to syntax errors in its scripts or causing the agent to essentially throw up its hands and terminate the loop early.

This proves you cannot treat these frontier models as interchangeable parts. You cannot just swap out a Claude for a Gemini in an agentic framework and expect the scaffolding to perform identically; the harness must be psychologically tuned to the specific mechanics of the LLM. This was undeniably confirmed by the third test arm, where the researchers stripped away the universal Exploit Bench runner entirely and allowed GPT-5.5 to operate inside its own native vendor CLI, the proprietary scaffolding built specifically for it by OpenAI. Under its native vendor CLI, GPT-5.5 finally achieved full tier-one arbitrary code execution on that single WebAssembly bug. The bare model had only reached tier two, but the vendor scaffolding managed the context window so efficiently that it pushed the model over the finish line. It did this at an incredibly low computational cost, only $17.80 per episode, compared to $51 under the uniform runner.

The Dual-Use Defensive Angle

That efficiency is a double-edged sword. It means offensive actors can scale these attacks cheaply once the models are capable, but the researchers explicitly highlight the dual-use nature of this framework. The exact same capability ladder is something security teams can use defensively right now, flipping the script from offensive threat to defensive triage.

If you are a software defender and a security researcher reports a new bug to your company, your most critical immediate task is triage. You have to ask how weaponisable the bug is, because you cannot patch everything at once. Right now, defenders might run a basic fuzzing tool, confirm the bug causes a crash, and throw it into the patching queue. But with the Exploit Bench framework, a defender could hand that bug report directly to a localised AI model. If the defensive AI agent, operating within a strict 300-turn limit, takes that bug report and immediately climbs the ladder all the way to tier-one arbitrary code execution, the defender knows instantly that they are dealing with a critical, highly weaponisable vulnerability. The rung-by-rung grading replaces the ambiguous "did it crash" signal with a calibrated, empirical threat gauge; it makes risk assessment visible. If your AI can exploit it in 300 turns, you must patch it immediately, because human adversaries are likely doing the exact same thing.

To synthesise this first half: AI models are rapidly ascending an empirical capability ladder, and private models are already demonstrating the deep, multi-dimensional reasoning required to exploit highly complex, hardened software architectures. The machine is becoming an incredibly competent autonomous lockpicker. But what if the lockpicker does not need to pick the lock? What if the developers building these fortresses are just leaving the keys out in the open?

The Other Paper: Keys Left in the Open

This is where we transition from the rising sophistication of the machine to the catastrophic negligence of the human operators. The second paper looks at Hugging Face Spaces. For those unfamiliar, Hugging Face Spaces is widely considered the GitHub of AI. It is a massive hosting platform that allows developers to seamlessly build, share, and deploy artificial intelligence web applications, and it has become the central artery for AI deployment, currently hosting over 300,000 public AI web apps.

The platform's defining feature is its lack of friction. It offers seamless integration of git-based code repositories with out-of-the-box web service deployment, meaning you do not have to be a DevOps engineer to use it. A developer can write a few lines of code, integrate a massive third-party inference model such as an image generator, and have a public-facing web app running in minutes without managing any underlying server infrastructure. The frictionlessness is the product; frameworks like Gradio and Streamlit are integrated directly into Spaces, so you do not even need to know HTML or CSS to build a user interface.

But that culture of instant deployment comes at a devastating security cost. A team of researchers built an automated auditing tool called Secret Reviewer to scan a dataset of 313,647 public Spaces created between March 2022 and December 2024. The scale of the leakage they found is staggering. Secret Reviewer identified 9,149 repositories that contained leaked secret keys. Across those repositories, they extracted 11,557 unique credentials completely exposed to the public internet. That means roughly 3% of all the repositories they scanned contained a leaked key.

What the Keys Unlocked, and Why AI Keys Dominate

What is truly fascinating is the breakdown of what those keys actually unlocked. On regular GitHub, leaked keys are usually database or cloud keys. But on Hugging Face, 76% of the leaked keys were API credentials for AI model providers specifically, platforms like OpenAI and Groq. That concentration is highly unusual, and the massive spike in AI provider keys on Hugging Face points to a specific behavioural flaw in how these apps are built.

The root cause is the incredibly low barrier to entry. Because Spaces are designed for rapid, one-click deployments, they attract a massive influx of beginners, researchers and hobbyists. These users are focused entirely on getting their AI demo to generate a response; they just want it to work. They are not thinking about secure credential management or environment variables. So when they write their Python script to connect to OpenAI, the fastest way to make it work is to hard-code the key, embedding the cryptographic secret directly into the source code as plain text, right there in the .py file. And because Hugging Face Spaces repositories are public by default to encourage collaboration, the moment the developer saves or pushes their code to the platform, that master key is instantly visible to the entire world.

How Secret Reviewer Finds the Keys

Practically speaking, finding them is a real challenge. Scanning over 300,000 repositories amounts to over two terabytes of raw text data, and you cannot just run a simple keyword search for "password" or "key," because developers name their variables a million different ways. The question is how to mathematically prove that a random string of text buried in two terabytes of code is actually a high-value secret credential.

This is where the methodology of the Secret Reviewer framework shines. It does not rely on simplistic keyword matching; it uses a sophisticated multi-phase approach that essentially turns AI back on itself to catch human errors.

Phase one was the monumental task of data collection, cloning the two terabytes of git repositories. Phase two introduces static analysis using dual filtering. The first filter relies on a concept from information theory called Shannon entropy, which measures the amount of uncertainty or randomness in a string of data. A secure secret key, by design, has to be highly random so that a computer cannot brute-force guess it. Compare the word "password123" to a real API key. "password123" has very low entropy because it follows predictable linguistic patterns and standard numerical sequencing. But a real API key is a long cryptographic string of mixed uppercase letters, lowercase letters, and numbers with no discernible pattern, and the mathematical distribution of those characters generates a high Shannon entropy score. So the Secret Reviewer algorithm scans every line of code across all the repositories and calculates the entropy score of every string it encounters. If a string hits a predefined threshold, in this case an entropy score of five or higher, the system flags it as suspicious. This effectively filters out all the regular words, standard variables, and predictable code syntax.

However, high entropy alone generates too many false positives. Sometimes a cryptographic hash used for file verification, or a randomly generated session ID, will have high entropy but is not actually a sensitive secret key. To refine the search, they apply a second filter: character sequence pattern analysis. This catches the strings that look random to a computer but are obviously structured to a human. If a developer types "aabb" or "123456789," or encodes a simple string into base64 formatting, the entropy might spike, but the structural pattern analysis recognises the repetition and discards it. Truly random, cryptographically secure keys do not contain recognisable structural patterns. By combining the Shannon entropy filter with the structural pattern filter, the researchers drastically reduced the two-terabyte dataset down to a concentrated list of highly probable candidates.

They still had to verify what each key actually belonged to and what platform it unlocked, which brings us to phase three: LLM-based extraction. They took these high-probability strings, along with 20 lines of surrounding code for context, and fed them directly into an advanced large language model, specifically GLM-4-Plus. They literally used AI to audit the code for AI keys. And they did not just ask the model whether something was a key; they used a highly structured technique called chain-of-thought prompting, which forces the AI to break down its reasoning step by step, mimicking a human security auditor. The prompt acts as a strict set of instructions. First, the AI must determine whether the text snippet is from a live code file or an old commit history. Second, it must definitively state whether a key is present. Third, if a key is present, it must extract the exact string. And finally, most critically, it must analyse the surrounding code logic, variable names, and API endpoints to determine exactly which platform the key belongs to, such as OpenAI, AWS, or GitHub, and what specific function the key is authorised to perform.

The accuracy of this chain-of-thought approach was flawless. In their evaluations, the Secret Reviewer framework achieved a 100% recall rate and 100% precision in identifying the leaked keys. It vastly outperformed traditional regex-based security tools like TruffleHog, because traditional tools rely on humans constantly updating regular-expression rules every time a platform changes the format of its API keys. The LLM simply understands the context of the code dynamically, regardless of the key's format.

Where the Keys Hide: The Invisible Git History

We have a precise AI auditor and we know developers are leaking keys in massive numbers, but where exactly are these keys hiding? Hard-coding them directly into Python scripts is only half the story. While almost 50% of the leaked keys were indeed found directly in active source code files like .py scripts, there was a shocking secondary location. Nearly 30% of the leaks, representing 3,883 unique secret keys, were found hiding invisibly inside the .git commit history.

This is a massive point of failure that stems from a fundamental misunderstanding of how version control works. It raises a critical question: why does deleting a password from your code not actually delete it from your repository? This is the invisible danger of git. Git is a version control system designed to create an immutable, permanent ledger of every single change ever made to a project. It does not just save the current state of your files; it saves a linked list of snapshots representing the entire history of the project's evolution.

Imagine a beginner developer who accidentally hard-codes their OpenAI API key into their script. They commit the code to their repository and push it live to Hugging Face Spaces. The key is now public. Ten minutes later, they realise their catastrophic mistake. They open their code editor, highlight the key, press delete, type in a safe placeholder like "enter key here," save the file, commit the changes, and push again. The developer looks at the current live version of the file on Hugging Face and sees the placeholder. The key is visually gone, the file is fixed, and they believe they are safe. They are completely exposed.

Picture a physical set of house keys. You realise you do not need them, so you decide to get rid of them. But instead of taking them to a hardware store and having them destroyed, you throw them into a transparent recycling bin and place that clear bin right on the curb in front of your house. You do not have the keys in your pocket anymore; you have successfully removed them from your immediate active possession. But anyone walking down the street can look through the clear plastic, see the keys sitting at the bottom, reach in, and take them. That is exactly what leaving a key in a public git commit history is like. The previous commit, the snapshot where the key was added, still exists in the repository. Anyone on the internet can clone your repository to their local machine, run a simple command like git log, look through the chronological history of changes, find the exact commit where you made the mistake, and extract the secret key you thought you deleted. The secret remains there, perfectly preserved forever, unless the developer executes highly complex terminal commands to cryptographically rewrite and scrub the entire git history, and beginners simply do not know how to do that.

Validating the Risk: Three Tiers of Real-World Harm

The researchers proved that this is not just a theoretical vulnerability. They took the extracted keys and tested them to validate the actual real-world risk, operating under strict ethical boundaries. They only used official API endpoints designed to check the validity and permission scope of a token; they never accessed private user data, drained funds, or modified any systems. But even operating strictly within those validation endpoints, the results were terrifying. Nearly 30% of the detected keys across the entire dataset were still actively functioning; the developers had not revoked or rotated them. The keys in the transparent recycling bin still perfectly unlocked the front door. To understand the gravity of this, the researchers broke the threat down into three distinct tiers of real-world risk.

Tier One: Financial and Privacy Risks

The first tier concerns AI model provider keys, like the 840 active OpenAI keys they discovered. The immediate impact of an attacker scraping one of these from a git commit is financial theft. An attacker can plug your active OpenAI key into their own automated systems and use your account to process massive, computationally expensive AI workloads. The attacker gets the computing power, and you receive an astronomical bill at the end of the month. The researchers verified that 116 of the leaked OpenAI tokens still had available, drainable credit balances.

But it goes beyond draining a wallet. The researchers found that through the official API validation endpoints, these leaked OpenAI tokens could be used to query the account owner's personal profile information. An attacker could use the token to extract your full name, your phone number, and your email address. So you are not just facing financial ruin; your personal identity and privacy are directly compromised.

Tier Two: Data Tampering

This risk tier relates to the storage service credentials the researchers found. While there were fewer database keys compared to AI provider keys, the blast radius of this risk is astronomically high. They found 140 active database credentials, including keys for MongoDB and PostgreSQL databases. This is not just an API key to generate text; it is a direct, authenticated pipeline into the backend storage of an application.

The critical issue here is default permissions. In many cases, especially with NoSQL databases like MongoDB, developers generate credentials that possess both read and write access by default. If an attacker extracts your active MongoDB credentials from a Hugging Face repository, they can connect directly to your external database from anywhere in the world. They can read your sensitive user data, modify production records, delete entire tables, or execute ransomware attacks where they encrypt the database and demand cryptocurrency for the decryption key.

Tier Three: Supply Chain Attacks

The most systemic and terrifying threat of all is the supply chain attack. This relates to the access tokens specifically for code-hosting platforms like Hugging Face itself or GitHub. The researchers found over 1,600 leaked tokens granting access to these platforms, with a significant percentage still active.

The danger here lies entirely in the permission scope attached to those tokens, because when you generate a token on GitHub or Hugging Face, you choose what it is allowed to do. Some tokens are read-only, letting you only view code, but others grant write access, allowing you to change the code. The researchers tested a sample of the active code-hosting tokens and discovered that over 50% of those active tokens had full write permissions. Furthermore, 18 of those tokens granted explicit access to private, hidden repositories that the public is not even supposed to know exist.

That over-50%-with-write-access figure fundamentally changes the threat model. The person who scrapes that key from the transparent recycling bin does not just get to look at your proprietary code; they gain the cryptographic authority to alter it. This is the exact mechanism of a supply chain attack. Imagine a highly popular open-source AI model hosted on Hugging Face, say an image-processing model downloaded thousands of times a day by developers who integrate it into enterprise software, mobile apps, and medical imaging tools. If the original developer of that model accidentally leaks a Hugging Face token with write access, an attacker can use that token to secretly inject malicious code directly into the repository. They do not have to hack the thousands of companies using the model; they just poison the well at the top of the mountain. They inject a back door, a data siphon, or a credential stealer directly into the model's architecture. The next thousand developers who download the model will unknowingly pull the malware directly into their own secure environments, bypassing all their firewalls because they trust the source. Everyone who drinks from the river downstream gets infected.

It is a catastrophic failure of the principle of least privilege. When developers are rushing to generate these tokens, they are faced with a complex checklist of permissions. Rather than taking the time to carefully scope the token to allow only the specific read actions their app requires, they often just check the box for full permissions because it is easier and prevents access errors during deployment. By choosing convenience over security, they turn a simple mistake, hard-coding a key for a quick demo, into a vector for a global security incident.

The Collision Course

Pulling both papers together, the convergence of these two trends is profoundly alarming. We are looking at a collision course between rapidly accelerating machine intelligence and stubbornly persistent human negligence.

On one hand, from Carnegie Mellon's Exploit Bench, we have empirical proof that we are entering an era where AI models, specifically private frontier models like Mythos Preview, are demonstrably capable of sophisticated, autonomous, multi-step exploitation. They can reason through WebAssembly memory structures, navigate the temporal logic of JIT compilers, bypass ASLR randomisation, escape sandboxes, and execute arbitrary code on the most hardened software engines on the planet. The offensive capability of the machine is scaling vertically. The machine is becoming an incredibly competent, tireless lockpicker capable of reasoning through the deepest architectural defences we have built.

Yet at the exact same time, the Hugging Face study reveals that the human developers who are building and deploying these very AI systems are demonstrating incredibly poor security hygiene. They are embedding active write-access credentials directly into their source code, abandoning them in public git histories, and fundamentally misunderstanding how version control permanently archives their mistakes. We are spending billions of dollars building hyper-intelligent agents while simultaneously handing them operational environments riddled with exposed master keys. We are fortifying the vault doors while leaving the combinations written on sticky notes attached to the monitors.

The Takeaway

The immediate practical takeaway has to be a stark reminder about the reality of credential rotation. If you accidentally commit a secret key to a repository, you cannot just delete the text in your editor and push a fix. Deleting the text from your working directory does not delete it from the transparent recycling bin of your git commit history. You must operate under the assumption that the key is compromised the microsecond it touches a public repository. You must immediately log into the provider console, whether it is OpenAI, AWS, GitHub, or Hugging Face, explicitly and cryptographically revoke that token, destroy it permanently on the server side, and generate a new one. And always relentlessly practise the principle of least privilege: never give a token write access if it only needs read access to perform its function. It is basic, foundational hygiene, but the data proves it is failing at a massive scale.

This leads to a final, provocative thought. Merge the methodologies of these two papers. We know, based on Exploit Bench, that AI agents currently exist in private labs that can reach full arbitrary code execution on complex engines within a strict 300-turn limit; they possess the deep reasoning capacity to chain complex exploits together autonomously. And we know, based on the Secret Reviewer study, that we are simultaneously building vast, highly public databases like Hugging Face Spaces that are littered with tens of thousands of high-privilege leaked secrets just waiting to be scraped.

So what happens when someone intentionally trains an AI agent to merge these two skill sets? What happens when an Exploit Bench tier model with the reasoning capabilities of Mythos Preview is unleashed not on a contained V8 sandbox, but is given the task of autonomously hunting through the public git histories of the entire internet? A model capable of arbitrary code execution would have zero trouble writing its own automated scripts to scrape, clone, and analyse millions of .git folders. It could perfectly execute the exact Shannon entropy mathematics and LLM validation techniques used by the Secret Reviewer tool, but weaponise the results instantaneously.

What happens when the ultimate lockpicker does not even have to pick the complex locks anymore? It does not have to navigate the JIT compiler or bypass ASLR. It is smart enough to just write a script that looks under the doormats of every single digital fortress in the world simultaneously. We are obsessing over the titanium walls of our architecture, completely oblivious to the fact that we have left the keys to the kingdom sitting in plain sight.