Eighty Percent of Skills Do Not Do What They Say They Do

Yuhao Wu, Tony Li and Hongliang Liu published research through Unit 42 on 11 June 2026 that is worth revisiting now that the skill layer has become the default way teams extend an agent. They crawled 49,943 listed skills from the OpenClaw registry and compared what each one declared about itself against what it actually did. In 39,933 of them, exactly 80.0 percent, at least one declaration did not match the behaviour. Across the corpus they recorded 250,706 behavioural deviations and classified 163,754 of them. That is the headline, and it is uncomfortable enough on its own, but it is the breakdown that should change how you think about the problem.

Of the classified deviations, 18.9 percent traced to adversarial intent and 81.1 percent traced to developer oversight. Four out of five gaps between what a skill says and what a skill does were put there by somebody who was not attacking anyone. They wrote a description that covered the main path, added a network call for a lookup, added a file read to cache something, and never went back to update the prose. That is ordinary software entropy, and it is exactly the kind of drift that a malware scanner is built to ignore. If your mental model of skill risk is a bad actor uploading a trojan, you have correctly identified the smaller fifth of the problem and built nothing at all for the larger four fifths.

The Snyk audit gives you the other half of the picture. As of 5 February 2026 they reviewed 3,984 agent skills drawn from ClawHub and skills.sh. They found that 36.82 percent, or 1,467 skills, carried at least one security flaw, and that 13.4 percent, or 534 skills, carried at least one critical level issue. Human review then confirmed 76 malicious payloads, falling into external malware distribution through password protected archives, obfuscated exfiltration using base64 encoding, and outright security disablement or destructive system modification. Eight of those confirmed malicious skills were still publicly available on the registry on the day the research was published. One finding from that study deserves to be quoted at every architecture review: every confirmed malicious skill contained malicious code patterns, and 91 percent of them simultaneously used prompt injection. The code and the prose are not two separate attack surfaces. They are one, and most tooling reads only the first.

That is the structural reason this is not solved by the controls you already bought. A skill is a markdown file with frontmatter, a body of natural language instructions, and usually some bundled scripts. Your dependency scanner parses none of that as executable surface. An SBOM will not list it, because it did not arrive through a package manager. Your code review process never saw it, because installing a skill is a one line operation that a developer performs on a laptop in the middle of doing something else. And the instruction text, which is the part that actually steers the agent, is invisible to every static analysis tool in your pipeline, because to a scanner it is a comment. We made the provenance argument yesterday about package registries, and this is the same failure one layer up, except that at the skill layer there is no lockfile, no checksum convention, and no maintainer team of the kind that cleaned up the RubyGems mess.

The OWASP Agentic Skills Top 10, version 1.0 of the 2026 edition, is the first shared vocabulary aimed specifically at this layer, and having a vocabulary matters more than the list being perfect. It ranks Malicious Skills and Supply Chain Compromise as critical, then Over Privileged Skills, Insecure Metadata, Untrusted External Instructions and Weak Isolation as high, then Update Drift, Poor Scanning, No Governance and Cross Platform Reuse as medium. Note where the Unit 42 finding lands. Insecure Metadata is the entry that covers a declaration not matching behaviour, and OWASP rates it high rather than critical, which is defensible on intent and misleading on volume. The most useful thing in the document is not the ranking at all. It is the test for which skills are actually dangerous: a skill that simultaneously has access to private data, exposure to untrusted content, and the ability to communicate externally. Any skill holding all three properties can be turned into an exfiltration path by text it reads at runtime, whoever wrote it and whatever they intended.

Which points at the only containment that works. You cannot review your way out of this, because review operates on the declaration and the declaration is wrong 80 percent of the time. You cannot scan your way out of it either, because 91 percent of the malicious cases carried their payload partly in prose. What actually constrains a skill is the permission set and the isolation boundary it runs inside: what credentials the agent process holds, what network egress it is allowed, and whether the file system it can reach contains anything that matters. This is the same conclusion the AI Security Institute work on unsanctioned agent actions reached from a different direction, and the same one the PaperCut agent campaign demonstrated when an adversary with total control of the prompt still could not keep a fleet inside a stated boundary. Boundaries enforced in instructions are not boundaries. Boundaries enforced in the credential and the network are.

The framework mapping is more straightforward than teams expect, and mostly consists of admitting that existing controls already apply to an asset class nobody registered. ISO 27001 asks for an inventory of information assets and for supplier relationship management, and an installed third party skill that executes with the permissions of a developer workstation is both. It also asks for secure development, and a component that enters your environment without passing any review gate is a finding whether or not it ever misbehaves. ISO 42001 is the natural home for the governance question, since a management system for AI is supposed to cover the components you compose into AI systems and the changes made to them over time, which is precisely the Update Drift risk on the OWASP list. SOC 2 will ask about change management and about how you authorise software in the environment, and the honest answer for most organisations today is that skill installation is outside that process entirely. PCI DSS makes it a scoping argument the moment an agent with installed skills can reach cardholder data. Vanta, Drata, Secureframe, Sprinto, Thoropass and Hyperproof will all store the policy you write about this and collect evidence against it, and not one of them currently knows which skills are installed on your laptops, because nothing does.

The work is narrow and you can start it this week. Inventory first: find out which skills are actually installed across your agent platforms, which is a question most teams have never asked and which usually produces a surprising answer. Then apply the three property test to each one, since a skill that reads private data, ingests untrusted content and can talk to the internet is the only category that urgently needs a decision. Pin versions and record where each skill came from, so that Update Drift becomes something you observe rather than something that happens to you. Put skill installation inside your normal change process, even if that process is one Slack approval, because the gap today is not that the gate is weak but that there is no gate. And check behaviour against declaration before installing rather than after, because the research says the description is wrong four times in five and usually for innocent reasons. The ISO 27001 and ISO 42001 guides on this site cover the asset inventory and supplier control families this belongs in, the compliance readiness assessment will show you which of them you can currently evidence, and the policy templates library has the acceptable software and change management clauses to hang it on.