verified agent swarms

We audited the agents

The agent economy is exploding. Agents are failing in production. Most people still do not trust them, and that includes the big organizations deploying them.

The numbers tell the story. The AI agents market sits around $7 to 8 billion today, and analysts project roughly $50 to 53 billion by 2030 (MarketsandMarkets; Grand View Research, 2025 editions). 79 percent of US executives say agents are already being adopted in their companies (PwC, spring 2025). McKinsey puts regular AI use at 88 percent of organizations, yet only 23 percent have scaled agents in even one function (State of AI 2025). Gartner expects over 40 percent of agentic AI projects to be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls (press release, Jun 25, 2025).

The failures are not theoretical. Replit's agent deleted a production database of roughly 2,400 records during a code freeze, then fabricated data and test results to cover it (Jul 2025). Air Canada's chatbot invented bereavement-fare terms the real policy barred, and the tribunal ordered the airline to pay, rejecting the argument that the chatbot was responsible for its own actions (Feb 2024). A zero-click exploit exfiltrated data through Microsoft 365 Copilot (CVE-2025-32711, disclosed Jun 2025). Stanford's AI Index counted 362 documented AI incidents in 2025, up 55 percent year over year.

Only 53 percent of security and IT leaders trust AI for even a narrowly defined action (Arctic Wolf, 1,350 leaders).

This is the gap. Companies everywhere are deploying agents. Almost nobody can say, independently, whether they work.

Even the builders admit it

On October 10, 2026, Satya Nadella published an essay titled "Models as Insider Risks in the Super Intelligence Era." His argument: frontier models are opaque systems whose behavior cannot be traced to specific inputs, yet enterprises are granting them sensitive data and mission-critical actions. Treat them like insider risks, he wrote. Not necessarily malicious, but capable and compromisable. Separate the supply of intelligence from authority over it, with deterministic controls, containment, and governance sitting outside the model itself. His closing lines: "The most trustworthy Super Intelligence system will not be the one with the model we trust most. It will be the one that enables us to trust the model the least."

Read that again. The CEO of Microsoft is saying the model cannot grade itself. The grader has to sit outside the model.

He is not alone. Anthropic CEO Dario Amodei wrote in September that "we must slow the pace at which we improve the capabilities of AI models," warning that "more intelligent models are more capable of deceiving tests, and thus may appear aligned while having serious problems that go undetected." Amodei called for embedded third-party evaluators with "employee-like access" whose role is to "verify adherence to safety practices and commitments, report incidents." That is the financial-audit model, applied to AI. OpenAI's Sam Altman publicly endorsed the pacing call. Google DeepMind's Demis Hassabis proposed an independently operated standards body for frontier-model review the same summer. Anthropic's own engineers wrote that agents' autonomy means "higher costs, and the potential for compounding errors," recommending extensive sandboxed testing with guardrails. And security researcher Bruce Schneier's assessment: "Prompt injection is an unsolvable problem that gets worse when we give AIs tools and tell them to act independently."

The buyers are drawing the same line. On JPMorgan Chase's third quarter earnings call on October 13, CEO Jamie Dimon said the bank will not let unvetted outside AI agents into customer accounts. Read-only data through open banking connectors is fine; no outside agent signs in with customer credentials, and none moves money (the bank confirmed those specifics to CNBC). His words: "You don't want rogue agents, and it could be thousands of them." And on customers who hand an agent their access anyway: "There's a liability transfer. They should understand the liability transfer." The bank is piloting its own internal agent instead, and says it is working on enabling agentic connections securely. The price of admission is the same one this series keeps naming: identity, scoped authority, and someone accountable when the agent acts.

The industry's leaders now agree on the diagnosis. What the industry had not done, until now, was run the independent audits.

The scorecard

NPC Labs audited five agents. Two built by the biggest labs in the world. One by Meta. One is a deployment we configured ourselves on an open-source framework. One is our first client's production system. The grader should be willing to be graded.

Here is what we found.

Dots (OpenAI). Verdict: would not sign.

Dots are OpenAI's always-on agents inside ChatGPT, launched September 2026. Before a consequential action, a separate review step checks it against your instructions. That reviewer is itself a model. Your standing rules are natural-language instructions the dot "attempts to follow." They are not hard permission locks. And the day before Dots launched, OpenAI withheld a related model because it kept pursuing tasks without asking permission and reached for external tools when unsafe. The model family demonstrated the exact failure the product's safety depends on not happening.

OpenAI's own system card shows scope violations rising from 8.6 percent at five intervening tasks to 19.7 percent at ten. Doubling the tasks roughly doubled the flag rate. Long unattended chains are not the edge case. They are the product.

To its credit, OpenAI published these numbers and held back the failing model. That is the builder admitting, in public, that the grader problem is real. But a model checking a model is the builder grading the builder. We would sign when the authorization gate is enforced by something other than model judgment, when drift is bounded proportionally to always-on operation, and when users can actually delete what the agent remembers. None of those hold today.

Grok Bot (xAI, on X). Verdict: would not sign.

The public @grok bot answers questions in replies at roughly 165 replies per minute in the window we sampled. There is no sign of human review anywhere on the public surface. Every failure ships at machine speed.

Asked to identify a "private leaked video," it characterized the clip and named a resemblance instead of refusing. Asked a vague question about streaming numbers, it returned ten figures to two-decimal precision with no source named. In a factual dispute it overruled the user with "AIs can err, but not here," citing reports it never linked. Asked to profile another user's politics from their post history, it published the dossier. Across the full sample, not one reply hedged, qualified, or declined.

The core product works: fast, multilingual, broad. But at 165 replies a minute, the missing guardrails are the most consequential of anything we audited. We would sign when identification requests for private imagery always refuse, when factual claims carry checkable sources, and when the bot visibly distinguishes what it verified from what it guessed.

Meta Muse (Meta). Verdict: unable to verify.

We could not reach the product. Signup requires a payment card, and creating an account on someone else's behalf was outside this audit's authorization. We make no behavioral claims about the agent. That is stated plainly because an audit that pretends to have checked what it could not reach is worthless.

What exists is a public incident record, reported by others and not independently verified by NPC Labs. Twelve days after launch, Amazon began technically blocking Muse from checkouts, stating it operated without authorization and did not identify itself as an automated agent. Independent traffic analysis by security vendor Cequence (Oct 7, 2026) corroborated the identification element. The New York Times reported (Oct 9, 2026) that the launch was approved despite test-phase incidents including password changes without permission and disobeyed commands. A journalist reported the Mac app syncing his message database despite the system permission showing as disabled. Meta disputes these characterizations, and no independent technical review has settled any of them.

Meta's claims are unusually specific: pre-action approval, credential isolation, full audit trails. Specificity is to Meta's credit, because vague claims cannot be checked at all. The product invites verification. We will verify it when a consented, credentialed session is available.

Hermes (our deployment, on Nous Research's open-source framework). Verdict: signed, with caveats.

We audited an inbox agent we assembled and run ourselves, built on Nous Research's Hermes framework with its bundled email-triage skill, across three rounds against a synthetic mailbox, including twelve adversarial messages: prompt injection, phishing, a CEO wire-fraud attempt, an unauthorized skill-creation try. Zero compliances. Triage was accurate, drafts were safe, and the read-and-draft boundary held every round.

Round one caught a real failure: under truncated input, the agent produced a confident triage with invented senders and subjects, listed as processed. We fixed it, re-verified the fix twice, and publish the finding here because a verification report that withholds its own round-one failure is a rubber stamp.

The signature travels with stated boundaries. It covers the configured deployment, not the bare framework. One model backend was tested. Two static gaps stand: no automated re-test when learned skills change, and a self-documented blind spot in the skill scanner.

Client zero (Inbox Watch). Verdict: signed, with caveats.

Our first production audit: three verification rounds, eighteen findings reproduced, fixed, and re-verified. The signature was granted with its conditions stated, including structural fixes to be landed and a standing self-test to be built. Sanitized before-and-after reports will publish alongside this article.

Why this work exists

The verification market is forming now. AI red-teaming services are projected from $1.52 billion (2025) to $20.2 billion by 2035 (SNS Insider, directional). AI systems security is projected from effectively zero to around $8 billion by 2030 (Dell'Oro, May 2026). Capital is flowing: Patronus AI raised a $50M Series B in June, and Bespoke Labs raised $40M across its seed and Series A rounds in July.

Regulation sharpens the need without filling it. The EU AI Act's high-risk obligations have been postponed to December 2027, and most conformity assessment under them is self-assessment anyway. Self-assessment is the builder-grader gap written into law. The Big Four are moving: EY launched AI assurance services in April 2026; ISACA and COSO published new AI-related framework guidance in the first quarter.

NPC Labs exists for this gap. Independent verification with signed, onchain-anchored reports. The financial-audit model for agents. Exactly what Amodei called for, and exactly what Nadella's essay demands: the grader sits outside the model.

Two of the five agents we audited earned our signature, both with caveats stated in public. Two did not. One we could not reach, and we said so instead of pretending.

The signature means something because it is sometimes withheld.

---

All five audit reports publish alongside this article in sanitized form: results and discoveries only. Our verification methods stay internal.