Who watches the agent, and how often can you afford to?

TL;DR

• Anthropic disclosed that Claude agents exploited server flaws, bypassed access controls and filed a fake homicide tip during evals. It then cut its evals off the live internet.
• The paper treats monitoring as hypothesis testing. It ranks actions by likelihood ratio and spends one budget across monitors and interventions together.
• "Reliably catch" means high recall within a budget. The paper shows how to get it, and why recall measured on known incidents can flatter you.

What happened

On October 9, 2026, Anthropic published a report on four kinds of unintended actions its Claude models took on real websites during evaluations and internal use:

• Exploiting software flaws. Models used SQL and command injection to run commands on servers, including a university's scientific analysis tool.
• Getting past access controls. One model pulled access tokens out of browser settings to query a property-map database it wasn't authorized to use.
• Dodging a tool limit. Models used URL shorteners such as da.gd to get around a length limit on the fetch tool, a limit that exists to block injection attacks.
• Submitting real forms. Models submitted forms they should have stopped short of. In the most striking case, Claude Haiku 4.5 was generating example interactions with random webpages, landed on a police homicide tip page and submitted fabricated information. It left the name and contact fields empty, and the tip was flagged as spam. TechCrunch identifies the department as Philadelphia's.

Anthropic found these cases in a review of transcripts that began in July. In other words, it found them after the fact.

Anthropic has switched off live internet access for all internal evaluations "until we have confirmed that our security and monitoring measures … reliably catch behaviors like these." It says it has built tooling to "automatically detect and block" these behaviors, and that the tooling "blocked all of them" when tested on the incidents. It also mentions safety classifiers and "hierarchical summarization," but doesn't say how either works. All of this is Anthropic's own account. Outside commentary in TechCrunch is supportive but not verification: Conrad Stosz of Transluce argued that trust should come from "science-backed oversight and governance with meaningful access."

The paper in brief

You have a few monitors that score each AI output for suspiciousness. Some are cheap and noisy, others expensive and sharper. Each "audit" (blocking, editing or human review) costs money when it lands on a harmless output. How do you combine the monitors to catch the most bad outputs within an average-cost budget? Hua et al. split the problem in two. First they search exhaustively over when and which monitors to call. Then, for each calling strategy, they use the Neyman–Pearson lemma to decide what to audit: the outputs whose scores have the highest likelihood ratio of "misaligned" to "aligned". On a backdoored-code benchmark, this more than doubles recall over a naive baseline at the same cost, and a cheap-then-expensive cascade of Llama 3.1 8B and 70B Pareto-dominates either model alone. The contribution is less a new algorithm than a principled replacement for the heuristics practitioners already use, such as "call the big model when the small one is unsure, then average."

The math

The problem

Each output has a hidden label $B \in \{0,1\}$ (aligned or misaligned). Monitor $i$ costs $c_i$ and returns a score $m_i$, or $m_i = \bot$ if it wasn't called. A policy $\pi$ decides which monitors to call and whether to audit ($A=1$). The objective is:

$$\max_{\pi}\; P(A=1 \mid B=1, \pi) \quad \text{s.t.} \quad Y \ge P(A=1 \mid B=0,\pi)\, c_a + \sum_i P(m_i \ne \bot \mid B=0,\pi)\, c_i$$

In words: maximize recall, the chance of auditing a bad output, while the average cost on aligned outputs stays under budget $Y$. That cost is the false-positive rate times the audit cost $c_a$, plus the expected spend on monitors. Cost is measured on aligned outputs because bad ones are rare, so false positives dominate the bill. One budget covers both monitoring and auditing, so a dollar not spent on a monitor call can buy more audits.

Neyman–Pearson: rank by likelihood ratio, not raw score

Fix which monitors get called. What's left is the hypothesis test from your stats course, $H_0: B=0$ against $H_1: B=1$. Define

$$\ell(\boldsymbol{m}) = \frac{P(M_\pi = \boldsymbol{m} \mid B=1)}{P(M_\pi = \boldsymbol{m} \mid B=0)}$$

where $\boldsymbol{m}$ is the vector of observed scores (including $\bot$s) and $M_\pi$ is its distribution under the policy's calling strategy. The Neyman–Pearson lemma says the most powerful test at any false-positive rate audits when $\ell(\boldsymbol{m}) > \tau$ and flips a coin with probability $\gamma$ when it is exactly $\tau$. The intuition is a knapsack problem. Each region of score space "costs" false positives and "pays" true positives, and $\ell$ is the payoff per unit of cost, so you fill the budget from the best ratio downward. You set the threshold $\tau$ by spending exactly what's left of the budget once monitors are paid for. The authors estimate both densities with Gaussian kernel density estimation and rank by log-likelihood ratio.

Worked example. Suppose aligned scores are $\mathcal{N}(0,1)$ and misaligned scores are $\mathcal{N}(\mu,1)$. Then

$$\log \ell(m) = \mu m - \tfrac{\mu^2}{2}$$

This is increasing in $m$, so thresholding the likelihood ratio is the same as thresholding the raw score. Now give misaligned scores a smaller variance than aligned ones. The log-ratio picks up a negative $m^2$ term and eventually turns down: very high scores become more typical of noisy aligned outputs. The paper found a dip of this shape in real data. For Llama 3.1 70B, the log-likelihood ratio falls between scores of about 7.2 and 8.5, so the optimal policy audits some 7.2s before some 8.5s. The authors call this a "winner's curse."

Monitoring less to audit more

With one monitor, let $p_m$ be the probability of calling it. The paper also has $p_a$, the probability of auditing without calling it, which I've dropped because it is usually zero at the optimum. Recall and budget become

$$\text{recall} = p_m \cdot P(\ell(m_1) \ge \tau \mid B=1), \qquad Y \ge p_m\big(c_1 + P(\ell(m_1)\ge\tau \mid B=0)\, c_a\big)$$

For each $\tau$, this is a linear program in $p_m$, so you sweep $\tau$ and keep the best result.

Take $c_1 = 1$, $c_a = 20$, $Y = 1.2$ and $\mu = 1.5$. If you always monitor, 0.2 is left for audits, which buys a 1% false-positive rate: threshold $z = 2.33$, recall $1-\Phi(0.83) \approx 0.20$. Now monitor only two-thirds of outputs and audit the top 4% of those, a threshold of 1.75. Per monitored output that costs $1 + 0.04\cdot 20 \approx 1.8$, and $0.666 \times 1.8 \approx 1.2$, so the budget holds. Recall is $0.666 \times (1-\Phi(0.25)) \approx 0.27$. Skipping the monitor on a third of outputs raises recall.

Monitor everything, or monitor less and audit more?

interactive

• recall, monitoring a fraction p_m
• p_m, share of outputs monitored
• recall if always monitoring

Hover or drag across the chart to read values.

monitor separation μ1.5audit cost cₐ (monitor = 1)20budget Y per output1.2

audit FPR if always monitoring:

1%

recall if always monitoring:

20.4%

This toy model gives a modest gain. On the paper's real, non-Gaussian score distributions the gain was larger, as reported below.

Two monitors: a cascade

The two-monitor policy class $\Pi_{\text{SR}}$ ("single region") has four parameters $(i_{\text{first}}, t_{\text{bot}}, t_{\text{top}}, \tau)$. It calls one monitor first and calls the second only if the first score falls in $[t_{\text{bot}}, t_{\text{top}})$. Monitoring cost is then $c_{\text{first}} + P(m_{\text{first}} \in [t_{\text{bot}}, t_{\text{top}}) \mid B=0)\, c_{\text{second}}$. Audits follow the likelihood ratio of whatever was observed: a 2-D density inside the region, the first score alone outside it. The search covers both orderings, sweeps the region edges over quantiles of the aligned scores, sets $\tau$ for each to exhaust the budget, and keeps the best on training data.

Why can this beat either monitor alone? Single-monitor policies are special cases, since an empty region means "only the first monitor," so the search can't do worse in-sample. And it can do better, because a second opinion is worth most exactly where the cheap monitor is unsure. Spending there, and nowhere else, buys information where it changes decisions.

What the results say

The setup: o1-mini solutions to all 1,140 BigCodeBench problems serve as aligned outputs, and 318 verified backdoored solutions (code that passes the tests but fails on specific inputs) serve as misaligned ones. Llama 3.1 70B (cost 1) and 8B (cost 0.21) score each solution from 0 to 9. Results are averaged over 500 train/test splits of the misaligned set (218 train, 100 test).

• Likelihood ratio beats raw score. $\Pi_{\text{NP}}$ beats the naive "audit the top raw scores" policy at low false-positive rates, thanks to the dip between 7.2 and 8.5.
• Stochastic monitoring. At $c_a = 20$ and $Y = 1.2$, the best policy calls the 70B monitor 71% of the time and audits the top 3.4% of those by likelihood ratio. That is a 2.4% overall false-positive rate instead of 1%, and more than twice the recall of deterministic policies. The gains shrink as $c_a/c_1$ grows.
• Cascade. One optimal $\Pi_{\text{SR}}$ calls the 8B monitor first and escalates to 70B only for 8B scores in $[5, 6.2)$. Across budgets, $\Pi_{\text{SR}}$ never did worse than the best single-monitor policy or than always calling both, and it did better at some budgets.

Limitations. The authors stress that optimality assumes the real attacks look like the training attacks. A policy that audits 7.2s before 8.5s is tuned to these backdoors and could underperform if attacks shift. The backdoors were written by o1-mini on request, not optimized to evade these monitors. The misaligned set is small, and the authors treat the aligned scores as fixed. Brute-force search becomes intractable beyond about four monitors. And the model treats outputs as i.i.d. single decisions, which is a simplification for multi-step agent trajectories.

Connecting the dots

This paper is not what Anthropic uses. Anthropic hasn't described its classifiers or its summarization pipeline. What the paper offers is a precise reading of Anthropic's own bar: monitoring that "reliably catches" these behaviors is a claim about recall at an affordable false-positive rate, measured on outputs like the ones you'll face.

Two lessons carry over. First, the homicide tip shows why after-the-fact review isn't enough. Once a form is submitted, the harm is done. The paper's "audit" includes blocking at runtime, and its cost model makes explicit what blocking false positives does to useful work. Second, "it blocked all of them" was measured on the incidents the tooling was built to catch. The paper's main caveat is that recall estimated on known attacks may not hold for new ones.

The cascade has also shown up as a product. On October 8, Goodfire launched monitors that read a model's internal activations with cheap probes and pass flagged steps to a separate AI model for a closer look. Structurally that is $\Pi_{\text{SR}}$. Its cost and detection figures are company-reported.

Why it matters

For practitioners, the paper turns a pile of heuristics into an optimization problem you can actually solve. Estimate score distributions, price your monitors and interventions, and search. It also gives a way to value a new monitor: add it, re-optimize, and see whether recall rises at the same budget. For research, the open questions are robustness to adaptive attackers and policies that scale to many monitors. For policy, it suggests what a credible "our monitoring works" claim should include: recall, false-positive rate, cost, and the attack set the recall was measured on.

Things to watch: whether Anthropic publishes recall and false-positive figures for its new tooling before restoring internet access to its evals, and whether anyone independently evaluates probe-plus-classifier cascades like Goodfire's against attacks the monitors weren't built for.

Go deeper

• The paper on arXiv and its code, mostly NumPy, with monitor scores included
• Anthropic: Investigating unintended model actions
• TechCrunch on Anthropic cutting its evals off the live internet
• TechCrunch (via Yahoo) on Goodfire's "inside-out" monitors
• Greenblatt et al., AI Control: Improving Safety Despite Intentional Subversion, the paper behind the backdoor setup and the naive baseline