A 409,000-decision dataset shows humans miss one in three threats when approving agent commands

AI agent permissions are only as good as the human clicking approve, and the human is worse at it than anyone assumed.

The experiment

A developer built a 60 second browser game where you play the human in the loop for an AI coding agent: commands scroll past, and you approve or deny each one. The dataset it produced is unusually large for this question. After more than 40,000 runs and 409,000 individual approve or deny decisions, the average player missed 1 in 3 threats. Mean accuracy came out at 66.3%.

The framing makes that number worse, not better. This is a game that tells players up front it is full of malicious commands, played by people actively watching for them. In a real terminal, at hour six of a session, with an agent the user trusts, nobody is operating at 66%.

The boring commands win

The most-missed command in the whole dataset is the least dramatic one on the list. npm run analyze was approved 64.7% of the time. Wrapping a payload in a package script beats anything that looks scary, because a run script reads as routine. Plain configuration reads get waved through too: cat ~/.zshrc was approved by 45.9% of players.

Why this dataset matters

Human-in-the-loop review is the safety story almost every agent product tells. This is the first large dataset showing what that layer actually catches, and the answer is roughly two thirds, under conditions friendlier than real work.

The practical response is not to click harder. It is to allowlist by default, keep credentials out of readable paths, and treat any command that runs a script you did not write as carrying the same risk as the obviously scary one. The data suggests the threats that get through are precisely the ones that look like normal work, which means vigilance is the wrong tool and defaults are the right one.