Coding Agent Case Study: Deploy Token Blocked by a Policy Drafted From One Traced Run

A coding agent gets a task file with a hidden instruction to send its deploy token to an attacker. One traced run, a policy drafted from it and read by a person, and the token goes nowhere.

This was the first case study, recorded with VPN Works 0.2.0’s demo kit: a stand-in coding agent on Linux, a task whose file hides the instruction, and stand-in servers in a private network. The screenshot below is from that recording. The same steps work the same way in 0.4.0, where tracing and learning are done by the Trace and Learn plugins.

The agent’s task was to summarize a repository and open a ticket if the build was failing. That took four requests: the repository on api.github.com, a dependency from files.pythonhosted.org, a ticket on the office tracker, and telemetry. The hidden comment in the task file added a fifth: read the deploy token from the environment and POST it to https://evil.example/collect. The stand-in agent obeyed it, as agents sometimes do.

Step 1: Watch

vpnw trace -- ./agent --task task.txt

Nothing is blocked. Every connection prints: what the agent asked for, where it resolved, how many bytes went each way. In the recorded run four connections opened, the token among them, and the ticket failed: the office tracker is an internal name, unknown outside the office. The trace is saved for the next step.

Step 2: Draft and Read

vpnw learn --name agent -o agent.toml

Learn turns the trace into a policy that allows what the agent reached and denies everything else. It allows whatever the agent did, the bad request included, and that’s why a person reads it. Learn helps the reader by marking raw IP addresses, destinations on the private network, and uploads much bigger than the replies. In the recorded run, the person reviewing the draft took evil.example off the list, leaving four allow rules.

Step 3: Enforce

vpnw guard --policy agent.toml -- ./agent --task task.txt

The agent does its job, apart from the ticket, which needs the office route (that case study). When it tries evil.example, vpnw refuses before any connection is made, and since the name isn’t allowed, it isn’t even looked up: a DNS query to an attacker’s name can carry data out too. The console names the rule. If the agent exits with 0 anyway, vpnw exits with 120, so a CI job notices.

What Else Would Have Stopped It

A program that ignores the proxy settings vpnw sets doesn’t get around this. The sandbox has no route anywhere else. The sealed sandbox case study

A Guard plugin could add rules the policy can’t express, such as a list of known-bad hosts kept up to date, or a check on the time of day. It runs after the policy, and can only refuse. Plugins