Why does agent testing need a real-world sandbox?
An agent's real risk is not in any single call. It is in the sequence of calls across live services that hold state, enforce permissions, and fail in ways you did not plan for. You cannot see those failures against mocks, because mocks never push back. A real-world sandbox recreates enough of the environment for the agent's genuine failure modes to appear before production does.
The risk lives between systems
Think about hiring a brilliant new employee. They can use Slack, update Salesforce, process refunds, and open GitHub issues. You would never judge them on a written test alone. You would watch what happens when they actually do the work, because that is where things go wrong. Agents are the same. The danger is not in one action, it is in the handoffs between them.
The empty office problem
Testing an agent against mocks is like training that new employee in an empty office. Every door opens, every request is approved, nobody else is editing the same record, and nothing ever rate limits them. They look perfect. Then you put them in the real building, where permissions block them and the system is slow at the worst moment. The mock never prepared them for any of it.
Approaches to building the environment
There are a few ways to get a realistic environment. Vendor test modes cover single services. Research environments like WebArena and benchmarks like tau-bench simulate agent tasks for evaluation. And dedicated cross-service sandboxes recreate several real services together with shared state, which is what a multi-system agent actually runs against.