TLDR: If every AI agent action requires human intervention, organisations cannot realise the full productivity benefits of autonomous AI. If agents operate without meaningful constraints, organisations cannot confidently deploy them against business-critical work. This blog explores ways to preserve agent autonomy while giving humans proactive control over cost, context, and governance for agent work.
An engineering team in a large insurance organisation sought to introduce AI agents to support infrastructure incident diagnosis, with resolution as the longer-term ambition.
They wanted AI agents to lower the engineering effort required for each incident, reducing MTTD and MTTR metrics.
To realise that value, agents needed to operate across existing systems overnight, without an engineer continuously monitoring their progress.
However, initial solutioning exposed two critical gaps.
Uncontrolled cost: Agents continued consuming tokens long after the incremental value of work became questionable, creating significant overnight spend.
Limited accountability: There was no reliable record of what an agent had investigated, which decisions it had made or how it had arrived at a resolution, creating audit risk.
The organisation needed controls on agentic behaviour, equivalent to the ones it had for human-led work.
Running AI Agents in a Glass Box
At V2 AI, we introduce a glass box around the otherwise opaque activity of AI agents. All agents run within predetermined cost, quality, and scope boundaries.
Agents are told exactly what they need to do and what the success and failure criteria are. They act autonomously but must escalate to humans when they hit control boundaries.
Configurable, deterministic, programmatic controls enforce boundaries, governing and limiting agent behaviour throughout its lifecycle. Rules apply to everything from agent context and activity to decision tracking and monitoring for audit trails.
The glass box environment supports the broader principle for enterprise AI: autonomy and control are not opposing objectives. The right controls are what make greater autonomy possible.
A 56-day snapshot of one engineer’s environment showed 51 human approvals, with approximately $5,400 of spend under control and no emergency stops or committed work lost. More than 57 hours of allocated work was completed without an engineer present and producing 63 pull requests for subsequent human review.
The key principles for implementing the glass box are explained below.
Context to Make Consistent Decisions
A persistent context management system gives every agent access to the relevant project history, enabling work to continue across agents and across time. This is foundational to the glass box environment.
The context management system records both information about the work done and the decisions made. Each decision is stored with the options considered and why the others were rejected, and it can be queried later.
For example, if an AI agent is investigating a suspected infrastructure issue, the context system captures:
Systems and logs it investigated
Symptoms it identified
Remediation options it considered
Actions it ultimately took.
This means a subsequent agent can pick up the investigation without starting from scratch. It gives the ability to reconstruct not only what happened, but also decision-making and impact.
This is key to support both output consistency and ongoing learning and improvement when running agents at scale.
Cost Boundaries on Agentic Work
Model selection and explicit cost authorization ensure predictable costs.
An AI agent works iteratively, calling tools and launching other agents until it determines a task is complete. Teams are billed on the tokens it consumes. Cost boundaries limit agent runs to permissible limits, such as how many agents a run may launch and the amount of tokens they consume between human checks.
Reduce Total Spend with Enforced Plan Selection
We programmatically enforce a leader-worker model in which a leader agent breaks down complex tasks into smaller sub-tasks completed by worker agents. The leader agent runs on more expensive, cutting-edge AI models, while workers run more cost-efficiently on models best suited for their task.
For example, for a coding task, the leader assigns every worker a detailed brief with design decisions already made. The brief includes a test suite, a minimum coverage target, a secret scan and other machine gates that determine when the work is done.
Every pull request is a draft that only a human can approve and merge.
Ensure Budget Limits with Explicit Cost Authorisation
To limit runaway spending, it is important to enforce rules on:
Maximum tokens consumed
Maximum number of agents
Maximum number of concurrently running agents
When a limit is reached, the agent must escalate to a human to authorise an exception or raise the budget. Built-in guardrails prevent agents from authorising or auto-approving budget changes.
For example, our system waits for manual entry on a terminal. The system refuses an approval supplied by a program.
The solution can also be extended to stop long-running agents if they exceed budget limits.
Auditable Record of Agent Activity
A cryptographic ledger, built on blockchain principles, can track agent activity alongside approvals, expenditure, and other significant decisions for auditing purposes.
Traditional application logs are not sufficient on their own. They can show that an application was accessed or that an API call occurred, but they do not necessarily provide a complete record of the work performed by an autonomous agent.
For mission-critical work, organisations need to be able to answer basic operational and audit questions:
What system changes were made and by whom?
When did it happen?
How much did it cost?
The ledger provides this record. Each unit of work is assigned a unique ID and tracked through a defined lifecycle like “assigned”, “in progress”, “blocked”, “done”. Approvals, expenditure and other significant events are recorded against the job, creating a chronological history of the work.
For example, when an agent is assigned an infrastructure incident, the ledger records the:
Incident ID
The agent responsible
When investigation started
When a resolution was recommended for human action
Human approvals requested and received
The final cost of the run.
Boost Ongoing Intelligence Capture
The ledger also creates a second benefit: organisational learning. Agents can use the activity history to inform and improve future action. It also supports cost management. For example, agents can record task type, model used and actual cost and this data can later inform future cost estimation.
Planned Handoffs for Enhanced Agentic Productivity
Systematic human-AI handover within cost boundaries allows organisations to safely extend an AI agent’s operating window beyond human work hours.
The glass box environment includes a process that lets engineers allocate agent runs outside business hours.
The engineer has to first plan the task with the agent. The agent assesses the work and raises any questions that require human input. It also estimates a task budget, which is manually approved by the engineer.
The agent then operates autonomously within the agreed boundaries.
The context system gives the agent the information required to continue the work.
Cost boundaries limit expenditure - the agent waits for human input once pre-approved budget limits are reached.
The ledger records activity and decisions for subsequent review.
At the end of the run, the agent writes a handoff note that is programmatically validated. If work is unfinished, the agent must record the reason for failure.
From Agent Autonomy to Enterprise Control
These mechanisms aim to make agentic autonomy bounded, observable, and accountable.
Agents can work for extended periods without continuous human supervision, as the organisation retains control over cost, context and authority. Organisations can run AI agents confidently at scale while deriving maximum productivity and efficiency benefits.
