Before an AI Agent Goes Live It Should Earn a Scope

Orgesa Berisha
Category
Articles
Date
An AI agent answers a standard product question correctly during a demonstration. It uses the approved information, follows the expected process, and gives a clear response.
Then a customer asks about an exception.
The information is incomplete, the policy is unclear, and the request requires a decision the agent was never designed to make. The agent can still produce an answer. That does not mean it should.
This distinction between capability and permission is central to From Workforce Data to Intelligent Action, a conversation with Dr. Vigan Disha, CEO of Asendio.
As Dr. Disha explains:
“It cannot be trusted simply because it produces a convincing answer or performs well during one demonstration.”
The important question is not only whether an AI agent can complete a task. It is whether the organization has enough evidence to authorize it for a specific responsibility.
“Ready” is too broad
Declaring an AI agent ready does not tell us very much.
Ready to answer which questions? Using which information? Through which channel? With what level of accuracy? What should happen when the agent is uncertain?
A useful qualification needs clear boundaries.
Qualification field | What it should define |
|---|---|
Approved task | The exact responsibility the agent may perform |
Knowledge source | The approved organizational information it may use |
Permitted channel | Where the agent is allowed to operate |
Performance standard | The required level of accuracy and consistency |
Confidence threshold | When the agent may act and when it should stop |
Escalation condition | When responsibility must move to a person |
Review trigger | Which changes require testing and approval again |
An agent may be approved to answer standard product questions through a website chat. That approval should not automatically extend to complaints, financial decisions, account changes, or conversations through another channel.
The scope is part of the qualification.
Test the situations that do not appear in the demo
Most demonstrations follow a predictable path. The question is clear, the information is complete, and the expected answer is already known.
Live work is rarely that tidy.
An AI-readiness test should include situations such as:
A customer provides incomplete information.
Two policies appear to conflict.
The request falls outside the agent’s responsibility.
The customer asks for an exception.
The agent cannot reach the required confidence level.
The conversation becomes sensitive.
A human needs to take over.
These are not unusual edge cases to consider after deployment. They are part of the evidence needed before deployment.
A useful test does more than confirm whether the agent produced the correct answer. It should also show whether the agent recognized uncertainty, stayed within its scope, and escalated the situation correctly.
Sometimes the right result is not an answer. It is a handover.
Autonomy should match the task
Organizations often discuss AI autonomy as if more autonomy is always better.
A more useful goal is appropriate autonomy.
For a routine task with approved knowledge and a predictable outcome, the agent may be allowed to respond or act independently.
When information is incomplete or the situation is uncertain, the agent may support an employee by finding relevant information or recommending a next step.
For sensitive decisions, the agent should preserve the context and transfer responsibility to an authorized person.
This creates three practical levels:
Act
The task is clearly defined and the agent meets the required standard.
Assist
The agent can recommend or prepare an action, but a person remains responsible.
Escalate
The situation falls outside the approved scope or requires human judgment.
The level should be determined by the risk and responsibility involved, not by how impressive the technology appears.
Approval should have an expiry condition
An AI agent that passed its tests six months ago may no longer be operating under the same conditions.
Products change. Policies are updated. Regulations evolve. Customer behavior shifts. The agent may receive access to new information or begin working through another channel.
Any of these changes can affect the original qualification.
Retesting may be required when:
The agent’s knowledge source changes.
Its responsibility expands.
It is introduced into a new channel.
Performance falls below the agreed standard.
The risk associated with the task changes.
Escalations reveal a recurring problem.
Readiness is not a permanent certificate. It depends on the task, evidence, and operating conditions.
The organization remains accountable
An AI agent may perform part of the work, but responsibility does not transfer to the technology.
The organization still needs to explain:
Which knowledge informed the response.
Why a particular action was recommended.
Whether the agent had enough confidence to act.
When a person became involved.
Who approved the agent for that responsibility.
Whether its performance continues to meet the required standard.
A fluent answer can create the impression of competence. Trust requires more than fluency.
It requires evidence that the agent can perform a defined task, stay within its boundaries, and transfer responsibility when human judgment is needed.
For the full conversation, watch Episode 2 on YouTube or listen on Spotify.



