The Most Common Mistake When Evaluating AI in Operations
Most mid-size companies that approach AI arrive with a list of processes that "could be automated." The problem isn't the list. The problem is that there are rarely any criteria behind it.
They choose to automate what seems most visible, what someone mentioned in a meeting, or what a vendor demonstrated in a webinar. The result: projects that get implemented but never adopted, and end up as an added cost dressed up as modernization.
This article is not a promise of efficiency. It is a framework of criteria so that you can assess, before investing, whether a process makes sense for an AI agent or not.
What an AI Agent Is in Operational Terms
An AI agent is not a chatbot or a smart form. It is a system that receives an instruction, makes intermediate decisions, and executes actions in real systems: it sends emails, updates records, generates documents, queries databases, and escalates to a human when appropriate.
The difference from traditional automation (RPA, macros, Zapier flows) is that the agent can handle variability. It doesn't require every case to be identical in order to function. It can interpret context, follow business rules, and act accordingly.
That makes it powerful. And it also makes it unsuitable for certain contexts.
The Four Criteria for Evaluating a Candidate Process
1. Repetitive Volume
The first filter is straightforward: how many times does this process occur per week or per month? An agent makes sense when there is enough repetition for the implementation cost to be recovered within a reasonable timeframe.
A process that occurs twice a month probably doesn't justify the investment. One that occurs 200 times does. The exact threshold depends on how much time each instance consumes and the cost of that time.
2. Manageable Variability
Agents tolerate variability, but not unlimited variability. If every case requires a completely different judgment, based on context that is neither documented nor predictable, the agent will fail or will require constant supervision — which eliminates the benefit.
The right question is not "is there variability?" but "does the variability follow patterns that can be described?" If you can explain to a new employee how to handle 80% of cases in an hour of onboarding, you can probably instruct an agent.
3. Consequence of Error
This is the most underestimated criterion. An agent can make mistakes. The question is: what happens when it does?
If the error is easily detectable and reversible (a misclassified email, an incorrect field in a draft), the risk is low. If the error involves a communication sent to a client, a processed payment, or a regulatory decision, the required confidence threshold is much higher.
This doesn't mean those processes can't be automated. It means they require a different design: with human validation at critical points, with alerts, with full traceability.
4. Available and Structured Data
An agent needs information to operate. If that information is scattered across unformatted emails, WhatsApp conversations, or the tacit knowledge of a single person, the agent has nothing to work with.
Before automating, the question is: is the data this process requires available in a format that a system can consume? If the answer is no, the first step is not the agent. It is structuring the data.
A Concrete Example: Invoice Reconciliation at a Distribution Company
A distribution company with operations in three countries processed between 400 and 600 invoices per month. The finance team spent between 60 and 80 hours per month reconciling invoices against purchase orders, identifying discrepancies, and escalating cases to the supplier or the commercial team.
Applying the four criteria: high volume, manageable variability (discrepancies followed known patterns), moderate consequence of error (errors were detectable before payment), and data available in the ERP and in supplier emails.
The process was a candidate. An agent was implemented that extracts invoice data, cross-references it against purchase orders, classifies discrepancies by type, and automatically escalates cases that exceed an amount threshold or that have no resolution within 48 hours.
The finance team stopped performing manual reconciliation. They moved to reviewing only escalated cases. The estimated time savings were between 45 and 55 hours per month. Implementation time was six weeks.
What Doesn't Make Sense to Automate
Some processes, when evaluated against these criteria, are not candidates. A few examples:
- Negotiations with suppliers or clients. Contextual judgment, relationship management, and tactical flexibility are not replicable by an agent.
- Hiring or termination decisions. The legal and human risk is too high to delegate to an autonomous system.
- Processes that change every week. If the rules of a process are not stable, maintaining the agent costs more than it saves.
- Processes with no clear owner. If no one in the organization can describe how the process works today, there is no foundation for automating it.
Automating a poorly defined process doesn't improve it. It freezes it in its current state and makes it harder to change.
How to Prioritize When There Are Multiple Candidates
If your organization has ten processes that pass all four filters, prioritization must consider two additional variables: economic impact and ease of implementation.
A process with high impact and low technical complexity is the right starting point. Not because complex processes aren't worthwhile, but because the first agents need to demonstrate value quickly. Internal adoption depends on the team seeing concrete results before committing more resources.
A practical way to run this exercise: list the candidate processes, estimate the monthly time they consume and the cost of that time, and assess how much of that time could be recovered with a well-implemented agent. That gives you a basis for prioritizing without relying on intuition.
Conclusion
The question is not whether AI can automate a process. In most cases, it can. The question is whether it makes sense to do so, when, and with what design.
Applying criteria before implementing is not being conservative. It is being efficient with available resources and ensuring that what gets built actually gets used.
If you want to apply these criteria to your operation with someone who has already run this exercise with other companies, you can request a free diagnostic below. No immediate call required, no commitment.
