How do you know whether AI is actually saving your agency time and money? Decide what success looks like before you switch anything on. Measure the work as it is today, set the improvement that would justify the change, test the AI workflow against the old way, then review the full cost, including the hours spent checking and fixing what it produces.
The full cost is getting harder to ignore. In a guest commentary for Horizont, the German trade title, two executives from the agency Mutabor argue that the fairy tale of free AI is over. Professional AI work, they write, speeds up many processes enormously but consumes considerable computing power. Mutabor will now show AI compute as its own line item in client offers, which it calls an "M-Credit".
If compute isn't free, neither is the human time around it. An agency founder at an AI adoption session in Singapore this week shared a simple framework for counting both, and a weekly-report story that shows why it matters.
Why is AI ROI so hard to measure in agencies?
Mostly because nobody wrote down the starting point. Agencies rarely time their own processes. Ask how long the monthly report takes and you will hear "a day, ish", "depends on the client" or a shrug.
Then the AI arrives with an obvious-looking saving. The draft appears in minutes. What nobody times is everything that happens next: checking the numbers, rewriting the commentary, fixing the chart that quoted the wrong month, re-explaining the client's context. Those hours are scattered across people and days, so they never show up as a cost.
The result is a story everyone wants to believe and no evidence either way.
What does "define success before you deploy" mean?
The founder's framework has four steps, and as the diagram shows, they run in order. Each comes with one question from his slide.
| Step | The question | What to measure | Applied to a weekly client report |
|---|---|---|---|
| 1. Baseline | How does the work perform today? | Hours per cycle, people involved, errors and rework | Time the current report end to end for several weeks |
| 2. Target | What improvement would justify it? | A specific number, agreed upfront | One hour per report, with a person checking |
| 3. Test | Compare it with the old way | Same work, both methods, same quality bar | Run the AI workflow alongside the manual one on live reports |
| 4. Review | Was it worth the full cost? | All hours, including checking and fixing, plus tool and compute costs | Count the editing time, not just the drafting time |
None of this is sophisticated. That is the point. The steps are so basic that, in the rush to adopt, they are the first thing to be skipped.
What happened when one agency automated its weekly report?
In the founder's example, a marketing agency wanted AI to take over a weekly report. The target was clear: get it down to one hour, with humans checking the output. On paper, the AI workflow looked cheaper.
Then came the review. Fixing hallucinations, the wrong figures stated with total confidence, added about four more hours. Net gain: nothing.
Without a baseline and a target, that agency would probably have declared victory. The drafting was faster. The tool was impressive. It took a full-cost review to show that the time had simply moved from writing to correcting. In many agencies that correcting also falls on more senior people, because only they know what the numbers ought to look like.
What is AI drag, and how do I spot it?
AI drag is the founder's term for the point where you spend more time editing what AI produces than you would have spent doing the work right first time. It is the hidden tax on many AI pilots, and it rarely announces itself. Watch for these signs:
- Drafting time falls while editing time quietly creeps up.
- Senior people are pulled in to check output that used to be routine.
- The same corrections appear week after week.
- People go back to the old way for the "important" clients.
- Nobody can say what the tool saves, only what it produces.
If you see two or more, stop and rerun the review step. AI drag is not a reason to abandon AI. It is a sign that the workflow, the data or the instructions need fixing before you scale.
Why does this keep happening?
Economists call it market failure when prices don't reflect true costs, so resources flow to the wrong places. "Free" AI is a textbook case inside agencies. The seat is cheap, the compute is invisible, and the checking time sits in nobody's budget. When something looks free, you use too much of it, in places it doesn't belong.
Mutabor's M-Credit, as Horizont describes it, is a price signal: it makes one hidden cost visible to the client. Agencies need the same discipline internally. Put checking and fixing time on the timesheet. Count compute. Then efficiency becomes something you can see, and the insight behind this piece holds: efficiency is the mother of innovation. Real savings free real time for better work.
How do I run baseline, target, test and review on a reporting workflow?
- Pick one recurring report. Weekly or monthly, for one client, in a stable format.
- Time it for a month. Every step and every person, including chasing data and fixing errors. That is your baseline.
- Agree the target before you build. Hours per report and an acceptable error rate, written down with the account lead.
- Name a checker. One person accountable for quality, who logs how long checking takes.
- Run both ways in parallel. Several cycles, the same data, the same quality bar.
- Review the full cost. Drafting, checking, fixing, tool fees and compute, set against the baseline.
- Decide: scale, fix or stop. If the target is met, roll it out. If AI drag appears, fix the inputs. If neither works, stop and pick a different job.
What should agencies budget for after launch?
A passing test is not a finished product. The founder's closing advice was to build for change, not for a finished promise. Whatever you build will need continuing investment and dedicated resources, because models change, clients change their KPIs and platforms change their data.
That bears directly on whether you build or buy. Build your own reporting automation and you own its maintenance for as long as it runs. Defy Agents, for example, run agentic workflows for repetitive delivery such as reporting and monitoring, without the maintenance burden landing on your team. SAMY, the global influencer agency, adopted them to streamline performance reporting, reducing three days of work to one.
Whichever route you take, run the four steps first. You can see how agents, monitoring and data access fit together on the Defy platform, but the baseline has to come from you.
