What’s best practice for analysing data with AI in marketing and making well-founded decisions? The golden rule fits on one line: treat every AI conclusion as a hypothesis until you can show the data behind it. Everything else (framing the question properly, bringing sources together, reviewing as a team) exists so that rule gets followed without turning every analysis into an audit.
The issue has already reached the contracts. According to Digiday, agencies are working AI into their master service agreements clause by clause rather than rewriting them. Three things trigger the changes: a client asking for a seat in the agency’s proprietary AI tool, a client asking for specific AI language to be added, or a new AI-powered tool being used on a client project. Keri Bruce, a partner at the law firm Reed Smith, puts it simply: “Ultimately, people want disclosure. They want to know what tools are being used.”
Knowing which tool was used is the first step. The next one, which every account director has heard in a meeting, is more uncomfortable: “And how do we know that?” If the answer is “the AI says so”, the insight is worth very little, however good it is.
Why do AI-driven analyses breed mistrust?
Because they often arrive without a trail. A tidy summary, three confident conclusions and no route back to the original data. The client can’t tell whether they’re looking at a solid reading or a plausible synthesis that nobody has checked.
Then add the silos. Social listening lives with one team, campaign data with another, the CRM at the client’s end and competitor ads as screenshots in a deck. AI only analyses what you feed it, so partial sources produce partial conclusions, and every team turns up to the meeting with its own version of reality.
The result is all too familiar: hours spent reconciling figures, meetings to decide which number wins, and a client who, just to be safe, trusts none of them.
Information asymmetry: the problem underneath
Information Asymmetry Theory, which the economist George Akerlof popularised with his famous “market for lemons”, explains what happens when one side knows far more than the other. The buyer, unable to tell good from bad, distrusts everything and pays less for everything.
AI works the same way. If the client can’t see how a conclusion was reached, they discount the value of all of them, the good ones included. And the timing could hardly be worse: according to Digiday, AI reduces the hours the work requires and upends the agency billable-hours model. If the work takes fewer hours and inspires less confidence, margin gets squeezed from both sides.
The asymmetry exists inside the agency too. The analyst knows how the analysis was done; the account director doesn’t; the creative team knows even less. The way out is the same in both cases: share the evidence. Collaboration conquers inefficiency when everyone is looking at the same data and can check the same things.
What makes an AI conclusion verifiable?
Use this table as a checklist before any analysis goes to a client.
| Criterion | “Black box” conclusion | Verifiable conclusion |
|---|---|---|
| Origin | “The AI says so” | Links to specific posts, ads or records |
| Scope | No sources, volume or time period | States sources, volume analysed and dates |
| Reasoning | Mixes observation and interpretation | Separates what was seen from what is inferred |
| Challenge | Only supporting evidence | Includes what contradicts or qualifies it |
| Review | One person, no record | Reviewed by strategy, media and accounts |
| Reproducibility | Impossible to repeat | Someone else on the team reaches the same result |
| In the meeting | Defended with authority | Defended with examples |
AI data analysis best practice, step by step
- Start with the decision, not the data. Write down the question and what you’d do differently depending on the answer. If no answer would change anything, you don’t need the analysis.
- Bring the sources together in one place. Social listening, competitor ads, campaign data and, where possible, client data. The less fragmented the base, the less biased the conclusion.
- Separate observation from interpretation. “Lots of comments mention price” is an observation. “Price is holding back purchase” is an interpretation. Ask the tool to present them separately.
- Insist on traceability back to the original data. Every conclusion should be traceable to the posts, ads or records that support it. In Defy Monitor, for example, every AI conclusion leads back to the real posts behind it, because strategy is reverse-engineered from what audiences and competitors actually publish, not from surveys.
- Go looking for contrary evidence. Ask what contradicts the conclusion. If nothing turns up, be suspicious: either the sample is small or the question was loaded.
- Review as a team before presenting. Fifteen minutes with strategy, media and accounts looking at the same examples saves weeks of misunderstanding.
- Document tools, sources and dates. That’s exactly what, according to Digiday, clients are starting to ask for: which tools are being used and how their data is handled.
If you’d like to see how this workflow is organised, from gathering the data to querying it as a team, take a look at the Defy platform.
How do you break down data silos in an agency?
Not with another weekly meeting, but with a shared evidence base. When strategy, media, creative and accounts consult the same sources, the arguments stop being about which number is right and start being about what to do with it.
It helps enormously if querying that base doesn’t depend on one person. If only the analyst can drive the dashboard, the silo is still there; it’s just human now. Tools that let you ask the data questions in plain language spread access and, with it, the responsibility for checking.
What should go into the contract and the presentation?
Contracts are already moving. According to Digiday, Mile Marker has adjusted clauses on metadata management, intellectual property in AI-generated content and transparency about the tools used, while Go Fish has added brand-safety and data-protection clauses in the past six months. Clients, the outlet sums up, want to know how their data is stored and used, what human oversight there is and which tools are involved.
It makes sense to get ahead with three commitments that needn’t wait for the next renewal:
- An evidence appendix with every deliverable, showing the sources and examples behind each conclusion.
- An oversight line stating who reviewed the analysis and when.
- A tool inventory: which AI was used, for what and with which data.
VML’s Brian Yamada warns in the same piece that a standard cost structure for AI has to be agreed before standard contract language can be written, and advises building in flexibility to revisit the terms. Until that standard arrives, transparency is the best clause no contract contains yet.
