What happens when AI agents act on out-of-date company knowledge
Agents that read old policies act on them with a correct-looking citation. This post covers why it happens, what it costs, and six controls that keep agents on current knowledge.
When an AI agent acts on out-of-date company knowledge, it does the wrong thing with complete confidence and a citation that looks fine. It approves an invoice at last year's contract rate, tells a new starter the leave rules from a handbook HR replaced in 2023, or sends a customer terms that were withdrawn in March. The citation is accurate, since that's what the old document says. Nothing in the answer looks wrong until someone who knows the current policy reads it.
This is a bigger risk with agents than with chatbots. A chatbot gives a person a wrong answer, and the person might check it. An agent acts on the wrong answer itself, often before anyone looks.
Why agents pick up old information
The causes are the ordinary state of most shared drives.
- Old versions are never removed. "Leave Policy FINAL v3" sits beside "Leave policy (OLD, do not use)" and the current one, and a search treats all three as equally valid.
- Search ranks by how closely the words match. An older document that's longer, or uses the same words as the question, can outrank the one that replaced it.
- The right answer depends on a date. The rate that applies to an invoice is the one in force on the invoice date, which isn't always the newest rate. An agent that always takes the latest figure gets these invoices wrong too.
- Each agent has its own copy. A support agent built by one team and a finance agent built by another each index their own folder. When the policy changes, one copy gets updated and the other doesn't.
- Nobody owns the conflict. Two current documents disagree, say a travel policy that allows 14 days for claims and an expense procedure that allows 30. The agent picks one, and nobody is told there was a choice.
What it costs
The cost depends on what the agent is allowed to do. An agent that drafts replies for a person to send creates rework. An agent that approves payments, changes records or replies to customers directly creates liability. In February 2024 British Columbia's Civil Resolution Tribunal held Air Canada responsible for a refund policy its website chatbot had described wrongly, and rejected the argument that the chatbot was responsible for its own words.
There's also a slower cost. Once staff catch an agent using an old rule a couple of times, they start checking everything it does, and the time the agent was meant to save goes into checking it.
Six controls that keep agents on current knowledge
| Control | What it does |
|---|---|
| One shared knowledgebase | Every agent reads the same source, so a correction or a replaced document reaches all of them at once. |
| Replaced documents are marked | When a document replaces another, the old one is marked as replaced and the agent is told, instead of finding both. |
| Rules that apply as at a date | The agent uses the rate or rule in force on the date of the transaction, and records which one it used. |
| Conflicts go to an owner | When two current documents disagree, a named person is asked to decide, and the decision is kept. |
| Agents stop when unsure | If the knowledgebase can't support a step, the agent stops and hands it to a person instead of guessing. |
| Every step cites its source | A reviewer can see which document and version each decision relied on. |
The first control does most of the work. If you're deploying several agents across departments, give them one knowledgebase to read from, and keep the per-agent differences in their instructions and permissions.
What this looks like in a run
Here's an invoice-approval agent working through a supplier invoice, written out the way Certant records it. The supplier and figures are made up for this example.
- It matches "Harlow Civil Pty Ltd" on the invoice to "Harlow Civil" in the accounts and to the party on contract 2026-114.
- It finds the invoice is $18,400 against a contract rate of $17,900, and cites schedule B on page 22 of the contract.
- It uses the rate in force on 12 May 2026, the invoice date, and records that a later schedule exists and wasn't used.
- It notes that payments over $10,000 need the finance manager, citing section 3.2 of the delegations policy.
- It finds no purchase order on file, so it stops and sends the invoice to a person.
Each step names the document it relied on. If the contract schedule had been replaced and the agent had used the old one, a reviewer would see that at step 3, before it turned into a supplier dispute three months later.
How Certant handles it
Certant reads your documents into one knowledgebase, and the same knowledgebase sits behind its chatbots, the Agent Builder, Analytics, Strata and the AI FDE. When you correct it or replace a document, each of them uses the change. The page on a knowledge base for AI agents shows how that works.
It flags policies that contradict each other and shows the two passages side by side, and it does the same for procedures that have been replaced and guidance saved more than once. Ask whether casual staff get long service leave and the answer cites the current agreement, then tells you the older handbook said something different and has been replaced. Corrections your people make go into an append-only ledger and are applied again every time new documents arrive; I wrote about that in Teach it once: the corrections ledger.
Agents can stop at a human approval step. When they do, the action waits in an approvals inbox with the reason attached, and approving it carries out exactly the plan the reviewer saw.
A quick test for your own agents
Pick a policy you replaced in the last year. Ask each of your agents, or the assistant behind them, a question where the old and new versions give different answers. Then ask a question where two current documents disagree. A well-built agent gives the current answer, cites it, and tells you about the conflict. If any of them gives the old answer, find out where it's reading from before you let it act on anything.
Frequently asked questions
What happens when AI agents act on unverified company knowledge?
They make decisions that look properly sourced but rest on old, conflicting or wrong documents, such as paying at a superseded rate or quoting withdrawn terms. The organisation is still responsible for the result.
How do you stop multiple AI agents using outdated information?
Give them one shared knowledgebase, mark replaced documents, apply rules as at the relevant date, send conflicts to an owner, and have agents stop and ask a person when the knowledgebase can't support a step.
Can a knowledge base separate current policies from archived versions?
Yes, if it tracks which document replaced which. Certant flags replaced and contradictory documents and tells the reader when an older version says something different. You can test it with the free trial, or book a call to run it against your own policy library.
