My AI made something up. Now what?

Early on, one of my own client automations wrote about 82 wrong records into their CRM in a single import. Not because it was broken. Because it was confident.

It read the import file, matched the fields it thought were right, and wrote them in without stopping to check any of it. Eighty-two records, wrong, and nothing flagged a single one. The automation didn’t know it was wrong. It just knew what a normal-looking record looks like, and wrote one.

That’s the same failure a hallucination is. AI making something up, or matching something wrong, fluently and with zero hesitation. Not a glitch. Not a bug you can point to. Just a confident wrong answer, delivered in the same tone as a correct one.

What a hallucination actually is

AI doesn’t look things up the way you’d hope. It predicts what a good answer sounds like, one word at a time, based on patterns it’s seen before.

Most of the time that produces something true, because true things and well-written things overlap a lot. But sometimes the most “normal-sounding” answer is just wrong. There was no lookup. There was no check. It read like an answer, so it shipped as one.

This is why hallucinations feel worse than a typo. A typo looks broken. A hallucination looks finished.

This already got a real company sued

Air Canada found this out the expensive way. A customer asked their support chatbot about bereavement fares after a family death. The bot told him he could book a full-price flight and apply for a partial refund afterward. That’s not how the airline’s actual policy worked. He booked it, applied, got denied, and took the airline to a tribunal.

Air Canada argued the chatbot was “a separate legal entity” responsible for its own words. The tribunal didn’t buy it. The company had to pay out, and the ruling made the point plainly: if your AI says it, your business said it.

That’s the part every owner needs to sit with. You don’t get to blame the bot.

Where it bites Main Street hardest

I think about this in three zones.

1. Dates and numbers. Prices, hours, deadlines, quantities. Anything with a specific digit is a place AI will happily guess if you haven’t given it the real number.
2. Policies you never actually wrote. Return windows, warranty terms, cancellation rules. AI has read thousands of other companies’ policies. It’ll borrow one and hand it to your customer as yours.
3. Anything “quoted.” A law, a stat, a source, a quote from a real person. AI can invent a citation that looks exactly like a real one, complete with a name and a year.

If your business touches any of these unsupervised, and most do, you’ve got exposure right now.

I’d add a fourth zone, less obvious than the first three: anything that sounds legal or official. AI is trained on a lot of contracts and terms-of-service language, so it’s genuinely good at producing text that sounds binding. That’s dangerous precisely because it reads so convincingly, whether or not it means anything.

Why this feels new even though it isn’t

Hallucinations aren’t a 2026 problem. They’ve been in every version of every chat model since the first one shipped. What’s new is how many businesses are now handing AI unsupervised, customer-facing jobs. A chatbot answering pricing questions on your site. An email assistant drafting replies your team sends without a second look. A booking tool nobody’s double-checking.

The risk was always there. It just used to be theoretical, because AI mostly lived in a chat window where a human read every word before doing anything with it. Now it’s writing the thing that goes out the door, which is exactly why the review habit below matters more this year than it did two years ago.

The habit that fixes 90% of it

Ask AI to cite where it got the answer, or tell it to say “I don’t know” instead of guessing.

That one instruction changes its behavior more than you’d expect. Something like: “If you’re not certain, say so. Don’t fill in a plausible-sounding answer.” It won’t catch everything, but it moves AI from “always confident” to “confident only when it should be,” which is most of the fix right there.

A 60-second check before anything goes out the door

Before a customer-facing AI reply gets sent, I run it through the same four checks every time.

1. Any date or number in here. Does it match my real records?
2. Any policy claim. Is that actually my policy, word for word?
3. Any quote or citation. Does that source exist, and does it say that?
4. Any promise. Would I keep it myself if a customer held me to it?

If all four pass, send it. If one’s shaky, that’s the one line you fix before it leaves your hands.

That’s the entire routine. Not a compliance department, just a minute of reading before you hit send.

When to loop in a human no matter what

Some things I don’t let AI touch unsupervised, full stop. Anything with legal language, a contract, or tax advice with real numbers behind it goes to an actual lawyer or accountant first. AI can help you draft the question you’ll ask them. It shouldn’t be the answer you act on.

The line I use: AI is great at getting you 90% of the way to a first draft. The last 10%, the part where being wrong actually costs you money or a lawsuit, still needs a human who’s looking at your specific situation.

I’d rather look overly cautious than explain to a client why the AI promised something my business never agreed to. That trade is easy for me.

I run most of my own business on AI systems at this point. An AI that handles my scheduling, one that calls me when a decision needs a human, automated meeting notes that file themselves. The thing that makes all of it safe to run isn’t that the AI is smart. It’s that nothing customer-facing goes out without a check first. That’s the boring part nobody talks about, and it’s the part that actually matters.