In February 2024, the British Columbia Civil Resolution Tribunal (Canada) ordered Air Canada to compensate a passenger because the chatbot on its website invented a refund policy that did not exist. The airline’s defence was memorable: it argued that the chatbot was “a separate legal entity that is responsible for its own actions”. The tribunal rejected it with a line every marketing department should frame: the company is responsible for all the information on its website, “whether it comes from a static page or a chatbot”.
Moffatt v. Air Canada is small in damages and enormous as a precedent. It sets the standard by which courts and regulators are treating conversational assistants: your chatbot speaks on behalf of your company, and what it says binds you.
If you have a chatbot in production, or are about to launch one, this guide walks through exactly what risk you are taking on and which concrete measures reduce it.
What the Air Canada case actually said
The facts are mundane, which is exactly why they are so applicable. A customer asked Air Canada’s chatbot about bereavement fares (discounts following the death of a family member). The chatbot replied that he could buy the ticket at the normal price and claim the difference back within the following 90 days. That policy did not exist: the real policy required requesting the fare before flying. When the customer claimed his refund, the airline refused, and the matter ended up before the Civil Resolution Tribunal.
Three ideas from the ruling are worth keeping:
- It doesn’t matter where the text comes from. The tribunal drew no distinction between “official” content and generated content: if it’s on your website, it’s yours.
- The standard of care is accurate information. Air Canada could not explain why the customer should have trusted a static page more than the chatbot on the airline’s own website. Nobody can require users to audit your channels.
- Hallucination is no defence. That the mistake was made by a language model, rather than an employee, turned out to be legally irrelevant.
None of this is exotic for European law either: pre-contractual information is binding, advertising forms part of the contract, and consumer protection rules shield the customer from incorrect information supplied by the business. The chatbot merely adds a new channel through which your company can assert things.
What the EU AI Act adds
On top of the contractual risk comes the regulatory one. The European Artificial Intelligence Act (AI Act) imposes transparency obligations on AI systems that interact with people: users must know they are talking to a machine, and generated content must be identifiable as such. We have already written in detail about labelling AI-generated content and its deadlines.
For a customer support chatbot, the practical reading is twofold:
- Mandatory transparency: the “hi, I’m the virtual assistant” stops being a courtesy and becomes a requirement. Patterns that pretend to be a human agent are sanctionable territory.
- Expected traceability: even if a support chatbot is not a high-risk system in the Act’s sense, the regulatory logic pushes in one clear direction: being able to reconstruct what the system said, when, and based on what.
The real risk isn’t the fine: it’s the stray claim
When we analyse chatbots in production, the risk pattern is almost never one big catastrophe, but an accumulation of small, uncontrolled assertions:
- An ecommerce bot that promises a more generous return window than the terms and conditions allow.
- A clinic’s bot that offers guidance on a symptom instead of referring to a professional.
- A SaaS bot that “confirms” the basic plan includes a feature that belongs to the higher tier.
- A bot that verbally applies a discount nobody has approved.
Every one of those sentences is a potential contractual obligation, a consumer claim or, in the healthcare case, something worse. And the underlying problem is always the same: nobody knows how many of those sentences the bot is saying, because nobody reviews the conversations or can trace which document each answer came from.
Checklist: seven measures that reduce the risk
None of these measures requires halting the project. All of them are implementable, and we have applied them in assistants in production.
1. Answers with cited sources
The measure with the best effort-to-benefit ratio. If every answer cites the document it comes from, two things happen: the system can only assert what is written down somewhere, and when something goes wrong you can audit in seconds whether the error was in the documentation or introduced by the model. An assistant that doesn’t cite sources is indefensible, in the literal sense: you cannot reconstruct why it said what it said.
2. An explicit topic perimeter
Define by design what the assistant may and may not talk about: prices yes, medical advice no; order status yes, delivery-time promises no. Anything outside the perimeter should produce a handoff, not an improvisation.
3. Knowing when to stay quiet
The behaviour under uncertainty must be to say “I don’t know” and escalate to a human — never to fill in the gap. This gets tested: in every iteration we include batteries of questions with no answer in the documentation, and we check that the assistant doesn’t make them up.
4. A complete conversation log
Store every conversation with a timestamp and the sources used in each answer. It is your evidence if there is a claim, your audit material and, as a bonus, your best continuous improvement tool.
5. Human review by sampling
Someone — your team, your vendor or both — must periodically review a sample of conversations against the actual documentation. Every error on the list above gets caught this way, before an angry customer catches it.
6. Clear identification as AI
The assistant introduces itself as an assistant, doesn’t pretend to be a person, and always offers a route to a human. It complies with the AI Act and, on top of that, manages customer expectations better.
7. Documentation as the single source of truth
The assistant answers from your versioned documentation, not from the model’s general knowledge. If the returns policy changes, you change the document and the assistant changes with it, with a date. On how to prepare that documentation we wrote a dedicated guide.
The question for your vendor
If you already have a chatbot or are evaluating platforms, the liability conversation boils down to one request: “Show me any answer from last week and prove which document it came from”. If the response is an awkward silence, your legal exposure is bigger than you think.
It is, not by coincidence, one of the reasons we built our AI assistant around mandatory source citations and conversation reviews: after reading the Air Canada case, it seemed to us the only defensible design. In our report Deflection is not resolution we develop the metrics for auditing any assistant, including ours.
This article is informational and does not constitute legal advice. For your specific case, consult a lawyer specialising in technology and data protection.