Skip to main content
Back to blog
IA responsable AI Act Human Oversight Product

Human oversight in AI: how to tell if anyone really reviews

If a person approves 99% of what the AI proposes in four seconds, there is no oversight. How to design and measure it under Article 14 of the EU AI Act.

JM
Javier Manzano
CEO & Co-founder • November 16, 2026
Human oversight in AI: how to tell if anyone really reviews

There is a way to comply with human oversight on paper that achieves nothing: put a person in front of a queue of system proposals with an “approve” button. After a week they approve almost everything, in a few seconds, because the system is right most of the time and reviewing each case carefully pays off for nobody.

That effect has a name, automation bias, and it has been studied for decades in aviation and medicine. The EU AI Act cites it explicitly. This post explains what the law requires, why oversight fails and how to measure whether it works in your system.

What the EU AI Act requires

Article 14 applies to high-risk systems, which for Annex III (recruitment, credit, insurance, education, essential services) become mandatory on 2 December 2027 after the delay introduced by the Digital Omnibus. The system has to be designed so that the person overseeing it can:

  1. Understand its capabilities and limitations, and monitor its operation to detect anomalies.
  2. Remain aware of the tendency to over-rely on what the system proposes, automation bias, especially when the system gives recommendations for a person to decide on.
  3. Correctly interpret its output.
  4. Decide not to use it, or ignore, override or reverse what it proposes.
  5. Intervene or stop it.

Article 26 adds the part that falls on the company using it (the deployer): it has to assign oversight to people with the necessary competence, training and authority, and give them the support they need. This is where it meets AI literacy.

Why oversight fails

In 2010, Raja Parasuraman and Dietrich Manzey reviewed the research on how people use automated systems in aircraft cockpits, medical diagnosis and industrial control (Complacency and Bias in Human Use of Automation). Two of their conclusions apply directly to AI:

  • The more reliable the system, the less it gets checked. If it is right 95% of the time, the person stops looking for the other 5%. Approving without looking is the strategy that works best day to day, until the day it fails.
  • Experience and training do not remove it. Experts are affected too. What reduces the effect is the design of the task: what information is shown, in what order, and what it costs to get it wrong.

The best-known European case is the Dutch childcare benefits scandal. The tax administration used a risk model to flag possible fraud, and the model’s flags were treated as evidence: thousands of families had to pay back benefits they were entitled to. The government resigned in January 2021, and the data protection authority later fined the tax administration for using nationality in the processing. There were people in the process, but in practice the system was making the decisions.

How to design it so it works

Show the evidence, not just the conclusion. “Candidate rejected” cannot be reviewed. “Rejected because they lack degree X: the CV shows Y” can. If the system cannot explain what it is basing its output on, the person has nothing to check.

Ask for the decision before showing the proposal, in a share of the cases. On a sample, the person decides first and only then sees what the system proposed. If they almost always agree, good. If the person agrees much more often when they see the proposal first, they are copying.

Separate what cannot go unreviewed. Rejecting a candidate, denying a service or flagging someone as a suspected fraudster are decisions that harm a person, and there the review has to be complete. Classifying a ticket or suggesting a reply can be reviewed by sampling.

Make disagreeing cost the same as approving. If approving is one click and rejecting means filling in a five-field form, the design pushes people to approve. Ask for a reason in both cases, or in neither.

Cap the workload. A person with 400 cases a day oversees nothing. Work out how long a properly reviewed case takes and size the team on that number.

Add a stop button. Someone has to be able to switch the system off and go back to the manual process without asking three departments for permission. And that manual process has to keep existing.

How to measure whether it works

Three metrics, from the easiest to the most revealing:

MetricWhat it showsWarning sign
Approval rateHow much the system gets correctedConsistently above 98-99%, when you know the system is wrong more often than that
Time per reviewWhether there is time to review anythingSeconds on cases that require reading a document
Detection of control casesWhether the person finds the errorsFewer than half of the deliberately planted errors detected

The third one is what actually tells you something. It means slipping into the queue, every so often, system proposals you know are wrong (a well-qualified candidate marked as rejected, a valid application flagged as fraud) and counting how many the person catches. It is the same idea airport security uses with test bags. If the control cases go through without anyone noticing, the real errors go through too.

These metrics have to be logged per person and per period, and reviewed. They are not for penalising the people doing the oversight: a low detection rate usually points to a problem with task design, workload or training.

Where to start

If you already run a system with human review, pull the approval rate and the average time per case for the last three months. That is usually enough to see whether the review is real. If the numbers look bad, the fix starts with the design of the review screen, before training.

Measuring human oversight is one of the tests in our responsible AI audit, together with bias testing and the transparency review. You will find the full timeline of obligations in the EU AI Act guide.

Don't miss a thing

JM

Javier Manzano

CEO & Co-founder at Soamee

Passionate about technology and software development. Sharing knowledge and experiences to help other developers grow.

Did you enjoy this article?

If you need help with your development project, we are here for you.

Human Oversight in AI and Automation Bias

Tell us your challenge. We'll propose a solution.

No commitment. We reply within 24 hours, and within 48 hours you have a proposal with scope, timeline and budget. No fine print.

Book a free call →