Human in the loop: designing supervision for AI agents

HITL: supervision, not build

Human in the loop is about deciding where humans stay in the loop of an automated system. The build side (the flow, the tools, the business KPIs) is covered in how to create an AI agent. Here, we talk about validation queues, thresholds, mental load and capture.

Three intents often mix: safety (blocking a dangerous action), quality before a customer send, and learning (producing labels to improve the system). If you do not separate them, you end up over-validating everything or letting the risk through.

Autonomy works like a dial: it goes from pure suggestion to automatic action on a strictly defined subset. Many serious business agents start between 20 and 60% effective autonomy by volume.

Without the third brick (control), you just have a POC / a demo.

Where to place humans in the flow

Place HITL where the error cost multiplied by its probability peaks. A standard follow-up email carries a low risk. A price change, a legal send or a VIP client carries a high one.

Then map each action to its supervision level:

Risk number one remains review fatigue. If 100% of outputs need validation, the team ends up clicking through without reading. Only queue amber and red cases, with a clear diff: what changed, the sources, the score.

The same logic holds for code: AI review speeds up the first pass, and humans keep ownership on auth and payments (see AI code review).

Thresholds and business policies

A model confidence score is not enough. Combine it with deterministic rules: an amount above a threshold, a VIP client, an uncovered language, a missing document, detected PII, a tool outside the allowlist.

Calibrate those thresholds on an annotated sample of 50 to 200 cases, measuring precision and recall of the green class. Too low a threshold floods the queue; too high a threshold lets errors through. Recalibrate after every prompt or model change.

Write the policy in plain language, readable outside engineering: "The agent may auto-send J+7 follow-ups if the quote is under €5k and the client is not VIP. Otherwise, sales validation."

Version that policy like code, with the date, the author, the reason and the impacted metric.

Capitalize on validations

Every accept, edit or reject is useful data. Store the input, the agent output, the human action, the comment and the review time.

That log then serves four purposes:

• Feeding corrected few-shot examples
• Identifying recurring error classes
• Spotting autonomy candidates, those cases always accepted without edits
• Training the validators

On the GDPR side, minimize PII in the logs, set a retention period and restrict access. HITL does not excuse an ungoverned data lake.

Ship HITL in 30 days

The rollout fits in four weeks:

1. Map the actions and classify the risks
2. Ship a minimal validation UI and the logs
3. Calibrate the thresholds on a real corpus
4. Track the metrics and expand the green cases a first time

Three roles are enough: a business owner for the rules, a tech owner for the tools and logs, and validators trained with a short checklist.

HITL health shows in the median review time, the share of green cases, the share of edits, post-action incidents and mental load. If the load explodes, shrink the automatic scope.

If you want clean HITL on a real process, we can scope it in 20-40 minutes.