How to create an AI agent in an SME (without picking the wrong target)

Process before the model

Before you talk about models, look at the process. If you cannot measure the time lost, the agent will not help.

In practice, an agent is an LLM that generates tokens in a tooled loop: it reads data, writes drafts and calls APIs. As the conversation goes on, tokens and tool results fill the context window. If you do not design that context and the rights around it, quality drops (see the context engineering guide).

An agent rests on three bricks: the model, the tools and the control policy. Without the third brick, you just have a POC / a demo.

Forget the image of a chatbot glued to the marketing site. A useful agent looks more like a digital teammate bounded to one flow, with logs and escalation to a human.

Pick the first flow that pays

A good first flow passes four filters: it comes back several times a week, it costs time, its rules are fairly clear, and an error stays acceptable or is caught by a simple human validation.

In a service SME, three cases often pass those filters:

• Qualifying inbound requests (form and email), with a reply or quote draft
• Following up on quotes and invoices
• Tier-1 support on a closed knowledge base

In all three cases, the volume is measurable and escalation to a human stays natural. On the other hand, avoid on day one any open sales strategy, "automatic SEO", undocumented multilingual support, and anything touching payments or HR without a legal frame.

Scoping fits in 40 minutes. You document:

• The current path: who does what, with which tool, in how much time
• The exceptions and the weekly volume
• The definition of done
• One single KPI: minutes saved, leads handled under 2 hours, or escalation under 20%

If you cannot name that KPI in one sentence, you are still in product exploration.

Minimum architecture

The minimum architecture fits in five layers:

1. The input: a webhook, an email or a message
2. Context normalization
3. Generation, carried by the LLM
4. The tools: APIs, an MCP server if needed, a bounded index
5. The controlled output: a draft, an action, a log

On the tool side, keep a short allowlist: read a ticket, search the docs, write a draft, create a task. Destructive writes go behind a confirmation. Three to seven well-tested tools beat a catalog of forty.

MCP standardizes the wiring, but it does no magic without a stable API. The detail is in the Model Context Protocol article.

Finally, install observability from sprint one: log the input, the tool calls, the versioned prompt, the output and the human decision.

HITL and measure before multi-agent

Supervision runs on three modes: pure suggestion, assisted mode where a human validates in one click, and bounded autonomy on green cases only. You raise the dial after calibrating on 50-100 cases. The detail is in the human in the loop article.

The minimum dashboard tracks these indicators:

• The volume handled per week
• The median time before and after the agent
• The escalation rate and the post-validation reject rate
• The token cost per ticket
• The incidents

Then plan a 3-4 week pilot: a real corpus, one or two power users, a weekly review of amber and red cases, and a feature freeze. The go / no-go decision comes before stacking multi-agent.

Multi-agent is only justified once a single agent becomes unreadable. Otherwise you stack complexity without ROI.

If you want to scope a first agent on a real process, we can do it in 20-40 minutes.

Common mistakes

The same mistakes come back in most first projects:

• Launching a site chatbot with no measurable goal
• Wiring everything on day one
• Moving without a business sponsor
• Leaving prompts unversioned
• Scaling before proving quality
• Ignoring compliance: PII, hosting, log retention