How to build an AI agent that answers customer questions

Before any tool, write two lists: what the agent answers, and what it never answers. This single decision shapes everything that follows, and skipping it is the reason most agents get switched off within a month.

The first list is usually the easy one — the questions a new hire would learn in their first week:

• Hours, locations, contact details, and what you actually do.
• Prices and what each price includes, when they are fixed and published.
• Availability, lead times, and how the process works.
• Policies: returns, cancellations, guarantees, coverage areas.

The second list is the one that protects the business. Anything that commits you, anything that requires professional judgment, and anything whose wrong answer is expensive:

• Custom quotes and negotiated discounts — a number the agent invents is a number a customer will hold you to.
• Medical, legal, financial, or safety advice, in any form.
• Complaints, refunds, and anything about an unhappy customer's specific case.
• Anything about a named individual's account or record before identity has been established.

The out-of-scope list is not a limitation you apologise for. It is what makes the agent trustworthy enough to put in front of customers at all.

The agent is only as good as the material it answers from, and that material almost never exists in one place. It is spread across a website written three years ago, a PDF price list, a spreadsheet somebody maintains, and the head of the person who has been there longest.

So the real first task is unglamorous: write the answers down, and settle which version is right. Where two sources disagree — the website says one price, the spreadsheet another — decide which one wins before the agent starts quoting either. A person who finds a contradiction asks a colleague. An agent picks one and says it with total confidence.

This is the same source-of-truth problem that decides whether any process can be automated, covered in which processes are worth automating. It applies with more force here, because the output goes straight to a customer.

The most common misunderstanding about these projects is expecting the model to know your business. It does not, it cannot, and anything it says about you without being told is a guess dressed as an answer. There are three ways to give it the facts, and most real agents use all three.

• Put the facts in the instructions. Works when the set is small and stable — hours, locations, a short price list. Simplest to build and simplest to check.
• Retrieve from a document store. When the material is too large for that, or changes often, the agent searches your documents for the passages relevant to each question and answers from those. This is what people mean by RAG, and it is the usual choice once there is more than a page or two of material.
• Look it up live. When the answer changes by the minute — stock, availability, the status of an order — the agent has to call the system that knows, at the moment it is asked. This is a normal integration with normal failure modes; APIs and webhooks covers how those calls behave.

A useful rule when choosing: if a fact could be wrong tomorrow, do not write it into the instructions. Retrieve it or look it up, so correcting it is an edit in one place rather than a rebuild.

Pick the channel that already carries the volume, not the one that is most fun to build. A widget on a website that nobody visits will answer nothing; the same agent on the WhatsApp number that receives forty messages a day starts earning its cost immediately.

The channel also sets the constraints, and they are not small. WhatsApp in particular requires the platform rather than the free Business app the moment an agent is answering — the decision is worked through in WhatsApp Business app vs Cloud API. Email gives you more room and more latitude on response time. An internal channel like Slack is the safest place to start, because the first audience is your own team.

Most instructions people write are half tone-of-voice and half wishful thinking. The instructions that hold up read like an operating procedure for a new employee, and they are specific about the failure cases rather than the happy ones.

• What the business is, what it sells, and who is asking — in two or three sentences.
• The scope from step one, both halves, stated plainly.
• Where answers come from, and the rule that anything not found there is not answered.
• What to do when it does not know: say so and hand over. This instruction does more for reliability than any other single line.
• Never invent prices, dates, availability, or policies. Name those categories explicitly — a general instruction to be accurate does not cover them.
• How to behave when someone pushes: a customer insisting on a discount does not change what the agent may offer.

Any competent language model will follow this — ChatGPT, Claude, Gemini and the rest are all far past the point where the model is the constraint. What separates a good agent from a bad one is whether someone wrote down what it should do.

An agent with no exit is worse than no agent, because a customer who cannot reach a person leaves rather than complaining. Design the handover first, then the conversation.

It should escalate when:

• The question falls outside the scope list.
• The customer asks for a person — immediately, without negotiating.
• The conversation is going badly: frustration, repetition, or two or three turns without progress.
• The answer was not found in the material, rather than being answered approximately.

Then decide where it lands and who owns it. A handover that reaches an inbox nobody watches is not a handover. This pattern — rules and AI handling what they can, a person taking the rest — is what makes a high-volume agent workable, and it is exactly how this lead response automation handles roughly fifty thousand messages a month without pretending to cover every case.

Answering questions is safe: the worst outcome is a wrong sentence a person can correct. The moment the agent books, cancels, charges, or writes to a customer record, the cost of a mistake changes completely.

Actions are worth adding — an agent that answers a price and then books the appointment is worth far more than one that answers and stops, which is what this WhatsApp booking agent does. But each action needs its own decision about whether it runs unattended or waits for a person, and the automation fit assessment produces exactly that split from how reversible each step is.

The short version: reading, reporting and drafting can run on their own. Publishing, messaging at scale, moving money and deleting records earn an approval step.

The questions you imagine customers ask are cleaner, better spelled, and more reasonable than the ones they actually send. Take the last hundred real enquiries from the channel you picked, run them through the agent, and sort the results into four piles.

• Answered correctly — the target.
• Escalated correctly — also a success, not a failure.
• Should have escalated but answered anyway — the dangerous pile, and the one to work on first.
• Answered wrongly — count these honestly, because this is the only number that decides whether it can launch.

Then try to break it on purpose. Ask for a discount it should not give. Ask something firmly out of scope. Ask in a second language. Ask the same thing three different ways and check the three answers agree — an agent that contradicts itself across phrasings has a knowledge problem, not a wording problem.

Start on one channel, or during hours when someone is available to catch what goes wrong, or with a subset of topics. A narrow launch that works builds the confidence to widen it; a broad launch that misfires in week one usually ends the project.

Log every conversation from day one and read the escalations weekly. They are the most useful document the project produces: every escalation is either a gap in the material, a question that belongs in scope after all, or a genuine limit worth keeping. Build the alerting for the failures too — an agent that stops answering at 2am should wake something, which is the practical side of failure-ready automations.

Prices change, hours change, policies change, and an agent will quote last year's figure with the same confidence it quoted this year's. Knowledge bases rot faster than anyone expects, and the rot is invisible until a customer acts on a wrong answer.

• Give the material a named owner — the person who changes prices should be the person who updates the agent.
• Make corrections possible without a developer. If fixing a wrong answer needs a ticket, wrong answers stay live.
• Review the escalation log on a fixed cadence rather than when someone remembers.
• Re-run the hundred real questions after any significant change, so a fix to one answer does not quietly break another.

An agent is worth building when the same questions arrive often, the answers are stable enough to write down, and there is somewhere for the hard cases to go. Where the questions are all different, or every answer depends on judgment, the honest conclusion is that the work around the conversation is automatable and the conversation is not.

That is a judgment worth making before the build rather than during it. The automation fit assessment asks the same questions in a few minutes, and the general method behind all of this — define the problem, map it, design the failure cases, roll out gradually — is set out in the automation playbook.