How to Build an AI Chatbot: A Developer’s Guide (2026)

In 2026 the practical way to build an AI chatbot is not to train a model — it is to connect an existing large language model (LLM) to your own content. The flow has three parts: split your knowledge base (FAQs, product docs, procedures) into chunks and store them in a vector database; find the chunks closest to the user’s question and hand them to the model as sources — this is RAG (retrieval-augmented generation); stream the model’s answer back to your interface. A support bot that answers from your own documents typically goes live in 2-4 weeks; a bot that can actually do things — look up an order, book an appointment — takes 6-12 weeks. Below: the architecture, working code, what it really costs, and the mistakes that sink these projects.

First decision: which kind of chatbot?

What drives cost and timeline is not the model but what you want the bot to do. There are four types, and complexity compounds at every step:

• Rule-based (flow) bot: Moves through predefined buttons and menus. No AI, cheap, predictable — and stuck the moment a user steps outside the script.
• Knowledge-based (RAG) bot: Answers from your documents. The most common option for customer support, product questions and internal knowledge bases, and the best cost/benefit ratio by a wide margin.
• Action-taking bot (tool/function calling): Looks up orders, books appointments, updates records. The model now calls your APIs, which brings authentication, authorisation and rollback scenarios into scope.
• Autonomous agent: Plans and executes multiple steps on its own. Powerful, but it demands supervision, logging and cost control — we covered that model in our guide to AI agents.

Most companies should start with the second type: a bot that answers correctly from your own content also builds all the infrastructure you need to move to the third.

Architecture: what an AI chatbot is made of

The pieces are the same regardless of provider:

• The interface: a widget on your site, a screen in your app, or a WhatsApp/Slack integration. Streaming the answer token by token makes a large difference to perceived speed.
• The orchestration layer: code on your server. It takes the question, gathers sources, calls the model and returns the result. The API key lives here and only here — never in the browser.
• A vector database: stores embeddings (numeric representations) of your documents. pgvector, Qdrant and Pinecone are all options; if you already run PostgreSQL, pgvector is more than enough for most projects.
• The LLM API: the model that writes the answer. Keep it behind a single service file so the provider stays swappable.
• Logging and analytics: every question, the sources used and the answer should be logged. The only way to improve a bot is to read the list of questions it could not answer.
• Human handoff: the path that transfers the conversation, with its history, to a live agent when the bot is stuck. Projects that skip this lose user trust in the first week.

Code: an endpoint that answers from your own content

Below is the core of a RAG flow in a Next.js API route. The critical part is that the model gets retrieved sources rather than free rein, and is told explicitly not to invent anything that isn’t in them:

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic(); // key is read from the ANTHROPIC_API_KEY env var

const SYSTEM = `You are a customer support assistant.
Answer only from the information in the SOURCES section.
If the answer isn't there, do not invent one — say you'll hand the user to a person.
Keep answers short and reply in the language the user wrote in.`;

export async function POST(req: Request) {
  const { question, history } = await req.json();

  // 1) Retrieve the document chunks closest to the question (RAG)
  const chunks = await searchKnowledgeBase(question, { topK: 5 });

  const context = chunks
    .map((c, i) => `[${i + 1}] ${c.title}\n${c.text}`)
    .join('\n\n');

  // 2) Call the model with those sources
  const response = await client.messages.create({
    model: 'claude-opus-5',
    max_tokens: 1024,
    system: [
      { type: 'text', text: SYSTEM },
      // Long, stable text gets cached: on repeat requests the input cost
      // drops to roughly a tenth.
      { type: 'text', text: `SOURCES:\n${context}` },
    ],
    output_config: { effort: 'low' }, // chat reply: keep latency down
    messages: [...history, { role: 'user', content: question }],
  });

  // 3) Join the text blocks (content is a union type)
  const answer = response.content
    .filter((b): b is Anthropic.TextBlock => b.type === 'text')
    .map((b) => b.text)
    .join('');

  return Response.json({ answer, sources: chunks.map((c) => c.title) });
}

Three things go on top of this skeleton: streaming so the answer appears as it is written, a window on the conversation history (old messages should not be carried forever), and a per-user rate limit. If you also want the bot to talk to your own systems — order status, stock, appointments — the next step is function calling; we covered the general mechanics in what API integration is.

Chatbot projects nearly always fail at the same point: the model states something confidently that was never in the sources. Two measures handle most of it — write the “don’t answer if it isn’t in the sources” rule explicitly into the system instruction, and show the document behind every answer. A bot that cites its source is one whose mistakes users can actually catch.

Cost: how token billing adds up

LLM providers price input and output tokens separately; the logic holds whichever provider you pick. A concrete calculation: a typical support message consumes roughly 2,000 input and 300 output tokens once sources are included. At $5 per million input and $25 per million output tokens, that is about $0.018 per message — roughly $175 for 10,000 messages a month. With a smaller, faster model (say $1 / $5 per million), the same volume lands near $35. On top: vector database hosting (effectively free at small scale on your existing PostgreSQL), one-off embedding generation, and your server.

• Use prompt caching: caching the fixed system instruction and frequently used procedure text cuts input cost dramatically.
• Match the model to the question: a small model for classification and routing, a large one for the hard answers. Using one big model for everything is the most expensive route.
• Cap the conversation history: resending the whole transcript on every message grows cost linearly.
• Set a per-user daily message limit; abuse turns into a billing surprise in month one.
• The big line is development, not the API: the RAG pipeline, interface, handoff and admin panel are the one-off project cost.

The five most common mistakes

• Loading documents without cleaning them: mangled text copied out of PDFs, an outdated price list, three versions of the same page — all of it goes straight into the answers. Answer quality is source quality.
• Picking a chunking strategy at random: chunks that are too small lose context, chunks that are too large lose precision. Splitting on headings with a small overlap is the right starting point for most documents.
• Not designing the human handoff: a bot that traps a user in a dead end costs more than the phone call it was meant to replace.
• Not logging: without knowing which questions went unanswered you cannot improve the bot. That list also doubles as your best content and FAQ roadmap.
• Leaving data protection until later: if conversations contain personal data, retention periods, the privacy notice and where the data is processed all need deciding upfront.

How long does it take, and where do you start?

A realistic calendar: gathering and cleaning sources, 3-5 days; building the RAG pipeline and first tests, 1 week; interface and human handoff, 1 week; validation against a question set before go-live, 3-5 days. Two to four weeks in total. Moving to an action-taking bot adds API integrations, authorisation and refund/cancellation scenarios, pushing it to 6-12 weeks. The right starting scope is a narrow bot covering the 30-50 most repeated questions: widening coverage is easy, winning back trust is not.

Conclusion

The hard part of building an AI chatbot is not the model — it is the data and the flow: well-chunked content, source-grounded answers, logs you actually read, and a path to a human. Get those four right and even a small model performs surprisingly well; get them wrong and the strongest model on the market will still lose users’ trust. We covered pricing and the off-the-shelf-vs-custom comparison in what an AI chatbot costs. If you want us to build a bot that answers from your own content, see our custom software service; for an organisation-wide rollout, see corporate solutions, or send us the size of your document set and get a free quote.