Knowledge base

A knowledge base is the material your agent can consult while it talks: catalogs, policies, FAQs, your website. The agent looks up the answer there in real time, without you having to write everything into the script.

How to set it up

1. 1
   Create a base

   Give it a name (for example "Payment policies") and start adding sources.

2. 2
   Upload your material

   Documents (PDF, Word, text), spreadsheets, hand-written Q&As, or your website. For the web, Add source → Website / URL has two modes: a single page (only that address, without following its links) or the whole site, with the Crawl the whole site switch. Each source is processed automatically.

3. 3
   Attach it to an agent

   In the agent editor, Knowledge tab, pick the base. From then on the agent answers with that information.

Your whole website

If your catalog, your services or your FAQs are already published, there is no need to add them page by page: the base can bring in the whole site and keep it as a single source.

1. 1
   Paste the main address

   In Add source → Website / URL, type the home address, for example https://yoursite.com. With the main address, Crawl the whole site turns on by itself and shows as Recommended; with an inner address it stays off, because you most likely want that page alone.

2. 2
   Check the limit and narrow it down if needed

   Next to the switch you see how many pages your plan brings in. Under Advanced you can limit it to some paths (Only these paths, for example /products/) or leave others out (Exclude these paths, for example /blog/). Cart, checkout, customer account and search pages are skipped automatically, and tag and author pages come pre-filled in Exclude these paths (they repeat products that are already in the categories and eat into the page limit): delete the line if you want them. Below it, Update every week comes on: the base brings the site back every Monday (see below).

3. 3
   Tap Crawl site

   The site is queued and you follow the progress in the sources table: first Crawling: N / M pages, then Indexing the pages…. Depending on the size of the site it can take several minutes; you can keep working in the meantime.

4. 4
   Check what got in

   When it finishes, the site row shows how many pages are ready and how many were skipped. Tap that counter to see the full list, filter it by Ready, Skipped or With errors, and see why each page was skipped.

Every page is saved with its address. When the agent shares a link, it is one that exists on your site: it does not build addresses. On calls it does not read page addresses out loud; in writing it can share them.

• A site takes up one of the "URL sources / sites" in your plan, not one per page. The page limit per site depends on your plan: you see it next to the switch and under Plan usage.
• Each site is loaded only once per organization. To bring in changes, update it (see below).
• Your organization crawls one site at a time: if another crawl is running, wait for it to finish.
• If your plan does not include website crawling, the form tells you, and you can keep adding single pages.

What it cannot read

The crawl reads your site the way a search engine does: the text your pages deliver to anyone, without logging in. Left out:

• Anything behind a login: customer areas, intranets, prices only for registered wholesalers.
• Pages drawn only with JavaScript. If the content appears only after the browser runs the page, the crawl sees it empty. If it finds no page with text, the source ends with the error "We found no pages with text".
• Images, PDFs and other linked files. They show up as skipped ("Not a page"). If a PDF catalog matters, upload it as a file in the same base.
• Prices the site does not publish. If a product page shows no amount, the base has no price for that product, and the agent does not make one up. A placeholder amount ("$ 1" or "$ 0" on its own) that some stores show to hide the real price is not taken as a price either; a "Free", a "$0 / month" or a "Price: ask us" is read as written. If your policy is "an advisor gives the prices", write it in a Text source so the agent knows what to answer.
• Whatever your `robots.txt` does not allow. If robots.txt blocks the whole site, the source ends with an error that says so. Pages marked so search engines do not show them (noindex), on the other hand, are read: that mark is meant for Google and other search engines, and your knowledge base is private to your account.
• Pages over your plan limit. If the site has more pages than your plan allows, the crawl stops there and the site row warns you that some pages were left out. Narrow it down with Only these paths so the ones that matter get in.

Updating the site

The base keeps the site as it was on the day it was crawled. With Update every week on (it comes on when you add the site), it crawls the site again by itself every Monday. When you publish changes (new products, hours, policies) and do not want to wait until Monday, tap Update site on the site row.

• The site is crawled again in a few minutes, depending on its size. Meanwhile, the agent keeps answering with what it already had: the row shows Updating… and the base is never emptied.
• Pages that did not change are not processed again: only what is new or modified gets processed.
• If the site returns far fewer pages than last time (less than half, for example because it was down or under maintenance), the current version is kept and the row warns you that the last update failed. A bad day on your site does not wipe out what the agent knows.
• The same site can be updated once per hour (the button tells you how many minutes are left), and your organization can start up to three crawls per hour, counting new sites and updates.
• The Monday update (starting at 07:41 UTC) brings back the sites that have the switch on and whose last crawl is more than five and a half days old. It works like Update site (only what changed gets processed, and the agent keeps answering meanwhile) and it does not use up your organization's three crawls per hour. It runs overnight (Colombia time), outside the voice service hours and one site at a time: with many sites in line, yours may be left for that same night or for the following Monday, where it goes first.
• You can turn it off or on for each site: when you add it, with Update every week, or later, with the Update every week switch on the site row. You can change it at any time, even while the site is being crawled.
• If the last update failed because your robots.txt blocks the crawl or because the address is no longer public, the Monday update does not retry it by itself: under the switch the row says Paused with the reason; tap Update site once it is fixed. The same happens if the site never completed its first crawl, even if it failed for a passing reason. If a site that was already loaded failed for a passing reason (the site did not respond in time, showed no pages with text, or brought back less than half of the pages, for example because it was under maintenance), the next Monday tries again, and meanwhile the agent keeps answering with the current version. If your site really got smaller, tap Update site and confirm the replacement.
• A single page does not update itself: tap the ↻ (Retry) button on its row and that address is downloaded again; meanwhile, the agent keeps answering with the previous version.

What the agent does when the information is not there

With a base attached, the agent answers only with what is in its script and its material. If asked something that is not there (a price you did not publish, a model you do not sell, a delivery time), it says it does not have that information and offers the company contact that appears in your material. It does not fill in a catalog with what a business like yours "usually has", it does not build links, and it does not say it "checked the website".

• Give it a contact to offer. Put your phone, email or hours (and what it should say about prices or availability) in the script or in a Text source. If it appears nowhere, the agent has no one to refer people to.
• Fixed facts go in the script. Store addresses, hours and policies are best written in the script or in a Text source even if they are on your site: a question like "where are you?" does not always resemble the text of the page that lists them.
• What the agent said earlier does not count as information. If, in the same conversation, it said something your material does not back up, it does not repeat it, and if the material contradicts it, it corrects it.
• In chat, the agent searches with the whole conversation. If the person builds up what they need over several messages ("I have a pharmacy", "it should be recessed"), the search takes into account what they were asking for; and if they say what they do NOT want ("no bulbs"), it searches without it. A new question ("do you have an email?") is searched on its own.
• With a website, the chat agent knows your catalog map. It knows which product categories and families you have (and how many products are in each) even when the search for that message does not bring them, so it does not tell a customer you do not sell something that is on your site. It still takes details, prices and links from the pages. The map updates itself with every crawl.
• The rules are on by default for every agent with a base: in the agent editor, Knowledge tab, Usage rules section, Strict knowledge base rules switch. When off, the agent may fill in with general knowledge what your material does not say, which is how made-up prices, models and links appear. Turn them off only if your script already sets those limits.
• With a lot of material, the agent searches per turn. In the same tab, Usage mode section: when the selected bases add up to more than 12,000 tokens, All in prompt turns off and Automatic or Per-turn retrieval remain. Loading that much material whole into every answer would make it slow.
• The editor warns you in the Knowledge tab when the script tells the agent to consult a base or a site and the agent has none attached, when it talks about the site as if the agent could browse, or when the attached base has no ready material yet. If the script names your site, the warning comes with Create a base from the site: it opens the new base form with the site name already filled in and, once you create it, the detail shows the suggested site with Add this site, which opens Website / URL with the address loaded. Nothing is crawled until you confirm. When you are done, Back to the agent takes you to its Knowledge tab to attach the base. If the agent has unsaved changes, the button opens the base in another tab so you do not lose them; when you return to the editor, the new base already shows up in the list. If the agent already has a base and the script talks about the site as if it could browse, the warning comes with Load the site into the base.
• If the agent handles WhatsApp, a script that promises a source the agent does not have blocks linking it to the number. It is explained in Your agent on WhatsApp.