An AI knowledge assistant gives accurate answers when four things are in place: it only reads documents you have chosen, those documents are current with one version of each, they are written so a section makes sense on its own, and every answer shows which document it came from. Most inaccurate assistants are not a model problem. They are a document problem.
Here is how to get the documents right.
1. Choose the sources deliberately
Do not point the assistant at your whole shared drive. It will find the 2019 price list, the draft policy nobody approved and someone's personal notes, and treat them all as truth.
Start with a short list:
- Standard operating procedures and process guides
- Current service descriptions and pricing
- Policies: refunds, returns, HR, security
- Frequently asked questions, from customers and from staff
- Templates and approved wording
For each one, name an owner who keeps it up to date.
2. One version of everything
If there are three versions of the onboarding checklist, the assistant may quote any of them. Before loading anything:
- Delete or archive old versions.
- Put a "last reviewed" date on every document.
- Where two documents disagree, fix the disagreement at the source.
This step alone fixes a surprising share of wrong answers, and it helps your team whether or not you ever use AI.
3. Write so each section stands on its own
Assistants find and quote sections of documents, not whole files. A section that says "as above" or "see the previous page" loses its meaning when quoted alone. Good habits:
- Clear headings that say what the section is about: "Refunds for annual plans", not "Other".
- Name the thing instead of using "it" or "this" across paragraphs.
- One topic per section. Long pages mixing three processes produce muddled answers.
- Tables for facts such as prices, limits and dates, so numbers are not buried in paragraphs.
- Spell out acronyms the first time in each document.
4. Make every answer show its source
An answer your team cannot check is an answer your team cannot trust. The assistant should quote or link the document and section it used, so anyone can click through and confirm. It also shows you immediately when an answer came from the wrong document.
5. Teach it to say "I don't know"
The most important instruction is what to do when the documents do not contain the answer. The assistant should say so plainly and suggest who to ask, rather than filling the gap with something plausible. Test this directly: ask it questions you know are not covered.
6. Control who can see what
Not everyone should get answers from every document. Salary bands, client contracts and disciplinary policies need the same access rules in the assistant as they have in your files. Set this up before launch, not after someone asks the wrong question.
7. Keep it current
- When a document changes, the assistant should read the new version, not keep answering from the old one.
- Review questions the assistant could not answer. Each one points to a gap in your documents.
- Re-test a standard set of questions after big changes, the same way you would test any AI agent.
A simple preparation checklist
- List the 20 questions your team asks most often.
- Find the document that answers each one. Where none exists, write it.
- Remove old versions and add review dates and owners.
- Fix headings and split long mixed-topic pages.
- Decide who can see which documents.
- Load them, then test with the 20 questions plus ten it should not be able to answer.
What it gives you back
Done well, your team finds answers without interrupting the one person who knows, new starters get up to speed faster, and knowledge stays in the business when people leave. Our AI knowledge assistants page covers how we build them, including answers that cite their source and review before anything is sent.
