An internal knowledge assistant should retrieve and cite only material the person asking is authorized to see. To make that dependable, preserve access rules through search, generation, and citation—not just in the source folder—and test what happens when permissions or documents change. Begin with a small, well-owned document collection; if you cannot reliably identify a user’s access rights, do not include restricted sources yet.
Treat access control as part of retrieval
An internal knowledge assistant should only retrieve and cite documents the current user is allowed to see. If the system searches a shared index without applying document permissions, it can disclose restricted information even when the answer sounds harmless.
A citation is not protection by itself. The assistant must enforce access before content reaches the model, and the interface must avoid revealing restricted titles, snippets, or metadata. Treat the entire path—from source system to search result to generated answer—as one security boundary.
Map the sources and their rules first
Inventory the repositories the assistant will search: for example, policy files, support records, engineering notes, or shared folders. For each, identify the owner, authoritative copy, access model, and how edits or deletions are detected. Multiple copies of a policy can produce conflicting answers even if each file is technically accessible.
Then map those rules to the assistant’s identity model. Can it know who is asking? Can it check group membership or record-level permissions at query time? If permissions vary by document, a broad “employees only” filter is not sufficient. Where access cannot be represented reliably, exclude the source until it can.
Choose where permission checks happen
A common design is to filter candidate documents using the user’s identity and access rights before their text is passed to the language model. This reduces exposure and keeps citations aligned with what that user can open. Depending on the source systems and search technology, permissions may be synchronized into the index or checked live against the source; each approach has tradeoffs.
A synchronized index can make search practical, but access changes may lag unless updates are reliably propagated. Live checks can reflect current permissions more directly, but add dependency and response-time considerations. Whichever method is chosen, define how the system behaves when an authorization check fails: it should fail closed, not quietly search without the check.
Make sources useful and auditable
A useful citation lets a person identify and open the relevant source they are authorized to view. Store stable source identifiers and enough location detail—such as a section or page reference—to help users verify the answer. Do not rely on a generated title or an old copied URL as the sole reference.
The assistant should distinguish a sourced statement from an inference. If retrieved material does not support an answer, it should say that it cannot find sufficient evidence, rather than filling the gap with confident-sounding general knowledge. Human review remains important for consequential decisions, and users need a clear route to flag missing, stale, or misleading sources.
Test the boundary, not just answer quality
Build tests around both permitted and prohibited access. Use accounts with different roles, then ask questions that should return a document for one account but not another. Check the full response: answer text, citations, snippets, search suggestions, and logs can all disclose information.
Include operational changes. Revoke a user’s access, move or delete a source file, and update a document while the index is being refreshed. Verify how quickly the assistant reflects each change and what happens when synchronization is interrupted. Also test ambiguous questions and conflicting versions; retrieval quality depends on whether the right, current material is found, not just whether a plausible response is generated.
Start narrowly, or skip the assistant
A small pilot around one well-maintained collection is easier to validate than indexing every shared drive at once. Choose a real user task, such as finding the current procedure for a routine internal request. Measure whether users can locate the right source, whether answers are supported, how often searches return nothing useful, and whether access checks behave as intended. Review failures before adding more repositories.
An assistant may be unnecessary when a reliable search function already meets the need, documents are few and easy to browse, or permissions cannot be mapped safely. In those cases, improving document ownership, naming, and ordinary search may be the better first step.
Before deploying an internal knowledge assistant
- Map each document source to its owner, access rules, and update schedule.
- Confirm whether user identity and document permissions are preserved during retrieval.
- Test questions against documents users may access and documents they must not see.
- Require clear source references and make unsupported answers visibly uncertain.
- Test access changes, removed files, stale copies, and failed synchronization.
- Give users a way to report a bad answer and route it for review.
Questions about this approach
Can an assistant cite a document a user cannot open?
Only if it can verify that the user is permitted to open it. A citation to a restricted document can expose its name, contents, or existence. Filter before retrieval and ensure the citation target remains protected by the source system.
What if a document’s permissions change after indexing?
It depends on how access is enforced. A synchronized index can lag behind permission changes, while live checks add reliance on the source system. Test the chosen approach with revoked access and ensure failures do not bypass authorization.
Should we launch if our source permissions are inconsistent?
No. If it cannot reliably identify the user and apply the relevant document rules, keep restricted sources out of scope. A smaller searchable collection or conventional search is safer until access can be represented and verified.
For help applying this approach, explore our AI Development & LLM Integration services.
Discuss your project with Full Blown Studio
Tell us what needs to work better and what you have tried. Discuss your project, email bob@fullblown.com, or call (661) 429-0940.
