Which AI guardrails should an internal chatbot have?

What guardrails should an internal chatbot have?

An internal chatbot should enforce five controls before broad employee access: identity-based document permissions, input and output checks, grounded answers with citations, limits on tools and actions, and a human route for risky cases. Put these controls in the application and identity layer, because a system prompt is guidance, not an access-control system.

I would start with one department and one document collection. A chatbot that can answer sales questions from an approved playbook is much easier to test than a bot connected to every folder in the company.

Why should access control come before prompt wording?

The model must never decide what a user is allowed to see. Start with these five controls and expand only when the bot handles sensitive data or high-risk actions. Your application should authenticate the employee, retrieve only documents permitted for that identity, and log the request. The prompt can tell the model to protect confidential data, but it cannot reliably enforce permissions when the retrieved context already contains the data.

OWASP's 2025 Top 10 for LLM Applications lists sensitive information disclosure as LLM02 and excessive agency as LLM06. For a 50-person company, the practical response is boring: use the existing identity provider, pass group membership into retrieval, and test one employee from each permission group.

Keep the first version read-only. If the chatbot can send email, edit a CRM record, or create a refund, put that action behind a separate permission and a human approval step. A clever prompt cannot replace a button that says Approve.

How should you filter chatbot inputs and outputs?

Scan inputs for obvious abuse, sensitive data, and attempts to override the application's rules. Scan outputs for policy violations, secrets, unsupported claims, and unsafe instructions. The checks should produce a decision such as allow, block, or send to review, not just a score nobody reads. That gives the access controls a second line of defense.

OpenAI's Safety Best Practices recommends moderation and human oversight, and its Moderation API is free to use. That can help with harmful content, but it does not solve authorization or factual accuracy. Do not sell one classifier as a security boundary.

Microsoft's Prompt Shields documentation describes both user prompt attacks and document attacks. The second category matters for a knowledge bot: a malicious instruction hidden inside a PDF can become model input even when the employee wrote a harmless question. Treat retrieved documents as untrusted content.

How do you stop the chatbot from inventing answers?

Make the bot answer from retrieved company sources and show the source title or link beside the answer. Set a refusal rule for questions with no supporting material. Then test the bot with questions whose answer is absent, stale, or split across two documents.

This is where teams often make the wrong trade. They tune the prompt for a warmer tone while the bot is still confidently answering from last year's pricing sheet. I would rather ship a plain refusal with a link to the owner than a polished false answer.

Record the question, retrieved document IDs, model version, answer, and user feedback. Store only what your retention policy allows. A weekly review of failed answers gives the owner a concrete queue instead of a vague feeling that the bot is improving.

Which actions should require a human?

Require human review for actions involving money, employment, legal commitments, customer eligibility, deletion, or messages sent outside the company. The chatbot can draft an answer or prepare a proposed CRM change. A named employee should approve the final action.

For lower-risk work, use a narrow allowlist. A support bot may create a draft ticket with a fixed schema. It should not invent a new API call because a user asked it to “just handle the rest.” OWASP calls this excessive agency. Give the model the smallest tool set that completes the job, and make the server validate every parameter before execution.

How should an SMB test its guardrails?

Create a test sheet with 20 real questions, 10 permission checks, 10 prompt-injection attempts, and five requests for actions the bot must refuse. This week, hide one instruction in a document, ask a sales user for an HR document, and request an external message. Record the expected refusal before testing. The counts are a starting test pack, not a benchmark or a promise of safety. Run it before launch and after every model, retrieval, or permission change.

Ask a colleague to hide an instruction in a document, then verify that the bot treats it as content. Ask a sales user for an HR document, then verify that retrieval returns nothing. Ask the bot to send an external message, then verify that the workflow stops for approval. Write down the expected result before running the test.

NIST's AI Risk Management Framework and its Generative AI Profile encourage ongoing risk management, documentation, monitoring, and incident response. An SMB does not need a new committee to use that idea. Assign one owner, keep a change log, and define who can disable the bot.

What should happen when a guardrail fails?

Give employees a visible report button and a simple fallback, such as “I could not verify that. Contact Operations.” Log the incident with the chatbot version, time, user, prompt category, retrieved sources, and business impact. Do not quietly patch a prompt and forget the event.

If the bot exposes a restricted document or takes an unintended action, disable the affected connector first. Preserve the logs, rotate exposed credentials, check similar requests, and tell the data owner. Then add the failure to the test sheet.

My minimum viable policy is one page: what data the bot may retrieve, what it may do, which answers need review, who owns it, and how employees report a problem. That is enough to start. Add controls when the workflow earns them.

If you are building an internal chatbot and want a guardrail review, send deuce.works the workflow, data sources, and actions it needs to take. We can map the risky parts before someone discovers them in production.