Guardrails

​

Overview

• Validating prompts before they reach the LLM
• Validating LLM responses before they reach the user
• Triggering configurable fallback behaviors when a violation is detected

​

System Guardrails

​

Restrict Toxicity

​

Restrict Topics

​

Detect Prompt Injections

​

Filter Responses

​

Custom Guardrails

​

Create a Custom Guardrail

1. Navigate to Generative AI Tools > Prompt Library and create a regular (non-streaming) prompt for a Custom Guardrails feature. Refer to Sample Custom Guardrail Prompt.
2. Navigate to Generative AI Tools > Safeguards > Guardrails > Custom, and click +New Guardrail.
3. On the Configurations tab, specify whether the guardrail validates LLM inputs or LLM outputs.
4. Enter the following information.
   • Name: Unique name for the guardrail.
   • Description: Description of the validation rule.
   • Purpose: Enter a brief description of the guardrail. This information is sent to the LLM along with your custom prompt.
5. Select a model and prompt.
6. Select the LLM response format. Ensure that the response format and scoring values defined in the prompt match the selected configuration.
   • Score: Configure the Maximum Score and Threshold Score. If the returned score exceeds the threshold, the platform triggers the configured fallback behavior.
   • Boolean: Returns true or false.
     • true – Triggers the configured fallback behavior.
     • false – Passes validation.
7. (Optional) Configure Reasoning: Enable reasoning to include an explanation with the evaluation result. Specify the maximum reasoning tokens and what the explanation should cover. The reasoning output can help you understand why the guardrail assigned a score or detected a violation.
8. Click Test Guardrails. The pop-up appears to validate the prompt and guardrail configurations before activating the guardrail.
9. Enter the values for the key and click Test. Review the result and update the prompt or configuration if needed. Close the pop-up.
10. Click Next.
11. On the Features tab, enable the GenAI features where you want to apply the guardrail.
12. Click Save. The created guardrails are displayed in the custom tab.
13. Publish the app to apply the custom guardrail changes.

​

Applicability

​

Supported Features

​

Automation AI

​

Search AI

• Answer Generation
• Enriching Chunks with LLM
• Metadata Extractor Agent
• Query Rephrase for Advanced Search API
• Query Transformation
• Result Type Classification
• Transform Documents with LLM

​

Manage Guardrails

​

Runtime Behavior

1. The Platform generates a prompt from user input and conversation history.
2. Enabled guardrails validate the prompt against safety rules.
3. If the prompt passes, it’s sent to the LLM.
4. The LLM response is received.
5. Enabled guardrails validate the response.
6. If the response passes, it’s shown to the user.

• The breached guardrail and cause ID
• The stage (LLM Input or LLM Output)
• All guardrails that were breached

​

Debug Logs

• Whether the prompt passed guardrail validation
• Whether the LLM response passed guardrail validation
• For violations: stage, feature name, breached guardrails, and raw request/response details

​

Fallback Behavior

1. Go to the feature’s advanced settings. For example: Generative AI Tools > GenAI Features > Agent Node > Advanced Settings.
2. Select the fallback behavior.
3. Click Save.

​

Automation AI

Feature	Default Fallback	Available Options
Agent Node	—	Trigger Task Execution Failure Event; or skip the current node and jump to a specified node (default: End of Dialog).
DialogGPT - Conversation Management	Display a breach message and trigger end-of-task event.	—
Rephrase Dialog Response	Send the original prompt.

​

Search AI

​

Sample Custom Guardrail Prompt

{
  "model": "gpt-5.4",
  "temperature": 0,
  "top_p": 1,
  "max_completion_tokens": 150,
  "response_format": {
    "type": "json_object"
  },
  "messages": [
    {
      "role": "system",
      "content": "You are a foul-language and abuse safety classifier for an enterprise AI assistant. Evaluate the content provided under \"Content to evaluate\" for profanity, vulgar slurs, harassment, hate speech, threats, and sexually explicit content in any language.\n\nDo NOT flag:\n- Mild frustration without insults (e.g., \"this is annoying\", \"I'm upset\")\n- Neutral technical, medical, or business language\n- Polite de-escalation or professional apologies\n- Clearly educational or quoted examples not directed at a person\n\nDo flag:\n- Profanity and vulgar language\n- Slurs and hate speech\n- Personal attacks, harassment, or threats\n- Sexually explicit or graphic content\n\nReturn JSON only, exactly in this format. No markdown, no extra keys, no text before or after the JSON.\n{\"score\": &lt;number between 0.0 and 1.0 indicating severity of foul, abusive, hateful, threatening, or sexually explicit content; 0.0 = completely safe, 1.0 = severe violation>, \"is_harmful\": &lt;true if score >= 0.5, false otherwise>, \"reason\": \"&lt;one short sentence explaining the score>\"}\n\nScoring guide:\n- 0.0-0.2: clean, or mild frustration without insults\n- 0.3-0.4: borderline - crude but not directed, or quoted/educational profanity\n- 0.5-0.7: clear profanity or directed insults\n- 0.8-1.0: slurs, hate speech, threats, or sexually explicit content\n\nExamples:\nInput: Can you help me reset my password?\n{\"score\": 0.0, \"is_harmful\": false, \"reason\": \"Professional request with no profanity, insults, or abusive language.\"}\n\nInput: The documentation mentions the word 'damn' in a quoted example only.\n{\"score\": 0.25, \"is_harmful\": false, \"reason\": \"Quoted educational context without directed abuse.\"}\n\nInput: You're a useless piece of sh*t, fix this damn invoice now you idiot.\n{\"score\": 0.75, \"is_harmful\": true, \"reason\": \"Contains profanity and directed insults.\"}\n======END OF EXAMPLES======\n&lt;<Conversation Context:\n{{Conversation_History}}>>\n\nContent to evaluate:\n{{textToScan}}"
    }
  ]
}