# Smart Flagging ### Understanding Smart Flagging #### What is Smart Flagging? Smart Flagging is Alhena's intelligent monitoring system that automatically identifies AI responses that may require human review. Rather than requiring administrators to manually review every conversation, the system uses advanced algorithms to detect specific signals that warrant attention. The feature works by scanning conversations in real-time, analyzing AI responses against your approved knowledge base, and flagging only those that meet certain criteria for human review. This targeted approach transforms your quality assurance process from reviewing 100% of conversations to focusing only on the small fraction (typically less than 1%) that might need attention. #### How Smart Flagging Works The Smart Flagging system uses several detection mechanisms to identify potential issues in AI responses: 1. **Outside Knowledge Detection**: Identifies when the AI references information that isn't directly inferred from your approved knowledge base or documentation 2. **Low Confidence Responses**: Detects when the AI shows signs of uncertainty in its responses 3. **Customer Frustration Signals**: Recognizes interactions where sentiment analysis detects potential user dissatisfaction 4. **Policy and Compliance Topics**: Flags discussions involving sensitive areas like refunds, warranties, or other topics where accuracy is critical 5. **System Fallbacks**: Identifies cases where the AI couldn't generate an appropriate response When any of these conditions are met, the conversation is automatically flagged for review, with a clear explanation of why the flag was raised. This allows your QA team to focus exclusively on conversations that truly need attention, rather than sifting through hundreds of routine interactions. ### Accessing and Using Smart Flagging #### Identifying Flagged Conversations When using Alhena's Smart Flagging feature, you'll notice visual indicators that help you quickly identify conversations that require attention: **Orange Flag Indicators in Conversation Lists** In your conversation list, flagged messages are marked with a orange flag icon. This makes it easy to spot which conversations have been automatically identified as potentially needing review. **Filtering for Flagged Conversations** To view only flagged conversations: 1. Navigate to the "Conversations" section in your Alhena dashboard 2. Click on the three-dot menu (⋮) in the top-right corner of the conversation list 3. Select "Flagged" from the dropdown menu This filtering option allows you to focus exclusively on conversations that require your attention, rather than scrolling through all conversations. #### Reviewing Flagged Content When you open a flagged conversation, you'll see a notification banner at the top of the message that says "This message has been flagged. Learn why." This banner provides access to detailed information about why the message was flagged. **Understanding Flag Reasons** To see why a message was flagged: 1. Click on the "This message has been flagged. Learn why." banner 2. The banner will expand to show the specific reason(s) for flagging 3. Review the explanation, which typically identifies statements that are not directly inferred from your knowledge base The system identifies specific statements that are not directly inferred from the ingested knowledge. This transparency helps you quickly understand what potential issues might exist in the AI's response. **Providing Feedback on Flags** After reviewing a flagged message, you can provide feedback to help improve the system: 1. At the bottom of the expanded flag explanation, you'll see the question "Was this message flagged correctly?" 2. Click the thumbs-up (👍) button if you agree with the flag 3. Click the thumbs-down (👎) button if you believe the flag was unnecessary This feedback mechanism helps train the system to better understand what should and shouldn't be flagged in the future, continuously improving the accuracy of the Smart Flagging feature. ### Best Practices for Smart Flagging #### Maximizing the Value of Smart Flagging To get the most out of Alhena's Smart Flagging feature, consider implementing these best practices in your quality assurance workflow: **Establish a Regular Review Cadence** Consistency is key to effective quality assurance: * **Daily Quick Reviews**: Spend 15-30 minutes each day reviewing newly flagged conversations * **Weekly Deep Dives**: Schedule a weekly session to analyze patterns and trends in flagged content * **Monthly Reporting**: Track metrics on flag frequency, accuracy, and resolution to measure improvement over time A regular cadence ensures that flags are addressed promptly while also providing insights into longer-term trends. **Use Flags as Learning Opportunities** Each flagged conversation is an opportunity to improve your AI: * **Update Knowledge Base**: When you encounter legitimate flags, update your AI's knowledge base to address the gap * **Refine Guidelines**: Use patterns in flagged content to refine your AI's guidelines and personality settings * **Document Common Issues**: Maintain a log of frequently flagged issues and their resolutions for team reference **Provide Consistent Feedback** The feedback mechanism is crucial for improving the system: * **Always Respond**: Make it a practice to provide thumbs-up or thumbs-down feedback for every flag you review * **Be Consistent**: Establish clear criteria for what constitutes a "good" flag to ensure consistency across your team * **Explain Disagreements**: When marking a flag as incorrect, note why for your team's reference --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://alhena.gitbook.io/docs/dashboard/conversations/quality-assurance/smart-flagging.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.