diff --git a/ai-for-service/analytics/quality-ai/reports/interaction-conversation-analytics-report.mdx b/ai-for-service/analytics/quality-ai/reports/interaction-conversation-analytics-report.mdx index d1f120f84..e5ff93a5a 100644 --- a/ai-for-service/analytics/quality-ai/reports/interaction-conversation-analytics-report.mdx +++ b/ai-for-service/analytics/quality-ai/reports/interaction-conversation-analytics-report.mdx @@ -86,6 +86,10 @@ Custom fields appear as optional columns and update dynamically based on the sel ![Schedule](/ai-for-service/analytics/quality-ai/reports/images/interaction-run-on-demand.png) + + + The report returns all interactions matching the selected filters. Use the **Duration Status** filter to narrow results by evaluation eligibility. Interactions ingested before duration threshold support may display a `null duration_status` value. + --- ### Processing Status Options in Reports diff --git a/ai-for-service/analytics/quality-ai/reports/scheduled-reports.mdx b/ai-for-service/analytics/quality-ai/reports/scheduled-reports.mdx index d3fd1045c..c87fa5431 100644 --- a/ai-for-service/analytics/quality-ai/reports/scheduled-reports.mdx +++ b/ai-for-service/analytics/quality-ai/reports/scheduled-reports.mdx @@ -127,8 +127,9 @@ This allows you to add filters to refine report data. - **Language**: Select a language configured for Quality AI. Defaults to the application's default language. - **Duration Status**: Filter by evaluation eligibility: Evaluated or Below Threshold. - **Contact Direction**: Filter by Inbound or Outbound to segment report data by contact direction. When applied, the report output includes only interactions matching the selected direction, and the exported file reflects the active direction filter. +- **Processing Status**: Filter interactions by conversation processing state. - Reports exclude Below Threshold interactions by default. +Reports exclude **Below Threshold** interactions by default. To include them, select **Below Threshold** in the **Duration Status** filter. Manually evaluated interactions are always included, and you can select **Delete** to remove a filter row. ##### Configure Optional Fields diff --git a/ai-for-service/quality-ai/analyze.mdx b/ai-for-service/quality-ai/analyze.mdx index f91ec0a83..5d2688656 100644 --- a/ai-for-service/quality-ai/analyze.mdx +++ b/ai-for-service/quality-ai/analyze.mdx @@ -31,7 +31,6 @@ Gain deeper insight into contact center performance, customer experience, and co | Feature | Description | |---|---| -| [Conversation Intelligence](/ai-for-service/quality-ai/analyze/conversation-intelligence) | Post-interaction analytics dashboard covering agent efficacy, customer experience, and contact center efficiency metrics. | | [Customer Experience (CX) Insights](/ai-for-service/quality-ai/analyze/cx-insights) | Post-interaction view of customer experience combining sentiment, resolution, churn risk, and topic-level drivers. | | [Performance Insights](/ai-for-service/quality-ai/analyze/performance-insights) | Time-based visualization of agent communication quality, empathy, language trends, evaluation scores, and sentiment. | | [Topic Discovery](/ai-for-service/quality-ai/analyze/topic-discovery) | Interactive bubble canvas that maps conversation topics to sentiment, resolution rates, and handle times for trend analysis. | diff --git a/ai-for-service/quality-ai/analyze/adherence-heatmap.mdx b/ai-for-service/quality-ai/analyze/adherence-heatmap.mdx index e1698e82d..b5cb4e35c 100644 --- a/ai-for-service/quality-ai/analyze/adherence-heatmap.mdx +++ b/ai-for-service/quality-ai/analyze/adherence-heatmap.mdx @@ -3,9 +3,11 @@ title: Adherence Heatmap sidebarTitle: Adherence Heatmap --- -The Adherence Heatmap provides a visual view of adherence trends across evaluation forms, queues, agents, channels, languages, contact direction, and date ranges to help supervisors identify quality gaps, monitor fatal interactions, compare inbound and outbound performance, and support targeted coaching through metric-level analysis. +The **Adherence Heatmap** provides a visual view of adherence trends across evaluation forms, queues, agents, channels, languages, contact direction, and date ranges to help supervisors identify quality gaps, monitor fatal interactions, compare inbound and outbound performance, and support targeted coaching through metric-level analysis. -This supports shared filters and role-based queue access, where users with Cross Queue Data Access can view all queues, while other users can access only their assigned queues. By default, the heatmap includes only evaluated interactions eligible for aggregation, excluding Below Threshold and Duration Unavailable interactions unless manually evaluated. +By default, only eligible evaluated interactions are included. Interactions marked **Below Threshold** or **Duration Unavailable** are excluded unless manually evaluated, ensuring adherence metrics reflect valid and comparable data. + +The heatmap supports role-based access and shared filters. Users with **Cross Queue Data Access** can view all queues, while others are limited to assigned queues. All visuals update dynamically based on selected filters. Whereas, the **Handled By** filter helps supervisors to refine adherence analysis by selecting **AI Agent**, **Human Agent**, or **Both**, enabling focused or combined views based on conversation type and routing context. ## Key Capabilities @@ -19,10 +21,11 @@ This supports shared filters and role-based queue access, where users with Cross | **Interaction Review** | Opens failed interactions directly in Conversation Mining via the Duration Status filter. | | **Scorecard-based Duration Rules** | Applies scorecard-specific duration thresholds to determine interaction eligibility. | | **Manual Override** | Includes flagged interactions in adherence metrics after manual evaluation. | +| **Handled By Filtering** | Enables filtering by AI Agent, Human Agent, or Both to analyze adherence by conversation type and routing context.| ## Access Adherence Heatmap -Navigate to **Quality AI** > **Analyze** > **Adherence Heatmap**. +Navigate to **Quality AI** > **AutoQA** > **Adherence Heatmap**. ![Adherence Heatmap Landing Page](/ai-for-service/quality-ai/analyze/adherence-heatmap/images/adherence-heatmap-landing-page.png) @@ -135,6 +138,15 @@ Supports multi-select filtering based on configured languages. When applied, the Language options depend on the selected evaluation form. +### AI Agent + +Enable supervisors filter using Experience Flow tags (for example: Payment Status, Account Balance, Address Update) with search, multi-select, and expandable `Show all` options. + +### Human Agent + +Allow filtering by Queues and Agents, both supporting multi-select tagging, search, and `Show all` expansion for easier selection. + + --- ## Heatmap Interactions diff --git a/ai-for-service/quality-ai/analyze/adherence-heatmap/images/adherence-heatmap-landing-page.png b/ai-for-service/quality-ai/analyze/adherence-heatmap/images/adherence-heatmap-landing-page.png index da48a184e..a4b89a12e 100644 Binary files a/ai-for-service/quality-ai/analyze/adherence-heatmap/images/adherence-heatmap-landing-page.png and b/ai-for-service/quality-ai/analyze/adherence-heatmap/images/adherence-heatmap-landing-page.png differ diff --git a/ai-for-service/quality-ai/analyze/agent-dashboard/images/language-settings.png b/ai-for-service/quality-ai/analyze/agent-dashboard/images/language-settings.png index 5b9d778c0..c86a0f8c1 100644 Binary files a/ai-for-service/quality-ai/analyze/agent-dashboard/images/language-settings.png and b/ai-for-service/quality-ai/analyze/agent-dashboard/images/language-settings.png differ diff --git a/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-coaching-insights.png b/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-coaching-insights.png new file mode 100644 index 000000000..809b66746 Binary files /dev/null and b/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-coaching-insights.png differ diff --git a/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-evaluation.png b/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-evaluation.png index df3ef8200..15fef6ab1 100644 Binary files a/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-evaluation.png and b/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-evaluation.png differ diff --git a/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-review-dispute.png b/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-review-dispute.png index 10a4be0d6..74ccbbd54 100644 Binary files a/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-review-dispute.png and b/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-review-dispute.png differ diff --git a/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-review-resolved.png b/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-review-resolved.png index 4f6abc8fa..7faff8627 100644 Binary files a/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-review-resolved.png and b/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-review-resolved.png differ diff --git a/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-view.png b/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-view.png index 4690a2cd4..4ea8f5bf4 100644 Binary files a/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-view.png and b/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-view.png differ diff --git a/ai-for-service/quality-ai/analyze/ai-assisted-manual-audit.mdx b/ai-for-service/quality-ai/analyze/ai-assisted-manual-audit.mdx index 0ccc9c260..732e426f6 100644 --- a/ai-for-service/quality-ai/analyze/ai-assisted-manual-audit.mdx +++ b/ai-for-service/quality-ai/analyze/ai-assisted-manual-audit.mdx @@ -3,9 +3,13 @@ title: AI-Assisted Manual Audit sidebarTitle: AI-Assisted Manual Audit --- -AI-Assisted Manual Audit is where supervisors and QA teams evaluate voice and chat interactions using a hybrid approach that combines AI-generated insights with manual scoring in a single workspace. When **Agent Accept & Dispute** is active, agents use the screen as a workspace to review evaluation outcomes, acknowledge metric scores, and submit disputes. Supervisors then re-evaluate each dispute metric by metric. +AI-Assisted Manual Audit enables supervisors and QA teams to evaluate voice and chat interactions using a hybrid workflow that combines AI-generated insights with manual scoring in a single workspace. The workspace provides transcripts, interaction metadata, timeline navigation, AI-generated summaries, topics, sentiment, resolution status, evaluation metrics, and scorecards to support consistent quality assessments. Evaluators can review AI recommendations, confirm or override scores, add comments, and submit final audit results for coaching and compliance. -The workspace brings together transcripts, interaction metadata, timeline navigation, and AI insights such as summaries, topics, sentiment, resolution status, and metric-level evaluations. Evaluators review AI-generated recommendations, validate or override scores, add comments, and submit audit results to support coaching and compliance processes. The framework supports inbound and outbound interactions, applies duration-based scoring, and uses structured QA forms to drive consistent evaluations. +When **Agent Accept & Dispute** is enabled, agents can use the audit workspace to review evaluation results, acknowledge scores, and dispute individual metrics. Supervisors can then review each disputed metric and complete the re-evaluation workflow. + +When **Scheduled Processing** is enabled for supported conversation sources (Agent AI, CCAI, and Automation AI), conversations become available in AI-Assisted Manual Audit immediately after they end, even before Quality AI processing begins. Until processing completes, only conversation metadata is available. AI-generated insights, transcript analysis, evaluation metrics, and scoring appear after processing is complete. + +Note: **When Scheduled Processing** is enabled, supervisors can open **Pending** and **Validation Failed** conversations directly from **Conversation Mining** before Quality AI processing completes. **Key Capabilities:** @@ -38,7 +42,7 @@ Before using AI-Assisted Manual Audit, confirm the following: - **Role-Based Access**: Appropriate permissions assigned based on your organizational role. - **GenAI Settings**: Enable sentiment, emotions, and topic modeling features as required. - **Metric Settings**: Enable Speech, Playbook, Hold Etiquette, and related metrics when needed. -- **Agent Dispute**: Enable Agent Accept & Dispute in **Quality AI** > **Settings** > **Quality AI General Settings** to activate the metric-level dispute and acknowledgment workflow. +- **Agent Dispute**: Enable **Agent Accept & Dispute** in **Quality AI** > **Settings** > **Quality AI General Settings** to activate the metric-level dispute and acknowledgment workflow. --- @@ -46,7 +50,7 @@ Before using AI-Assisted Manual Audit, confirm the following: Navigate to **Quality AI** > **Analyze** > **Conversation Mining** > **Interactions** > **AI-Assisted Manual Audit**. -![AI-Assisted Manual Audit Page](/ai-for-service/quality-ai/analyze/ai-assist-manual-edit/images/configured-topics.png) +![AI-Assisted Manual Audit Page](/ai-for-service/quality-ai/analyze/ai-assist-manual-edit/images/audit-review-pending.png) You can open interactions from: @@ -84,6 +88,7 @@ The header identifies the conversation, current audit stage, and available workf | **Submit** | Saves the evaluation or re-evaluation and routes it according to the configured workflow. | --- + ## Audit Review Tabs The system displays Audit Review statuses based on the selected Allocations tab and audit workflow stage, helping you track review progress from Assigned through Pending Supervisor Review, Resolved, and Closed. @@ -97,6 +102,18 @@ The system displays Audit Review statuses based on the selected Allocations tab | **Resolved** | Shows that the reviewer or supervisor completed the review process and reached a resolution. | **Audit Review → My Evaluations** (after resolution). | | **Closed** | Shows that the system completed the audit review workflow and no further action is available. | The system finalizes the audit review and prevents further actions. | +---- + +# Processing Status + +Processing Status indicates whether Quality AI has completed conversation processing. + +| Status | Description | +|--------|-------------| +| **Processed** | Quality AI completed conversation processing. All audit insights and evaluation data are available. | +| **Pending** | The conversation is waiting for processing. Only conversation metadata is available. | +| **Validation Failed** | Quality AI could not process the conversation because validation failed. Conversation metadata remains available, but AI-generated analysis and evaluation data are unavailable. | + ## Audit Screen Tabs The Audit Screen provides role-based access to evaluation, transcript review, audit history, and dispute management. diff --git a/ai-for-service/quality-ai/analyze/conversation-intelligence.mdx b/ai-for-service/quality-ai/analyze/conversation-intelligence.mdx deleted file mode 100644 index 780f8a7e7..000000000 --- a/ai-for-service/quality-ai/analyze/conversation-intelligence.mdx +++ /dev/null @@ -1,355 +0,0 @@ ---- -title: Conversation Intelligence -sidebarTitle: Conversation Intelligence ---- - -The Conversation Intelligence dashboard provides post-interaction analytics to help admins and supervisors understand agent performance, customer experience, and interaction details. - -You can filter data by date and time range and by channel: **All**, **Voice**, **Chat**, or **Email**. Use the **Compare** toggle to highlight performance changes between the selected period and the previous one. - -Navigate to **Quality AI** > **Analyze** > **Conversation Intelligence**. - -![Conversation Intelligence Page](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/conversation-intelligence-page.png) - -**Dashboard sections:** - -| Section | What it shows | -|---------|---------------| -| **Contact Center Efficiency** | Average Speed to Answer, Abandonment Rate, CSAT, Transfer Rate. | -| **Agent Efficacy** | Empathy Score, CSAT, Crutch Word Score, Agent Performance Monitor, Agent Occupancy. | -| **Customer Experience** | Average Wait Time, NPS, Churn Risk, Sentiment Score, Customer Churn Monitor. | -| **Insights Mining** | Topic vs. Sentiment bubbles, Keyword Cloud, Emotions. | -| **Interaction Details** | Session date/time, Call ID, Agent Name/ID, Call Reason/Intent, Sentiment Score, Dispositions. | - ---- - -## Filters - -### Create a Filter - -1. Select **Filters**. - - ![Filters](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/dashboard-filter.png) - -2. Select **+ Add New Filter**. - - ![Add New Filter](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/add-new-filter.png) - -3. Choose relevant **Queues** or **Agents**, then select **Apply**. - - ![Filter Options](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/queues-filter.png) - - Selecting **Apply** saves the filter to the **Unsaved Filter** category, letting you review before saving permanently. - -4. Enter a **Filter Name** under **Save Filter**. -5. Enable **Make this the default view** if needed. -6. Select **Save & Apply**. - - ![Save Filter](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/save-filter.png) - -### Manage Saved Filters - -Select the **Filters** tab to view saved filters. - -![Saved Filters](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/saved-filters.png) - -Hover over a saved filter to access these options: - -![Filter Options](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/filter-options.png) - -| Option | Action | -|--------|--------| -| **Duplicate** | Creates a copy of the filter. | -| **Mark as Default** | Sets this filter as the default. | -| **Delete** | Removes the filter. | -| **Edit** | Modifies the filter. | - -#### Duplicate a Filter - -1. Select **Duplicate**. - - ![Duplicate Filter](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/duplicate-filter.png) - -2. Enter a name and select **SAVE**. - - ![Rename Filter](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/rename-filter.png) - - A confirmation message appears after the copy is created. - - ![Filter Cloned Success](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/filter-cloned-success.png) - -#### Mark as Default - -Select **Mark as Default** to set the filter as default. - -![Mark as Default](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/mark-as-default.png) - -![Filter Marked Default](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/filter-marked-default.png) - -#### Delete a Filter - -1. Select **Delete**. - - ![Delete Filter](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/delete-filter.png) - -2. Confirm by selecting **Delete** in the pop-up. - - ![Confirm Delete Filter](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/confirm-delete.png) - -#### Edit a Filter - -1. Select **Edit**. - - ![Edit Filter](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/edit-filter.png) - -2. Make your changes, then select **Save & Apply**. - - ![Save Edited Filter](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/save-edit-filter.png) - -### Clear Filters - -Select **Clear Filters** to reset all filters and show only the current day's data. - -![Clear Filters](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/clear-filter.png) - ---- - -## Date and Time Range - -Filter data by date and time range. The dashboard shows the current day's data by default. Select a range and select **Apply**. - -![Time Selection](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/time-duration.png) - ---- - -## Compare Toggle - -Enable **Compare** to view metric changes between the selected period and the previous equivalent period. The toggle is on by default. - -- **Green upward arrow**: Positive change. -- **Red downward arrow**: Negative change. - -Spike and dip indicators appear across metrics showing percentage change from the prior period. - -![Compare Functionality](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/compare-functionality.png) - ---- - -## Channel Filter - -Filter data by channel: **All**, **Voice**, **Chat**, or **Email**. This applies to the entire dashboard except Agent Occupancy (not tracked per channel). - -![Channels](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/channels.png) - ---- - -## Contact Center Efficiency - -### Key Performance Indicators (KPIs) - -| KPI | Channels | Description | -|-----|----------|-------------| -| **Average Speed to Answer (ASA)** | Voice, Chat, Email | Average time an agent takes to answer inbound calls from when callers join the queue. | -| **Abandonment Rate** | Voice, Chat | Percentage of customers who disconnect before reaching an agent. | -| **Transfer Rate** | Voice, Chat, Email | Percentage of interactions transferred to another resource. | -| **CSAT** | Voice, Chat | Customer Satisfaction score calculated from survey responses. | - -### Default Color Zones - -| KPI | Green | Yellow | Red | -|-----|-------|--------|-----| -| **ASA (Voice)** | Up to 28s | 28-40s | 40s+ | -| **ASA (Chat)** | Up to 35s | 35-50s | 50s+ | -| **Transfer Rate** | Up to 10% | — | Over 10% | -| **Abandonment Rate** | Up to 6% | — | Over 6% | -| **CSAT** | 8-10 | 6-8 | 1-6 | - -The dashboard shows the KPI average for the selected period alongside a percentage change from the previous period. Hover over a color zone to see call distribution by percentage and count. - -![Contact Center Efficiency](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/cc-efficiency.png) - ---- - -## Agent Efficacy - -### Empathy Score - -Measures whether agents respond empathetically when customers express distress. The system classifies customer utterances as empathy-seeking or non-empathy-seeking, then evaluates agent responses accordingly. - -### CSAT - -Displays the average customer satisfaction score on a scale of 1 to 5. - -### Crutch Word Score - -Measures how often agents use filler words (for example, `um`, `uh`, `like`, `you know`, `so`, `basically`). The system detects these across multilingual conversations. [Learn more](/ai-for-service/quality-ai/configure/language-settings). - ---- - -## Agent Performance Monitor - -Visualizes relationships among Empathy Score, Crutch Word Score, CSAT Score, and Sentiment Score to help supervisors make data-driven decisions. - -- **Y-axis**: Agent performance metrics (select one or more). -- **X-axis**: Customer experience metrics (CSAT and Sentiment Score). - -![Agent Performance Monitor](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/agent-performance-monitor.png) - -### Agent Occupancy - -Tracks the percentage of time agents spend handling interactions or performing work-related tasks. - -**Formula:** - -$$ -\text{Agent Occupancy (\%)} = \frac{\text{Total talk/chat time} + \text{Total ACW time}}{\text{Total logged-in time}} \times 100 -$$ - -| Component | Description | -|-----------|-------------| -| **Total talk/chat time** | Total duration of active customer interactions. | -| **Total ACW time** | Total time spent on post-call activities (notes, record updates). | -| **Total logged-in time** | Total time agents are logged in to Agent AI. | - -Agent statuses include `Available`, `Busy`, `Away`, `Break`, and any custom codes configured by the administrator. A pie chart shows distribution across these statuses. - -![Agent Occupancy](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/agent-occupancy.png) - -### Script and Playbook Adherence - -Tracks how consistently agents follow predefined scripts. Adherence displays as a bar graph, with each attribute showing a compliance percentage for the selected period, plus a comparison to the previous equivalent period. - -If no Agent AI playbooks are configured, the system measures adherence against these default conversation etiquettes: - -| Script | Example | -|--------|---------| -| **Greeting** | "Hello, My name is John Doe, and I am your customer support executive. How may I help you?" | -| **Branding** | "Thank you for contacting Mr. John." | -| **Privacy Policy** | "This call gets recorded for quality and training purposes." | -| **Hold Etiquette** | "May I place you on hold for a few minutes while I pull up some information?" | -| **Customer Verification** | "May I know your date of birth?" | -| **Proper Sign Off** | "Thank you for reaching out. It was a pleasure to assist you. Have a great day!" | - -![Script Adherence](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/script-adherence.png) - -Supervisors can select a custom playbook from the dropdown to view adherence to its specific steps. Administrators can configure playbook attributes. - -![Playbook Adherence](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/playbook-adherence.png) - -The **Playbook Adherence** tab is available only when playbooks are configured in Agent AI. - ---- - -## Customer Experience - -Filter customer experience data by language and date range. Available metrics: - -| Metric | Description | -|--------|-------------| -| **All Languages** | Multi-select filter; defaults to all languages. Shows only metrics for selected languages. | -| **Average Wait Time** | Total customer wait time ÷ customers served in the period. | -| **NPS Score** | Net Promoter Score measuring customer loyalty (scale: 0-10). | -| **Churn Risk** | Number of customers who stopped using your services in the selected period. | -| **Sentiment Score** | Average sentiment score per interaction, normalized on a 1-10 scale. | - -![Customer Experience](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/custom-experience.png) - -### Sentiment Monitor - -Classifies interactions as Positive, Neutral, or Negative and infers the likely emotion (for example, Happy, Satisfied, Disappointed). - -The bar chart shows sentiment distribution across intents or topics: - -| Color | Sentiment | -|-------|-----------| -| Green | Positive | -| Yellow | Neutral | -| Red | Negative | - -![Sentiment Monitor](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/sentiment-monitor.png) - -### Customer Churn Monitor - -Displays churn risk in a pie chart, comparing churn percentage with total calls and escalations. Hover over a section to see its value; select a section to view related calls. - -| Category | Description | -|----------|-------------| -| **No Churn/Escalation** | Interactions with no churn or escalation. | -| **Customer Churn** | Interactions where customer churn was detected. | -| **Escalation** | Interactions where the customer requested supervisor assistance. | - -![Customer Churn Monitor](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/customer-churn-monitor.png) - ---- - -## Insights Mining - -Shows the top 30 topics by volume. Use this to identify topics associated with sentiment, keywords, and emotions for the selected date range and channel. - -![Insights Mining Default Screen](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/insights-mining-default-screen.png) - -By default, this widget is blank. Select a date range and topic to populate sentiment scores. - -### All Languages - -Use the **All Languages** dropdown to filter by language. The system updates interaction sentiment scores and bubble plots across all widgets when you change language or channel. - -### Topics - -Displays graded sentiment scores (1-10) as bubbles. Each bubble represents a topic's volume and sentiment distribution. Hovering over a bubble shows the average emotion label (for example, angry, frustrated). - -**Bubble color by sentiment:** - -| Color | Sentiment Range | Meaning | -|-------|-----------------|---------| -| Green | 6-10 | Positive | -| Grey | 4-6 | Neutral | -| Red | 1-4 | Negative | - -Bubble size reflects interaction volume relative to the minimum and maximum for the selected period. - -Only the top 30 topics (by interaction volume) appear on the widget. - -**Example**: The **Payment** bubble shows Positive 68% (green), Neutral 10% (gray), Negative 22% (red). The circumference represents these percentages. Hovering shows the average emotion index. - -![Topics Percentage](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/topics-percentage.png) - ---- - -## Keyword Cloud and Emotions - -Shows relevant keywords for the selected topic, excluding stop words and common terms. - -![Insights Mining](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/insights-mining.png) - -### Keyword Search - -Use keyword search to find and analyze keywords across interactions by topic: - -- Select one topic or **All Topics**, then enter a keyword. -- Hover over a keyword to see total mentions and unique interactions. -- Select a topic bubble or use the dropdown to filter related keywords. -- Select **All Topics** to view keywords across all topics. -- Select a semantic variation to view its related interactions in the Interaction Details panel. - -![Keyword Search - Selected Topic](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/selected-topic.png) - ---- - -## Interaction Details - -Displays interaction data based on your selections. If no topic or keyword is selected, data is shown based on the highest sentiment score. - -| Field | Description | -|-------|-------------| -| Date/Time | Session date and time. | -| Call ID | Unique call identifier. | -| Agent Name/ID | Select from a dropdown of agent groups and agents. | -| Description | Text reference to the keyword cloud. | -| Call Reason/Intent | The topic or intent of the call. | -| Sentiment Score | Configurable from high to low. | -| Dispositions | Call outcome or disposition. | - -![Interaction Details](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/interaction-details.png) - ---- diff --git a/ai-for-service/quality-ai/analyze/conversation-mining-audit-allocations.mdx b/ai-for-service/quality-ai/analyze/conversation-mining-audit-allocations.mdx index 0cffb1aa1..a184a76b1 100644 --- a/ai-for-service/quality-ai/analyze/conversation-mining-audit-allocations.mdx +++ b/ai-for-service/quality-ai/analyze/conversation-mining-audit-allocations.mdx @@ -3,33 +3,62 @@ title: Audit Allocations sidebarTitle: Audit Allocations --- -Audit Allocations provides a workspace for QA managers and supervisors to assign customer interactions to auditors for manual evaluation. It complements AutoQA by supporting structured review workflows, balanced auditor workloads, and consistent quality standards. You can create, track, and manage audit assignments with visibility into allocation progress, ownership, completion status, and, when enabled, agent acknowledgments and disputes. +Audit Allocations provides a workspace for QA managers and supervisors to create, assign, and manage manual evaluation workflows for both Human Agent and AI Agent (bot) interactions. It supports structured sampling, controlled distribution, and real-time tracking of audit progress across conversations. -Configure a Minimum Duration Threshold to exclude contacts marked Below Threshold or Duration Unavailable from automated scoring and, by default, from audit allocation batches. Use the toggle to include these contacts for targeted review. The system flags these contacts before evaluation. +Allocation behavior is determined by the evaluation form configuration: -Manage allocations across all queues with the Cross Queue Data Access permission. Without this permission, you can manage allocations only for assigned queues. +- Experience Flow assignments create AI Agent allocations. +- Queue assignments create Human Agent allocations. +- A single evaluation form cannot be assigned to both Experience Flows and Queues. -Navigate to **Quality AI** > **Analyze** > **Allocations**. +For AI Agent evaluations, allocations are scoped using Experience Flows instead of agents or queues to ensure consistent bot evaluation coverage, while Human Agent allocations continue to rely on queues, agents, and agent groups. -![Audit Allocations](/ai-for-service/quality-ai/analyze/allocations/images/audit-allocations.png) +Audit Allocations supports full lifecycle management including creation, tracking, reassignment of pending work, and (when enabled) agent acknowledgment and dispute workflows. - Enable **Agent Accept & Dispute** in **Quality AI** > **Settings** > **Quality AI General Settings** to activate the My **Evaluations**, **Disputes**, **Resolved**, and the Review Status column in **My Allocations**. +--- + +## Why Use Audit Allocations? + +Even with AutoQA-driven evaluation, organizations rely on manual audits to validate quality outcomes, ensure evaluation accuracy, and support targeted coaching. **Audit Allocations** addresses operational gaps in traditional QA workflows by enabling structured, flexible, and transparent audit distribution across both AI Agent and Human Agent conversations. + +| **Challenge** | **How Audit Allocations Helps** | +| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | +| Allocation Visibility Gaps | Provides real-time tracking of allocation progress, including auditor-wise completion status and evaluation ownership. | +| Inflexible Assignment Management | Enables reassignment of pending interactions without impacting completed evaluations, supporting dynamic workforce changes. | +| Manual Workflow Inefficiencies | Allows creation of allocations using quick filters or saved filters directly within the workflow, eliminating dependency on pre-configured setups. | +| Limited Distribution Control | Supports precise allocation using percentage-based or fixed interaction distribution per auditor. | +| Fragmented QA Workspace | Centralized allocation workspace separate from Conversation Mining for clearer operational focus. | +| Limited Evaluation Transparency | Provides unified visibility into evaluation status, including acknowledgments and dispute states (when enabled for Human Agent evaluations). | --- ## Key Capabilities -- **Comprehensive Tracking**: Monitor all allocations with real-time completion rates per auditor, including acknowledgement and dispute status when disputes are enabled. -- **Flexible Reassignment**: Redistribute pending interactions from unavailable auditors without disrupting completed work. -- **Streamlined Creation**: Build allocations with quick filters in the assignment workflow. -- **Controlled Distribution**: Set exact interaction counts or percentages per agent. -- **Dual Assignment Methods**: Choose **Random** for unbiased sampling or **Custom** for targeted evaluations. -- **Dispute Queue**: The **Disputes** tab gives QAs a focused queue of all evaluations actively routed to them for re-evaluation. -- **Threshold-aware Allocation**: Include or exclude below-threshold contacts based on audit objectives. +- Real-time tracking of allocation progress with auditor-wise completion visibility +- AI Agent and Human Agent–aware allocation handling with context-based workflow adaptation +- Support for AI Agent (Experience Flow-based) and Human Agent (Queue/Agent-based) allocations +- Random and Custom allocation methods using saved or quick filters +- Percentage-based or fixed-count interaction distribution per auditor +- Dynamic reassignment of pending interactions without impacting completed audits +- Threshold-aware filtering for below-duration or ineligible interactions +- Multi-auditor assignment support for Human Agent evaluations +- Reassignment of pending interactions without impacting completed audits +- Dispute and acknowledgment workflow support for Human Agent evaluations (when enabled) +- Exclusion of dispute metrics for AI Agent-only allocations +- Flexible filter-driven allocation creation directly within workflow --- +## Access Audit Allocations + +Navigate to **Quality AI** > **AutoQA** > **Allocations**. + +![Audit Allocations](/ai-for-service/quality-ai/analyze/allocations/images/audit-allocations.png) + + + Enable **Agent Accept & Dispute** in **Quality AI** > **Settings** > **Quality AI General Settings** to activate the My **Evaluations**, **Disputes**, **Resolved**, and the Review Status column in **My Allocations**. + For steps to create an allocation, see [Creating Audit Allocations](/ai-for-service/quality-ai/analyze/creating-audit-allocations). For information on managing allocations and auditor views, see [Understanding Views and Permissions](/ai-for-service/quality-ai/analyze/understanding-views-and-permissions). diff --git a/ai-for-service/quality-ai/analyze/conversation-mining/images/conversation-mining-interactions.png b/ai-for-service/quality-ai/analyze/conversation-mining/images/conversation-mining-interactions.png new file mode 100644 index 000000000..743898a9a Binary files /dev/null and b/ai-for-service/quality-ai/analyze/conversation-mining/images/conversation-mining-interactions.png differ diff --git a/ai-for-service/quality-ai/analyze/cx-insights.mdx b/ai-for-service/quality-ai/analyze/cx-insights.mdx index 7505c17a6..1805ddb9e 100644 --- a/ai-for-service/quality-ai/analyze/cx-insights.mdx +++ b/ai-for-service/quality-ai/analyze/cx-insights.mdx @@ -5,15 +5,16 @@ sidebarTitle: CX Insights Customer Experience (CX) Insights provides a unified post-interaction view of customer experience across conversations, queues, channels, and contact directions. It helps supervisors, QA managers, and CX leaders analyze customer satisfaction, sentiment, resolution outcomes, churn risk, and key satisfaction drivers using real conversation data. -Users with Cross Queue Data Access can view and filter data across all queues; otherwise, the dashboard is limited to assigned queues. +The **Handled By** filter is available across all CX Insights views, including **Sentiment Distribution**, **Customer Churn Monitor**, **CSAT Drivers**, **DSAT Drivers**, and **Insights Mining**, enabling users to analyze interactions handled by AI Agents, Human Agents, or both. -The dashboard supports direction-aware analysis through the Channel filter. Selecting Inbound, Outbound, or Both updates all widgets and visualizations to display only matching interactions. +Users with **Cross Queue Data Access** can view and filter data across all queues. Whereas, other users can access only their assigned queues. The **Channel** filter supports direction-aware analysis by displaying interactions that match the selected **Inbound**, **Outbound**, or **Both** contact direction. ## Why Use CX Insights? | Use Case | How CX Insights Helps | | -------------------------- | ------------------------------------------------------------------------------------------------------------------ | | **Experience Monitoring** | Tracks resolution, wait time, and customer sentiment trends. | +| **AI Agent and Human Agent Analysis** | Analyze customer experience separately for AI Agent, Human Agent, or both using the Handled By filter. | | **Risk Detection** | Identifies churn risk and escalation patterns early. | | **Satisfaction Drivers** | Highlights topics that impact customer satisfaction positively or negatively. | | **Issue Prioritization** | Helps teams focus on high-impact customer pain points. | @@ -27,6 +28,7 @@ The dashboard supports direction-aware analysis through the Channel filter. Sele | Capability | Description | | ----------------------------- | ------------------------------------------------------------------------------------------------ | | **CX Metrics** | Provides a summary view of Average Wait Time, Resolution Rate, Sentiment, and Churn Monitor. | +| **Handled By Filter** | Filter CX Insights by AI Agent, Human Agent, or Both to compare customer experience across conversation types. | | **Overall Resolution Rate** | Displays the percentage of successfully resolved interactions. | | **CSAT Drivers** | Identifies topics that positively impact customer satisfaction using driver-impact scoring. | | **DSAT Drivers** | Identifies topics that negatively impact customer satisfaction with detailed warning indicators. | @@ -40,11 +42,39 @@ The dashboard supports direction-aware analysis through the Channel filter. Sele ## Access CX Insights -Navigate to **Quality AI** > **Analyze** > **CX Insights**. +Navigate to **Quality AI** > **AutoQA** > **CX Insights**. ![CX Insights Dashboard](/ai-for-service/quality-ai/analyze/conversation-intelligence/images/cx-insights.png) -Enable the **Conversational Intelligence** toggle under **Configure** > **Settings** to access CX Insights and CX trends. +Enable the **Conversational Intelligence** option under **Configure** > **Settings** to access CX Insights and CX trends. + +--- + +### Handled By Filter + +The **Handled By** filter shows as a dropdown in the upper-right corner of the **CX Insights** page, next to the **Date Range** and **Channel** filters. The label displays **Handled by**: followed by the current selection. + +The dropdown includes two checkboxes: **AI Agent** and **Human Agent**. Select an agent to view data for the other conversation type, and then select **Apply**. + +For conversations transferred from an AI Agent to a Human Agent: + +- **AI Agent** displays analytics calculated only from the AI Agent conversation segment. +- **Human Agent** displays analytics calculated only from the Human Agent conversation segment. +- **Both** display aggregated analytics across both conversation segments. + +### Per-Segment and Combined Analytics + +CX Insights computes analytics at the segment level and aggregates results based on the selected filter. + +- **Sentiment analysis** is calculated independently for AI Agent and Human Agent segments. +- **Topic classification** is performed separately for each segment using the shared L1/L2/L3 taxonomy. +- **CSAT and DSAT drivers** are derived from sentiment and topic outputs of the selected segment. +- When **Both** is selected, CX Insights aggregates segment-level results into a combined view. + +### Topic Classification + +- AI Agent and Human Agent conversations share the same L1/L2/L3 taxonomy. +- Each conversation segment is independently classified, and results are then aggregated for reporting. ## Filters @@ -206,7 +236,7 @@ Highlights the top 6 DSAT (Dissatisfaction) topics that negatively impact custom If no data is available, more conversations are needed to identify dissatisfaction drivers. -## Data Handling Rules +## CX Data Handling Rules CX Insights excludes `Below Threshold` or `Duration Unavailable` conversations from all aggregated metrics. Manually evaluated conversations remain included. Excluded interactions don’t affect totals, averages, distributions, or driver calculations. diff --git a/ai-for-service/quality-ai/analyze/dashboard.mdx b/ai-for-service/quality-ai/analyze/dashboard.mdx index db8304d9d..9b6855be0 100644 --- a/ai-for-service/quality-ai/analyze/dashboard.mdx +++ b/ai-for-service/quality-ai/analyze/dashboard.mdx @@ -3,16 +3,18 @@ title: Supervisor Dashboard sidebarTitle: Dashboard --- -The Supervisor Dashboard (QA Dashboard) provides near real-time insights into audit performance, agent quality trends, coaching activity, fatal interactions, adherence heat maps, fail statistics, sentiment, resolution trends, and detailed interaction-level analytics. It helps supervisors identify performance gaps, compare Auto QA and manual scores, track inbound and outbound performance trends, monitor failed interactions, and manage coaching activities. +The **QA Dashboard** provides a centralized view of quality performance across AI Agent and Human Agent conversations. It combines audit metrics, evaluation outcomes, AutoQA coverage, adherence trends, coaching insights, and interaction analytics to help supervisors monitor quality, identify performance gaps, and prioritize coaching opportunities. -When you enable agent disputes, supervisors can track pending reviews and active disputes. The dashboard provides role-based access and supports dynamic global filters such as language, date range, channel, contact direction, queue, and evaluation form for targeted analysis. +The dashboard uses shared global filters, including Handled By, allowing supervisors to analyze AI Agent, Human Agent, or combined conversation data. All widgets update automatically to reflect the selected filters. -Using the dashboard the supervisors can: -* Monitor overall quality performance -* Compare inbound and outbound performance trends -* Identify fatal and failed interactions -* Track audit progress and coaching workload -* Track pending reviews and active disputes (when agents can dispute decisions) +Using the QA Dashboard, supervisors can: + +- Monitor quality performance and evaluation trends. +- Compare AutoQA and manual audit results. +- Analyze failed and fatal interactions. +- Track audit progress, coaching activity, and pending reviews. +- Evaluate AI Agent and Human Agent performance separately or together. +- Identify agents requiring coaching. ---- @@ -21,6 +23,7 @@ Using the dashboard the supervisors can: | **Feature** | **Description** | | ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Real-time Performance Insights** | Provides near real-time visibility into audit performance, agent quality trends, coaching activity, and fatal interactions across conversations. | +| **Handled By Analysis** | Filter dashboard data by All, Human Agent, or AI Agent to analyze quality metrics separately or together. For handoff conversations, the dashboard attributes Bot evaluations to AI Agent and human evaluations to Human Agent. | | **Cross-queue Analytics** | Enables viewing agent performance across all queues without assignment restrictions. | | **Direction-aware Analysis** | Supports separate or combined analysis of inbound and outbound interactions for performance comparison. | | **Adherence Heatmap** | Visualizes adherence trends, fatal interactions, and evaluation performance based on selected filters. | @@ -32,18 +35,10 @@ Using the dashboard the supervisors can: | **Filter-based Analysis** | Supports global filters for language, date range, channel, and contact direction, with dynamic widget updates based on selected criteria. | | **Role-based Access** | Grants access based on user role. App Owners, App Developers, App Testers, and Supervisors with Cross Queue Data Access can view all queues, while other users can access only their assigned queues. | -## When to Use This Dashboard - -Use the Supervisor Dashboard to: - -- Monitor overall quality performance. -- Compare inbound and outbound trends. -- Identify fatal and failed interactions. -- Track audit progress and coaching workload. ## Access Supervisor Dashboard -Navigate to **Quality AI** > **Analyze** > **Dashboard**. +Navigate to **Quality AI** > **AutoQA** > **Dashboard**. ![Quality AI Dashboard](/ai-for-service/quality-ai/analyze/dashboard/images/qm-dashboard.png) @@ -53,16 +48,18 @@ Navigate to **Quality AI** > **Analyze** > **Dashboard**. #### Configuration Requirements -* Enable the required settings in **Quality AI** > **Settings** > **Quality AI General Settings** to access dashboard features and activate the Agent Accept & Dispute workflow. -* Assign the required permissions to access features such as Agent Scorecard, Auto QA, Evaluation Forms, Adherence Heatmap, Performance Monitor, and Agent Leaderboard. -* Enable Cross Queue Data Access for users to view data across all queues. -* Configure and publish evaluation forms and scorecards, and enable Auto QA (for automated evaluation scoring) in Quality AI General Settings. +- Enable the required settings in **Quality AI** > **Settings** > **Quality AI General Settings** to access dashboard features and activate the **Agent Accept & Dispute** workflow. +- Assign the required Quality AI permissions to access features such as **Agent Scorecard**, **AutoQA**, **Evaluation Forms**, **Adherence Heatmap**, **Performance Monitor**, and **Agent Leaderboard**. +- Enable **Cross Queue Data Access** for users to view data across all queues. +- Configure and publish evaluation forms and scorecards, and enable **AutoQA** (for automated evaluation scoring) in **Quality AI General Settings**. + --- ## Dashboard Filters The QA Dashboard supports shared global filters across all widgets, consistent with the Agent Dashboard and Audit views. Use the Filter panel to refine dashboard data by evaluation form, queue, language, date range, channel, contact direction, and agent for targeted quality and performance analysis. + By default, the dashboard displays data for All Evaluation Forms, All Queues, All Languages, Last 7 Days, All Channels, and Both Directions. All widgets update dynamically based on selected filters. Default: All Languages, Last 7 Days, All Channels, Both Directions. @@ -80,19 +77,22 @@ To filter by calendar, ### Filter Options -| Filter | Description | Affected Widgets | -| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Evaluation Forms** | Filters dashboard data using selected evaluation forms. Available queue options depend on the selected forms. | Total Audits, Fatal Interactions, Evaluation Score, Adherence Heatmap, Fail Statistics (Evaluation Form), and Performance Monitor (Evaluation Form). | -| **Queues** | Filters interactions using queues mapped to the selected evaluation forms. | Total Audits, Fatal Interactions, Evaluation Score, Adherence Heatmap, Fail Statistics (Evaluation Form), Performance Monitor (Evaluation Form), and Agent Leaderboard. | -| **Channel** | Filters interactions by Voice or Chat and by Inbound, Outbound, or Both directions. | All widgets. | -| **Language** | Filters interactions by one or more configured languages. The system selects all configured languages by default. | All widgets. | -| **Agents** | Filters dashboard data for selected agents. | Agent Leaderboard and coaching-related widgets. | +Select **Filter** in the upper-right corner of the dashboard to open the filter panel. + +| **Filter** | **Description** | **Affected Widgets** | +| ------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | +| **Evaluation Forms** | Filters dashboard data using the selected evaluation forms. Queue and Experience Flow options update dynamically based on the selected forms. | Total Audits, Fatal Interactions, Evaluation Score, Adherence Heatmap, Fail Statistics (Evaluation Form), Performance Monitor (Evaluation Form) | +| **AI Agent → Experience Flow** | Filters AI Agent conversations using the selected Experience Flows. Supports multi-select, search, **Show all**, and **Clear selection**. | Evaluation widgets, Fail Statistics, Adherence Heatmap, Performance Monitor | +| **Human Agent → Queues** | Filters Human Agent conversations using the selected queues. Supports multi-select, search, **Show all**, and **Clear selection**. | Evaluation widgets, Fail Statistics, Adherence Heatmap, Performance Monitor, Agent Leaderboard | +| **Human Agent → Agents** | Filters dashboard data for the selected Human Agents. Supports multi-select, search, **Show all**, and **Clear selection**. | Agent Leaderboard and coaching-related widgets | +| **Channel & Direction** | Filters interactions by channel (**Voice** or **Chat**) and contact direction (**Inbound**, **Outbound**, or **Both**). | All widgets | +| **Language** | Filters interactions by the configured languages. By default, all available languages are selected. | All widgets | -Select **Apply** to apply the selected filters and refresh the dashboard widgets. The button becomes available only when filter changes are pending. +Select **Apply** to apply the selected filters and refresh the dashboard widgets. This **Apply** option becomes available only when filter changes are pending. - ![Channel Options](/ai-for-service/quality-ai/analyze/dashboard/images/channel-directions.png) +![Channel Options](/ai-for-service/quality-ai/analyze/dashboard/images/channel-directions.png) - Only configured languages are available. The default view includes **All Languages**, **Last 7 Days**, **All Channels**, and **Both Directions**. + Only configured languages are available. The default view includes **All Languages**, **Last 7 Days**, **All Channels**, and **Both Directions**. --- @@ -112,7 +112,7 @@ Provides a high-level summary of audit activity, coaching workload, quality perf | **Fatal Interactions** | Displays the number of interactions that failed due to fatal evaluation rules. If an interaction meets any fatal criterion, the system assigns a score of zero to the entire scorecard. | - This metric is visible only when these settings are enabled in **Quality AI** > **Settings** > **Quality AI General Settings** and **Agent Accept & Dispute **disputes are configured for one or more evaluation forms. + This metric is visible only when these settings are enabled in **Quality AI** > **Settings** > **Quality AI General Settings** and **Agent Accept & Dispute** are configured for one or more evaluation forms. ### Audit Progress @@ -184,6 +184,25 @@ Displays failed interaction trends based on selected scorecard metrics. If you m ![Fail Statistics - Agent Scorecard](/ai-for-service/quality-ai/analyze/dashboard/images/fail-stat-agent-scorecard.png) +The widget updates automatically based on the applied dashboard filters. + +| **Element** | **Description** | +| ----------------------------- | -------------------------------------------------------------------------------------------------------------- | +| **Evaluation Form** | Selects the evaluation form used to calculate failure statistics. | +| **Queues & Experience Flows** | Filters results by Queues or AI Experience Flows. Supports separate tabs with search and multi-select options. | +| **No. of Fails** | Displays the total number of failed interactions and the percentage change compared to the previous period. | +| **Failed Interactions** | Shows trends of failed interactions over time. | +| **Fatal Interactions** | Displays interactions that triggered one or more configured fatal metrics. | +| **Trend Interval** | Allows selection of trend view by **Daily**, **Weekly**, or **Monthly** intervals. | + + + +* The widget updates automatically based on the selected dashboard filters. +* When **Handled By** is set to **AI Agent**, only AI Agent evaluation data is displayed. +* The **Agent Scorecard** view is unavailable for AI Agent analysis because scorecards apply only to **Human Agent** evaluations. + + + --- ## Performance Monitor diff --git a/ai-for-service/quality-ai/analyze/dashboard/images/qm-dashboard.png b/ai-for-service/quality-ai/analyze/dashboard/images/qm-dashboard.png index 301f5e2f0..39efc07ff 100644 Binary files a/ai-for-service/quality-ai/analyze/dashboard/images/qm-dashboard.png and b/ai-for-service/quality-ai/analyze/dashboard/images/qm-dashboard.png differ diff --git a/ai-for-service/quality-ai/analyze/my-dashboard-agent-view.mdx b/ai-for-service/quality-ai/analyze/my-dashboard-agent-view.mdx index 95d73c631..69e081880 100644 --- a/ai-for-service/quality-ai/analyze/my-dashboard-agent-view.mdx +++ b/ai-for-service/quality-ai/analyze/my-dashboard-agent-view.mdx @@ -12,7 +12,7 @@ By default, agents can only access data from their assigned queues. Cross-queue ## Access My Dashboard -Navigate to **Quality AI** > **Analyze** > **My Dashboard**. +Navigate to **Quality AI** > **AutoQA** > **My Dashboard**. ![My Dashboard - Agent View](/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-view.png) @@ -119,7 +119,7 @@ Selecting an attribute opens a detailed metric-level view. | **Red** | Very Low | Significant performance gap. | | **NA** | Not Applicable | Not triggered or evaluated. | -![Coaching Insights](/ai-for-service/quality-ai/analyze/dashboard/images/my-dashboard-coaching-insights.png) +![Coaching Insights](/ai-for-service/quality-ai/analyze/agent-dashboard/images/my-dashboard-coaching-insights.png) --- @@ -266,12 +266,14 @@ By selecting any reviewed conversations or interactions on the Evaluation page, The **Review** page gives agents a dedicated view of all their manually evaluated conversations, grouped by review state. -Navigate to **Quality AI** > **Analyze** > **Review**. - Agents can track evaluations awaiting action, monitor active disputes, and view completed or closed dispute outcomes from one place. --- +### Access Review Tabs + +Navigate to **Quality AI** > **Review**. + ### Review Tabs Tabs represent the different evaluation groupings available in the Review tab. @@ -311,14 +313,6 @@ The **Review Status** column reflects the current acknowledgment or dispute stat --- -### Review Tabs - -| Tab | Description | -|:-----|:-------------| -| **Pending Review** | Evaluations the agent hasn't acknowledged or disputed. Select a row to open the audit screen and respond at the metric level. | -| **Disputes** | Evaluations with an active dispute submitted by the agent. If no disputes exist, the tab displays: `No disputes raised`. | -| **Resolved** | Completed evaluations — both accepted outcomes and closed disputes. | - ### Review Status Values The Review Status column shows the current acknowledgment or dispute state for each manually evaluated conversation. @@ -381,6 +375,7 @@ After submission, the system routes the disputed metrics to the assigned QA for --- + ### Reviewing a QA Re-evaluation Response When a QA completes their re-evaluation, the evaluation status changes to **Re-evaluation Received**. The agent can: @@ -401,8 +396,6 @@ The **Disputes** tab provides agents with a single view of all evaluations they ### Access Disputes -Navigate to **Quality AI > Disputes**. - The **Disputes** menu item is visible only for users with Agent Dashboard permission when **Agent Accept & Dispute** is enabled in **Quality AI** > **Settings** > **Quality AI General Settings** and disputes are configured for at least one evaluation form. --- @@ -460,8 +453,6 @@ The **Resolved** tab displays completed evaluation reviews along with the evalua | **Dispute Rounds** | Number of dispute rounds completed before resolution or closure. | | **Review Status** | Final state of the evaluation — Resolved or Closed. | -![Resolved Tab](/ai-for-service/quality-ai/analyze/dashboard/images/my-dashboard-resolved.png) - --- ### Review Filter @@ -499,8 +490,8 @@ The Queues filter helps you to refine evaluations by selecting one or more queue --- -### Language Settings +### Settings -The **Language Settings** section is view-only for agents. +The **Settings** page displays all available options in view-only mode. -![Language Settings](/ai-for-service/quality-ai/analyze/agent-dashboard/images/language-settings.png) +![Settings](/ai-for-service/quality-ai/analyze/agent-dashboard/images/language-settings.png) diff --git a/ai-for-service/quality-ai/analyze/my-dashboard-coaching-assignments.mdx b/ai-for-service/quality-ai/analyze/my-dashboard-coaching-assignments.mdx index 2ee3ccc83..4ac9d6de8 100644 --- a/ai-for-service/quality-ai/analyze/my-dashboard-coaching-assignments.mdx +++ b/ai-for-service/quality-ai/analyze/my-dashboard-coaching-assignments.mdx @@ -11,7 +11,7 @@ When you enable **Agent Accept & Dispute**, the coaching assignment status also ## Access Coaching Assignments -Navigate to **Quality AI** > **Analyze** > **Coaching Assignments**. +Navigate to **Quality AI** > **Coaching** > **Coaching Assignments**. ![Coaching Assignments](/ai-for-service/quality-ai/coach/coaching-assignments/images/coaching-assignments-agent-view.png) diff --git a/ai-for-service/quality-ai/analyze/topic-discovery.mdx b/ai-for-service/quality-ai/analyze/topic-discovery.mdx index 021d8728f..a8542d64e 100644 --- a/ai-for-service/quality-ai/analyze/topic-discovery.mdx +++ b/ai-for-service/quality-ai/analyze/topic-discovery.mdx @@ -3,46 +3,52 @@ title: Topic Discovery sidebarTitle: Topic Discovery --- -Topic Discovery is an analytics dashboard that helps QA managers, supervisors, and CX teams analyze conversation trends through interactive visualizations and actionable insights. +Topic Discovery helps QA managers, supervisors, and CX teams analyze conversation trends through interactive visualizations. It identifies recurring issues, monitors customer sentiment and resolution trends, and supports data-driven coaching and process improvements. -It converts conversation data into actionable insights to help identify recurring issues, monitor sentiment and resolution trends, and support data-driven coaching and process improvements. +The dashboard supports analysis of Inbound and Outbound interactions through the Channel filter and AI Agent, Human Agent, or both conversation types through the Handled By filter. All visualizations and metrics update based on the selected filters. -The dashboard supports Inbound and Outbound interaction analysis through the Channel filter. When applied, all topic visualizations, metrics, and conversation data update based on the selected interaction direction. +Topic Discovery includes both configured and AI-generated intents for bot conversations. AI Agent and Human Agent conversations share the same L1/L2/L3 topic taxonomy, enabling consistent topic analysis across all conversation types --- ## Why Use Topic Discovery? -| Challenge | How Topic Discovery Helps | -|-----------|--------------------------| -| **Theme Discovery** | Identifies recurring conversation topics and patterns across interactions. | -| **Emerging Issues** | Detects new customer issues early before they escalate. | -| **Performance Trends** | Tracks sentiment, resolution, and AHT trends by topic. | -| **Coaching Focus** | Highlights low-performing topics for targeted coaching. | -| **Topic Analysis** | Supports drill-down analysis from topic trends to individual conversations. | -| **Taxonomy Expansion** | Discovers AI-generated themes outside the configured taxonomy. | +### Topic Analysis Capabilities + +| **Capability** | **Description** | +| ------------------------- | --------------------------------------------------------------------------------------------------------------- | +| **Pattern Recognition** | Identifies recurring conversation themes and trends across AI Agent, Human Agent, or combined conversations. | +| **Performance Analysis** | Correlates topics with key metrics such as sentiment, resolution rate, and Average Handle Time (AHT). | +| **Emerging Issue Detection** | Surfaces configured and AI-generated topics early to help teams address issues before they escalate. | +| **Targeted Coaching** | Highlights low-performing topics to guide training and process improvements. | +| **Conversation Segmentation** | Enables separate analysis of AI Agent, Human Agent, or both conversation types using the **Handled By** filter. | +| **Cross-Queue Visibility** | Users with Cross Queue Data Access can analyze topics across all queues without individual queue assignment. | + --- ## Key Capabilities -- **Topic Performance Analysis**: Analyze topics using sentiment, resolution, and AHT metrics. -- **Advanced Filtering**: Filter data by channel, direction, language, queue, and agent. -- **Intent Comparison**: Compare configured intents with AI-generated topics. -- **Trend Drill-down**: Move from topic-level trends to individual conversations. -- **Hierarchical Topic Analysis**: Track trends across L1, L2, and L3 topic levels. -- **Coaching Support**: Identify low-performing topics for targeted coaching and process improvement. -- **Proactive Detection**: Surface emerging issues early through AI-generated themes. -- **Direction-aware Analysis**: Filter all topic visualizations, metrics, and conversation lists by contact direction (Inbound or Outbound) within each channel. - +- **Trend Identification**: Identify high-volume topics and their performance impact to assess operational health. +- **Coaching Focus**: Pinpoint topics with low sentiment or resolution rates for targeted coaching and improvement. +- **Performance Monitoring**: Track topic-level metrics such as AHT, sentiment, and resolution rate for objective evaluation. +- **Proactive Management**: Detect emerging topics early using AI-generated insights to prevent escalation. +- **Handled By Filtering**: Filters analytics by All, Human Agent, or AI Agent segments; All combines both, while each mode shows only its respective segment (AI or human). +- **Direction-Aware Analysis**: Filter all topic views by Inbound or Outbound interactions. --- +## Prerequisite + +- Topic Discovery analyzes AI Agent conversations only when the Automation AI conversation source is enabled for the workspace. +- If the Automation AI conversation source is disabled, AI Agent conversations are excluded from Topic Discovery, while Human Agent conversations continue to be analyzed. +- AI Agent conversations are included automatically once both the Conversation Intelligence feature and Automation AI conversation source are enabled. + ## Access Topic Discovery -Navigate to **Quality AI** > **Analyze** > **Topic Discovery**. +Navigate to **Quality AI** > **AutoQA** > **Topic Discovery**. -![Topic Discovery Filter](/ai-for-service/quality-ai/analyze/topic-discovery/images/topic-discovery-landingpage.png) +![Topic Discovery Filter](/ai-for-service/quality-ai/analyze/topic-discovery/images/topic-discovery-landing-page.png) ## Topic Hierarchy @@ -55,21 +61,34 @@ Topic Discovery uses a three-level structure: | **L2** | Subtopic under L1 | Payment Problems | | **L3** | Granular subtopic under L2 | Credit Card Declined | +Note: Bot conversations, AI Agent conversations, and Human Agent conversations all use the same L1/L2/L3 taxonomy, ensuring consistent reporting across all interaction types. + +### Topic Classification for Handoff Conversations + +For conversations transferred from an AI Agent to a Human Agent: + +- The AI Agent segment is classified independently using the shared L1/L2/L3 taxonomy. +- The Human Agent segment is classified independently using the same taxonomy. +- Conversation-level topic analytics are generated by aggregating the independently classified segment results. +- No separate taxonomy exists for AI Agent conversations. + --- ## Filters -The **Top Filter Bar** is the central control for customizing the Topic Discovery view. Every adjustment instantly updates the visualization. +The top filter bar is the central control panel for customizing the Topic Discovery dashboard. Every adjustment updates the visualization in real time. ![Topic Discovery Filter](/ai-for-service/quality-ai/analyze/topic-discovery/images/topic-discovery-filter.png) -| Filter | What it does | How to use it | When to use it | -|--------|-------------|---------------|----------------| -| **Search Topic Names** | Locate topics in the visualization. | Start typing a keyword, and matching topics highlight instantly. | When looking for a specific issue (for example, "Payment Failure"). | -| **Configured or Generated Intents** | Switch between taxonomy-based topics and AI-discovered themes. | Select **Configured Intents** for your taxonomy and **Generated Intents** for blind spots. | Use Generated Intents to find new themes outside your taxonomy. | -| **Time Range** | Adjust the analysis period. | Options: 7 days (default), 28 days, 30 days, 90 days, custom. | Compare weekly vs. monthly trends to spot recurring issues. | -| **Sentiment** | Focus on conversations by sentiment. | Adjust the score range slider (0-10). The default is full range. | Narrow to low-sentiment conversations for quality monitoring. | -| **Resolution** | Filter by resolution success rates. | Adjust the score range slider (0-100). The default is full range. | Zero in on unresolved or low-resolution conversations. | +### Filters Overview + +| **Filter** | **What it does** | **How to use it** | **When to use it** | +| --------------------------------------- | ---------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | +| Search Topic Names | Locates topics within the visualization. | Start typing a keyword; matching topics are highlighted instantly. | Use when looking for a specific customer issue (e.g., `Payment Failure`). | +| Configured Intents or Generated Intents | Switches between taxonomy-based topics and AI-discovered themes. | Select **Configured Intents** if your taxonomy is set, or **Generated Intents** to surface blind spots. | Use **Generated Intents** to uncover new themes not yet in your taxonomy. | +| Time Range Selector | Adjusts the analysis period. | Choose from 7 days (default), 28 days, 30 days, 90 days, or a custom date range. | Compare weekly vs. monthly trends to identify recurring issues. | +| Sentiment Filter | Focuses on conversations by sentiment. | Adjust the score range slider (0–10). Default is full range. | Narrow results to low-sentiment conversations for quality monitoring. | +| Resolution Filter | Filters by resolution success rates. | Adjust the score range slider (0–100). Default is full range. | Focus on unresolved or low-resolution conversations. | ### Advanced Topic Filters @@ -88,81 +107,74 @@ Select **Filters** to access additional filtering options and refine the dashboa ## Bubble Visualization Canvas -Topics are displayed as color-coded bubbles to help analyze conversation volume, performance metrics, and topic relationships. - -### Bubble Attributes +The central visualization displays topics as interactive bubbles with meaningful visual encoding, providing an at-a-glance view of conversation volumes, performance metrics, and relationships between topics. This intuitive interface allows you to explore your conversation data spatially, with visual cues guiding you to areas that need attention. -Each bubble encodes key topic attributes visually: +For handoff conversations, bubble metrics are computed according to the selected Handled By filter. +- AI Agent displays metrics derived only from AI Agent conversation segments. +- Human Agent displays metrics derived only from Human Agent conversation segments. +- All displays aggregated metrics across both conversation segments. -| Attribute | Meaning | -|-----------|---------| -| **Size** | Conversation volume — larger bubbles = more conversations. | -| **Color** | Topic performance based on the selected metric (sentiment or resolution). | -| **Position** | Groups related topics together to reveal patterns and clusters. | -| **Labels** | L1 topics: labels outside. L2 and L3: labels appear inside. | +### Bubble Chart Display +The central visualization displays topics as interactive bubbles, encoding meaningful data visually. -### Sentiment Color Coding +Each bubble encodes key topic attributes visually: -L3 sentiment aggregates to L2/L1 with color-coded trends. +* **Bubble Size**: Represents conversation volume. Larger bubbles indicate a higher number of conversations. +* **Bubble Color**: Indicates topic performance based on the selected metric (sentiment or resolution). +* **Positioning**: Groups related topics together to reveal patterns and clusters. +* **Labels**: L1 topics are labeled outside the bubbles. L2 and L3 topics are labeled inside. -| Color | Sentiment | -|-------|-----------| -| Green | Positive | -| Grey | Neutral | -| Red | Poor | +You can switch bubble coloring between sentiment and resolution, while continuing to filter topics using both metrics simultaneously (AND logic). -### Resolution Color Coding +![Bubble Visualization Canvas](/ai-for-service/quality-ai/analyze/topic-discovery/images/bubble-visualization-canvas.png) -Topics use color codes by resolution rate, supporting combined analysis with sentiment. +#### Sentiment Visualization Across Topic Hierarchy -| Color | Resolution Rate | -|-------|----------------| -| Red | 0-50% (Low) | -| Grey | 50-70% (Moderate) | -| Green | 70-100% (High) | +Customer sentiment is captured at the L3 level and aggregated across L2 and L1 topics. Topic Discovery uses color-coded indicators: green (positive), gray (neutral), and red (poor), helping identify sentiment trends across the topic hierarchy. -You can switch bubble coloring between sentiment and resolution while keeping both filters active simultaneously (AND logic). +#### Resolution-Based Visualization -![Bubble Visualization Canvas](/ai-for-service/quality-ai/analyze/topic-discovery/images/bubble-visualization-canvas.png) +When resolution-based coloring is applied, topics appear as red (0–50%), grey (50–70%), and green (70–100%). Changing the coloring mode affects only visualization; sentiment and resolution filters remain active together, enabling combined analysis of topic performance. --- -## Bobble Tooltips +### Bobble Tooltips -Hovering over a bubble shows a tooltip with key metrics for quick assessment and deeper analysis without leaving the main view. +Hovering over any bubble reveals a detailed tooltip that provides quick access to key metrics without leaving the main visualization. This instant feedback mechanism helps you assess topic performance and identify areas for deeper investigation. | Field | Description | |-------|-------------| | **Topic Name** | Full name if truncated in the visualization. | | **Conversation Count** | Total interactions for that topic. | -| **Total Conversations** | Conversation counts with trend indicators (spikes or dips in percentage). | -| **Average Sentiment** | Overall sentiment score with trend analysis. | +| **Total Conversations** | Total conversations with trend indicators (spike or dip) percentage. | +| **Average Sentiment** | Average of the overall sentiment score with trend analysis. | | **Sentiment Breakdown** | Distribution across positive, neutral, and negative interactions. | ![Hovering Tooltips](/ai-for-service/quality-ai/analyze/topic-discovery/images/hovering-tooltips.png) --- -## Configured vs. Generated Intents +### Configured vs. Generated Intents Topic Discovery provides two topic views to support different analysis needs. -| View | Description | When to Use | -|---|---|---| -| **Configured Intents** | Displays topics based on your organization’s predefined taxonomy and trained conversation categories. | Use for monitoring known business categories, tracking taxonomy performance, and comparing historical trends. | -| **Generated Intents** | Uses AI to automatically discover conversation themes that may not exist in the configured taxonomy. | Use for identifying blind spots, detecting emerging issues, exploring unexpected conversation patterns, and expanding the taxonomy. | +| **View** | **Description** | **When to Use** | +| ---------------------- | ----------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | +| **Configured Intents** | Displays topics based on your organization's predefined taxonomy and trained conversation categories. | Monitor known business categories, track taxonomy performance, and compare historical trends. | +| **Generated Intents** | Uses AI to automatically discover conversation themes that may not exist in the configured taxonomy. | Identify blind spots, detect emerging issues, explore unexpected conversation patterns, and expand the taxonomy. | To create or update your taxonomy, see [Taxonomy Setup](/ai-for-service/quality-ai/analyze/topic-discovery). + --- ## Topic View Detail Pane -Select **View Details** from any bubble tooltip to open the detail pane for comprehensive analytics on a specific topic. +Topic metrics reflect the selected **Handled By** filter and use data from the corresponding conversation segment. -The detail pane provides topic-level analytics, trend insights, and conversation-level drill-down. +Select **View Details** from any bubble tooltip to open the detail slideout for a specific topic. The detailed slideout provides comprehensive analytics for individual topics, combining historical trends, performance metrics, and qualitative insights. ### Overview Tab @@ -170,19 +182,31 @@ Shows the **Overview** tab to analyze topic performance over time. #### Time Granularity -Daily displays day-by-day trends for short-term monitoring, while Weekly aggregates data into weekly trends for pattern analysis. +Use this you can switch between Daily and Weekly views based on your analysis needs. + +#### Time Granularity + +Use the **Time Granularity** toggle to switch between **Daily** and **Weekly** views based on your analysis needs. + +| **View** | **Description** | **Best For** | +| ---------- | ------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------- | +| **Daily** | Displays day-by-day trends to monitor short-term fluctuations and immediate issues. | Tactical monitoring and identifying recent changes. | +| **Weekly** | Aggregates data into weekly trends to identify patterns, reduce noise, and support strategic analysis. | Trend analysis, pattern identification, and long-term decision-making. | #### Topic Metrics -| Metric | Description | -|--------|-------------| -| **Total Conversations %** | Topic's share of all conversations. | -| **Average Sentiment Score** | Overall sentiment with trend analysis. | -| **Sentiment Breakdown** | Distribution across emotional categories (Positive, neutral, and negative). | -| **Average Handle Time (AHT)** | AHT trend for the selected topic. | -| **Average Resolution %** | Resolution success rate analysis. | -| **Top Keywords** | Most frequent terms in topic conversations. | -| **Emotion Detection** | Top 6 emotions identified in conversations. | +Review topic-level performance metrics to understand conversation volume, customer sentiment, operational efficiency, and resolution outcomes over the selected time period. + + +| **Metric** | **Description** | +| ----------------------------- | --------------------------------------------------------------------------------------- | +| **Total Conversations (%)** | Shows the topic's share of all conversations. | +| **Average Sentiment Score** | Displays the overall sentiment score and its trend over time. | +| **Sentiment Breakdown** | Shows the distribution of conversations by sentiment (Positive, Neutral, and Negative). | +| **Average Handle Time (AHT)** | Displays the average handle time and its trend for the selected topic. | +| **Average Resolution (%)** | Shows the resolution rate and its trend over time. | +| **Top Keywords** | Lists the most frequently occurring keywords for the selected topic. | +| **Emotion Detection** | Displays the top six emotions identified in conversations for the selected topic. | ![Topic Detail Pane](/ai-for-service/quality-ai/analyze/topic-discovery/images/topic-detail-pane.png) @@ -202,15 +226,19 @@ Shows the **Conversations** tab with list and detailed views of interactions for | **Actions** | Conversation details | Access full interaction details. | #### Navigation and Access + +The Conversation List view displays individual interactions and lets you search, sort, and filter conversations to quickly identify patterns and locate specific interactions. + - **Sorting**: Most recent conversations first. - **Pagination**: 10 conversations per page. +- **Page Jumping**: Direct access to specific pages. - **All Conversations**: Opens [Conversation Mining - Interactions](/ai-for-service/quality-ai/analyze/conversation-mining-audit-allocations) with topic filters applied. ![Conversation Mining Interactions](/ai-for-service/quality-ai/analyze/topic-discovery/images/conversation-mining-interactions.png) -### Full Conversation View +#### Full Conversation View -Select a conversation from the **Conversations** tab to open the full interaction view. +Open the conversation icon from the **Conversations** Tab to view a detailed interaction breakdown with full thread, metadata, and analytics in one panel. #### Conversation Details @@ -221,6 +249,7 @@ Select a conversation from the **Conversations** tab to open the full interactio | **Metadata** | Channel, duration, resolution status, sentiment scores. | | **Timeline View** | Chronological conversation flow. | | **Context** | Queue, agent, and channel details. | +| **Handled By** | AI and Human agents. | #### Analysis Tools @@ -243,16 +272,16 @@ Select a conversation from the **Conversations** tab to open the full interactio 1. **Initial analysis** - Open Topic Discovery with the default 7-day view. - Scan L1 topics for large bubbles with negative sentiment. - - Identify "Technical Support" as a high-volume, low-sentiment topic. + - Identify `Technical Support` as a high-volume, low-sentiment topic. 2. **Drill-down investigation** - - Select "Technical Support" to reveal L2 topics. - - Notice "Software Installation" has poor resolution rates. - - Select "Software Installation" to see L3 subtopics. - - Identify "Driver Installation" as the primary problem area. + - Select `Technical Support` to reveal L2 topics. + - Notice `Software Installation` has poor resolution rates. + - Select `Software Installation` to see L3 subtopics. + - Identify `Driver Installation` as the primary problem area. 3. **Detailed analysis** - - Open the "Driver Installation" detail pane. + - Open the `Driver Installation` detail pane. - Metrics: 150 conversations, 45% resolution rate, average sentiment: 2. - Top keywords: `error`, `crash`, `incompatible`, `frustrated`. - Top emotions: Anger (40%), Frustration (35%), Confusion (25%). diff --git a/ai-for-service/quality-ai/analyze/topic-discovery/images/conversation-mining-interactions.png b/ai-for-service/quality-ai/analyze/topic-discovery/images/conversation-mining-interactions.png index b2fed786d..743898a9a 100644 Binary files a/ai-for-service/quality-ai/analyze/topic-discovery/images/conversation-mining-interactions.png and b/ai-for-service/quality-ai/analyze/topic-discovery/images/conversation-mining-interactions.png differ diff --git a/ai-for-service/quality-ai/analyze/topic-discovery/images/full-conversation-view.png b/ai-for-service/quality-ai/analyze/topic-discovery/images/full-conversation-view.png index b0a10256d..4b1b7eef4 100644 Binary files a/ai-for-service/quality-ai/analyze/topic-discovery/images/full-conversation-view.png and b/ai-for-service/quality-ai/analyze/topic-discovery/images/full-conversation-view.png differ diff --git a/ai-for-service/quality-ai/analyze/topic-discovery/images/topic-discovery-landing-page.png b/ai-for-service/quality-ai/analyze/topic-discovery/images/topic-discovery-landing-page.png new file mode 100644 index 000000000..c40c054f5 Binary files /dev/null and b/ai-for-service/quality-ai/analyze/topic-discovery/images/topic-discovery-landing-page.png differ diff --git a/ai-for-service/quality-ai/coach/coaching-assignments/images/coaching-assignments-agent-view.png b/ai-for-service/quality-ai/coach/coaching-assignments/images/coaching-assignments-agent-view.png index 46b120a4a..00e05caf7 100644 Binary files a/ai-for-service/quality-ai/coach/coaching-assignments/images/coaching-assignments-agent-view.png and b/ai-for-service/quality-ai/coach/coaching-assignments/images/coaching-assignments-agent-view.png differ diff --git a/ai-for-service/quality-ai/configure/conversation-intelligence.mdx b/ai-for-service/quality-ai/configure/conversation-intelligence.mdx index 947da2978..8ab02c4c6 100644 --- a/ai-for-service/quality-ai/configure/conversation-intelligence.mdx +++ b/ai-for-service/quality-ai/configure/conversation-intelligence.mdx @@ -3,137 +3,150 @@ title: Conversation Intelligence Settings sidebarTitle: Conversation Intelligence --- -The Settings section controls app-level configuration for Quality AI features. It includes three areas: +This extracts valuable insights from customer interactions across various communication channels. It captures information from spoken and written conversations, matches it with structured metadata about the interaction, and analyzes emotions and sentiments to understand customers' needs, opinions, and expectations. All post-interaction analytics from Contact Center AI interactions are reflected here. -- Conversation Intelligence -- Quality AI General Settings -- Language Settings +This helps you make app-level changes to **Conversation Intelligence**. It includes following three sections: ---- +- **Conversation Intelligence** +- **Quality AI General Settings** +- **Language Settings** -## Conversation Intelligence +--- -Conversation Intelligence extracts insights from customer interactions across voice and chat channels. It captures spoken and written conversation data, matches it with interaction metadata, and analyzes customer sentiment to surface needs, opinions, and expectations. +## Access Conversation Intelligence Navigate to **Quality AI** > **Configure** > **Settings** > **Conversation Intelligence**. -![Settings](/ai-for-service/quality-ai/configure/settings/images/default-settings.png) +![Conversation Intelligence](/ai-for-service/quality-ai/configure/settings/images/default-settings.png) -### Enable Conversation Intelligence +--- -Users can't view Conversation Intelligence Dashboard details until an administrator configures the settings. Permissions are based on user and agent roles. +## Enable Conversation Intelligence -1. Expand the **Conversation Intelligence** section. +1. Enable the **Conversation Intelligence Dashboard** toggle. +1. Configure script adherence settings as required. - ![Conversation Intelligence](/ai-for-service/quality-ai/configure/settings/images/converse-intelligence.png) +Role permissions in Settings are defined by Admin based on agent roles. -2. Enable the **Conversation Intelligence Dashboard** toggle. +--- - ![Conversation Intelligence Dashboard Toggle](/ai-for-service/quality-ai/configure/settings/conversation-intelligence/images/converse-intelligence-dashboard.png) +## Interaction Level Resolution Detection Methods - Enabling this option makes the Conversation Intelligence feature visible under the Analyze section. +This setting defines how the system determines contact-level resolution across conversations. Select the method that aligns with your quality standards and reporting needs. The system controls resolution calculation, while the Taxonomy Builder defines success and failure criteria for topics. --- -## Interaction-Level Resolution Detection +### Method 1: Topic-Based Resolution (Strict) -This setting defines how the system determines resolution across all conversations. The method you select applies organization-wide and affects dashboards, reports, and agent metrics. +The system evaluates resolution using **AND logic** across all L3 topics in a conversation. A contact is marked resolved only when every topic is successfully resolved. -### Method 1: Topic-Based Resolution (Strict) +#### Use when: -The system marks a contact as resolved only when **all** L3 topics in a conversation are resolved. +- You want strict resolution evaluation across all topics +- You want to prevent minor mentions from affecting resolution metrics +- You want structured control via Taxonomy Builder -**When to use:** -- All issues in a conversation must be resolved, including minor ones. -- Quality standards require complete issue closure. -- Compliance mandates full resolution tracking. +For example, if a payment issue is resolved but a rewards question isn't, the contact is still marked as resolved only if all L3 topics are resolved. -**Example:** A customer calls about a payment issue and asks a rewards question. If the payment is resolved but the rewards question isn't, the contact is marked unresolved. +#### Configuration: -**Configuration:** +1. Go to **Settings > Conversation Intelligence** +2. Select **Topic-Based Resolution (Strict)** +3. Select **Save** -1. Navigate to **Settings > Conversation Intelligence**. -2. Select **Topic-Based Resolution (Strict)**. -3. Select **Save**. +No additional configuration is required in Taxonomy Builder. -No further configuration is required. The Taxonomy Builder doesn't display a Resolution tab for this method. +--- ### Method 2: Overall Contact Level Resolution (Holistic Assessment) -The system uses an LLM to assess whether the customer's **primary** reason for contact was resolved, even if minor secondary issues remain. +The system uses an LLM-based evaluation to determine whether the primary reason for contact is resolved, even if minor issues remain unresolved. -**When to use:** -- Primary and secondary issues should be weighted differently. -- Agent performance should reflect resolution of the main customer concern. -- Minor mentions shouldn't affect resolution metrics. +#### Use when: -**Example:** A customer's payment issue is resolved, but a casual rewards question isn't. The contact is marked resolved. +- You want outcome-based evaluation of primary intent +- You want flexibility in handling secondary issues +- You want LLM-driven resolution assessment -**Configuration:** +For example, if a payment issue is resolved but a rewards question remains unresolved, the contact is still marked as resolved if the primary issue is addressed. -1. Navigate to **Settings > Conversation Intelligence**. -2. Select **Overall Contact Level Resolution (Holistic Resolution Assessment)**. -3. Select **Save**. -4. Navigate to **Quality AI > Configure > Taxonomy Builder**. -5. Open the **Resolution** tab. -6. Configure **Successful**, **Unsuccessful**, and **Overall Resolution** descriptions. -7. Select **Save**. +#### Configuration: - ![Interaction Level Resolution Detection](/ai-for-service/quality-ai/configure/settings/conversation-intelligence/images/interaction-level-resolution-detection-method.png) +1. Go to **Settings** > **Conversation Intelligence** +2. Select **Overall Contact Level Resolution (Holistic Assessment)** +3. Select **Save** +4. Go to **Quality AI** > **Configure** > **Taxonomy Builder** +5. Open the **Resolution** tab +6. Configure Successful resolution description, Unsuccessful resolution description, and Overall resolution description and save configuration. -This setting applies to all taxonomies, queues, and agents. Changing the resolution method affects all historical data interpretation, dashboards, reports, and agent performance metrics. +--- -### Settings vs. Taxonomy Configuration +## Impact of Changing Resolution Method -| Level | Scope | Who Configures | -|-------|-------|---------------| -| **Settings (Application level)** | Defines the global resolution detection method, applies organization-wide | Administrator | -| **Taxonomy Builder (Taxonomy level)** | Defines topic structures, resolution criteria, and holistic assessment descriptions | Administrator | +Changing the resolution method affects: +- All taxonomies in the organization +- All queues and agents +- Historical data interpretation +- Real-time dashboards and reports +- Agent performance metrics +- QA workflows + +--- + +## Settings vs Taxonomy Configuration + +### Settings (Application Level) + +- Defines global resolution detection method +- Applies across the entire organization +- Requires admin permissions +- Takes effect immediately system-wide + +### Taxonomy Builder (Taxonomy Level) + +- Defines topic structure and classification +- Configures topic-level resolution criteria (L3 only) +- Manages resolution descriptions (when enabled) +- Supports version-controlled updates --- ## Disable Conversation Intelligence -1. Toggle off **Conversation Intelligence Dashboard**. +1. Turn off the **Conversation Intelligence Dashboard** option. +2. Select **Confirm**. +3. Select **Save** to apply changes. + + + + + + + + + + + + + + + + - ![Disable Conversation Intelligence](/ai-for-service/quality-ai/configure/settings/images/disable-conversation-intelligence.png) - Disabling Conversation Intelligence hides all related insights — contact center efficiency, agent efficacy, and customer experience data — across the application. -2. Select **Confirm**. - ![Confirm Disable](/ai-for-service/quality-ai/configure/settings/images/confirm-conversation-intelligence.png) -3. Select **Save**. ---- -## Script Adherence Configuration -Script Adherence lets you define custom messages for each step of a conversation script. When enabled, the system checks agent adherence against your custom messages instead of the default generic script. -### Enable Script Adherence -1. Expand **Conversation Intelligence**, enable the **Conversation Intelligence Dashboard** toggle, then enable **Script Adherence Configuration**. - ![Script Adherence Toggle](/ai-for-service/quality-ai/configure/settings/images/script-adherence.png) -2. Configure messages for each script attribute: - | Field | Description | - |-------|-------------| - | **Attribute** | Script step (for example, Greeting, Branding) | - | **Action** | Toggle on to enable custom messages for that step | - | **Message** | Custom message agents should follow for that step | - ![Script Adherence Warning](/ai-for-service/quality-ai/configure/settings/conversation-intelligence/images/conv-warning-message.png) - If you enable the message toggle for an attribute, you must enter a message before saving. A warning appears if the field is left empty. Save each attribute before moving to the next. -3. Select **Save**. -When enabled, agents can view the following in their dashboard: -- **Conversation Intelligence Dashboard**-post-interaction analytics including brand tags, greetings, questions, emotions, and interactions. -- **Script Adherence**-the custom messages or scripts defined by the supervisor or administrator. ---- diff --git a/ai-for-service/quality-ai/configure/evaluation-forms.mdx b/ai-for-service/quality-ai/configure/evaluation-forms.mdx index f14c12a1f..301c90092 100644 --- a/ai-for-service/quality-ai/configure/evaluation-forms.mdx +++ b/ai-for-service/quality-ai/configure/evaluation-forms.mdx @@ -3,175 +3,251 @@ title: Evaluation Forms sidebarTitle: Evaluation Forms --- -Evaluation Forms standardize QA scoring across queues, channels, and interaction directions. QA Managers can configure metrics, scoring models, duration thresholds, and dispute workflows for Auto QA and manual evaluations. +Evaluation Forms provide a flexible framework for evaluating Human Agent and AI Agent interactions. They support configurable assignments, evaluation metrics, scoring models, duration thresholds, and version-controlled updates. QA Managers can create evaluation forms for contact center queues or Automation AI Experience Flows based on the selected assignment type. + +Evaluation Forms support two assignment types: + +- **Human Agent**: Evaluate human agent interactions routed through a contact center queue. The Assignments step presents Queues for selection. +- **AI Agent**: Evaluate bot interactions handled by an Automation AI flow. The Assignments step presents Experience Flows for selection. + + Set the assignment type during form creation and don’t change it later. Assign each evaluation form to either Queues (Human Agent) or Experience Flows (AI Agent), not both. AI Agent forms don’t support dispute workflows or speech metrics. + ## Key Features -| Feature | Description | -|------------------------------------|-------------------------------------------------------------------| -| **Multi-language Support** | Configure forms for multiple languages. | -| **Flexible Scoring** | Use percentage-based or points-based scoring. | -| **Fatal & Negative Scoring** | Configure pass thresholds, fatal metrics, and penalties. | -| **Direction-aware Evaluation** | Apply forms individually to inbound and outbound interactions. | -| **Channel-specific Configuration** | Separate settings for Voice and Chat. | -| **Queue and Channel Assignment** | Assign forms to specific queues and channels. | -| **Auto QA and Manual Audits** | Supports automated scoring and supervisor-led manual evaluations. | -| **Duration Threshold** | Exclude short or incomplete interactions from scoring. | -| **Dispute Configuration** | Configure acknowledgement and dispute workflows. | -| **Versioned Forms** | Apply updates only to future evaluations. | +| Capability | Description | +| ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **AI Agent Evaluation Forms** | Create evaluation forms for AI Agent conversations handled through Automation AI Experience Flows. | +| **Experience Flow Assignment** | Assign evaluation forms to one or more Experience Flows instead of contact center queues. | +| **AI Agent Metric Support** | Configure AI-compatible metrics, including **By Question (GenAI-Based)**, **By Question (Deterministic)**, and **By AI Agent** metrics. | +| **Flow-Based Assignment Rules** | Support one active evaluation form per Experience Flow and channel combination. Queue and Experience Flow assignments are mutually exclusive. | +| **Channel & Language Support** | Configure evaluation forms independently for supported channels and languages. | +| **Flexible Scoring** | Configure percentage-based or points-based scoring, pass scores, metric weights, and response logic. | +| **Fatal & Negative Scoring** | Define fatal error metrics and negative scoring based on evaluation requirements. | +| **Duration Thresholds** | Specify the minimum interaction duration required for evaluation eligibility. | +| **Cross-Segment Transcript Context** | Allow By Question metrics to evaluate transcript context across AI Agent and Human Agent segments in handoff conversations. | +| **Versioned Forms** | Maintain version history and an audit trail of evaluation form configurations and assignments. | +| **Default AI Agent Evaluation Form** | Automatically creates and assigns a default AI Agent evaluation form with predefined quality metrics to all Experience Flows. | --- ## Evaluation Forms Structure -Defines the overall evaluation and dispute workflow configuration. +Defines the overall configuration of an evaluation form. -| Section | Purpose | -|------------------------|------------------------------------------------------------| -| **General Settings** | Configure form details, scoring, language, and thresholds. | -| **Evaluation Metrics** | Define scoring metrics and outcomes. | -| **Assignments** | Map queues, channels, and contact direction. | -| **Dispute Allocation** | Configure dispute routing and re-dispute rules. | +| Section | Human Agent | AI Agent | +| ------------------------- | ------------------------------------------------------------------------------------- | ---------------------------------------------------------- | +| **General Settings** | Configure form details, language, channel, scoring, thresholds, and assignment type. | Configure the same settings for AI Agent evaluation forms. | +| **Assignments** | Assign evaluation forms to queues with Contact Direction and Conversation Source. | Assign evaluation forms to **Experience Flows**. | +| **Evaluation Metrics** | Configure all supported evaluation metrics. | Configure AI Agent-supported metrics only. | +| **Dispute Configuration** | Configure dispute and acknowledgement workflows. | Not supported. | --- + ## Access Evaluation Forms Navigate to **Quality AI** > **Configure** > **Evaluation Forms**. -![Evaluation Forms](/ai-for-service/quality-ai/configure/evaluation-criteria/evaluation-forms/images/evaluation-forms.png) +![Evaluation Forms](/ai-for-service/quality-ai/configure/evaluation-criteria/evaluation-forms/images/eva-forms.png). --- -## Evaluation Forms Elements +### Evaluation Forms List + +The Evaluation Forms page displays the following list: + +| Field | Description | +| --------------- | -------------------------------------------------------------------------------- | +| **Name** | Displays the evaluation form name. | +| **Description** | Displays the evaluation form description. | +| **Assigned In** | Displays the assigned **Queue** (Human Agent) or **Experience Flow** (AI Agent). | +| **Channel** | Displays the assigned channel (Voice or Chat). | +| **Pass Score** | Displays the minimum passing score. | +| **Status** | Indicates whether the evaluation form is active or inactive. | +| **Created By** | Displays the user who created the evaluation form. | +| **Search** | Searches evaluation forms by name. | + + To create **AI Agent** evaluation forms, also enable the **Automation AI Conversation Source**. -The Evaluation Forms display the following list of elements: +--- + +## Evaluation Processing -| Column | Description | -|--------|-------------| -| **Name** | Evaluation form name. | -| **Description** | Short description of the form.| -| **Queues** | Assigned and unassigned queues. | -| **Channel** | Channel mode assigned to the form (Voice or Chat). | -| **Created By** | Form creator. | -| **Pass Score** | Minimum score for the agent to pass. | -| **Status** | Enable or disable scoring for a form. | -| **Search** | Find evaluation forms by name. | +For conversations that transfer between an AI Agent and a Human Agent, By Question metrics can evaluate transcript context across both conversation segments. This provides additional context for AI-based evaluation while each conversation segment continues to use its own assigned Evaluation Form. -Enable Auto QA in Quality AI Settings before creating evaluation forms. +This behavior applies only to supported By Question metrics. --- ## Create a New Evaluation Form -Creating an evaluation form involves the following four sections: +Evaluation Form creation consists of three or four steps depending on the selected assignment type. -### General Settings +### Configure General Settings 1. Select the **Evaluation Forms** tab. 1. Select **+ New Evaluation Forms**. -1. Enter a **Name** and **Description** (Optional). -1. Select the required **Language**. -1. Select a **Channel** type: - * **Chat**: Displays only chat metrics (excluding speech and voice-specific Playbook metrics). - * **Voice**: Displays voice, speech, and Playbook-supported metrics. -1. Select a **Scoring Type** (**Percentage** or **Points**). +1. Enter a **Name** for the new form. +1. Enter a **Description** (Optional). +1. Select more than one **Language** if needed. +1. Select a **Handled By** agent to assign a queue name: **Human Agent** (Human Agent forms) or **AI Agent** (Experience Flow name). +1. Select a **Channel** type: + + * **Chat**: Display only chat metrics (excluding speech and voice-specific Playbook metrics). + * **Voice**: Display all applicable voice metrics, including speech and Playbook metrics. + +1. Select a **Scoring Type**: **Percentage** or **Points**. 1. (Optional) Set the minimum interaction duration required for evaluation by entering values in **MIN** and **SEC**. 1. Set a minimum **Pass Score** required for agents. 1. Select **Next**. -![General Settings Configuration](/ai-for-service/quality-ai/configure/evaluation-criteria/evaluation-forms/images/add-new-eva-forms2.png) + ![General Settings Configuration](/ai-for-service/quality-ai/configure/evaluation-criteria/evaluation-forms/images/add-new-eva-forms2.png) + +Assign at least one queue before proceeding to the next step, and make sure the selected direction determines which interaction types the system evaluates for the queue. + +### Configure Assignments + +The **Assignments** step defines where the evaluation form is used. Based on the selected **Handled By** type, assign the form to **Queues** (Human Agent) or **Experience Flows** (AI Agent). +#### Human Agent Assignments -### Assignments +1. Search for and select one or more **Queues**. +2. For each selected queue, configure the **Contact Direction** (Inbound, Outbound, or Both) and **Conversation Source** (such as Contact Center AI (CCAI), AgentAI, or Quality AI Express). +3. Review the selected queues. +4. Select **Next**. -Assign queues to the evaluation form and define the interaction direction for evaluation. + ![Human Agent Experience Flow](/ai-for-service/quality-ai/configure/evaluation-criteria/evaluation-forms/images/config-queues.png) -1. Search and add queues to the Evaluation Form. -2. Select the applicable contact direction for each queue: **Inbound**, **Outbound**, or **Both**. -3. Select a **Conversation Source**: - * **Quality AI Express**: Processes Express-based processing, - * **CCAI Integration**: Processes Contact Center AI interactions - * **Agent AI Integration**: Processes Agent AI interactions. -6. Select **Next**. +If an evaluation form includes both **CCAI** or **AgentAI** queues and **Quality AI Express** queues, **By Playbook** and **By Dialog** evaluation metrics are unavailable. - ![Queue Configuration](/ai-for-service/quality-ai/configure/evaluation-criteria/evaluation-forms/images/config-queues.png) +#### AI Agent Assignments - If CCAI or Agent AI queues are assigned together with Quality AI Express queues in the same form, **By Playbook** and **By Dialog** metrics are unavailable. +1. Search for and select one or more **Experience Flows**. +2. Review the selected Experience Flows. +3. Select **Next**. + + ![AI Agent Experience Flow](/ai-for-service/quality-ai/configure/evaluation-criteria/evaluation-forms/images/ai-agent-experience-flow.png) + +Experience Flows are retrieved from the configured Automation AI environment. + +#### Experience Flow Assignment Rules + +| Rule | Description | +|------|-------------| +| **One Form Per Flow Per Channel** | Each Experience Flow supports only one active evaluation form per channel. If a form exists for the same flow and channel, the system displays a validation error. | +| **Mutually Exclusive Assignment** | You can assign an evaluation form either to queues or to Experience Flows, but not both. | +| **Flow Source** | Experience Flows available for selection come from the Automation AI configuration in the same workspace. | #### Queue Assignment Rules -Each Evaluation Form supports only one unique Queue + Channel + Contact Direction assignment combination. Duplicate assignments aren't allowed, and only queues the user has access to are displayed. +| Assignment Type | Rule | +|-----------------|------| +| **Human Agent** | Assign each evaluation form to a unique **Queue + Contact Direction** combination. You can't assign the same combination more than once within a form. | +| **AI Agent** | Assign each Experience Flow to only one active evaluation form per channel. You can't combine Queue and Experience Flow assignments in the same evaluation form. | -If you don't select a direction, the system excludes the queue from evaluation. CCAI Chat queues don’t support the Outbound direction. +Experience Flows available for selection come from the Automation AI configuration in the same workspace. -Supported directions include Inbound (incoming interactions), Outbound (agent-initiated interactions), or Both (same form applied to both inbound and outbound interactions). +### Configure Evaluation Metrics -Assign at least one queue before proceeding to the next step, and make sure the selected direction determines which interaction types the system evaluates for the queue. +Evaluation metrics define the criteria used to evaluate interactions for audits and AutoQA scoring. The available metric types depend on the selected **Handled By** type. Manual metrics are human-scored, while AI-based metrics use configured AI evaluation methods. ---- +| **Human Agent Metrics** | **AI Agent Metrics** | +| ---------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1. Search for and select one or more **Evaluation Metrics**. | 1. Search for and select one or more **Evaluation Metrics**. | +| 2. Review the **Total Positive Weightage** and **Total Negative Weightage**. | 2. Select the **Edit** icon to configure the metric **Weightage** or **Points**, **Response Outcomes**, **Correct Response**, and **Fatal Error**, if applicable for compliance-critical metrics. | +| 3. Select the **Edit** icon to configure the metric. | 3. Review the **Total Positive Weightage** and **Total Negative Weightage**. | +| 4. Configure the **Weightage** or **Points**, **Response Outcomes**, and **Correct Response**. | 4. Reorder or delete metrics, if required. | +| 5. Reorder or delete metrics, if required. | 5. Select **Create** to save the evaluation form. | +| 6. Select **Next**. | — | + + ![AI Agent Evaluation Metrics](/ai-for-service/quality-ai/configure/evaluation-criteria/evaluation-forms/images/ai-agent-eva-metrics.png) + +### Supported Metric Types -### Evaluation Metrics +| **Human Agent** | **AI Agent** | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Supports all available evaluation metric types, including **By Question**, **By Playbook**, **By Speech**, **By Hold**, **By Transfer**, **By Value**, **Manual**, and other Quality AI metrics. | Supports all available evaluation metric types, including **By Question (GenAI-Based)**, **By Question (Deterministic)**, and **By AI Agent** metrics. | +| Manual metrics are available only with **Points** scoring. | AI Agent evaluation forms don't support **By Playbook Adherence**, **By Hold**, **By Value**, **By Transfer**, or **By Speech** (Dead Air, Cross-talk, Speaking Rate). These metric types don't display in the Evaluation Metrics list. | -Evaluation metrics define the criteria used for audits and Auto QA scoring. Manual metrics are human-scored and assess qualitative aspects such as tone, empathy, and judgment. +### Metric Card Configuration -#### Add Configure Metrics +#### Trigger Scoring Disabled -1. Add the required evaluation metrics to the form. -1. Select the **Edit** icon to configure the metric weight or points value, response logic, scoring outcomes, and optional **Fatal Error** settings for compliance-critical metrics. -1. Reorder or remove metrics as needed. -1. Select **Next**. +When **Trigger Scoring** is off, the metric card shows a Weightage field for score contribution and a **Fatal Error** toggle that fails the evaluation if the metric isn't met. - ![Points Metrics](/ai-for-service/quality-ai/configure/evaluation-criteria/evaluation-forms/images/points-scoring.png) +#### Trigger Scoring Enabled - Manual metrics are supported only in points-based scoring and are excluded from automated (Auto QA) scorecards. +When **Trigger Scoring** is on, the metric card expands to outcome-level scoring with Yes or No rows, each having a Weightage field, plus a **Correct Response** option and a **Fatal Error** toggle for non-adherent outcomes, with negative scoring set at the outcome level. -### Dispute Allocation +#### Outcome Configuration + +Defines **Yes** or **No** outcomes with positive, zero, or negative weights, where matching responses get positive weight and non-matching responses get zero or negative weight if configured. + + Configure negative scoring only at the outcome level. + +### Configure Dispute Allocation (Human Agents Only) -Configure how agents can acknowledge or dispute evaluations created under this form. +The **Dispute Allocation** step defines how the system routes evaluation disputes. -1. Turn on **Dispute Resolution Assignment** to make acknowledgement and dispute actions available to agents. -1. Select a routing option for dispute re-evaluation: - * **Same Auditor**: Routes disputes back to the original evaluator. - * **Different Auditor**: Routes disputes to another QA from the selected auditor list. -1. If you select **Different Auditor**: - * Search and select one or more auditors. - * Select **Add** to include them in the routing list. -1. Set the **Max. Number of Re-Disputes rounds** for agents after re-evaluation. -1. Select **Create**. +1. Turn on **Dispute Resolution Assignment**. +2. Select one routing option: + - **Same Auditor**: Routes disputes assigned back to the original evaluator. + - **Different Auditor**: Routes disputes assigned to another QA from the selected auditor list. +3. If you select **Different Auditor**, search and select one or more auditors. +4. Turn on **Allow Multi-Round Re-Disputes**, if required. +5. Select the **Maximum Number of Re-Disputes** rounds for agents after re-evaluation. +6. Select **Create**. -When you disable dispute resolution, the system marks completed evaluations as **Audited**, and agents can't acknowledge or dispute them. To enable acknowledgment or disputes, you must enable **Agent Accept & Dispute** in **Settings** > **Quality AI General Settings**. + If **Dispute Resolution Assignment** is turned off, agents can't acknowledge or dispute completed evaluations. -![Dispute Allocation](/ai-for-service/quality-ai/configure/evaluation-criteria/evaluation-forms/images/eva-metrics-dispute-allocation.png) +![Evaluation Metrics Dispute Allocation](/ai-for-service/quality-ai/configure/evaluation-criteria/evaluation-forms/images/eva-metrics-dispute-allocation.png) +#### Metric Rules + +- The combined positive weightage must equal 100% for percentage-based scoring. +- Configure the expected response and score for every metric. +- Human Agent forms support all available metric types, and Fatal Error settings apply only to supported metrics. +- AI Agent forms support only AI Agent metrics used during Experience Flow evaluations. + +## Evaluation Processing ### Evaluation Behavior -During evaluation, the system selects the applicable form based on queue, channel, and contact direction. The system skips conversations without matching assignments and supports manual metrics only in points-based scoring, excluding them from automated scorecards. +During evaluation, the system selects the applicable form based on queue, channel, and contact direction. Conversations without matching assignments are skipped. Manual metrics are supported only in points-based scoring and excluded from automated scorecards. -* **Trigger Scoring Disabled**: When Trigger Scoring is off, the metric card shows a Weightage field for score contribution and a Fatal Error toggle that fails the evaluation if the metric isn't met. -* **Trigger Scoring Enabled**: When Trigger Scoring is on, the metric card expands to outcome-level scoring with Yes or No rows, each having a Weightage field, plus a Correct Response option and a Fatal Error toggle for non-adherent outcomes, with negative scoring set at the outcome level. -* **Outcome Configuration**: Defines Yes or No outcomes with positive, zero, or negative weights, where matching responses get positive weight and non-matching responses get zero or negative weight if configured. +### Form Selection Logic - You must configure negative scoring at the outcome level. +During evaluation, the system selects the applicable Evaluation Form using a hierarchy of Queue > Channel > Contact Direction, ensuring the selected form matches the interaction’s configured queue, channel, and contact direction. -#### Outcome Configuration +An interaction is eligible for evaluation when it has a valid queue association and an active evaluation form mapped to that queue. + +Conversations linked to evaluation-enabled queues remain eligible for evaluation even if the handling agent is not directly assigned to the queue, provided they match the configured channel and contact direction. -For each metric, define the outcomes (for example, **Yes** or **No**) and assign a positive, zero, or negative weight based on the expected response. A matching response receives positive weight. A non-matching response receives zero or negative weight (if configured). +Evaluation eligibility is based on the conversation’s mapped queue and assigned Evaluation Form, not on agent-to-queue membership in agent or workforce management systems. -## Contact Duration +#### Contact Duration Evaluation The system evaluates contact duration before scoring. -| Contact Duration Status | Assigned Result | Notes | -|--------------------------------|----------------------|--------------------------------------------| -| **Meets or Exceeds Threshold** | — | Evaluated normally. | -| **Falls Below Threshold** | Below Threshold | Excluded from scoring and quality metrics. | -| **Duration Unresolved** | Duration unavailable | Excluded from evaluation. | +| Contact Duration | Status Assigned | Result | +|------------------|-----------------|--------| +| Meets or exceeds threshold | — | Evaluated normally | +| Falls below threshold | Below Threshold | Excluded from scoring and quality metrics | +| Duration unresolved | Duration unavailable | Excluded from evaluation | + + Threshold updates apply only to future interactions. A contact may qualify for one evaluation form but not another. + +## Scoring -Threshold updates apply only to future interactions, and a contact may qualify for one form but not another. +### Scoring Logic + +- **Pass:** Final score ≥ Pass Score threshold +- **Fail:** Final score < Pass Score threshold -### Scoring Formula (Points-Based) +The system calculates the final score using positive and negative metric weights. + +### Points-Based Scoring Formula Kore Evaluation Score calculates the weighted impact of met and not-met metrics, subtracts penalties, divides by total positive weights, and multiplies the result by 100. @@ -184,84 +260,105 @@ where, - **Myi**, **Wyi** = Adhered metrics and positive points. - **Mni**, **Wni** = Non-adhered metrics and negative points. -### Scoring Logic +### Percentage vs. Points Comparison -- **Pass**: Final score ≥ Pass Score threshold. -- **Fail**: Final score < Pass Score threshold. - -The system calculates the final score using positive and negative metric weights. - -### Scoring Systems Comparison - -Quality AI supports two scoring methods: Percentage-Based and Points-Based. - -| **Feature** | Percentage-based | Points-based | -| -------------------------- | -------------------------------------------------------------------- | --------------------------------------------------------------------------- | -| **Best for** | Fewer than 20 metrics. | 20 or more metrics. | -| **Total weight** | Must equal 100%. | No fixed maximum. | -| **Scalability** | Limited. | High. | -| **Weight per metric** | Decreases as metrics increase (for example, 40 metrics ≈ 2.5% each). | Assign any point value based on importance. | -| **Weight precision** | May require fractional values (for example, 2.5%). | Uses whole-number allocation (for example, 50 points for critical metrics). | -| **Negative scoring** | Managed within 100% structure. | Negative points allowed; can't exceed total positive points. | -| **Final evaluation score** | Direct percentage (0-100). | Normalized to percentage (0-100). | +Quality AI supports two scoring methods. +| Feature | Percentage-Based | Points-Based | +|----------|------------------|--------------| +| **Best For** | Fewer than 20 metrics | 20 or more metrics | +| **Total Weight** | Must equal 100% | No fixed maximum | +| **Scalability** | Limited | High | +| **Weight Per Metric** | Decreases as metrics increase (for example, 40 metrics ≈ 2.5% each) | Assign any point value based on importance | +| **Weight Precision** | May require fractional values (for example, 2.5%) | Uses whole-number allocation (for example, 50 points for critical metrics) | +| **Negative Scoring** | Managed within 100% structure | Negative points allowed; cannot exceed total positive points | +| **Final Evaluation Score** | Direct percentage (0–100) | Normalized percentage (0–100) | ### Weight Assignment Rules -| **Configuration** | Percentage-based | Points-based | -| ------------------------------------------- | ----------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | -| **If the expected correct response is Yes** | Positive % for Yes; zero or negative % for No. | Positive points for Yes; zero or negative points for No. | -| **If the expected correct response is No** | Positive % for No; zero or negative % for Yes. | Positive points for No; zero or negative points for Yes. | -| **Validation** | Total positive weight must equal 100%; negative weight allowed within the 100% structure. | No upper limit on total positive points; total negative points must not exceed total positive points. | -| **Manual Evaluation Metrics** | Not supported. | Supported. | - +| Configuration | Percentage-Based | Points-Based | +|--------------|------------------|--------------| +| Correct Response = Yes | Positive % for Yes; zero or negative % for No | Positive points for Yes; zero or negative points for No | +| Correct Response = No | Positive % for No; zero or negative % for Yes | Positive points for No; zero or negative points for Yes | +| Validation | Total positive weight must equal 100%; negative weight allowed within the 100% structure | No upper limit on total positive points; total negative points ≤ total positive points | +| Manual Evaluation Metrics | Not supported | Supported | ### Fatal Error Behavior Fatal error configuration remains the same for both scoring types. When a fatal metric fails, the system sets the final score to 0 and marks the interaction as failed, regardless of other metric scores. -### Form Selection Logic +## Manage Evaluation Forms -During evaluation, the system selects the applicable Evaluation Form using a hierarchy of **Queue** > **Channel** > **Contact Direction**, ensuring the selected form matches the interaction’s configured queue, channel, and contact direction. +### Edit a Form -An interaction is eligible for evaluation when it has a valid queue association and an active evaluation form mapped to that queue. +Select **Edit** from the Evaluation Forms list. -Conversations linked to evaluation-enabled queues remain eligible for evaluation even if the handling agent isn't directly assigned to the queue, provided they match the configured channel and contact direction. +You can modify: General Settings, Queue or Experience Flow assignments, Evaluation Metrics, and Dispute Allocation (Human Agent only). -The system determines evaluation eligibility based on the conversation’s mapped queue and assigned Evaluation Form, not on agent-to-queue membership in agent or workforce management systems. +Select **Update** to save changes. ---- +### Delete a Form -## Managing Evaluation Forms +Select **Delete** from the Evaluation Forms list. -This section guides you through editing and updating the existing evaluation forms. +Before deleting: -### Edit and Delete Existing Evaluation Forms +- Remove all Queue or Experience Flow assignments. +- Resolve dependent metric or attribute references. -Steps to edit or delete the existing evaluation forms: +If the form is in use, the system displays a warning before deletion. -1. Use the three-dot menu to **Edit** or **Delete** the evaluation form and update the required details. -1. Before deleting an evaluation form, remove linked queue assignments, dependent metrics (if required), and resolve attribute dependencies. If the form is still in use, the system displays a warning. -1. Select **Update** to save changes. +Select **Update** to save changes. ---- +### Versioning + +Every update creates a new version of the Evaluation Form. Existing evaluations continue to use the version that was active when the evaluation started, while future evaluations use the latest published version. + +The system also maintains a history of assignment and configuration changes. ## Troubleshooting -### Switching Scoring Types + +### Switching Scoring Type Changing the scoring type clears all existing metric weights and requires reconfiguration; percentage totals must equal 100% and points-based values must meet validation rules. -### Unsupported Language Error (Form Level) +### Unsupported Language Issues (Form-Level) -This error occurs when you add a new language to a form but one or more associated metrics don't support it. The system blocks the update until all metrics support the selected language. For example, if you add Hindi to a form while metrics support only English and Dutch, the system triggers this error. +This error occurs when a new language is added to a form but one or more associated metrics don't support it. The system blocks the update until all metrics support the selected language. For example, adding Hindi to a form with metrics that support only English and Dutch triggers this error. -To resolve it, review each metric’s language configuration, update the metrics to support the new language, and then add the language to the form once all metrics are compatible. +To resolve it, review each metric’s language configuration, update metrics to support the new language, and then add the language to the form once all metrics are compatible. + +### Metric-Level Language Limitation + +This warning occurs when a metric is added or updated that doesn't support a language configured on the form. + +Resolve it by configuring the required language on the metric or selecting a metric that supports all form languages. ### Language Selection Behavior -The system applies an AND condition to multi-language selection, so it displays only By-Question metrics available in all selected languages. For example, if you select English and Dutch, the system shows only metrics available in both languages. +Multi-language selection applies an AND condition, so only By-Question metrics available in all selected languages are displayed. For example, if you select English and Dutch, the system shows only metrics available in both languages. + +### Channel Mode Change + +Switching between Voice and Chat triggers a warning and removes speech-based metrics. Update remaining metrics and adjust weights before saving changes. + +### No Dispute Workflow + +AI Agent evaluation forms don't include **Dispute Allocation**. Agents can't acknowledge or dispute AI Agent evaluations. + +### Speech Metric Limitations + +Speech-based metrics aren't available for AI Agent evaluation forms because Experience Flow evaluations analyze bot interactions rather than voice quality. + +### Human Agent Considerations + +Human Agent evaluation forms: + +- Support all evaluation metric types. +- Support Dispute Allocation. +- Support Queue assignments with Contact Direction and Conversation Source. +- Support Manual Evaluation metrics only with Points-based scoring. +- Support speech-specific metrics for Voice channel evaluations. -### Channel Mode Change -Switching between Voice and Chat triggers a warning and removes speech-based metrics; update remaining metrics and adjust weights before saving changes. diff --git a/ai-for-service/quality-ai/configure/evaluation-metrics.mdx b/ai-for-service/quality-ai/configure/evaluation-metrics.mdx index d4d50cb3a..247f23234 100644 --- a/ai-for-service/quality-ai/configure/evaluation-metrics.mdx +++ b/ai-for-service/quality-ai/configure/evaluation-metrics.mdx @@ -52,11 +52,11 @@ After selecting a measurement type, you have to configure details such as metric Evaluates adherence to specific questions asked or answered during interactions. **Key features:** -- **Static Adherence**-applies to all conversations -- **Dynamic Adherence**-conditional evaluation triggered by specific events -- **GenAI Detection**-contextual understanding with no training samples required -- **Deterministic Detection**-semantic matching against predefined patterns -- **Flexible thresholds**-set different similarity scores per use case +- **Static Adherence**: Applies to all conversations +- **Dynamic Adherence**: Cconditional evaluation triggered by specific events +- **GenAI Detection**: Contextual understanding with no training samples required +- **Deterministic Detection**: Semantic matching using predefined patterns +- **Flexible thresholds**: Set different similarity scores per use case **Common use cases:** Script adherence, greeting compliance, policy verification, response quality. @@ -67,9 +67,9 @@ For full configuration details, see [By Question](/ai-for-service/quality-ai/con Analyzes speech characteristics during voice interactions. **Key features** -- **Crosstalk**-detects overlapping speech with configurable thresholds -- **Dead Air**-monitors silence periods (configurable duration) -- **Speaking Rate**-tracks Words Per Minute (WPM) +- **Crosstalk**: Detects overlapping speech with configurable thresholds +- **Dead Air**: Monitors silence periods (configurable duration) +- **Speaking Rate**: Tracks Words Per Minute (WPM) **Use cases:** Voice quality, conversation flow analysis, speaking pace optimization. @@ -77,13 +77,13 @@ For full configuration details, see [By Speech](/ai-for-service/quality-ai/confi ### By Value -Verifies customer-specific information shared by an agent against trusted data sources. +Verifies customer-specific information shared by an agent with trusted data sources. **Key features:** -- **API integration**-real-time verification with CRM and external systems -- **Business rules engine**-five rule types (first/last value, negotiated, strict matching, custom) -- **Compliance tracking**-detects deviations from expected values -- **Audit trails**-logs validation results for supervisory review +- **API integration**: Real-time verification with CRM and external systems +- **Business rules engine**: Five rule types (first/last value, negotiated, strict matching, custom) +- **Compliance tracking**: Detects deviations from expected values +- **Audit trails**: Logs validation results for supervisory review **Use cases:** Pricing accuracy, interest rate verification, account balance confirmation, compliance validation. @@ -94,9 +94,9 @@ For full configuration details, see [By Value](/ai-for-service/quality-ai/config Assesses completion and quality of specific tasks or workflows within a conversation. **Key features:** -- **Dialog agent selection**-choose which dialog agent to evaluate -- **Evaluation scope**-entire conversation or time-bound segment -- **Time parameters**-configurable in seconds (voice) or message count (chat) +- **Dialog agent selection**: Choose which dialog agent to evaluate +- **Evaluation scope**: Entire conversation or time-bound segment +- **Time parameters**: Configurable in seconds (voice) or message count (chat) **Use cases:** Workflow adherence, task completion verification, dialog flow optimization. @@ -107,9 +107,9 @@ For full configuration details, see [By Dialog Task](/ai-for-service/quality-ai/ Measures how well interactions follow predefined playbooks or procedures. **Key features:** -- **Entire Playbook**-assesses adherence across all playbook components -- **Specific Steps**-targets evaluation at specific stages or steps -- **Percentage thresholds**-define minimum adherence levels required +- **Entire Playbook**: Assesses adherence across all playbook components +- **Specific Steps**: Targets evaluation at specific stages or steps +- **Percentage thresholds**: Define minimum adherence levels required **Use cases:** Process compliance, procedure adherence, enforcement of standards. @@ -135,10 +135,10 @@ Manual Evaluation metrics enable QA teams to assess agent performance through hu **Key features:** -- **Human-Driven Assessment**-metrics are evaluated exclusively by QA auditors without Auto QA involvement. -- **Points-Based Only**-available only within points-based evaluation forms to ensure accurate scoring allocation. -- **No AI Dependency**-independent of GenAI, deterministic detection, triggers, and adherence thresholds. -- **Clear Visual Identification**-displays distinctly across Audit screens, Conversation Mining, Heatmaps, and Reports with the suffix (Manual Evaluation Metric). +- **Human-Driven Assessment**: QA auditors evaluate metrics manually, without Auto QA involvement. +- **Points-Based Only**: Available only within points-based evaluation forms to ensure accurate scoring allocation. +- **No AI Dependency**: Independent of GenAI, deterministic detection, triggers, and adherence thresholds. +- **Clear Visual Identification**: Displays distinctly across Audit screens, Conversation Mining, Heatmaps, and Reports with the suffix (Manual Evaluation Metric). **Use Cases:** Manual Evaluation is ideal for assessing complex soft skills (such as tone, empathy, and negotiation), regulatory scenarios requiring human judgment, dispute handling quality, escalation decisions, and high-risk or edge-case interactions. @@ -148,18 +148,18 @@ For full configuration details, see [By Manual Evaluation](/ai-for-service/quali ### By Hold -Evaluates how effectively agents manage customer hold scenarios during voice interactions, ensuring proper communication, timing, and resumption behavior. +Evaluates how well agents manage customer hold scenarios during voice interactions, ensuring proper communication, timing, and resumption behavior. **Key Features:** -* **Static Adherence**-applies consistently to all conversations with hold events -* **Event-driven Evaluation**-triggers automatically when hold events occur via telephony integration -* **Multi-instance Detection**-evaluates multiple hold events within a single interaction -* **GenAI Detection**-contextual, flexible evaluation using LLM-based understanding -* **Deterministic Detection**-embedding-based semantic matching against predefined utterances -* **Configurable Sub-criteria**-assess hold notification, duration compliance, and call resumption -* **Flexible Thresholds**-defines similarity scores, hold duration limits, and evaluation windows -* **Weighted Scoring**-assigns percentage-based contributions to each sub-criterion +* **Static Adherence**: Applies consistently to all conversations with hold events +* **Event-driven Evaluation**: Triggers automatically when hold events occur via telephony integration +* **Multi-instance Detection**: Evaluates multiple hold events within a single interaction +* **GenAI Detection**: Contextual, flexible evaluation using LLM-based understanding +* **Deterministic Detection**: Embedding-based semantic matching using predefined utterances +* **Configurable Sub-criteria**: Assess hold notification, duration compliance, and call resumption +* **Flexible Thresholds**: Defines similarity scores, hold duration limits, and evaluation windows +* **Weighted Scoring**: Assigns percentage-based contributions to each sub-criterion **Use Cases:** Hold etiquette compliance, agent coaching, customer experience improvement, regulatory adherence, and interaction quality monitoring during hold scenarios. @@ -174,4 +174,8 @@ For full configuration details, see [By Hold](/ai-for-service/quality-ai/configu 3. Choose **Edit** to modify or **Delete** to remove the metrics. 4. Select **Update** to save changes. + --- + + + diff --git a/ai-for-service/quality-ai/configure/settings/conversation-intelligence/images/default-settings.png b/ai-for-service/quality-ai/configure/settings/conversation-intelligence/images/default-settings.png index de9ebdb33..1f388e45c 100644 Binary files a/ai-for-service/quality-ai/configure/settings/conversation-intelligence/images/default-settings.png and b/ai-for-service/quality-ai/configure/settings/conversation-intelligence/images/default-settings.png differ diff --git a/docs.json b/docs.json index aeda02cb7..8b1bf1e92 100644 --- a/docs.json +++ b/docs.json @@ -635,7 +635,8 @@ }, "redirects": [ - { "source": "/ai-for-service/apis/agent-ai/hooks-api-for-internal-transfers", "destination": "/ai-for-service/apis/agent-ai/webhook-api-for-conversation-events" }, + { "source": "/ai-for-service/quality-ai/analyze/conversation-intelligence", "destination": "/ai-for-service/quality-ai/analyze/cx-insights" }, + { "source": "/ai-for-service/apis/agent-ai/hooks-api-for-internal-transfers", "destination": "/ai-for-service/apis/agent-ai/webhook-api-for-conversation-events" }, { "source": "/ai-for-service/agentai/agent-experience/access-custom-data-in-agent-ai", "destination": "/ai-for-service/agentai/agent-experience" }, { "source": "/agent-platform/models/open-source-models/importing-models/", "destination": "/agent-platform/v1/models/open-source-models#import-a-local-model"}, { "source": "/agent-platform/models/open-source-models/model-optimization/", "destination": "/agent-platform/v1/models/open-source-models#optimization-techniques"}, @@ -1050,4 +1051,4 @@ { "playground": { "proxy": false } } -} \ No newline at end of file +}