Designing Bot Analytics Dashboards for Tracking Containment, Escalation, and Drop-Off Rates

How-To-Articles
1

Designing Bot Analytics Dashboards for Tracking Containment, Escalation, and Drop-Off Rates

What This Guide Covers

This guide details the architectural design of analytics dashboards specifically calibrated to measure Conversational AI performance within Genesys Cloud CX. You will configure calculated metrics and reporting views that distinguish between successful containment events and necessary human escalations. The end result is a monitoring framework capable of identifying drop-off points where user intent diverges from bot capabilities, allowing for data-driven optimization of dialog flows.

Prerequisites, Roles & Licensing

To execute this configuration, the following environment state is required:

Licensing Tiers

  • Genesys Cloud CX: Enterprise or Professional edition with Digital Engagement enabled.
  • Conversational AI: The Conversational AI add-on license must be active for the specific application users are interacting with. Standard CCX licenses do not expose granular bot intent metrics without this add-on.
  • Workforce Management (Optional): WFM integration is required if you plan to correlate escalation rates with real-time staffing levels.

Permissions & Roles

  • Reporting > Admin: Required to create Custom Calculated Metrics and modify Report Definitions.
  • Digital Engagement > Conversation View: Required to access raw conversation data for verification of intent mapping.
  • API Access (Optional): analytics.conversations.read scope is required if you intend to query the Reporting API directly for custom dashboarding via BI tools like PowerBI or Tableau.

External Dependencies

  • Conversational AI Application: A deployed bot application with defined intents and escalation nodes must exist in the environment.
  • CRM/Backend Systems (Optional): If using CRM data to validate containment success, ensure integration endpoints are live.

The Implementation Deep-Dive

1. Metric Taxonomy and Semantic Mapping

The foundational error in bot analytics is relying on default conversation types without understanding their semantic definitions within the Genesys Cloud reporting schema. Default metrics often conflate “Successful” conversations with “Contained” conversations, leading to inflated success rates. You must explicitly define containment logic using Custom Calculated Metrics.

Architectural Reasoning:
In a standard telephony flow, a call is successful if it connects. In a conversational AI context, a session can connect but fail the user’s goal. A bot might successfully route a user to an agent (escalation), which counts as a “Successful Conversation” in default reporting but represents a containment failure from a cost-efficiency perspective. Therefore, you must construct metrics that isolate ccx.conversation.contained and compare it against ccx.conversation.escalated.

The Trap:
Many architects rely on the standard Conversation Type > Successful metric. This includes escalations to human agents as successes because the user eventually reached a live agent. If your goal is cost reduction via automation, this metric is misleading. It creates a false positive where the bot appears to be working perfectly while actually offloading work to expensive human resources unnecessarily.

Implementation Steps:

  1. Navigate to Admin > Reporting > Metrics.
  2. Create a new Custom Calculated Metric named Bot Containment Rate.
  3. Use the following logic for the calculation:
    (Sum(ccx.conversation.contained) / Sum(ccx.conversation.interacted)) * 100
  4. Define the scope to include only conversations where ccx.bot.id is not null. This filters out standard voice or email interactions that did not utilize the AI engine.

API Payload Reference:
If configuring via the Reporting API, the metric definition requires specific aggregation functions.

{
  "name": "Bot Containment Rate",
  "expression": "(SUM(ccx.conversation.contained) / SUM(ccx.conversation.interacted)) * 100",
  "aggregationType": "CALCULATED_METRIC",
  "metricId": null, 
  "filterConditions": [
    {
      "name": "botId",
      "type": "STRING",
      "operator": "IS_NOT_EMPTY"
    }
  ]
}

2. Building the Data Source and Visualization Layer

Once metrics are defined, the next step is constructing the dashboard view. You must separate real-time monitoring from historical analysis to avoid latency issues in decision-making. Real-time data in Genesys Cloud CX has a 30-second to 1-minute delay depending on the queue volume, which can distort drop-off rate calculations during peak traffic.

Architectural Reasoning:
Historical reporting is stored in the Data Warehouse and provides a complete picture of conversation states. Real-time reporting relies on streaming data from the interaction engine. For containment and escalation analysis, you should prioritize Historical Reporting because it captures the full lifecycle of the intent resolution. Real-time views are useful for immediate intervention during high drop-off events but lack the granularity to determine why a user dropped off without accessing raw logs.

The Trap:
A common misconfiguration is building dashboards that mix real-time and historical data sources on the same tab. This leads to confusion where a user sees 100% containment in real-time (because conversations are still active) but 40% containment historically (because sessions have closed). The dashboard must explicitly label the data source as Historical or Real-time.

Implementation Steps:

  1. Navigate to Analytics > Dashboards.
  2. Create a new Dashboard named Bot Performance Monitor.
  3. Add two distinct widgets:
    • Widget A (Containment): Use the Bot Containment Rate metric created in Step 1. Set the time range to Last 7 Days for trend analysis.
    • Widget B (Escalation Flow): Use a custom query on ccx.conversation.escalated. Filter by escalationReason = 'Bot Failure'.
  4. Configure the X-axis to display Intent Name or Flow Node ID. This allows you to see exactly which part of the conversation path is causing the most friction.

Validation Logic:
Ensure the ccx.conversation.contained metric counts are aligned with your definition of containment in the bot flow. If your bot has a “fallback” node that loops back to the start, does this count as contained or drop-off? You must configure your Reporting Filters to exclude looped conversations from containment calculations if they represent user frustration.

3. Calculating Drop-Off and Session Abandonment

Drop-off rates are the most critical indicator of bot usability but are often conflated with session timeouts. You must differentiate between a user who explicitly ends the chat (drop-off) and a user whose connection terminates due to network issues or system inactivity (timeout).

Architectural Reasoning:
In Genesys Cloud, ccx.conversation.abandoned typically refers to a user hanging up or closing the window before an agent connects. In conversational AI, this often includes users who say “Goodbye” but do not reach their goal. You need to isolate sessions where the conversation ends without a successful intent match and without escalation to a human. This requires analyzing the conversationState at the final node of the flow.

The Trap:
The most catastrophic error here is treating all abandoned conversations as bot failures. A user might drop off after successfully completing their task (e.g., checking a balance) because they have no further need for the channel. If you optimize to reduce this drop-off, you will artificially inflate bot usage metrics without solving actual user problems. You must filter drop-offs that occur after a ccx.conversation.successful state is recorded.

Implementation Steps:

  1. Create a Custom Metric named Bot Drop-Off Rate.
  2. Formula logic:
    (Sum(ccx.conversation.abandoned) - Sum(ccx.conversation.successful)) / Sum(ccx.conversation.interacted)
  3. Apply a filter to exclude sessions where the final intent was ccx.intent.fallback.
  4. In the Dashboard Widget, add a trend line for Time of Day to identify if drop-offs spike during specific hours, which could indicate staffing shortages in backend support teams that cause users to abandon the bot experience.

API Payload Reference:
To retrieve raw data for validation via API:

GET /api/v2/analytics/conversations/queries
Content-Type: application/json

{
  "filterCriteria": {
    "dateRange": {
      "startDate": "2023-10-01T00:00:00Z",
      "endDate": "2023-10-31T23:59:59Z"
    },
    "metricFilters": [
      {
        "metricName": "ccx.conversation.abandoned",
        "operator": "IS_NOT_NULL"
      }
    ],
    "conversationType": "DIGITAL"
  },
  "metrics": [
    "ccx.conversation.duration",
    "ccx.bot.id",
    "ccx.intent.name"
  ]
}

Validation, Edge Cases & Troubleshooting

Edge Case 1: Intent Ambiguity and the Fallback Loop

The Failure Condition:
Users enter a conversation where they do not understand the bot’s options. They repeatedly select fallback options without reaching an agent or resolving their issue. The analytics dashboard shows high containment because the user stayed in the session, but the operational reality is a failed interaction.

The Root Cause:
The ccx.conversation.contained metric counts any session that did not escalate to a human. If your bot is designed to loop indefinitely on fallback intents without ever triggering an escalation node, the system registers this as “contained” even though the user never achieved their goal.

The Solution:
Implement a “Max Fallback Count” logic in the conversational AI flow. Configure the bot to trigger an escalation if the ccx.intent.fallback count exceeds 3 within a single session. Update your reporting filter to exclude sessions where ccx.intent.fallback > 2 from the Containment calculation. This forces the analytics engine to recognize that looping is not containment, but rather a failed state masquerading as retention.

Edge Case 2: Cross-Channel Session Continuity

The Failure Condition:
A user starts on Chatbot, drops off, and resumes later via Voice. The analytics dashboard counts two separate interactions with potentially conflicting outcomes (e.g., “Contained” on Chat, “Escalated” on Voice).

The Root Cause:
Genesys Cloud treats Digital and Voice conversations as distinct entities in the Reporting API unless they are linked by a specific conversationId that persists across channels. Standard bot analytics do not automatically aggregate multi-channel journeys into a single “Customer Journey” view without custom data warehousing logic.

The Solution:
Use the Conversation ID to join datasets if utilizing an external BI tool. In native Genesys Dashboards, you must create separate widgets for each channel type (Digital vs Voice) and use the ccx.conversation.id field to correlate them manually during deep-dive troubleshooting. For high-level KPIs, accept that containment rates will be channel-specific until you implement a unified Customer Data Platform (CDP) layer.

Edge Case 3: API Rate Limiting in Real-Time Dashboards

The Failure Condition:
Real-time dashboards display stale data or return errors during peak traffic hours when the underlying API cannot keep up with polling requests.

The Root Cause:
Genesys Cloud Analytics APIs enforce strict rate limits on real-time endpoints (/api/v2/analytics/conversations). High-frequency dashboard refreshes (e.g., every 5 seconds) can trigger throttling responses, causing the dashboard to freeze or show outdated metrics.

The Solution:
Configure dashboard auto-refresh intervals to a minimum of 60 seconds for historical data and 30 seconds for real-time data. Avoid using the GET /api/v2/analytics/conversations/queries endpoint for live monitoring in favor of the optimized Reporting API endpoints designed for dashboards. Implement client-side caching for static configuration data (like intent names) so that only metric values are fetched dynamically during each refresh cycle.

Official References