Designing Agent Assist Analytics for Measuring Suggestion Acceptance Rates and Time Savings

How-To-Articles
1

Designing Agent Assist Analytics for Measuring Suggestion Acceptance Rates and Time Savings

What This Guide Covers

This guide details the architectural pattern for capturing, normalizing, and analyzing Agent Assist telemetry to derive actionable metrics on suggestion acceptance rates and calculated time savings. You will implement a robust data pipeline that moves raw interaction events from Genesys Cloud CX into a warehouse environment, enabling precise attribution of productivity gains to specific knowledge articles.

Prerequisites, Roles & Licensing

  • Licensing Tier: Genesys Cloud CX 1 or higher with Agent Assist add-on enabled. For advanced predictive analytics, CX 3 is recommended.
  • Permissions:
    • Analytics > Report > Read
    • Data > Data Export > Read
    • Administration > Configuration > Edit (for setting up custom attributes)
  • External Dependencies:
    • A data warehouse or BI tool (e.g., Snowflake, BigQuery, Tableau) capable of ingesting JSON payloads via the Genesys Cloud Data Export API.
    • Access to the Genesys Cloud Developer Portal for configuring API keys or OAuth clients.

The Implementation Deep-Dive

1. Architecting the Telemetry Capture Layer

The fundamental challenge in Agent Assist analytics is that the platform does not provide a single “Acceptance Rate” report out of the box. The system treats Agent Assist interactions as a stream of discrete events within an interaction transcript. To measure acceptance, you must reconstruct the user journey by correlating the Suggestion Displayed event with the Suggestion Accepted event.

Configuring Custom Attributes for Contextual Enrichment

Before relying on default event fields, you must ensure that every suggestion carries the metadata required for downstream analysis. If you are using a custom knowledge base connector, you must map the article_id and category fields in the Agent Assist configuration.

  1. Navigate to Admin > Analytics > Agent Assist.
  2. Select your specific Knowledge Source.
  3. In the Mapping tab, ensure the following fields are exposed in the suggestion payload:
    • articleId: The unique identifier of the knowledge base article.
    • title: The human-readable title of the article.
    • confidenceScore: The algorithmic confidence level assigned by the search engine.

The Trap: Relying solely on the title field for deduplication. Titles often change, are duplicated across different categories, or contain dynamic variables. If you aggregate analytics by title, your acceptance rate will be artificially deflated because the same logical article is treated as multiple distinct entities. Always aggregate by articleId or a stable knowledgeSourceId.

Understanding the Event Schema

Genesys Cloud emits two critical event types within the interaction export stream that pertain to Agent Assist. You must filter your data pipeline for these specific event types:

  1. agentAssistSuggestion: Emitted when a suggestion appears in the agent’s UI.
  2. agentAssistAction: Emitted when the agent interacts with the suggestion (click, accept, dismiss).

The payload for an agentAssistAction event typically looks like this:

{
  "type": "agentAssistAction",
  "timestamp": "2023-10-27T14:30:00.000Z",
  "interactionId": "i-12345678-1234-1234-1234-123456789012",
  "participantId": "p-87654321-4321-4321-4321-210987654321",
  "agentAssistSuggestionId": "s-11111111-1111-1111-1111-111111111111",
  "action": "accepted", 
  "metadata": {
    "articleId": "KB-ART-998877",
    "articleTitle": "How to reset a user password",
    "knowledgeSourceId": "ks-custom-wiki-01"
  }
}

Architectural Reasoning: We capture the agentAssistSuggestionId in the initial display event and match it against the agentAssistAction event. This linkage is critical. Without this correlation key, you cannot determine if an agent saw a suggestion but ignored it, or if they accepted a different suggestion later in the call.

2. Normalizing the Data Stream for Analysis

Raw data from the Genesys Cloud Data Export API is verbose and denormalized. To calculate accurate metrics, you must perform a two-pass normalization strategy in your data warehouse.

Step 2.1: Isolating Suggestion Impressions

First, extract all agentAssistSuggestion events. Create a table or view that maps interactionId + agentAssistSuggestionId to articleId and timestamp. This represents your “Impressions” denominator.

CREATE VIEW v_agent_assist_impressions AS
SELECT 
    i.interactionId,
    e.agentAssistSuggestionId,
    e.timestamp AS impressionTime,
    e.metadata.articleId,
    e.metadata.knowledgeSourceId
FROM 
    interactions i
JOIN 
    events e ON i.id = e.interactionId
WHERE 
    e.type = 'agentAssistSuggestion';

Step 2.2: Isolating Suggestion Actions

Next, extract all agentAssistAction events where action = 'accepted'. This represents your “Acceptances” numerator.

CREATE VIEW v_agent_assist_acceptances AS
SELECT 
    i.interactionId,
    e.agentAssistSuggestionId,
    e.timestamp AS acceptanceTime,
    e.metadata.articleId
FROM 
    interactions i
JOIN 
    events e ON i.id = e.interactionId
WHERE 
    e.type = 'agentAssistAction' 
    AND e.action = 'accepted';

The Trap: Counting “Dismissed” actions as negative data points in the acceptance rate calculation. A dismissal is not a rejection of the article’s utility; it is often a result of UI noise or timing. Including dismissals in the denominator skews the metric toward zero, making the feature appear ineffective. The standard industry metric for Suggestion Acceptance Rate (SAR) is:

$$ SAR = \frac{\text{Total Accepted Suggestions}}{\text{Total Unique Suggestions Displayed}} $$

Do not include dismissals in the denominator. Only count unique agentAssistSuggestionId values that were displayed.

3. Calculating Time Savings: The Attribution Model

Measuring time savings is significantly more complex than measuring acceptance. Genesys Cloud does not explicitly tag “time saved” in the event stream. You must infer time savings using a Baseline Comparison Model.

The Baseline Model

You must establish a baseline handling time for specific intent categories before Agent Assist was enabled, or use a control group of agents who do not have Agent Assist enabled for specific intents.

  1. Identify Intents: Use Genesys Cloud’s Intent Classification or NLP annotations to tag interactions with specific business intents (e.g., “Password Reset”, “Billing Inquiry”).
  2. Calculate Baseline AHT: Determine the average Handling Time (AHT) for these intents in the pre-Agent Assist era.
  3. Calculate Current AHT: Determine the current AHT for the same intents, filtered for interactions where at least one suggestion was accepted.

The Calculation Logic

The time saved per interaction is the difference between the baseline AHT and the current AHT, multiplied by the number of interactions where suggestions were accepted.

def calculate_time_savings(current_aht, baseline_aht, interaction_count):
    """
    Calculates total time saved in seconds.
    """
    time_saved_per_interaction = baseline_aht - current_aht
    if time_saved_per_interaction < 0:
        return 0 # No savings, potentially increased friction
    return time_saved_per_interaction * interaction_count

Architectural Reasoning: We use a Python-based transformation layer (or equivalent SQL logic) to perform this calculation because it requires joining multiple data sources: the Agent Assist event stream, the Interaction transcript metadata (for intent classification), and the historical AHT baseline table. Doing this in real-time within Genesys Cloud is not feasible due to the latency of the Data Export API and the complexity of the join operations.

The Trap: Attributing time savings to all interactions where a suggestion was displayed. If a suggestion is displayed but not accepted, the agent may have still spent time reading it, potentially increasing AHT. You must strictly filter for action = 'accepted' when calculating the “Agent Assist Enabled” AHT. Otherwise, you will dilute the metric with noise, leading to an underestimation of ROI.

4. Building the Analytics Dashboard

Once the data is normalized and calculated, you must present it in a way that drives behavior change. The dashboard should not just show numbers; it should show opportunity.

Key Metrics to Display

  1. Suggestion Acceptance Rate (SAR): By Knowledge Source and by Article.
  2. Top 10 Accepted Articles: To identify which content is most valuable.
  3. Top 10 Ignored Articles: To identify content that is irrelevant, poorly timed, or confusing.
  4. Estimated Time Savings: Aggregated by week, compared against the baseline.

Visualizing the “Ignored” Pipeline

Create a specific report for High-Volume, Low-Acceptance articles. These are articles that are frequently displayed but rarely accepted. This is the primary signal for content optimization.

SELECT 
    articleId,
    articleTitle,
    COUNT(DISTINCT impression.agentAssistSuggestionId) AS impressions,
    COUNT(DISTINCT acceptance.agentAssistSuggestionId) AS acceptances,
    (COUNT(DISTINCT acceptance.agentAssistSuggestionId) * 100.0 / COUNT(DISTINCT impression.agentAssistSuggestionId)) AS acceptance_rate
FROM 
    v_agent_assist_impressions impression
LEFT JOIN 
    v_agent_assist_acceptances acceptance 
    ON impression.agentAssistSuggestionId = acceptance.agentAssistSuggestionId
GROUP BY 
    articleId, articleTitle
ORDER BY 
    impressions DESC, acceptance_rate ASC
LIMIT 20;

The Trap: Ignoring the Confidence Score in the analysis. If an article has a low acceptance rate, check the average confidenceScore at the time of display. If the confidence score is low (e.g., < 0.6), the issue is not the content, but the triggering logic. The suggestion is appearing for the wrong intents. If the confidence score is high but acceptance is low, the issue is the content quality or UI placement.

Validation, Edge Cases & Troubleshooting

Edge Case 1: The “Rapid Fire” Suggestion Flood

The Failure Condition: Agents report being overwhelmed by suggestions appearing too quickly, leading to a drop in acceptance rates.
The Root Cause: The Agent Assist engine is configured with a low debounce time or high sensitivity threshold. It is emitting multiple suggestions for the same intent within seconds.
The Solution: Adjust the Debounce Interval in the Agent Assist configuration. Set a minimum interval of 5-10 seconds between suggestions for the same intent. Additionally, implement a Max Suggestions Per Interaction limit (e.g., 5) to prevent UI clutter.

Edge Case 2: The “Ghost” Acceptance

The Failure Condition: The analytics show high acceptance rates, but agents report no time savings.
The Root Cause: Agents are clicking “Accept” to dismiss the suggestion rather than actually using the content. This is a UX design flaw where the “Accept” button is the primary dismissal mechanism.
The Solution: Review the Agent Assist UI configuration. Ensure there is a distinct “Dismiss” or “Close” action that does not register as an “Acceptance.” In Genesys Cloud, this is controlled by the Action Mapping in the Knowledge Source configuration. Map the “Close” UI element to action: dismissed and the “Insert into Chat” or “Copy to Clipboard” element to action: accepted.

Edge Case 3: Cross-Channel Inconsistency

The Failure Condition: Acceptance rates are high in Chat but low in Voice.
The Root Cause: Voice interactions require agents to read suggestions aloud or paraphrase them, which is cognitively more expensive than copying text in Chat. The suggestions may be too long or complex for voice delivery.
The Solution: Implement Channel-Specific Content Templates. Create shorter, conversational snippets for Voice channels and detailed, step-by-step guides for Chat/Digital channels. Use the channel attribute in the Agent Assist filtering rules to serve the appropriate template.

Official References