Designing Agent Assist Feedback Loops Where Agents Rate and Improve Suggestion Quality

How-To-Articles
1

Designing Agent Assist Feedback Loops Where Agents Rate and Improve Suggestion Quality

What This Guide Covers

This guide details the architectural pattern for implementing a closed-loop feedback mechanism in Genesys Cloud CX where agents explicitly rate the relevance and accuracy of Agent Assist suggestions. The end result is a system that captures granular sentiment data per suggestion event, persists that data for model retraining or rule optimization, and dynamically suppresses low-performing suggestions to improve agent efficiency over time.

Prerequisites, Roles & Licensing

  • Licensing Tier: CX 2 or higher (Agent Assist is a CX 2 feature).
  • Roles & Permissions:
    • Integrations > Agent Assist > Edit (to configure the Assist Profile).
    • Integrations > Data Connectors > Edit (to configure the outbound webhook for feedback).
    • Admin > User > Edit (to assign agents to the Assist Profile).
  • External Dependencies:
    • A backend service or middleware capable of receiving HTTP POST requests (e.g., AWS Lambda, Azure Function, or a custom microservice).
    • A data store for logging feedback events (e.g., DynamoDB, PostgreSQL, or Elasticsearch).
  • OAuth Scopes: If using the API to push historical data or validate configurations, integration:agentassist:edit and integration:agentassist:view are required.

The Implementation Deep-Dive

1. Configuring the Agent Assist Profile for Feedback Capture

The foundation of any feedback loop is the ability to capture the agent’s intent regarding a specific suggestion. Genesys Cloud CX does not natively expose a “Good/Bad” button in the default Agent Assist UI. You must construct this interface using a combination of the Suggestion Content configuration and Data Connectors.

First, navigate to Integrations > Agent Assist and create or edit an existing Assist Profile. In the Triggers section, ensure you have defined the events that generate suggestions (e.g., Conversation Started, Topic Changed).

The critical architectural decision here is how you structure the suggestion payload. You cannot rely solely on the text of the suggestion. You must embed a unique identifier that links the suggestion to the specific knowledge base article, script step, or AI-generated insight.

The Trap: Using static identifiers or assuming the suggestionId provided by Genesys is persistent enough for long-term correlation. The suggestionId is scoped to the session. If you need to aggregate feedback across thousands of interactions to retrain a model, a session-scoped ID is useless. You must generate a global, unique ID (UUID v4) in your upstream knowledge source and pass it into the suggestion metadata.

To implement this, you must use a Data Connector to push the suggestion data to your backend when it is generated, or more efficiently, capture the feedback when the agent interacts with it. Since Genesys does not have a native “Rate” action, you simulate this by creating a custom UI overlay or using the Note feature coupled with a webhook trigger.

However, the most robust method for a “Rating” loop is to utilize the Agent Assist Integration to push suggestions that include clickable actions. If you are using a custom integration (via the Agent Assist API), you can define actions in the suggestion payload.

{
  "suggestionId": "kb-article-12345-uuid",
  "title": "Refund Policy for International Orders",
  "description": "Click here to view the full policy.",
  "actions": [
    {
      "type": "click",
      "label": "Relevant",
      "data": {
        "feedbackType": "positive",
        "suggestionRef": "kb-article-12345-uuid",
        "conversationId": "{{conversation.id}}"
      }
    },
    {
      "type": "click",
      "label": "Not Relevant",
      "data": {
        "feedbackType": "negative",
        "suggestionRef": "kb-article-12345-uuid",
        "conversationId": "{{conversation.id}}"
      }
    }
  ]
}

If you are using the native Knowledge integration rather than a custom API integration, you do not have direct control over the actions array in the same way. In this scenario, you must rely on the Data Connector to capture the Suggestion Viewed event and correlate it with subsequent agent actions.

Architectural Reasoning: We use a Data Connector to stream AgentAssistSuggestionViewed events because it decouples the feedback mechanism from the real-time conversation flow. This prevents latency issues in the UI. The agent clicks “Relevant,” the Data Connector sends the event to your middleware, and the middleware updates the score for kb-article-12345-uuid. The Genesys UI remains responsive.

2. Building the Feedback Ingestion Middleware

The Data Connector in Genesys Cloud will POST a JSON payload to your configured endpoint. You must design this endpoint to be idempotent and fast. The response time of this endpoint directly impacts the reliability of the Data Connector; if your endpoint times out, Genesys will retry, potentially creating duplicate feedback records.

The Trap: Synchronous processing of feedback. Do not attempt to update your database or retrain a model synchronously within the HTTP handler. The Data Connector expects a 200 OK response within milliseconds. Any heavy lifting must be offloaded to a message queue (e.g., SQS, Kafka).

Here is the expected JSON structure from the Genesys Cloud Data Connector for an Agent Assist event:

{
  "event": {
    "eventType": "agentassist.suggestion.viewed",
    "timestamp": "2023-10-27T14:30:00Z",
    "data": {
      "conversationId": "conv-123456",
      "agentId": "agent-789",
      "suggestionId": "kb-article-12345-uuid",
      "profileId": "assist-profile-001",
      "triggerName": "Topic Changed",
      "metadata": {
        "customFeedback": "positive" // This comes from your custom UI action or parsing
      }
    }
  }
}

Implementation Step:

  1. Create a Data Connector: Go to Integrations > Data Connectors. Create a new connector of type Outbound Webhook.
  2. Define the Trigger: Select Agent Assist as the source. Choose Suggestion Viewed and Suggestion Clicked as the events.
  3. Configure the Endpoint: Point to your middleware URL (e.g., https://api.yourcompany.com/genesys/feedback).
  4. Map the Payload: Ensure you map the suggestionId and agentId correctly. If you are using custom actions, you may need to parse the metadata field to extract the feedbackType.

Architectural Reasoning: We separate the ingestion layer from the processing layer. The ingestion layer (the HTTP endpoint) simply validates the JSON schema, acknowledges receipt, and pushes the message to a queue. This ensures that even if your database is down, the feedback events are not lost. Genesys Cloud’s Data Connector has a retry policy, but it is not infinite. You must guarantee delivery on your end.

3. Implementing Dynamic Suppression via API

The value of the feedback loop is realized when low-quality suggestions are suppressed. Genesys Cloud does not have a native “Block Suggestion” toggle based on agent feedback scores. You must implement this logic in your middleware and push the updated state back to Genesys or your upstream knowledge source.

There are two primary approaches to suppression:

  1. Upstream Filtering: If your suggestions come from a custom API integration, your API should maintain a score for each suggestion. When Genesys requests suggestions, your API checks the score. If the score is below a threshold (e.g., < 0.5), you omit the suggestion from the response.
  2. Knowledge Article Visibility: If you are using the native Knowledge integration, you can use the Genesys Cloud API to update the visibility or ranking of articles based on feedback.

The Trap: Suppressing suggestions globally based on aggregate feedback without considering context. A suggestion might be rated poorly in one department (e.g., Sales) but highly relevant in another (e.g., Support). You must segment feedback by Queue, Skill, or Language.

Implementation for Custom API Integrations:

Your middleware aggregates feedback. Let us say kb-article-12345-uuid has received 10 “Not Relevant” clicks from agents in the “Technical Support” queue.

# Pseudocode for Middleware Logic
def update_suggestion_score(suggestion_id, feedback_type, agent_queue):
    current_score = get_score(suggestion_id, agent_queue)
    if feedback_type == "negative":
        new_score = current_score - 0.1
    else:
        new_score = current_score + 0.1
    
    # Clamp score between 0 and 1
    new_score = max(0, min(1, new_score))
    
    save_score(suggestion_id, agent_queue, new_score)
    
    # If score drops below threshold, flag for suppression
    if new_score < 0.3:
        flag_for_suppression(suggestion_id, agent_queue)

When Genesys Cloud calls your Agent Assist API endpoint to fetch suggestions, your API checks the flag_for_suppression status. If flagged for the agent’s specific queue, the article is excluded from the suggestions array in the response.

Implementation for Native Knowledge Integrations:

If you are using native Knowledge, you cannot easily suppress individual articles in real-time via the Assist API. Instead, you use the Knowledge API to update the Article Rank or Visibility.

Endpoint: PUT /api/v2/knowledge/articles/{articleId}

Body:

{
  "rank": 100, // Lower rank = lower visibility
  "status": "published"
}

Architectural Reasoning: We prefer the Custom API approach for true real-time suppression because it allows for granular, context-aware filtering without modifying the underlying knowledge base structure. Modifying article ranks globally can have unintended side effects on search results for other users.

4. Closing the Loop with Agent Notifications

To reinforce the feedback behavior, agents should see that their input matters. You can achieve this by updating the Agent Assist UI dynamically or by sending a soft notification.

Since the Agent Assist UI is not fully customizable, the most effective method is to use Architect to detect when a suggestion is suppressed and inform the agent, or to provide a visual cue in the custom integration payload.

If using a custom integration, you can add a field to the suggestion payload:

{
  "suggestionId": "kb-article-12345-uuid",
  "title": "Refund Policy",
  "description": "This suggestion has been rated low by peers. Please verify before using.",
  "actions": [...]
}

This transparency builds trust in the AI system. Agents are more likely to provide feedback if they see that their previous feedback has led to visible changes.

The Trap: Over-notifying agents. Do not send a notification every time a suggestion is suppressed. This creates noise. Only notify when a significant change occurs (e.g., a previously high-ranking article is demoted).

Validation, Edge Cases & Troubleshooting

Edge Case 1: Feedback Spamming and Bias

The Failure Condition: A single agent, dissatisfied with a specific article, clicks “Not Relevant” 50 times in a row, causing the article to be suppressed for the entire queue.
The Root Cause: Lack of deduplication and rate-limiting in the feedback ingestion logic.
The Solution: Implement a deduplication key in your middleware. The key should be a composite of agentId, suggestionId, and conversationId. If the same agent rates the same suggestion multiple times in the same conversation, ignore subsequent ratings. Additionally, implement a weighting system where newer feedback has less impact than older, stable feedback, or use a Bayesian average to prevent small sample sizes from skewing results.

Edge Case 2: Data Connector Latency and Dropped Events

The Failure Condition: The middleware endpoint takes 5 seconds to respond, causing Genesys Cloud to drop the feedback event.
The Root Cause: Synchronous database writes or heavy processing in the HTTP handler.
The Solution: Ensure your endpoint responds with 200 OK immediately after accepting the request into a message queue. Use asynchronous workers to process the queue. Monitor the Data Connector health dashboard in Genesys Cloud for failed deliveries. Configure a dead-letter queue (DLQ) in your middleware to capture and retry failed events.

Edge Case 3: Contextual Mismatch in Feedback

The Failure Condition: An agent rates a suggestion as “Not Relevant” because the article was correct, but the agent misunderstood the customer’s issue. The system suppresses the article, harming other agents who would have benefited from it.
The Root Cause: Feedback is binary (Good/Bad) and lacks context.
The Solution: Introduce a third option: “Uncertain” or “Requires Review.” Or, implement a confidence score in the feedback payload. If the agent marks “Not Relevant,” prompt for a reason (e.g., “Outdated,” “Wrong Topic,” “Too Complex”). Use this metadata to categorize the failure. If the reason is “Outdated,” flag the article for content review rather than suppression. If the reason is “Wrong Topic,” suppress it for that specific trigger/queue combination.

Official References