Designing Peer Evaluation Programs With Anonymized Interaction Reviews

How-To-Articles
1

Designing Peer Evaluation Programs With Anonymized Interaction Reviews

What This Guide Covers

This guide details the architectural implementation of a peer evaluation workflow within Genesys Cloud CX. You will configure a closed-loop quality management process where agents review anonymized interaction recordings from their peers, submit structured feedback, and have that data aggregated into a dashboard without exposing PII or violating privacy boundaries. The end result is a scalable, privacy-compliant training mechanism that leverages agent bandwidth to increase evaluation coverage while maintaining strict data isolation.

Prerequisites, Roles & Licensing

Licensing Requirements

  • CX 1 or higher: Required for basic Quality Management (QM) capabilities.
  • CX 2 or higher: Recommended to unlock advanced workflow automation and deeper integration with WEM (Workforce Engagement Management) for scheduling peer review tasks.
  • WEM Add-on: Required if you intend to push peer review tasks directly into the agent desktop as “Wrap-up” or “After-Call Work” items.

Permission Requirements

  • Quality > Form > Edit: To create the specific evaluation form for peers.
  • Quality > Form > View: For agents to view the form structure (if allowed).
  • Interaction > Recording > View: Agents require explicit permission to view recordings. Note that standard agents do not have this by default. You must create a custom role or modify an existing role (e.g., Agent) to include Interaction > Recording > View.
  • User > User > View: Required to assign the evaluation to the correct agent if using API-driven assignment, though the QM workflow handles this internally.
  • Dashboard > Dashboard > View: To view the aggregated peer evaluation metrics.

External Dependencies

  • Data Masking Strategy: You must define which fields constitute PII. This guide assumes you are using Genesys Cloud’s native redaction features or a downstream masking service.
  • Agent Grouping: Agents must be segmented into logical groups (e.g., by team, language, or product line) to ensure peer reviews are relevant and fair.

The Implementation Deep-Dive

1. Architecting the Anonymization Layer

The foundational constraint of this program is privacy. You cannot simply grant agents access to raw recordings. The trap here is assuming that hiding the caller ID in the UI is sufficient. Agents can still hear names, credit card numbers, or internal employee identifiers in the audio.

The Trap: Granting Interaction > Recording > View to agents without enforcing strict redaction policies. If an agent hears a colleague’s name or a customer’s SSN during a peer review, you have violated HIPAA/PCI-DSS and internal privacy policies. Furthermore, if the recording metadata is exposed, agents can identify whose call it is, leading to bias or social friction.

The Solution: Implement a dual-layer anonymization strategy.

Layer A: Audio Redaction
You must configure a Speech Analytics rule to detect and redact PII in real-time or near-real-time.

  1. Navigate to Admin > Speech Analytics > Rules.
  2. Create a new rule targeting sensitive data patterns (Credit Card, SSN, Name).
  3. Set the action to Redact.
  4. Apply this rule to all queues involved in the peer review program.
  5. Ensure the Redaction setting is set to “Bleep” or “Silence” to prevent agents from guessing the redacted content.

Layer B: Metadata Masking via Custom Attributes
Genesys Cloud allows you to mask metadata. However, the QM interface inherently shows the Agent Name and Date/Time. To anonymize the source agent, you must rely on the Evaluation Form logic and Dashboard aggregation, not the raw recording player.

Architectural Decision: Do not attempt to hide the agent name in the recording player. Instead, design the workflow so that the agent reviewing the call does not see the “Recordings” view directly. They only see the Evaluation Form triggered by a workflow. The workflow selects a random recording from a pool. The form does not display the Agent Name field.

2. Constructing the Peer Evaluation Form

You need a distinct Quality Form for peer reviews. Do not reuse the QA form used by managers. The criteria, weightings, and confidentiality requirements differ.

The Trap: Using the same form for QA and Peer Review. QA forms often include compliance checks that require legal authority. Peer reviews should focus on soft skills, empathy, and process adherence. Mixing these creates noise in your QA scores and exposes agents to liability.

Configuration Steps:

  1. Navigate to Admin > Quality > Forms.
  2. Click Add Form.
  3. Name: Peer_Review_Internal.
  4. Set Form Type to Quality.
  5. Crucial Step: In the General tab, ensure “Allow agents to view this form” is unchecked if you are using a workflow to trigger it, or checked if you are using the “Evaluate” button in the desktop. For this architecture, we recommend unchecked and using a workflow to push the evaluation. This prevents agents from browsing the form library.
  6. Fields:
    • Add a Rating field: “Did the agent follow the opening protocol?” (Scale 1-5).
    • Add a Comment field: “What one thing could the agent improve?”
    • Exclude: Do not add any field that asks “Who was the agent?” or “Who was the customer?”.
  7. Scoring: Set the pass/fail threshold. For peer reviews, a score of 0 might mean “Not Applicable” or “Skipped”. Decide on a binary pass/fail or a weighted score. For simplicity, let us use a weighted score out of 100.

The Trap: Allowing agents to submit evaluations with no comments. This leads to “rubber stamping” where agents give everyone a perfect score to avoid conflict.

Mitigation: Make the Comment field Required. In the form designer, set the comment field to “Required”. The form will not submit if the comment is empty. This forces cognitive engagement.

3. Building the Anonymized Assignment Workflow

This is the core engine. You need a workflow that periodically selects a random, compliant recording and assigns it to an agent for review.

The Trap: Assigning recordings sequentially. If Agent A always reviews Agent B’s calls, it creates a bias loop. You must introduce randomness.

Workflow Design:

  1. Navigate to Admin > Flows > Architect.
  2. Create a new Workflow.
  3. Trigger: Set to Schedule. Configure it to run every 15 minutes.
  4. Data Source: Use a Query node.
    • Object: Interaction.
    • Filter: Type equals Voice.
    • Filter: Status equals Completed.
    • Filter: Date is within the last 24 hours.
    • Filter: Queue ID is in [Your Target Queues].
    • Filter: Has Recording equals True.
    • Filter: Is Redacted equals True (Ensure only redacted calls are eligible).
    • Sort: Random.
    • Limit: 1.
  5. Decision Node: Check if a record was returned. If no, end workflow.
  6. Query Node (Reviewer Selection):
    • Object: User.
    • Filter: Type equals Agent.
    • Filter: Group ID is in [Peer Review Group].
    • Filter: Status equals Available.
    • Sort: Random.
    • Limit: 1.
  7. Action Node: Create Quality Evaluation.
    • Interaction ID: From the first query.
    • Evaluator ID: From the second query.
    • Form ID: The ID of Peer_Review_Internal.
    • Score: Leave blank (to be filled by the agent).
    • Comments: Leave blank.
    • Status: Pending.

Architectural Reasoning: By using two separate queries with Random sorting, you decouple the source agent from the reviewer. The workflow does not pass the Agent ID of the source call to the reviewer’s UI. The reviewer only sees the evaluation task.

The Trap: Assigning evaluations to agents who are currently on a call. This causes distraction and context switching penalties.

Mitigation: The workflow filters for Status equals Available. However, “Available” in Genesys can mean “Idle” or “Ready”. Ensure your Agent Desktop configuration maps “Available” correctly. For higher precision, use the WEM integration to schedule the task as “After-Call Work” (ACW) when the agent wraps up a call. This leverages the natural downtime.

4. Configuring the Agent Desktop Experience

The agent must have a seamless way to access the pending evaluation.

The Trap: Forcing agents to navigate to the Admin portal or the Quality tab to find their tasks. This adds friction and reduces compliance.

Solution: Use the Work Item approach.

  1. In the Quality Evaluation action in the workflow, ensure the evaluation is created.
  2. Genesys Cloud automatically creates a Work Item for pending evaluations if the agent has the correct permissions.
  3. Configure the Agent Desktop:
    • Go to Admin > User Experience > Desktop.
    • Ensure the Quality app is enabled.
    • In the Quality app configuration, ensure the “Pending Evaluations” list is visible.
  4. Anonymization in UI:
    • When the agent opens the evaluation, the Interaction tab shows the recording.
    • The Participant tab shows the agent’s name. This is the vulnerability.
    • Workaround: You cannot easily hide the participant name in the native QM UI for the evaluator. However, you can mitigate this by:
      • Training agents to ignore the name.
      • Using a Custom Form Field to ask “Did you recognize the agent?” (No, this doesn’t help).
      • Better Approach: Use the API to create the evaluation and store the Agent ID in a hidden custom attribute on the evaluation, but do not expose it in the form. The native UI will still show it.
      • Critical Mitigation: If strict anonymity is required, you must not use the native QM UI for the review. Instead, build a Custom Widget using the Genesys Cloud Web Messaging SDK or a custom HTML5 app that fetches the recording URL via API and presents it without metadata. This is complex but necessary for true anonymity.
      • Pragmatic Compromise: For most enterprises, “Blind” review is sufficient. The agent sees the name but is instructed not to use it. The workflow ensures they do not know who they are reviewing until they open it, and the randomness prevents targeted bias.

5. Aggregating Results Without PII

The value of peer reviews is in the aggregate trends, not individual scores.

The Trap: Publishing individual peer review scores to agents. This leads to gaming the system and social retaliation.

Solution: Create a Dashboard that aggregates scores by Queue or Team, not by Agent.

  1. Navigate to Admin > Analytics > Dashboards.
  2. Create a new Dashboard.
  3. Add a Quality widget.
  4. Set Form to Peer_Review_Internal.
  5. Set Group By to Queue or Group.
  6. Set Metric to Average Score.
  7. Exclude: Do not add a widget that groups by Evaluator or Evaluated Agent.
  8. Share this dashboard with Managers and Team Leads, not individual agents.

Advanced Analytics: Use Speech Analytics to correlate peer review scores with call outcomes. For example, do calls with high peer review scores have higher CSAT? This validates the relevance of the peer review criteria.

Validation, Edge Cases & Troubleshooting

Edge Case 1: The “Friendly Fire” Bias

The Failure Condition: Agents consistently give each other perfect scores to maintain social harmony.
The Root Cause: Lack of accountability and clear rubrics.
The Solution:

  • Introduce a Calibration Session. Have managers review a subset of peer evaluations alongside the agents.
  • Add a Confidence Score field to the form. “How confident are you in this evaluation?” (Low/Med/High). Low confidence scores are discarded in the aggregate.
  • Use the API to detect outliers. If an agent’s average score deviates >2 standard deviations from the team average, flag the evaluator for review.

Edge Case 2: Recording Redaction Failure

The Failure Condition: An agent hears PII during a peer review.
The Root Cause: The Speech Analytics rule failed to catch a variant of PII (e.g., a new product code or internal slang).
The Solution:

  • Implement a Feedback Loop. Add a button in the Agent Desktop (via Custom Widget) that allows the reviewer to flag “PII Detected”.
  • This flag triggers an alert to the QA team to review the recording and update the Speech Analytics rule.
  • Automatically invalidate the evaluation score if a PII flag is raised.

Edge Case 3: Workflow Deadlock

The Failure Condition: The workflow fails to assign evaluations because no agents are “Available”.
The Root Cause: High call volume leaves no idle agents.
The Solution:

  • Change the workflow trigger to Event-Based. Trigger on Agent Status Change to Available.
  • When an agent becomes available, the workflow checks for pending unassigned evaluations and assigns one.
  • Alternatively, use WEM to schedule the task as “ACW”. This ensures the task is assigned when the agent has just finished a call and is naturally transitioning.

Official References