Implementing Coaching Effectiveness Tracking by Correlating Sessions with Subsequent Scores

How-To-Articles
1

Implementing Coaching Effectiveness Tracking by Correlating Sessions with Subsequent Scores

What This Guide Covers

This guide details the architectural design and implementation of a closed-loop coaching analytics workflow in Genesys Cloud CX. You will configure the linkage between Quality Management (QM) evaluations and subsequent performance metrics to quantify the delta in agent scores following a coaching intervention. The end result is a data pipeline that attributes score improvements directly to specific coaching sessions, enabling leadership to validate the ROI of their quality initiatives.

Prerequisites, Roles & Licensing

Licensing Requirements

  • Genesys Cloud CX 2 or CX 3: Required for access to the Quality Management module and advanced reporting capabilities.
  • Workforce Engagement Management (WEM): Optional but recommended if you intend to correlate coaching outcomes with adherence or shrinkage data in a unified dashboard.
  • Speech Analytics: Optional, but often used to trigger the initial coaching session based on negative sentiment detection.

Permissions & Roles

  • User Role: Administrator or a custom role with the following permissions:
    • Quality > Evaluation > Edit
    • Quality > Form > Edit
    • Reporting > Report > Create
    • Reporting > Report > View
    • Analytics > Data Warehouse > Export (if pushing data to an external BI tool)
  • API Access: OAuth client with scopes quality:evaluation:read, reporting:report:read, and analytics:export:read.

External Dependencies

  • Data Warehouse: For long-term historical analysis beyond the native 13-month retention in Genesys Cloud reports.
  • BI Tool (e.g., Tableau, PowerBI): For visualizing the correlation between coaching frequency and score improvement.

The Implementation Deep-Dive

1. Structuring the Quality Management Form for Traceability

The foundation of effective coaching tracking is the structure of the Quality Management (QM) form. Standard forms often treat coaching as a binary flag (Coached: Yes/No). This approach fails because it does not capture the content or timing of the coaching, making correlation impossible.

You must redesign the QM form to include specific data points that allow for temporal and thematic correlation.

Configuration Steps

  1. Navigate to Admin > Quality > Forms.
  2. Open the target evaluation form.
  3. Add the following custom fields to the form section associated with the call/chat interaction:
    • Coaching Date (Date Picker): Mandatory if Coaching Provided is true. This captures the exact timestamp of the intervention.
    • Coaching Topic (Dropdown): Pre-populated with common coaching areas (e.g., “Soft Skills,” “Product Knowledge,” “Compliance”). This allows you to correlate specific topics with score improvements in those categories.
    • Coaching Duration (Number): To analyze if longer sessions yield better results.
    • Target Score Delta (Number): The agent and coach agree on a target improvement percentage for the next review period.

The Trap: Storing coaching notes in a free-text field.
Free text is unstructured data. You cannot query “Show me all agents who received coaching on Compliance and improved by >5%.” You must use structured fields (Dropdowns, Picklists) for the topic and outcome to enable reporting. Free text should only be used for qualitative notes.

Architectural Reasoning

By embedding structured metadata directly into the evaluation record, you create a primary key for the coaching event. This record becomes the “before” state in your correlation model. Without this explicit timestamp and topic tagging, you are forced to rely on heuristic guesses (e.g., “The agent was coached sometime last month”) which introduces significant noise into your analytics.

2. Configuring the Evaluation Workflow for Immediate Feedback

Coaching effectiveness degrades exponentially with time. If an agent is evaluated on Tuesday and coached on Friday, the behavioral reinforcement is weakened. You must configure the workflow to encourage immediate or near-immediate coaching linkage.

Configuration Steps

  1. Navigate to Admin > Quality > Workflows.
  2. Create a new workflow or edit the existing one.
  3. Add a Condition node:
    • Check if Score < Threshold (e.g., 80%).
    • OR Check if Critical Error = True.
  4. Add an Action node:
    • Send Email: Notify the Team Leader and Agent.
    • Create Task: Automatically create a Task in the Team Leader’s queue with the subject “Coaching Required: [Agent Name] - [Topic].”
  5. Link the Task to the Evaluation ID.

The Trap: Relying solely on manual coaching initiation.
If you leave coaching initiation to the discretion of individual Team Leaders, you will experience inconsistent data entry. Some leaders will log coaching; others will not. By automating the task creation for low scores, you enforce a consistent data capture mechanism. The act of completing the Task should be tied to updating the Coaching Date and Coaching Topic fields in the original evaluation record.

Architectural Reasoning

This workflow ensures that every low-scoring evaluation generates a traceable coaching event. The Task serves as the bridge between the diagnostic (the evaluation) and the intervention (the coaching). When the Task is completed, the system records the intervention timestamp, allowing you to measure the delta between Evaluation_Date and Coaching_Date, and subsequently, Coaching_Date and Next_Evaluation_Date.

3. Building the Correlation Report in Genesys Cloud Reporting

Genesys Cloud’s native reporting engine is powerful but requires precise metric selection to avoid data skew. You will build a report that joins Evaluation data with subsequent Evaluation data for the same agent.

Configuration Steps

  1. Navigate to Analytics > Reports > Create Report.
  2. Select Quality as the data source.
  3. Add the following metrics:
    • Evaluation Score
    • Coaching Date
    • Coaching Topic
    • Agent ID
    • Evaluation Date
  4. Add a Calculated Field named Days Since Coaching:
    • Formula: DATEDIFF(Evaluation Date, Coaching Date, 'day')
    • Note: This calculates how many days after coaching the current evaluation occurred.
  5. Add a Calculated Field named Score Delta:
    • This is complex in native reporting. Native reporting does not easily allow self-joins on the same data source to compare Row N with Row N+1.
    • Workaround: Use the Advanced Reporting feature or export to Data Warehouse. For native reporting, you will create a segmented view:
      • Segment A: Evaluations where Coaching Date is within the last 30 days.
      • Segment B: Evaluations where Coaching Date is NULL.
    • Compare the average Evaluation Score of Segment A against the agent’s historical average.

The Trap: Comparing coached evaluations to the population average.
Do not compare the score of a coached agent to the average score of all agents. This is a statistical fallacy. You must compare the agent’s score after coaching to their own score before coaching (or their rolling average). The native report builder struggles with this self-referential comparison.

Architectural Reasoning

Native reporting in Genesys Cloud is row-based. It does not inherently understand “previous” or “next” records for the same entity. To accurately track effectiveness, you need a longitudinal view. While native reports can show “Average Score of Agents who were coached in the last 30 days,” this is a cohort analysis, not an individual effectiveness analysis. For true individual tracking, you must use the Data Warehouse API.

4. Implementing the Longitudinal Analysis via Data Warehouse API

To achieve masterclass-level tracking, you must bypass the limitations of the UI and use the Data Warehouse to construct a time-series analysis. This allows you to plot the trajectory of an agent’s score relative to coaching events.

API Payload Construction

You will use the POST /api/v2/analytics/quality/evaluations/queries endpoint to retrieve granular evaluation data.

POST https://{{org}}.mypurecloud.com/api/v2/analytics/quality/evalusions/queries

{
  "dateFrom": "2023-01-01T00:00:00Z",
  "dateTo": "2023-12-31T23:59:59Z",
  "groupBy": ["agentId", "evaluationDate"],
  "metrics": [
    "evaluationScore",
    "evaluationDate",
    "coachingDate",
    "coachingTopic"
  ],
  "filter": {
    "type": "AND",
    "predicates": [
      {
        "type": "EQUAL",
        "dimension": "status",
        "value": "APPROVED"
      }
    ]
  },
  "limit": 1000
}

The Trap: Ignoring the status filter.
Always filter for APPROVED evaluations. Draft or Pending evaluations do not reflect final scores and will skew your delta calculations. Additionally, ensure you handle coachingDate null values correctly in your downstream processing; a null coaching date means the evaluation was not part of a coaching loop.

Downstream Processing Logic

  1. Ingest: Pull the JSON response into your data pipeline (e.g., Azure Data Factory, AWS Lambda).
  2. Sort: Sort records by agentId and evaluationDate ascending.
  3. Window Function: Use a SQL-like window function to identify the “Previous Score” for each agent.
    SELECT 
    agentId,
    evaluationDate,
    evaluationScore,
    coachingDate,
    coachingTopic,
    LAG(evaluationScore) OVER (PARTITION BY agentId ORDER BY evaluationDate) AS previousScore,
    evaluationScore - LAG(evaluationScore) OVER (PARTITION BY agentId ORDER BY evaluationDate) AS scoreDelta
FROM evaluations
  4. Filter for Coaching Impact: Select only records where coachingDate is NOT NULL AND evaluationDate > coachingDate.
  5. Calculate Effectiveness:
    • Positive Impact: scoreDelta > 0
    • No Impact: scoreDelta = 0
    • Negative Impact: scoreDelta < 0

Architectural Reasoning

This approach decouples the data storage from the analysis. Genesys Cloud stores the raw events; your data warehouse constructs the narrative. This allows you to apply complex statistical models (e.g., regression analysis to control for seasonality or product launch effects) that are impossible in the native UI. It also ensures data persistence beyond the Genesys Cloud retention policy.

Validation, Edge Cases & Troubleshooting

Edge Case 1: The “Hawthorne Effect” Skew

The Failure Condition: Agents show immediate score improvements after coaching, but the improvement plateaus or reverses after 3-6 months, regardless of further coaching.

The Root Cause: The Hawthorne Effect occurs when individuals modify an aspect of their behavior in response to their awareness of being observed. The initial coaching session acts as a “reset” button, causing a temporary spike in compliance and attention. This is not a sustainable improvement driven by skill acquisition.

The Solution:

  1. Time-Decay Weighting: In your analysis, weight the scoreDelta by the time elapsed since coaching. A 5% improvement at 1 week is less valuable than a 5% improvement at 12 weeks.
  2. Repeat Coaching Tracking: Track agents who require repeated coaching on the same Coaching Topic. If an agent receives coaching on “Soft Skills” three times in a quarter with diminishing returns, flag this for a different intervention (e.g., role change or advanced training) rather than standard coaching.

Edge Case 2: Data Latency and Approval Delays

The Failure Condition: The Coaching Date is recorded, but the subsequent Evaluation Score is not visible in reports for several weeks due to slow calibration and approval processes.

The Root Cause: Quality Management workflows often involve multiple steps: Evaluation → Calibration → Team Leader Review → QA Manager Approval. If the approval chain is slow, the “After” data point is delayed, breaking the temporal correlation.

The Solution:

  1. SLA Monitoring: Implement a monitoring alert for evaluations stuck in “PENDING” status for >48 hours.
  2. Proxy Metrics: If evaluation data is delayed, use interim proxy metrics such as Customer Satisfaction (CSAT) or First Contact Resolution (FCR) for the same time window. While not identical to QM scores, these metrics often update in near real-time and can provide an early signal of coaching effectiveness.

Edge Case 3: Topic Misalignment

The Failure Condition: The report shows a negative scoreDelta after coaching, leading to the conclusion that coaching was ineffective.

The Root Cause: The coaching topic did not align with the weighted sections of the evaluation form. For example, an agent is coached on “Product Knowledge” (10% of the form), but their overall score drops because they failed “Compliance” (40% of the form) which was not addressed in the coaching.

The Solution:

  1. Section-Level Delta: Do not rely solely on the overall Evaluation Score. Calculate deltas for each section of the form.
  2. Topic-Section Mapping: Map Coaching Topic dropdown values to specific QM Form sections.
    • Example: Coaching Topic = “Compliance” maps to Section = “Regulatory Adherence”.
  3. Targeted Reporting: Generate reports that compare the delta only in the mapped section. If the “Compliance” section score improves after “Compliance” coaching, the coaching was effective, even if the overall score dropped due to other factors.

Official References