Designing Outbound Campaign A/B Testing Frameworks for Script and Timing Optimization

How-To-Articles
1

Designing Outbound Campaign A/B Testing Frameworks for Script and Timing Optimization

What This Guide Covers

You are implementing an A/B testing framework for Genesys Cloud Outbound campaigns to scientifically determine which strategies yield the highest conversion rates. Instead of guessing whether a “Direct” or “Inquisitive” script works better, or whether calling on Tuesday morning is better than Thursday afternoon, you will use Contact List Splitting and Campaign Partitioning to run controlled experiments. When complete, you will have a data-driven dashboard that shows statistically significant differences in Hit Rate, Conversion Rate, and Revenue per Lead between your “Control” and “Test” groups.

Prerequisites, Roles & Licensing

  • Genesys Cloud: CX 1, 2, or 3 with Outbound.
  • Permissions:
    • Outbound > Contact List > Create/Edit
    • Outbound > Campaign > Create/Edit
  • Mathematics: Basic understanding of statistical significance and p-values.

The Implementation Deep-Dive

1. The Experiment Design

A proper A/B test requires two identical campaigns running against two randomized, mutually exclusive subsets of the same data.

The Split Logic:

  • Group A (Control): 50% of the list. Standard script, standard timing.
  • Group B (Test): 50% of the list. Modified script (or different calling window).

2. Randomizing the Contact List

Do not split the list alphabetically (A-M / N-Z) as this can introduce bias (e.g., ethnic or geographic bias). Use a random hash of the Phone Number or External ID.

Python Split Script:

import pandas as pd
import hashlib

def split_list(csv_file):
    df = pd.read_csv(csv_file)
    
    # Create a hash for each record to ensure random but stable splitting
    df['hash'] = df['phone_number'].apply(lambda x: int(hashlib.md5(str(x).encode()).hexdigest(), 16) % 100)
    
    # Assign to Group A or B
    df['group'] = df['hash'].apply(lambda x: 'A' if x < 50 else 'B')
    
    df[df['group'] == 'A'].to_csv('list_a_control.csv', index=False)
    df[df['group'] == 'B'].to_csv('list_b_test.csv', index=False)

split_list('master_leads.csv')

3. Campaign Configuration

  1. Upload the two separate contact lists.
  2. Create two identical campaigns (Campaign_A and Campaign_B).
  3. Variable Assignment:
    • For Script Testing: Assign Script_Standard to Campaign A and Script_Revised to Campaign B.
    • For Timing Testing: Assign the same script to both, but give Campaign B a different Callable Time Set (e.g., Late Afternoon only).

4. Tracking Conversions

Use a specific “Wrap-Up Code” or “Participant Data” attribute to flag successful conversions.

Architect Logic:

[Agent Selects Wrap-Up: "Sale Completed"]
    |
    v
[Set Participant Data: "Campaign_Group" = Task.CurrentCampaignGroup]

5. Analyzing Results (Statistical Significance)

After 1000+ dials per group, calculate the conversion rate difference.

Significance Check:
Use a Chi-Squared test to determine if the 2% lift in Group B is “Real” or just noise.

Metric Group A (Control) Group B (Test) Lift
Dials 5,000 5,000 -
Live Answers 500 (10%) 550 (11%) +10%
Conversions 50 (1.0%) 75 (1.5%) +50%

Validation, Edge Cases & Troubleshooting

Edge Case 1: Agent Bias

If Group A is handled by your “A-Team” agents and Group B by “New Hires,” the results are invalid.
Solution: Ensure both campaigns use the same Agent Pool (Queue). Genesys Cloud will naturally distribute calls from both campaigns to the same agents, neutralizing individual agent performance bias.

Edge Case 2: Contact List Refreshing

If you add new leads to the master list mid-test, you must apply the same randomization logic to the new leads to ensure the 50/50 split remains balanced.
Solution: Automate the splitting logic using a Lambda function that triggers whenever a new contact is added to the “Master” list via API.

Edge Case 3: The “Winner” Take-All

Once Group B is proven to be 20% better, you want to shift 100% of the volume to the new strategy immediately.
Solution: Do not delete Campaign A. Instead, update the “Dialing Weight” of Campaign B to 100% and Campaign A to 0% within the campaign group to preserve the historical performance data for the experiment.

Official References