Implementing Automated Utterance Generation Using Large Language Models for NLU Training

How-To-Articles
1

Implementing Automated Utterance Generation Using Large Language Models for NLU Training

What This Guide Covers

This guide details the configuration of automated utterance generation workflows within Genesys Cloud CX AI to expand intent training data using Large Language Models (LLMs). The end result is a production-ready Natural Language Understanding (NLU) model with significantly improved confidence scores and coverage of long-tail linguistic variations without manual transcription.

Prerequisites, Roles & Licensing

To execute this implementation, the environment must meet specific licensing and permission requirements. Automated utterance generation relies on Genesix AI capabilities, which are available only under specific tiers.

Licensing Requirements:

  • Genesys Cloud CX Premium or Enterprise License. Basic licenses do not include advanced NLU training features or LLM access for data augmentation.
  • Genesix Add-on (if applicable). Depending on the region and contract date, this feature may reside under the Genesix AI suite rather than standard WEM.
  • LLM Provider Access. If using custom external models via API, ensure network egress is permitted to the LLM provider endpoints (e.g., AWS Bedrock, Azure OpenAI, or Genesys-hosted models).

Granular Permissions:
The user account executing this configuration requires the following permission scopes:

  • AI > NLU > Edit: Allows modification of intent definitions and training data.
  • AI > NLU > Train: Permits triggering model retraining cycles.
  • Admin > System > Read: Required to view model statistics and generation logs.

External Dependencies:

  • CRM/Context Integration. The LLM requires context about the business domain to generate relevant utterances. Ensure a glossary or domain definition document is available.
  • API Access. For programmatic management, OAuth 2.0 tokens with ai.intents.read and ai.intents.write scopes are mandatory.

The Implementation Deep-Dive

1. Intent Baseline Definition and Context Injection

Before generating data, the system must have a defined “seed” intent. You cannot ask an LLM to generate training data for a concept it does not understand.

Configuration Steps:

  1. Navigate to Admin > AI > NLU > Intents.
  2. Create a new Intent or select an existing one that requires augmentation.
  3. Ensure at least three distinct, high-quality seed utterances are present in the Training Utterances field. These serve as the few-shot prompting examples for the LLM.
  4. Define Entity Definitions precisely. If your intent involves dates or specific product codes, define these entities explicitly before generation begins.

The Trap:
A common failure mode occurs when users create an Intent with only a label (e.g., “CheckBalance”) but zero seed utterances and no entity definitions. The LLM will hallucinate generic phrases that do not match actual customer speech patterns. This results in a model that scores high on synthetic data but fails catastrophically in production because the semantic distribution is skewed toward formal language rather than colloquial customer speech.

Architectural Reasoning:
We use seed utterances as anchors for the probability distribution of the LLM. The generation algorithm calculates vectors based on these seeds to maintain semantic proximity. Without them, the model drifts into abstract definitions that users never speak. Always validate that seed utterances cover different sentence structures (declarative, interrogative, imperative) before proceeding.

2. Configuring the Automated Generation Parameters

This step involves setting the constraints for how the LLM generates new data. You must balance volume with quality control.

Configuration Steps:

  1. Within the Intent editor, locate the Automated Generation tab.
  2. Select the Model Provider. Choose the Genesys-hosted model for compliance and latency optimization, or an external provider if specific domain tuning is required.
  3. Set the Generation Count. Recommended starting value is 50 new utterances per intent. Do not exceed 100 in a single batch without validation.
  4. Configure Diversity Parameters. Enable “Synonym Variation” to ensure the model generates phrases using different vocabulary while maintaining the same semantic intent.
  5. Define Negative Constraints. Input phrases or patterns that should be excluded (e.g., specific competitor names, prohibited words).

API Payload Reference:
When managing this configuration via REST API, the request body structure for initiating generation follows this pattern:

POST /api/v2/conversation/ai/intents/{intentId}/generate-utterances

{
  "modelType": "GENESIX_PRODUCTION",
  "count": 50,
  "parameters": {
    "temperature": 0.7,
    "diversityLevel": "HIGH",
    "excludePatterns": ["competitor_name", "spam_keyword"]
  },
  "contextInjection": {
    "domainGlossaryId": "glossary_123456",
    "includeEntities": true
  }
}

The Trap:
Users frequently set the temperature parameter too high (above 0.8) in an attempt to maximize variety. In NLU training, high temperature leads to hallucinations where the LLM generates grammatically correct but semantically unrelated phrases. For example, a request for “Balance” might generate “I want to transfer money,” which is a different intent. Always keep the generation temperature between 0.5 and 0.7 for NLU augmentation tasks.

Architectural Reasoning:
The temperature parameter controls randomness. Lower values ensure the LLM sticks closer to the seed distribution, preserving the intent boundary. Higher values explore the latent space but risk crossing into adjacent intents. For production training data, we prioritize precision over recall during the generation phase because false positives are more costly than missed opportunities in a contact center environment.

3. Review, Validation, and Integration

Generated utterances are not automatically added to the live model. They exist in a staging state that requires human validation. This step ensures compliance and accuracy.

Configuration Steps:

  1. Navigate to the Draft tab of the Intent editor after generation completes.
  2. Inspect the generated list. Filter by confidence score if available within the preview pane.
  3. Select specific utterances for inclusion. Use the checkbox mechanism to curate the final dataset.
  4. Click Review and Train. This action triggers the model retraining pipeline.

The Trap:
The “Select All” behavior is a significant risk vector. Users often accept all generated data to save time. This bypasses the quality gate and introduces noise into the training set. A single batch of hallucinated data can degrade the overall intent classification boundary, causing the system to misroute calls more frequently than before the update.

Architectural Reasoning:
Human-in-the-loop validation is non-negotiable for LLM-generated training data. The model generates candidates, not ground truth. The architect must verify that the generated utterances do not violate compliance policies (e.g., PII handling) and that they align with the specific business context. For regulated industries like finance or healthcare, this review step is also a regulatory requirement to ensure no unauthorized data patterns are introduced into the model.

4. Model Training and Performance Monitoring

Once validated, the intent must be trained to incorporate the new utterances.

Configuration Steps:

  1. Initiate the Train action on the Intent.
  2. Monitor the Training Progress dashboard. This process is asynchronous and may take several minutes depending on the dataset size.
  3. After training completion, run a Test Conversation within the NLU testing sandbox. Input known edge-case phrases to verify classification stability.
  4. Deploy the changes to the live environment by pushing the Intent definition to the active conversation flow.

API Payload Reference:
To trigger training programmatically after validation:

POST /api/v2/conversation/ai/intents/{intentId}/train

{
  "version": "v1",
  "environment": "PRODUCTION"
}

The Trap:
Deploying immediately after generation without sandbox testing is a common failure. The LLM may introduce edge cases that break existing conversation flows or confuse the routing logic. If an intent becomes ambiguous with another similar intent, call duration and abandonment rates will spike. Always test against the regression suite before deployment.

Architectural Reasoning:
NLU models are stateless but retraining is computationally expensive. The system caches the model weights during training. Deploying changes ensures that the inference engine uses the updated probability distributions. Monitoring post-deployment metrics (Confidence Score, Intent Match Rate) for at least 48 hours is critical to detect drift or degradation caused by the new data.

Validation, Edge Cases & Troubleshooting

Edge Case 1: Semantic Drift in Long-Tail Intents

The Failure Condition:
After generating utterances for a specific intent (e.g., “Report Fraud”), the system begins misclassifying calls intended for “Change Address” or “Account Lock”. The confidence scores for both intents drop below the routing threshold.

The Root Cause:
The LLM generated utterances that shared semantic features with adjacent intents but did not include enough negative constraints. This is often caused by the model inferring too much from generic phrases like “I need to report an issue” without specific contextual anchors.

The Solution:

  1. Revert the recent generation batch for the affected intent.
  2. Add Negative Constraints to the Intent configuration to explicitly exclude keywords associated with competing intents.
  3. Increase the number of seed utterances manually to anchor the semantic vector more strongly.
  4. Regenerate data using a lower temperature (0.5) and review the output for semantic overlap.

Edge Case 2: PII Leakage in Generated Data

The Failure Condition:
Automated generation produces training phrases that include placeholder names or account numbers that look real (e.g., “My account number is 123456789”). This triggers PCI-DSS compliance violations during data logging.

The Root Cause:
The LLM attempts to fill entity placeholders with realistic-looking data rather than the required generic tokens (e.g., {{accountNumber}}).

The Solution:
Configure the Entity Formatting Rules in the Intent settings before generation. Ensure that all entities are marked as Redacted or Masked in the training pipeline. When using the API, enforce the entityFormat parameter to ensure placeholders are used instead of synthetic data values. Verify logs immediately after generation to confirm no PII is present in the text field.

Edge Case 3: Cost and Latency Overhead

The Failure Condition:
Generation requests become expensive or slow due to excessive batch sizes or high-frequency calls to the LLM provider.

The Root Cause:
Unbounded generation counts without monitoring usage quotas. Each token generated incurs a cost, and large batches increase latency in the NLU configuration interface.

The Solution:
Implement Rate Limiting on the generation API calls. Cap the generation count per intent to 50-100 utterances initially. Schedule generation tasks during off-peak hours to avoid contention with production inference workloads. Monitor usage metrics in the Admin > AI dashboard to track token consumption against budget.

Official References