Does anyone know how to properly map the legacy confidence thresholds from our old Zendesk Answer Bot to the new Genesys Cloud Virtual Agent intents? We are currently in the final stages of migrating our digital support layer, and the intent matching behavior feels completely different than what we had configured in Zendesk.
In Zendesk, we relied on a simple confidence score where anything above 0.7 was accepted as a match, and lower scores triggered a fallback to human agents. However, in Genesys Cloud Virtual Agent, the intent node seems to be rejecting valid queries that were previously handled correctly. We are seeing a high volume of interactions falling through to the ‘No Match’ path, even when the user’s utterance is a direct hit on our training data.
We are using Genesys Cloud version 24.2 and have replicated all our Zendesk bot answers as intents in the Virtual Agent. The training data includes approximately 50 utterances per intent. When testing via the Virtual Agent simulator, the confidence scores for these intents are hovering around 0.4 to 0.5. In Zendesk, these same phrases would have scored 0.85. Is there a specific configuration in the Intent Node settings that adjusts the sensitivity or the matching algorithm?
We have already checked the ‘Confidence Threshold’ field in the intent node properties, but it seems to only control the minimum score required to proceed, not the scoring mechanism itself. We are worried that our customers will experience a drop in resolution rates if we lower the threshold too much, leading to false positives. Any advice on how to calibrate the NLP model to match the previous Zendesk performance would be greatly appreciated. We want to ensure the migration feels seamless for our end users.
I’d suggest checking out at how Genesys Cloud handles intent confidence differently than Zendesk, because the 0.7 threshold is not a direct 1:1 mapping. In Genesys, the Virtual Agent uses a more complex scoring model that includes entity extraction and context, so a raw confidence score of 0.7 might be considered low if the intent has many similar siblings. Instead of relying on a global threshold, configure the intent-specific confidence thresholds in the Virtual Agent builder. For each intent, set the “Confidence Threshold” property to a value that matches your legacy Zendesk behavior, but test it with your actual dataset. If you are using the API to manage these settings, you can update the intent configuration via the Virtual Agent API. Here is an example of how to adjust the threshold for a specific intent: PUT /api/v2/virtualagent/bots/{botId}/intents/{intentId} with the payload {"confidenceThreshold": 0.75}. Also, check the “Fallback” node in your flow. If the confidence is below the threshold, ensure the fallback is correctly configured to transfer to a human agent or trigger a clarification question. This approach gives you more granular control over the matching behavior. Note that for legal discovery purposes, the intent confidence scores are included in the chat transcript metadata, which is important for audit trails. If you are exporting these chats for legal hold, ensure that the metadata includes the intent confidence scores so you can review why certain intents were matched or missed. This helps in validating the migration accuracy. You might also want to review the “Similarity” settings in the NLP engine, as this can affect the confidence scores. Adjusting these settings can help align the behavior with your legacy system. Finally, test the migration with a sample dataset to validate the new thresholds before going live. This ensures that the intent matching behaves as expected and reduces the risk of misrouting chats.
If I remember correctly, pushing intent thresholds too high during migration risks silent failures. The system may drop valid requests to fallbacks faster than expected. Monitor the WebSocket throughput carefully. Sudden spikes in fallback traffic can saturate connection limits, just like our JMeter tests showed at 500 concurrent calls.
If I remember correctly, pushing intent thresholds too high during migration risks silent failures. The system may drop valid requests to fallbacks faster than expected. Monitor the WebSocket throughput carefully.