AI Bot Intent Confidence Threshold Mismatch in EU-West BYOC

Architect & Flows
1

Anyone know why the Architect flow logic diverges from the reported NLP confidence score in the Performance Dashboard? We observe agents receiving calls with low confidence scores (0.45), yet the flow logs indicate a successful handoff. Our bot configuration is set as follows:

nlp:
 model_version: v2.4.1
 intent_threshold: 0.75
 fallback_action: transfer_to_queue

This occurs in our EU-West BYOC environment. The dashboard metrics suggest the threshold is being ignored.

2

check if you’re using a decision block right after the nlp module. sometimes the threshold check gets bypassed if you’re just checking the intent name instead of the confidence score. also make sure your byoc model version actually supports dynamic thresholding. looks like a config mismatch to me

3

the decision block suggestion is spot on. we hit this same issue last month while wiring up our ServiceNow webhook triggers. the nlp module sets the confidence variable, but if the decision block is just checking IntentName == "Transfer", it ignores the threshold entirely.

make sure you’re actually evaluating the score in the condition:

{Conversation.AI.Intent.Confidence} >= 0.75

if you’re only checking the intent match, low-confidence hits will still pass through to the queue action, which explains why the dashboard shows 0.45 but the flow logs show success. also double-check that the BYOC model version isn’t caching old predictions during the update window.

1 Like
4

yeah, that check is critical. but also watch out for latency issues if you’re routing to APAC regions. sometimes the nlp response gets cached or delayed, causing the score to look stale in the dashboard while the flow already moved on. just a heads up for au teams dealing with cross-region calls.

3 Likes
5

that confidence variable check is the right track, but you’re missing the actual gating logic. just checking {Conversation.AI.Intent.Confidence} >= 0.75 in a decision block doesn’t stop the flow from continuing if you don’t wire the “No” path correctly. in Twilio, you’d just hang up or redirect in the same response. here, the NLP module outputs to Conversation.AI.Intent.Name and Conversation.AI.Intent.Confidence, but it doesn’t auto-fail.

you need to explicitly route the low-confidence leg. try this setup in your Architect flow:

  1. NLP Module: Set to “Get Intent”.
  2. Decision Block:
  • Condition: {Conversation.AI.Intent.Confidence} < 0.75
  • True Path: Route to your fallback_action (transfer_to_queue).
  • False Path: Continue to the intended action.

if you’re seeing agents get calls with 0.45 confidence, your “True” path is likely broken or pointing to a dead end, or you’re logging the score after a transfer that happened regardless. also, verify you’re not using Conversation.AI.Match.Confidence by mistake. that’s the match score, not the intent confidence. they’re different variables.

check the flow logs for the specific interaction ID. if the NLP module returns null for intent, the confidence is usually 0. make sure your model isn’t silently failing to parse the speech. if it is, the threshold check is moot.