Hello. I am a performance engineer executing load tests for a new Genesys Dialog Engine voice bot. We are generating 500 concurrent synthetic calls into the platform. The synthetic caller uses standard text-to-speech to say ‘I want to pay my bill’. However, we observe that the bot fails to recognize the speech in nearly 40 percent of the calls. The bot simply responds ‘I did not understand’. When a real human speaks the exact same phrase, the recognition rate is perfect. Why does the conversational AI struggle to process audio generated by another computer system?
Do not get me started on this. I manage speech analytics and topic detection, and the exact same thing happens with automated answering machines and synthetic voice testing. The problem is that the Genesys Cloud acoustic models are trained exclusively on human speech patterns.
Computer-generated text-to-speech lacks natural human cadence, breath sounds, and vocal tract resonance. The natural language understanding engine evaluates the audio stream, decides it sounds robotic or anomalous, and rejects the transcription.
You basically cannot use cheap text-to-speech generators for load testing AI models.
That is an incredibly interesting architectural observation regarding the acoustic models! However, from my experience managing BYOC SIP trunks globally, there is another technical factor you must investigate. When you blast 500 concurrent synthetic calls through your session border controllers, you might be experiencing packet loss or severe jitter on the RTP media streams. Dialog Engine is extremely sensitive to RTP packet degradation.
Even a tiny amount of packet loss will scramble the audio payload enough that the bot cannot transcribe the words, whereas a human listener can effortlessly fill in the missing gaps mentally. Please verify your network metrics!
Both of those points are totally valid, but this actually opens up a really exciting opportunity! If the acoustic models can detect the difference between a synthetic voice and a real human, you can use that exact same mechanism for security! I have been exploring the new Voice Biometrics APIs, and they rely heavily on anti-spoofing technology to prevent fraudsters from using generated voices to bypass authentication! The fact that the bot rejects your synthetic load test is actually proof that the platform is incredibly secure against synthetic audio injection attacks! It is a brilliant feature!