Running a JMeter script to validate the scalability of our AI bot flows on Genesys Cloud US1. The goal is to simulate high-concurrency chat sessions to identify bottlenecks in the NLU processing and response generation.
The test setup involves 200 concurrent WebSocket connections initiating chat sessions. Each session triggers a specific Architect flow that relies heavily on the ‘Extract Information’ node and subsequent API actions to fetch user data.
After approximately 50 active sessions, the Genesys Cloud backend starts returning HTTP 429 Too Many Requests errors. This happens specifically when the bot attempts to call external APIs via the ‘Make HTTP Request’ node. The error logs indicate that the rate limit for external API calls from the bot context has been exceeded.
The JMeter config uses a constant throughput timer to maintain a steady request rate of 10 requests per second per thread group. The WebSocket connections remain open, but the response time for bot replies increases significantly, leading to timeouts.
The environment is Genesys Cloud US1. The bot is configured with default settings for concurrency limits. No custom rate limiting policies are applied to the bot flow.
Is there a way to increase the rate limit for external API calls from bot flows? Or is this a hard limit on the platform? Has anyone encountered similar issues during load testing?
Sharing the JMeter config details:
- Thread Group: 200 threads
- Ramp-Up Period: 60 seconds
- Loop Count: Infinite
- WebSocket Listener: Connected to wss://webchat-api.genesiscloud.com
- HTTP Request Sampler: Used to simulate bot API calls
The error occurs consistently after the 50th session. The WebSocket connection remains stable, but the bot response is delayed or fails.
Need help ASAP with this. The current setup prevents us from validating the bot’s performance under load. Any insights or workarounds would be appreciated.