Architect flow api 429s during jmeter load test from singapore

Architect & Flows
1

looking for advice on handling rate limiting when stress testing a simple architect flow. running jmeter from singapore to simulate concurrent inbound calls hitting a specific ivr path. the goal is to validate capacity planning for a peak volume event.

setup:

  • jmeter 5.6.2 running locally in asia/singapore.
  • thread group: 200 threads, ramp up 10s, loop count 5.
  • using the /api/v2/architect/flows/{flow_id}/simulate endpoint to mimic call flow execution.
  • auth: oauth2 client credentials grant, refreshing token every 55 minutes.

issue:
when concurrent threads exceed 50, i start seeing 429 too many requests responses. the response headers include retry-after: 2. initially, i assumed this was just the standard api rate limit (100 requests per second per client id). however, even with a constant throughput timer set to 80 requests per second, the 429s persist.

i noticed that the error rate spikes specifically when the simulate endpoint tries to process the transfer action within the flow. is there a different rate limit bucket for flow simulation versus standard api calls? or is the bottleneck related to the underlying websocket connections required for real-time flow execution?

i have tried:

  • adding a random timer (50-150ms) to stagger requests.
  • splitting the load across two different client ids with separate oauth tokens.
  • reducing payload size in the simulate request body.

none of these seem to help. the 429s continue to appear around the 60-70 concurrent request mark.

is there a way to increase the rate limit for load testing purposes? or should i be using a different endpoint/method for high-volume flow simulation? any insights on how to accurately model architect flow capacity without hitting these limits would be appreciated. currently blocked on finalizing our capacity report.

2

The root of the issue is that the simulate endpoint is designed for development and debugging, not for high-concurrency load testing. It processes requests synchronously and has strict, undocumented rate limits that are significantly lower than the production routing engine. When you fire 200 threads from a single IP block in Singapore, the platform’s API gateway interprets this as a potential DDoS attack or misconfigured bot, triggering immediate 429 responses to protect the shared infrastructure.

To accurately validate capacity for a peak volume event, you need to bypass the simulation layer entirely. Instead, use the POST /api/v2/architect/flows/{flow_id}/execute endpoint. This endpoint is optimized for actual traffic ingestion and respects the standard rate limits associated with your organization’s tier. However, even with execute, you must implement exponential backoff in your JMeter script. A simple retry logic without backoff will only exacerbate the issue.

Here is a sample configuration for the JMeter HTTP Request Defaults to handle this correctly:

{
 "endpoint": "/api/v2/architect/flows/{flow_id}/execute",
 "headers": {
 "Authorization": "Bearer {{access_token}}",
 "Content-Type": "application/json"
 },
 "body": {
 "inputs": [
 {
 "name": "CallerId",
 "value": "6590000001"
 }
 ]
 }
}

Additionally, consider distributing your JMeter agents across multiple geographic regions or using a cloud-based load testing service that supports distributed execution. This mimics real-world traffic patterns more accurately and reduces the likelihood of IP-based throttling. If you continue to hit 429s, check the Retry-After header in the response to determine the exact wait time before resuming requests. This approach aligns with how our AppFoundry integrations handle bulk operations at scale.