Dealing with a very strange bug here with the Quality Management API. When JMeter threads exceed 200 concurrent requests to /api/v2/quality/evaluations, the US1 environment returns 429 Too Many Requests. This happens even with valid OAuth tokens and standard pagination.
The rate limit headers show remaining quota but the error persists. Is there a specific throttle for quality endpoints under load? Using SDK v2.8. Need to validate bulk evaluation workflows.
Make sure you implement exponential backoff in your JMeter logic. The Quality API enforces strict per-second limits distinct from standard REST endpoints. Ignoring the Retry-After header triggers immediate 429s regardless of token validity.
<RandomTimer/>
<ConstantThroughputTimer/>
Adjust these to spread requests evenly. Bursting >10 req/s per tenant usually hits the hard cap. Check x-ratelimit-remaining before sending the next batch.
Generally speaking, the Quality API throttles based on tenant-level concurrency, not just OAuth validity. Implementing a retry loop with exponential backoff is mandatory for bulk operations to avoid hitting the hard cap.
{
"retry_strategy": "exponential_backoff",
"max_retries": 3,
"base_delay_ms": 1000
}Check your JMeter thread group configuration for any static delays that might be causing synchronized request bursts. While the previous suggestions about exponential backoff are spot on for handling the 429s, I’ve seen this specific issue in our weekly schedule publish workflows where the initial burst overwhelms the gateway before the backoff logic even kicks in. The Quality API, like the WFM scheduling endpoints, has a strict per-tenant concurrency cap that doesn’t care about your token validity.
In our environment, we found that spreading the initial load using a Gaussian Random Timer in JMeter helps smooth out the request curve before the backoff strategy takes over. Here is a snippet of how we structure the timer to avoid the “thundering herd” effect:
<GaussianRandomTimer>
<stringProp name="GaussianRandomTimer.constantThroughput">false</stringProp>
<stringProp name="GaussianRandomTimer.gaussianDeviation">500</stringProp>
<stringProp name="GaussianRandomTimer.gaussianMean">1000</stringProp>
</GaussianRandomTimer>
Warning: Do not set the deviation too low, or you risk creating micro-bursts that still trigger the rate limiter. The key is to ensure that your average request rate stays well below the hard cap of ~10 req/s per tenant, as mentioned earlier. We usually aim for a 70% utilization of the rate limit headroom during bulk operations to account for any network latency spikes.
Also, verify that your JMeter script is correctly parsing the Retry-After header from the 429 response. If you ignore this header and just rely on a fixed backoff, you might still hit the limit if the server is under heavy load. The server tells you exactly how long to wait, so listening to it is crucial. This approach has saved us countless hours during our Friday schedule publishes, where timing is everything. Hope this helps stabilize your bulk evaluation tests!