Historical Reporting - Job Status Endpoint Throttling

Hey all,

Something’s up with the historical reporting API again. Been testing the job status endpoint - trying to poll job status every second, figured it’d be fine. It isn’t. Getting 429s way too quickly. I thought it was a burst limit initially, but it’s…different.

Ran a methodical test. The first few requests go through fine, then it just slams the brakes on. It’s not a clean cut-off at, say, 10 requests/second. It’s more…erratic. Sometimes it lets through three requests after the initial burst, sometimes zero. It feels sustained.

We’re on v12.1.0 Tokyo. I’m using the Node SDK - version 8.0.0. The Architect flow triggering this is pretty simple - just a Data Action after a reporting job is created. The Data Action makes the call to that status endpoint.

Here’s a snippet of the error, though it’s not a full log, just what I grabbed quickly:

{
 "message": "Rate limit exceeded. Please try again later.",
 "code": 429,
 "status": 429,
 "details": [
 {
 "field": null,
 "message": "Rate limit exceeded"
 }
 ]
}

Apologies if this is basic - maybe I’m missing something obvious? The documentation is…sparse on details about this endpoint specifically. It just says “subject to rate limits”. I’m trying to understand what the sustained rate is here. It’s killing our job monitoring.

1 Like

No. Polling every second is a great way to get throttled. We’ve got around 1800 agents and our Celery workers use exponential backoff for /api/v2/workforcemanagement/adherence/historical/jobs/{jobId}.

Try this logic instead:

import time

delay = 1
while job_status != 'COMPLETED':
 status = get_job_status(job_id)
 if status == 'RUNNING':
 time.sleep(delay)
 delay = min(delay * 2, 60)
1 Like

the earlier reply is right. 429 is always limit problem. In java if you poll /api/v2/workforcemanagement/adherence/historical/jobs/{jobId} too fast, you get throttled.

// fix for rate limit
if (response.getStatus() == 429) {
 long retryAfter = Long.parseLong(response.getHeader("Retry-After"));
 Thread.sleep(retryAfter * 1000);
}

sometimes the quota is small for this endpoint.

The observed 429 responses indicate a violation of the rate limit associated with the /api/v2/workforcemanagement/adherence/historical/jobs/{jobId} endpoint. High-frequency polling at one-second intervals exceeds the permissible request quota.

Similar behavior was documented in INC-4471. The optimal resolution involves implementing a jittered exponential backoff strategy to modulate request frequency.

2 Likes

the fix above is spot on. tbh, if you’re running this through a Postman collection (which is how i usually test these things), just stick a pm.execution.setNextRequest in your tests with a setTimeout (or just a delay if using Newman) to avoid hitting that wall. iirc, those job endpoints are notoriously picky about polling frequency.