Running the genesyscloud python SDK v2.20.0. We have a background worker that processes failed webhook deliveries by re-signing the payload and posting it to our internal dead letter queue.
The issue is that when the access token expires mid-batch, the SDK’s automatic refresh mechanism seems to lock up or hang indefinitely if the initial request fails with a 401 before the refresh completes.
Here’s the rough flow:
client = genesyscloud.rest_api_client.RestApiClient(config)
webhooks_api = genesyscloud.webhooks_api.WebhooksApi(client)
try:
response = webhooks_api.get_webhook(id=webhook_id)
except Exception as e:
print(f"Error: {e}")
If get_webhook hits a 401, the SDK internally calls the refresh endpoint. But if we have multiple threads hitting different endpoints at the exact same time, sometimes we get a TimeoutError on the refresh call itself, and the subsequent calls just hang waiting for the lock to release on the token manager.
We’ve tried setting timeout=30 on the client config, but it doesn’t seem to apply to the internal OAuth refresh loop. The logs show the refresh request going out, but no response comes back, and the main thread just sits there.
{
"timestamp": "2023-10-27T02:14:12.000Z",
"level": "WARNING",
"message": "OAuth token refresh failed or timed out",
"details": "Connection reset by peer"
}
Is there a way to force a synchronous refresh before the batch starts, or configure the SDK to fail fast instead of hanging? We don’t want our worker threads blocking for 60+ seconds just because one token expired.
Also, we’re in Sydney, so our server time is UTC+11. Not sure if that affects the token expiry window calculation in the SDK cache, but thought I’d mention it.