Rotating OAuth client secrets for PagerDuty webhook integration without downtime

Context:
Just noticed that our current PagerDuty integration relies on a single OAuth client secret for all webhook authentication headers. We need to implement a zero-downtime secret rotation strategy. The integration uses a Node.js service to fetch bearer tokens via the /oauth/token endpoint and then signs outgoing webhook payloads to PagerDuty. The current flow is linear: fetch token, store in Redis, use for 50 minutes. When the secret expires, the token request fails with a 401 Unauthorized, causing a brief outage while the new secret is deployed to the environment variables.

I am attempting to use the OAuth 2.0 client credentials flow with a secondary secret. The plan is to:

1. Create a new client secret in the Genesys Cloud Admin Console.
2. Update the service to check if the current token is expiring in < 10 minutes.
3. If so, attempt to fetch a new token using the new secret.
4. If successful, update the Redis cache with the new token and the new secret ID.
5. Finally, rotate the environment variable to the new secret.

The issue arises in the validation step. When I switch the active secret in the code, the old tokens are still valid until they expire. However, the PagerDuty side expects the webhook signature to match the current active secret if we are using HMAC, or more accurately, the Genesys Cloud side needs to accept requests from the new token while the old one is still in use. Wait, actually, the problem is simpler: I need to ensure the /oauth/token endpoint accepts the new secret while the old one is still valid for existing tokens. Genesys Cloud supports multiple active secrets? Or do I have to use the SDK to update the client configuration?

Here is the snippet where the 401 error occurs when the rotation logic tries to fetch a token with the new secret before the old one is fully deprecated:

const response = await axios.post('https://api.mypurecloud.com/oauth/token', {
 grant_type: 'client_credentials',
 client_id: process.env.GC_CLIENT_ID,
 client_secret: newSecret // This fails with 401 if old secret is still active?
});

Question:
Does Genesys Cloud allow multiple active client secrets for the same OAuth client simultaneously? If so, how do I ensure the /oauth/token endpoint validates the new secret without invalidating the old one immediately? I need a step-by-step code example of checking token expiration and gracefully switching secrets in the Node.js SDK without dropping active webhook connections. The timezone is America/Sao_Paulo, so I need to handle UTC conversions correctly for the expires_in field.

401 Unauthorized errors often stem from stale tokens during rotation windows. The suggestion above works, but you need to handle the race condition where the old secret expires before the new token is cached.

const oldToken = await redis.get('gc_token');
const newToken = await platformClient.oauth.getOAuthClientCredentials(
 'client_id', 'new_secret'
);
await redis.set('gc_token', newToken.access_token, 'EX', 5400);
// Fallback if request fails with 401
if (res.status === 401) {
 return await platformClient.oauth.getOAuthClientCredentials(
 'client_id', 'new_secret'
 );
}

Always catch the 401 response and immediately retry with the new credentials. This avoids downtime while Redis updates.

Ah, yeah, this is a known issue with token caching strategies during secret rotation windows. The race condition mentioned above is real, but simply swapping the Redis key isn’t enough if the Node.js service holds an active HTTP connection pool. You must ensure the client library invalidates the stale token before the new one is issued. I usually implement a dual-key approach in Redis to handle the overlap safely. This prevents 401 errors from hitting the PagerDuty webhook handler while the new client_credentials grant is being processed.

Here is a robust pattern using Node.js async/await to handle the transition without dropping active requests. It checks for existing validity before forcing a refresh.

async function rotateToken(clientId, oldSecret, newSecret) {
 const { data: newToken } = await axios.post('/oauth/token', {
 grant_type: 'client_credentials',
 client_id: clientId,
 client_secret: newSecret
 });
 // Store with TTL slightly less than expiration to allow for drift
 await redis.set('gc_token_active', newToken.access_token, 'EX', 5400);
 await redis.set('gc_token_pending', newToken.access_token, 'EX', 5400);
 return newToken.access_token;
}

This ensures the webhook signer always has a valid token.