OAuth client_credentials token returns 401 after secret rotation

Our OAuth client_credentials token started returning 401 Unauthorized after we rotated the client secret.

We updated the secret in our application config. The old secret was revoked. But every token request fails with 401. We triple-checked the new secret.

After rotating the secret, there’s a propagation delay of up to 60 seconds.

The new secret must propagate through GC’s OAuth infrastructure. If your application retries immediately after rotation, it may hit a node still using the old secret cache. Wait 60 seconds after rotation before making the first token request.

Under SOC2, OAuth client secret rotation must occur at least quarterly.

Document the rotation in your change log: old secret revocation time, new secret activation time, and verification that the application successfully authenticates with the new secret. Auditors specifically check for evidence of regular credential rotation.

We automated secret rotation with a Lambda function.

def rotate_gc_secret():
    new_secret = gc_api.regenerate_oauth_client_secret(client_id)
    ssm = boto3.client('ssm')
    ssm.put_parameter(Name='/gc/client_secret', Value=new_secret,
                      Type='SecureString', Overwrite=True)
    time.sleep(60)  # Wait for propagation
    verify_authentication(client_id, new_secret)

Runs quarterly via CloudWatch Events.

We once rotated the secret during peak hours and caused a 3-minute authentication outage.

The 60-second propagation delay, combined with our application’s aggressive retry-on-failure logic, triggered the GC rate limiter on the OAuth endpoint. We now schedule rotations during our maintenance window (Sunday 2 AM).

Also verify you are not including whitespace or newline characters in the secret.

# Common mistake: trailing newline from copy-paste
client_secret = os.environ['GC_SECRET'].strip()  # Always strip!
token = api.post_oauth_token(
    client_id=client_id,
    client_secret=client_secret,
    grant_type='client_credentials'
)

A single trailing \n in the secret causes 401 every time.