We’re seeing a spike in 5xx errors on our webhook consumer endpoint during peak traffic. The Genesys Cloud platform retries these failures, but the exponential backoff isn’t enough when our service is genuinely overwhelmed. We need to implement a dead letter queue (DLQ) pattern to capture these failed deliveries for later processing or alerting.
The webhook endpoint is a simple Express.js server. When it returns a 500, Genesys retries. We want to intercept these retries locally and push them to an AWS SQS DLQ instead of failing again. Here’s the current setup:
app.post('/webhooks/conversations', (req, res) => {
try {
// Process event
processEvent(req.body);
res.status(200).send('OK');
} catch (error) {
// Log error and send to DLQ
sendToDLQ(req.body, error);
res.status(500).send('Internal Server Error');
}
});
The issue is that Genesys retries the 500 response, causing the same event to be processed multiple times and sent to the DLQ repeatedly. We need a way to acknowledge the failure to Genesys while ensuring the event is only sent to the DLQ once. Is there a way to configure Genesys webhooks to stop retries after a certain number of failures or a specific time window? Or should we implement idempotency checks on the DLQ side?
We’ve tried setting the retry policy in the webhook configuration, but it seems to be fixed. Any insights on how to handle this gracefully would be appreciated. We’re looking for a code-level solution that integrates with the Genesys webhook delivery mechanism.