We’re seeing a spike in 504 Gateway Timeout errors on our /api/v2/analytics/icap webhook endpoint. The docs say if the consumer doesn’t respond with 200 OK within the timeout window, Genesys retries the delivery. Our current C# Azure Function handles the POST, deserializes the JSON, and pushes to the CRM queue. It usually takes 800ms, but under load, it hits 2.5s and Genesys kills the connection.
The issue is that we don’t have a proper dead letter queue (DLQ) for these failed attempts. We just want to capture the failed JSON payload and store it for manual retry later, instead of letting Genesys hammer our endpoint.
Here’s the current function signature:
[FunctionName("GenesysWebhookConsumer")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "post", Route = null)] HttpRequest req,
ILogger log)
{
try
{
var content = await new StreamReader(req.Body).ReadToEndAsync();
var data = JsonConvert.DeserializeObject<GenesysPayload>(content);
// Simulate CRM sync delay
await Task.Delay(1000);
return new OkObjectResult("Received");
}
catch (Exception ex)
{
log.LogError(ex, "Failed to process webhook");
// Currently returning 500, which triggers Genesys retry
return new StatusCodeResult(500);
}
}
The docs state: “If the target endpoint returns a 5xx status code, the system will retry delivery with exponential backoff.”
We want to change this. If the processing takes too long or fails, we want to return a 200 OK to Genesys (to stop retries) but save the content string to an Azure Storage Queue for our DLQ processor.
Is this the right approach? Or should we be using the X-Genesys-Event-Id header to check for duplicates first? The header seems to change on every retry, so deduplication is tricky.
Also, how do we handle the case where the CRM is down? We don’t want to block the webhook thread waiting for the CRM. Should we fire and forget the CRM call and just acknowledge the webhook immediately?
Any code examples for the DLQ pattern in C# Azure Functions would be helpful. We’re using the v4 SDK for the CRM part, but this is just the webhook listener.