We’ve set up an EventBridge integration to push Genesys Cloud events to an Azure Function written in C#. The goal is simple: capture routing.queue.conversation.wrapup events and update our internal CRM.
The problem is the delivery. We’re seeing duplicate events hitting the function. Sometimes it’s two identical payloads within milliseconds. Other times, it’s a retry after a 200 OK. The docs say EventBridge provides “at least once” delivery.
Here’s the relevant part of the Azure Function:
[FunctionName("ProcessGenesysEvent")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "post", Route = null)] HttpRequest req,
ILogger log)
{
var body = await new StreamReader(req.Body).ReadToEndAsync();
var eventDetail = JsonConvert.DeserializeObject<EventBridgePayload>(body);
// Check for duplicates in Cosmos DB
var exists = await _cosmosClient.CheckDuplicateAsync(eventDetail.Id);
if (exists) return new OkObjectResult("Duplicate skipped");
await _crmService.UpdateContact(eventDetail.Data);
await _cosmosClient.MarkAsProcessed(eventDetail.Id);
return new OkResult();
}
The eventDetail.Id is the eventId from the EventBridge payload. I assumed this would be unique per event. But we’re seeing the same eventId come in twice. The first request gets a 200. The second one comes in, checks Cosmos, and correctly skips it. But the overhead of checking and logging duplicates is killing us. And if Cosmos is down, we process twice.
Is there a better way to handle this in the HTTP handler? Should I be relying on the eventTime instead? Or is there a header I can check? The Genesys Cloud documentation on EventBridge delivery guarantees is pretty vague on the exact retry logic.
Also, the timezone difference (we’re in Sao Paulo, UTC-3) means the eventTime has to be handled carefully if we’re doing any windowing. But the main issue is the duplicate eventId.
Has anyone dealt with this? Is there a standard pattern for deduplication in this setup? I’m considering adding a distributed lock, but that seems like overkill for a simple webhook. What’s the least painful way to ensure idempotency without adding too much latency?