Is it possible to process high-volume interaction events from Genesys Cloud EventBridge without hitting AWS Lambda concurrency limits while maintaining New Relic APM visibility?
We are currently instrumenting our Genesys Cloud platform API performance using New Relic. Our architecture consumes interaction events via EventBridge, which triggers a Node.js Lambda function. This function is responsible for enriching the event data and creating custom events in New Relic for downstream analytics and alert policies.
Recently, during peak call center hours, we have observed significant throttling. The Lambda function hits its reserved concurrency limit, causing EventBridge to retry failed deliveries. This results in duplicate custom events and increased latency in our NRQL dashboards.
Here is the current handler structure:
exports.handler = async (event) => {
for (const record of event.Records) {
const gcEvent = record.body;
await sendToNewRelic(gcEvent);
}
return { statusCode: 200 };
};
The 429 Too Many Requests errors from the New Relic ingest API are exacerbating the issue, as we are not batching requests efficiently. We attempted to increase the Lambda concurrency, but this only shifted the bottleneck to the New Relic ingestion limits.
We need a code-level solution to batch these events before sending them to New Relic, or a way to leverage DynamoDB Streams or SQS for buffering. However, we must ensure that the instrumentation logic remains intact for accurate APM tracing.
Has anyone implemented a robust batching mechanism within the Lambda handler that respects both Genesys Cloud event frequency and New Relic ingest limits? Specifically, how should we structure the asynchronous calls to avoid overwhelming the concurrency pool while ensuring no events are dropped?