Trying to pipe Genesys Cloud Streaming Analytics events into our internal Kafka cluster using a Node.js consumer. The goal is to get near-real-time queue wait times for a dashboard.
The Kafka topic is receiving messages, but about 20% of the events seem to be disappearing or arriving out of order compared to what I see in the Genesys UI. I’ve verified the consumer group is healthy and offsets.commit() is being called correctly.
Here’s the basic consumer setup:
const consumer = new KafkaConsumer({
clientId: 'genesys-analytics',
groupId: 'analytics-group-1',
topics: ['genesys.stream.analytics']
});
consumer.subscribe().then(() => {
consumer.consume((err, messages) => {
if (err) throw err;
messages.forEach(msg => {
// process msg.value
});
consumer.commit();
});
});
The JSON payload from Genesys looks standard:
{
"eventId": "abc-123",
"eventType": "queue-stat",
"timestamp": 1678886400000,
"data": { ... }
}
I’m wondering if there’s a specific header or ordering guarantee I’m missing with the Genesys Cloud Kafka integration. The documentation mentions exactly-once semantics, but my consumer feels like it’s struggling with the volume during peak hours. Is there a recommended way to handle backpressure or ensure I’m not dropping commits? I’ve tried increasing max.poll.records but it didn’t help much.