Problem
We’re trying to stream analytics notification events to a Kafka topic using a quick Node.js consumer script. The WebSocket connection to the Genesys Cloud notification endpoint stays open. It just keeps dropping payloads after a few minutes. The message queue fills up with truncated JSON strings. Looks like the batch size is too large for the producer to handle.
Code
const WebSocket = require('ws');
const { Kafka } = require('kafkajs');
const ws = new WebSocket('wss://api.mypurecloud.com/api/v2/analytics/notifications/stream');
const kafka = new Kafka({ brokers: ['kafka-broker:9092'] });
const producer = kafka.producer();
ws.on('message', async (data) => {
const payload = JSON.parse(data.toString());
await producer.send({
topic: 'genesys-analytics-events',
messages: [{ value: JSON.stringify(payload) }]
});
});
Error
The producer throws a MALFORMED_MESSAGE_SET error when the payload hits around 8MB. The Genesys notification stream sends batched events, but the JSON structure looks completely valid. We’re getting a 429 Too Many Requests on the WebSocket reconnect loop when the buffer finally clears out. Honestly the timing feels off.
Question
How do we handle the batching logic properly without choking the Kafka producer? The SDK doesn’t have a notification stream helper for Node anyway.