We are trying to stream Analytics Notification events to a Kafka topic using a Node.js consumer. The goal is to get real-time conversation metrics into our data lake without polling the REST API. We have the WebSocket connection established and it works fine for low traffic.
The issue is that under high load, the consumer drops events or the connection resets. We are using the ws library in Node.js. Here is how we are handling the connection and message parsing:
const WebSocket = require('ws');
const kafka = require('kafka-node');
const ws = new WebSocket('wss://api.mypurecloud.com/api/v2/analytics/events');
ws.on('open', () => {
console.log('Connected to Analytics WebSocket');
});
ws.on('message', (data) => {
try {
const event = JSON.parse(data);
// Send to Kafka
producer.send([
{ topic: 'analytics-events', messages: JSON.stringify(event) }
], (err, data) => {
if (err) console.error(err);
});
} catch (e) {
console.error('Parse error', e);
}
});
ws.on('close', () => {
console.log('WebSocket closed');
// Reconnect logic here
setTimeout(connect, 5000);
});
We are seeing 1006 close codes frequently when the event rate spikes. We tried adding a heartbeat ping/pong, but it did not help much. Is there a specific backpressure mechanism we need to implement in the WebSocket client? Or is there a rate limit on the notification endpoint that we are hitting?
We also noticed that the event payload sometimes contains nested objects that are very large. We are not sure if that affects the WebSocket frame size. Any ideas on how to stabilize this connection?