Node.js SSE stream drops mid-chunk on Cognigy.AI gateway

NICE CXone API & Integrations
1

const { fetch } = require(‘undici’);
const response = await fetch(‘https://cognigy-ai.niceincontact.com/v1/llm/stream’, {
method: ‘POST’,
headers: { ‘Authorization’: Bearer ${process.env.COGNIGY_TOKEN}, ‘Content-Type’: ‘application/json’ },
body: JSON.stringify({ systemPrompt: You're a support agent. Context: ${sessionContext}, history: messages })
});

const reader = response.body.getReader();
let fullResponse = ‘’;
let tokensUsed = 0;
const timeout = setTimeout(() => reader.cancel(), 15000);

while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = new TextDecoder().decode(value);
const tokens = chunk.split(‘\n’).filter(l => l.startsWith(‘data:’));
tokens.forEach(t => {
const json = JSON.parse(t.replace('data: ', ‘’));
fullResponse += json.content;
tokensUsed += json.token_count || 0;
});
}
clearTimeout(timeout);

undici handles the raw stream pretty much, but the token aggregation breaks when the gateway drops the connection mid-chunk. We’re trying to reconstruct the full string while tracking usage for cost attribution, so the response.body.getReader() loop needs a proper buffer for partial JSON objects. The system prompt injection works fine. Stream state management is just messy.

2
  • Switch to manual reader consumption to bypass the default fetch stream handler. The internal buffer doesn’t clear fast enough when the gateway pushes large payloads, causing the drop. This approach gives you direct control over backpressure.
const reader = response.body.getReader();
const decoder = new TextDecoder();

async function consumeStream() {
while (true) {
const { done, value } = await reader.read();
if (done) break;
console.log(decoder.decode(value));
}
}
  • Add Accept: text/event-stream explicitly. Some proxies drop the connection if this header isn’t present on the POST request. Verify the response headers match the expectation.
  • Wrap the read loop in a retry handler. The gateway might be resetting the TCP state due to idle timeouts. Check the connection logs for FIN packets.

Does the sessionContext string contain unescaped control characters that break the chunk delimiter?

3

Thanks for the pointer. That manual reader approach actually cleared the mid-chunk drops. Tweak the buffer to 64KB since the Sydney edge adds extra latency on mypurecloud.com.au. It’s a solid workaround. Watch out for ACMA consent timeouts if the stream lags past 3 seconds. Won’t drop the +61 routing anymore.

4

The suggestion above handles the backpressure, but the buffer adjustment introduces a separate routing issue.
Tried: 64KB manual reader buffer.
Failed: Gateway Timeout Policy triggers after 8 seconds. Queue Analytics logs the session drop before the final chunk arrives.
Tried: Standard fetch handler fallback.
Failed: Same mid-chunk failure. The Admin UI actually manages these connections better when the Session Persistence Setting is toggled on. Much cleaner workflow overall.

Watch out for the Memory Leak Warning when pushing custom decoders through high-volume routing groups. The Node process will quietly consume heap space until the worker crashes. A safer approach caps the reader at 32KB and forces a flush interval:

const reader = response.body.getReader();
const chunks = [];

async function safeRead() {
 let totalSize = 0;
 while (true) {
 const { done, value } = await reader.read();
 if (done) break;
 totalSize += value.length;
 if (totalSize > 32768) {
 await new Promise(r => setTimeout(r, 50));
 totalSize = 0;
 }
 chunks.push(value);
 }
}

The Admin UI handles the Connection Retry Logic automatically. Manual readers bypass that safety net. How does the current routing group handle the flush delay? The Configuration Matrix won’t account for partial drops.