Bulk-Ingest NICE Cognigy.AI Entity Synonyms via REST API with TypeScript

What You Will Build

• This script constructs, validates, and posts bulk synonym payloads to the Cognigy.AI REST API, managing entity ID references, language directives, and synonym matrices.
• The implementation uses the Cognigy.AI v2 REST API surface with TypeScript, Axios, and Zod for schema enforcement.
• The programming language covered is TypeScript (Node.js 18+ runtime).

Prerequisites

• Cognigy.AI API key or OAuth2 bearer token with entities:write and entities:read scopes
• Cognigy.AI API v2 endpoint base URL (e.g., https://<YOUR_SERVER>.cognigy.ai)
• Node.js 18 or higher
• Dependencies: npm install axios zod typescript ts-node @types/node

Authentication Setup

Cognigy.AI accepts API keys or OAuth2 bearer tokens in the Authorization header. The following setup demonstrates token caching and refresh logic for long-running bulk operations. The code uses a simple in-memory cache with expiration tracking to avoid unnecessary token requests during batch processing.

import axios, { AxiosInstance } from 'axios';

const COGNIGY_BASE_URL = 'https://<YOUR_SERVER>.cognigy.ai';
const OAUTH_TOKEN_URL = 'https://<YOUR_AUTH_PROVIDER>/oauth/token';

interface TokenCache {
  accessToken: string;
  expiresAt: number;
}

let tokenCache: TokenCache | null = null;

async function getAuthToken(): Promise<string> {
  if (tokenCache && Date.now() < tokenCache.expiresAt) {
    return tokenCache.accessToken;
  }

  const response = await axios.post<OauthTokenResponse>(OAUTH_TOKEN_URL, {
    grant_type: 'client_credentials',
    client_id: process.env.COGNIGY_CLIENT_ID,
    client_required_scopes: 'entities:write entities:read'
  });

  tokenCache = {
    accessToken: response.data.access_token,
    expiresAt: Date.now() + (response.data.expires_in * 1000) - 60000 // 1 minute buffer
  };

  return tokenCache.accessToken;
}

interface OauthTokenResponse {
  access_token: string;
  expires_in: number;
  token_type: string;
}

The entities:write scope is mandatory for synonym ingestion. The entities:read scope enables index status verification and entity metadata retrieval. Token expiration is tracked with a sixty-second buffer to prevent mid-batch 401 failures.

Implementation

Step 1: Schema Validation and Payload Construction

The Cognigy.AI ingestion engine enforces strict batch size limits and language directive formats. The maximum batch size for synonym ingestion is 500 items per request. Exceeding this limit triggers a 400 Bad Request response. The payload must contain valid entity ID references, an ISO 639-1 language code, and a synonym matrix. Zod enforces these constraints before any network call occurs.

import { z } from 'zod';

const SynonymBatchSchema = z.object({
  entityId: z.string().uuid('Entity ID must be a valid UUID'),
  language: z.string().length(2, 'Language directive must be a 2-character ISO 639-1 code'),
  synonyms: z.array(z.string().min(1).max(255)).max(500, 'Maximum batch size is 500 synonyms')
});

type SynonymBatch = z.infer<typeof SynonymBatchSchema>;

function constructPayload(entityId: string, language: string, rawSynonyms: string[]): SynonymBatch {
  const validated = SynonymBatchSchema.parse({ entityId, language, synonyms: rawSynonyms });
  return validated;
}

The Zod schema rejects malformed entity IDs, invalid language codes, and oversized batches. The max(500) constraint aligns with the Cognigy.AI ingestion engine limit. The schema validation occurs synchronously, preventing network overhead for invalid data.

HTTP Request Cycle Example:

POST /api/v2/entities/550e8400-e29b-41d4-a716-446655440000/synonyms/bulk HTTP/1.1
Host: <YOUR_SERVER>.cognigy.ai
Authorization: Bearer <TOKEN>
Content-Type: application/json

{
  "language": "en",
  "synonyms": ["apple", "macintosh", "fruit"]
}

Expected Response (200 OK):

{
  "status": "success",
  "ingested_count": 3,
  "skipped_duplicates": 0,
  "entity_id": "550e8400-e29b-41d4-a716-446655440000",
  "index_status": "pending_rebuild"
}

Step 2: Character Normalization and Duplicate Checking Pipeline

Synonym ingestion requires strict character normalization to prevent index fragmentation. The pipeline applies Unicode NFKC normalization, lowercasing, whitespace trimming, and punctuation stripping. Duplicate detection uses a Set structure to guarantee O(1) lookup performance. This step prevents semantic drift and index corruption during scaling operations.

function normalizeSynonym(input: string): string {
  return input
    .normalize('NFKC')
    .toLowerCase()
    .replace(/[^\p{L}\p{N}\s]/gu, '')
    .replace(/\s+/g, ' ')
    .trim();
}

function deduplicateAndValidate(rawSynonyms: string[]): string[] {
  const normalized = rawSynonyms.map(normalizeSynonym).filter(s => s.length > 0);
  const unique = [...new Set(normalized)];
  
  if (unique.length !== normalized.length) {
    console.warn(`Removed ${normalized.length - unique.length} duplicate or empty synonyms after normalization.`);
  }
  
  return unique;
}

The normalize('NFKC') call converts compatibility characters to their canonical equivalents, ensuring fi becomes fi and accented characters map to standard forms. The regular expression /[^\p{L}\p{N}\s]/gu strips non-alphanumeric characters while preserving Unicode letters and numbers. The pipeline guarantees that identical semantic values map to a single index entry.

Step 3: Atomic POST Operations and Index Rebuild Triggers

The Cognigy.AI API treats bulk synonym ingestion as an atomic operation. The request either succeeds entirely or fails with a rollback. The code implements exponential backoff retry logic for 429 Too Many Requests responses. After successful ingestion, the code triggers an explicit index rebuild to force immediate vocabulary expansion.

interface IngestionResult {
  status: string;
  ingested_count: number;
  skipped_duplicates: number;
  entity_id: string;
  index_status: string;
}

async function postSynonymBatch(client: AxiosInstance, batch: SynonymBatch, maxRetries: number = 3): Promise<IngestionResult> {
  const url = `/api/v2/entities/${batch.entityId}/synonyms/bulk`;
  
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      const response = await client.post<IngestionResult>(url, {
        language: batch.language,
        synonyms: batch.synonyms
      });

      if (response.status !== 200 && response.status !== 201) {
        throw new Error(`Unexpected status: ${response.status}`);
      }

      return response.data;
    } catch (error: any) {
      if (error.response?.status === 429 && attempt < maxRetries) {
        const delay = Math.pow(2, attempt) * 1000;
        console.warn(`Rate limited (429). Retrying in ${delay}ms...`);
        await new Promise(resolve => setTimeout(resolve, delay));
        continue;
      }
      throw error;
    }
  }
  throw new Error('Max retries exceeded for synonym ingestion.');
}

async function triggerIndexRebuild(client: AxiosInstance, entityId: string): Promise<void> {
  try {
    await client.post(`/api/v2/entities/${entityId}/index/rebuild`);
    console.log(`Index rebuild triggered for entity ${entityId}`);
  } catch (error: any) {
    console.error(`Index rebuild failed for ${entityId}:`, error.response?.data || error.message);
  }
}

The retry loop handles 429 responses with exponential backoff. The postSynonymBatch function respects the atomic nature of the endpoint. The triggerIndexRebuild function calls the explicit rebuild endpoint to synchronize the search index immediately. This prevents stale query routing during high-volume ingestion.

Step 4: Latency Tracking, Audit Logging, and Webhook Synchronization

Production bulk ingestion requires governance tracking. The code measures request latency, records success rates, generates immutable audit logs, and synchronizes with external dictionaries via webhook notifications.

interface AuditLog {
  timestamp: string;
  entityId: string;
  batchId: string;
  synonymCount: number;
  ingestedCount: number;
  skippedDuplicates: number;
  latencyMs: number;
  success: boolean;
  webhookSynced: boolean;
}

async function syncWithExternalDictionary(webhookUrl: string, log: AuditLog): Promise<boolean> {
  try {
    await axios.post(webhookUrl, {
      event: 'synonyms_ingested',
      payload: log
    }, { timeout: 5000 });
    return true;
  } catch {
    console.error('Webhook sync failed. Dictionary alignment delayed.');
    return false;
  }
}

async function ingestBulkSynonyms(
  token: string,
  entityId: string,
  language: string,
  rawSynonyms: string[],
  webhookUrl: string
): Promise<AuditLog[]> {
  const client = axios.create({
    baseURL: COGNIGY_BASE_URL,
    headers: { Authorization: `Bearer ${token}` }
  });

  const normalized = deduplicateAndValidate(rawSynonyms);
  const batchSize = 500;
  const chunks: string[][] = [];
  
  for (let i = 0; i < normalized.length; i += batchSize) {
    chunks.push(normalized.slice(i, i + batchSize));
  }

  const auditLogs: AuditLog[] = [];

  for (const chunk of chunks) {
    const batchId = crypto.randomUUID();
    const start = Date.now();
    const success = true;
    let ingested = 0;
    let skipped = 0;
    let webhookSynced = false;

    try {
      const batch = constructPayload(entityId, language, chunk);
      const result = await postSynonymBatch(client, batch);
      ingested = result.ingested_count;
      skipped = result.skipped_duplicates;

      await triggerIndexRebuild(client, entityId);
      
      const latency = Date.now() - start;
      const log: AuditLog = {
        timestamp: new Date().toISOString(),
        entityId,
        batchId,
        synonymCount: chunk.length,
        ingestedCount: ingested,
        skippedDuplicates: skipped,
        latencyMs: latency,
        success,
        webhookSynced: false
      };

      webhookSynced = await syncWithExternalDictionary(webhookUrl, log);
      log.webhookSynced = webhookSynced;
      auditLogs.push(log);
    } catch (error: any) {
      const latency = Date.now() - start;
      auditLogs.push({
        timestamp: new Date().toISOString(),
        entityId,
        batchId: crypto.randomUUID(),
        synonymCount: chunk.length,
        ingestedCount: 0,
        skippedDuplicates: 0,
        latencyMs: latency,
        success: false,
        webhookSynced: false
      });
      console.error(`Batch failed:`, error.response?.data || error.message);
    }
  }

  return auditLogs;
}

The chunking logic splits large synonym lists into 500-item batches. Each batch generates a unique batchId for traceability. Latency measurement captures the full request cycle including retries. The webhook synchronization pushes ingestion events to external dictionary services for alignment. Audit logs record success rates and skip counts for governance review.

Complete Working Example

The following module combines all components into a runnable script. Replace the environment variables with your Cognigy.AI credentials.

import * as crypto from 'crypto';
import axios from 'axios';
import { z } from 'zod';

// --- Authentication ---
const COGNIGY_BASE_URL = process.env.COGNIGY_BASE_URL || 'https://<YOUR_SERVER>.cognigy.ai';
const OAUTH_TOKEN_URL = process.env.OAUTH_TOKEN_URL || 'https://<YOUR_AUTH_PROVIDER>/oauth/token';

interface TokenCache {
  accessToken: string;
  expiresAt: number;
}
let tokenCache: TokenCache | null = null;

async function getAuthToken(): Promise<string> {
  if (tokenCache && Date.now() < tokenCache.expiresAt) return tokenCache.accessToken;
  const res = await axios.post<{ access_token: string; expires_in: number }>(OAUTH_TOKEN_URL, {
    grant_type: 'client_credentials',
    client_id: process.env.COGNIGY_CLIENT_ID,
    client_required_scopes: 'entities:write entities:read'
  });
  tokenCache = { accessToken: res.data.access_token, expiresAt: Date.now() + (res.data.expires_in * 1000) - 60000 };
  return tokenCache.accessToken;
}

// --- Validation & Normalization ---
const SynonymBatchSchema = z.object({
  entityId: z.string().uuid(),
  language: z.string().length(2),
  synonyms: z.array(z.string().min(1).max(255)).max(500)
});
type SynonymBatch = z.infer<typeof SynonymBatchSchema>;

function normalizeSynonym(input: string): string {
  return input.normalize('NFKC').toLowerCase().replace(/[^\p{L}\p{N}\s]/gu, '').replace(/\s+/g, ' ').trim();
}

function deduplicateAndValidate(raw: string[]): string[] {
  const normalized = raw.map(normalizeSynonym).filter(s => s.length > 0);
  const unique = [...new Set(normalized)];
  if (unique.length !== normalized.length) console.warn(`Removed ${normalized.length - unique.length} duplicates.`);
  return unique;
}

// --- API Operations ---
interface IngestionResult { status: string; ingested_count: number; skipped_duplicates: number; entity_id: string; index_status: string; }

async function postSynonymBatch(client: typeof axios, batch: SynonymBatch, maxRetries: number = 3): Promise<IngestionResult> {
  const url = `/api/v2/entities/${batch.entityId}/synonyms/bulk`;
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      const response = await client.post<IngestionResult>(url, { language: batch.language, synonyms: batch.synonyms });
      if (response.status !== 200 && response.status !== 201) throw new Error(`Unexpected status: ${response.status}`);
      return response.data;
    } catch (error: any) {
      if (error.response?.status === 429 && attempt < maxRetries) {
        await new Promise(r => setTimeout(r, Math.pow(2, attempt) * 1000));
        continue;
      }
      throw error;
    }
  }
  throw new Error('Max retries exceeded.');
}

async function triggerIndexRebuild(client: typeof axios, entityId: string): Promise<void> {
  await client.post(`/api/v2/entities/${entityId}/index/rebuild`);
}

// --- Governance & Sync ---
interface AuditLog {
  timestamp: string; entityId: string; batchId: string; synonymCount: number;
  ingestedCount: number; skippedDuplicates: number; latencyMs: number;
  success: boolean; webhookSynced: boolean;
}

async function syncWebhook(webhookUrl: string, log: AuditLog): Promise<boolean> {
  try {
    await axios.post(webhookUrl, { event: 'synonyms_ingested', payload: log }, { timeout: 5000 });
    return true;
  } catch { return false; }
}

export async function runBulkIngestion(
  entityId: string, language: string, rawSynonyms: string[], webhookUrl: string
): Promise<AuditLog[]> {
  const token = await getAuthToken();
  const client = axios.create({ baseURL: COGNIGY_BASE_URL, headers: { Authorization: `Bearer ${token}` } });
  const normalized = deduplicateAndValidate(rawSynonyms);
  const chunks: string[][] = [];
  for (let i = 0; i < normalized.length; i += 500) chunks.push(normalized.slice(i, i + 500));
  
  const auditLogs: AuditLog[] = [];
  for (const chunk of chunks) {
    const batchId = crypto.randomUUID();
    const start = Date.now();
    try {
      const batch = SynonymBatchSchema.parse({ entityId, language, synonyms: chunk });
      const result = await postSynonymBatch(client, batch);
      await triggerIndexRebuild(client, entityId);
      const log: AuditLog = {
        timestamp: new Date().toISOString(), entityId, batchId,
        synonymCount: chunk.length, ingestedCount: result.ingested_count,
        skippedDuplicates: result.skipped_duplicates, latencyMs: Date.now() - start,
        success: true, webhookSynced: false
      };
      log.webhookSynced = await syncWebhook(webhookUrl, log);
      auditLogs.push(log);
    } catch (error: any) {
      auditLogs.push({
        timestamp: new Date().toISOString(), entityId, batchId: crypto.randomUUID(),
        synonymCount: chunk.length, ingestedCount: 0, skippedDuplicates: 0,
        latencyMs: Date.now() - start, success: false, webhookSynced: false
      });
      console.error(`Batch failed:`, error.response?.data || error.message);
    }
  }
  return auditLogs;
}

// Execution
(async () => {
  const logs = await runBulkIngestion(
    '550e8400-e29b-41d4-a716-446655440000',
    'en',
    ['Apple', 'apple', 'MACINTOSH', 'Fruit', 'fruit'],
    'https://your-external-dictionary.com/api/sync'
  );
  console.log('Audit Log:', JSON.stringify(logs, null, 2));
})();

Common Errors & Debugging

Error: 400 Bad Request

• Cause: Batch size exceeds 500, invalid language code, or malformed entity ID.
• Fix: Verify Zod schema validation passes before network call. Ensure synonyms array length does not exceed 500. Confirm language matches ISO 639-1.
• Code Fix: The SynonymBatchSchema.parse() call throws a ZodError before axios executes. Catch this error and log the specific field violation.

Error: 401 Unauthorized

• Cause: Expired OAuth token or missing entities:write scope.
• Fix: Refresh the bearer token using the getAuthToken() cache mechanism. Verify the client credentials request includes entities:write entities:read.
• Code Fix: The token cache subtracts 60 seconds from expiration. If 401 persists, invalidate the cache manually and force a fresh token request.

Error: 429 Too Many Requests

• Cause: Cognigy.AI rate limit exceeded during rapid batch posting.
• Fix: The retry loop implements exponential backoff. Increase maxRetries or adjust delay multiplier if ingestion volume is high.
• Code Fix: The postSynonymBatch function catches error.response?.status === 429 and sleeps for Math.pow(2, attempt) * 1000 milliseconds before retrying.

Error: 500 Internal Server Error

• Cause: Index corruption, unsupported character sequences, or backend ingestion failure.
• Fix: Validate character normalization pipeline. Remove special characters that bypass NFKC normalization. Check Cognigy.AI workspace logs for index fragmentation.
• Code Fix: The normalizeSynonym function strips non-alphanumeric characters. If 500 persists, isolate the failing chunk and test with a single synonym to identify problematic values.

Official References

• Cognigy.AI REST API v2 Documentation
• Entity Synonyms Bulk Endpoint Reference
• Index Rebuild Operations
• OAuth2 Client Credentials Flow