Does anyone know if the /api/v2/analytics/conversations/details/query endpoint strictly requires cursor-based pagination for high-volume historical queries? I am building a pipeline to ingest conversation details for sentiment model training. The documentation mentions both pageSize/pageNumber and nextPageToken. My current implementation uses pageNumber with a size of 5000. It works for the first few calls but starts returning empty arrays or stale data after ~20 pages. The API returns 200 OK, so there is no obvious error code to catch. I suspect the dataset is shifting or the endpoint silently falls back to a different indexing strategy that does not support deep offset pagination reliably.
Here is the relevant snippet from my Python script using requests. I am passing pageNumber and pageSize in the query parameters. I also tried switching to nextPageToken when provided in the response headers, but the token often seems to expire or become invalid if the request takes too long. Is there a specific pattern for handling this? I need to ensure I capture every conversation ID without duplicates or gaps for my NLP pipeline. Any insights on the internal mechanics of this endpoint would be appreciated.