What is the recommended pattern for handling pagination when pulling conversation details via the REST API? We are building a Terraform data source to validate historical call data against our IaC state, so we need to fetch every record for a given date range.
The documentation mentions a page parameter, but the response object also includes a nextPageToken. I have seen mixed advice in older forum threads about whether to rely on the token or increment the page count manually.
Here is the current loop structure we are testing in our helper script:
page = 1
token = None
while True:
params = {'page': page}
if token:
params['nextPageToken'] = token
resp = requests.get(url, params=params)
data = resp.json()
process(data['entities'])
if not data.get('nextPageToken'):
break
token = data['nextPageToken']
page += 1
We are noticing that when the dataset is large, the nextPageToken seems to be more reliable than the page number alone, but we are unsure if passing both causes conflicts. The API returns a 200 OK, but we want to ensure we are not missing records or hitting rate limits due to inefficient pagination logic. Is the token the primary driver here?