Need some troubleshooting help with a daily analytics export job that writes to S3 using Python and boto3. I am building a script to pull conversation metrics from Genesys Cloud and push them to our data lake. The issue arises when the dataset exceeds a few thousand records. The Genesys Cloud Python SDK seems to hang or timeout during the pagination loop, even though the initial request succeeds.
Here is the core logic I am using to fetch the data:
analytics_client = platform_client.analytics_api
body = analytics_conversations_query_parameters(
date_from='2023-10-01T00:00:00.000Z',
date_to='2023-10-02T00:00:00.000Z',
interval='PT1H',
size=1000
)
response = analytics_client.post_analytics_conversations_queries(body=body)
I then iterate through response.entities and upload each chunk to S3 using boto3.client('s3').put_object(). The upload works fine for small sets, but for larger exports, I get a requests.exceptions.ReadTimeout after the first page.
- Increased the
timeoutparameter in theGenesysCloudPlatformClientconstructor to 60 seconds, but the error persists at the same interval. - Verified that the S3 bucket permissions are correct by testing a direct upload from a static JSON file, which succeeded without issues.
Is there a recommended pattern for handling large dataset pagination in the Python SDK to avoid these timeouts?