Snowflake Data Action retry logic failing on warehouse suspension

NICE CXone API & Integrations
1 
body = DataActionRunRequest(input={'sql': 'SELECT * FROM t', 'params': [flow_val]})
resp = api.post_data_actions_run(body)

Throwing 503 when the Snowflake warehouse suspends. JDBC connection uses IAM role auth but session pooling doesn’t hold during retries. Dynamic type coercion for flow inputs fails on the Snowflake dialect while trying to map variables. Cursor management for pagination is messy. Output validation against JSON schema it’s timing out.

2

Problem

Warehouse suspension kills the session pool. It’s dropping your request before the retry kicks in.

Code

# POST /api/v2/data-actions/runs (scope: data-actions:write)
resp = platformClient.post_data_actions_runs(body={'input': {'sql': 'SELECT 1'}})

Error

You’ll still hit 503 if the IAM role misses resume privileges on that warehouse.

Question

Check the outbound rules for the resume endpoint.

3 
curl -X PATCH /api/v2/data-actions/{data_action_id} \
 -H "Authorization: Bearer {access_token}" \
 -H "Content-Type: application/json" \
 -d '{
 "config": {
 "retryCount": 3,
 "retryDelay": 5000
 }
 }'

The suggestion above hits the nail. 503s stem from the action config. Push retry logic to the resource via the API. Don’t patch this in Python. State drift kills prod.

4

Problem

The retry logic resets the context. When the warehouse suspends, the span closes on the first attempt. The retry starts a new trace, so the 503 error becomes an orphan in Jaeger. You lose the correlation to the original flow. The config patch works, but it hides the root cause in the backend.

Code

Wrap the retry in a context manager to keep the span active. Inject the headers manually if the SDK doesn’t carry them over.

from opentelemetry import trace
from opentelemetry.propagate import inject

tracer = trace.get_tracer("snowflake-retry")

# Keep the span open across retries
with tracer.start_as_current_span("snowflake-query") as span:
 headers = {}
 inject(headers)
 
 for attempt in range(3):
 try:
 # Pass headers to ensure trace context sticks
 resp = api.post_data_actions_run(body, headers=headers)
 break
 except Exception as e:
 span.record_exception(e)
 span.set_status(trace.Status(trace.StatusCode.ERROR, str(e)))
 if attempt == 2:
 raise

Error

Orphaned traces with status_code: 503. The parent span shows success while the child retry fails silently in the logs. This usually blows up the dashboard metrics.

Question

Verify the trace ID matches in Jaeger after the retry. Context propagation fails if the headers don’t stick.

5 
{
 "retryOnStatus": [503, 504],
 "maxRetries": 3,
 "resumeWarehouseOn503": true,
 "retryDelay": 12000
}

HTTP 503 Service Unavailable usually means the retry policy is bound to the client session instead of the Data Action resource. That config mistake forces the runner to spin against a suspended cluster and drop the trace context. The patch above handles the basic backoff, but you’re missing the warehouse resume trigger in the Snowflake dialect mapping. Setting resumeWarehouseOn503 to true tells the platform to poll the Snowflake REST API for the active status before firing the next attempt. Without that flag, the retry just burns your rate limit. The tracing gap mentioned earlier closes once the retry scope stays tied to the original execution ID. It doesn’t hurt to bump the retryDelay past 10 seconds either. Check the timeout config and verify it isn’t hardcoded to 5000ms. That kills the resume window.