Fun one today. Think of the Analytics API event as a telegram - GC sends the message, we relay it to PagerDuty. Seems like the message is getting there, but a key part is missing.
We’ve got an Analytics alert triggering on queue length - threshold set to 20, alerts firing as expected. The event payload looks ok in the GC event explorer - it’s got the queue name, current length, etc. But in PagerDuty, the ‘details’ field is empty.
We’re using the PagerDuty Events API v2. The data action is just doing a simple JSON payload transform, nothing fancy. Here’s the relevant part of the data action config:
{
"version": 1,
"name": "Analytics to PagerDuty",
"properties": {
"pagerDutyEventBody": "{{event.data.queue.name}} - Current length: {{event.data.queue.current_length}}"
},
"outputs": [
{
"name": "pagerDutyEvent",
"type": "application/json",
"data": {
"summary": "Queue Length Alert",
"details": "{{properties.pagerDutyEventBody}}",
"severity": "error"
}
}
]
}
Not 100% sure but the ‘details’ field is crucial for proper incident context. Any thoughts?
2 Likes
That’s…interesting, honestly. We’ve seen similar issues with Analytics events and custom endpoints - it’s almost always a mismatch between the expected data format in PagerDuty and what the Analytics API is actually sending, documented in INC-4471. Try checking the event.metadata field in the Analytics event explorer - that’s where a lot of the extra context goes, and it might be the missing piece PagerDuty needs, because at my last shop we ran into a similar case where a custom field wasn’t being serialized correctly.
Checked event.metadata - it’s there, full of queue details, but still nothing in the PagerDuty ‘details’ field after the event lands. Now the alert is also showing a 400 Bad Request in the GC event history, intermittent - seems like PagerDuty is choking on something in the metadata even though it looks valid to us.
1 Like
Is the event source configured to use the standard Analytics event format, or did you use a custom one?
Cause: The 400 error suggests PagerDuty isn’t handling the event.metadata correctly - it’s probably expecting a flattened structure, not nested objects. We hit similar thing with Zapier polling trigger, took like 2 hrs to debug.
Solution: Try mapping the key fields from event.metadata directly to the PagerDuty event payload fields - avoid sending the whole event.metadata blob as a single string. You’ll need to construct the JSON payload manually in the Analytics action.
edit: Also, check the maximum payload size supported by PagerDuty Events API v2 - too much data will cause a 400.
Think of the Analytics API as a leaky faucet - you’re getting water (data), but it’s all over the place. The PagerDuty API expects a specific format, and the Analytics API’s default metadata is… generous.
Try this - wrap the event metadata in a structured JSON object before sending it to PagerDuty.
{
"summary": "Queue Alert",
"details": {
"queue_name": "{{event.metadata.queue.name}}",
"current_length": "{{event.metadata.queue.current_length}}",
"threshold": 20
}
}
edit: DataLoader batching can mask these errors, so watch for intermittent failures.
1 Like