POST /api/v2/conversations/calls 400 - Malformed participant address

Trying to create a call via the REST API. Getting a 400 Bad Request with error code malformed_participant_address. Here is the payload:

{
 "to": { "phoneNumber": "+15550199" },
 "from": { "phoneNumber": "+15550100" }
}

The numbers look valid. Tried removing the +. Tried using uri instead of phoneNumber. Still getting the same error. Is there a specific format for the participant object in this endpoint?