How to manage OAuth client secrets in Terraform without them appearing in the state file

How-To-Articles
1

How to manage OAuth client secrets in Terraform without them appearing in the state file

What You Will Build

  • You will create a Terraform configuration that provisions a Genesys Cloud CX OAuth Client and stores its secret in a remote secret store (such as HashiCorp Vault or AWS Secrets Manager) instead of the Terraform state file.
  • This tutorial uses the Genesys Cloud CX Terraform Provider and the HashiCorp Vault Terraform Provider.
  • The programming languages covered are HCL (HashiCorp Configuration Language) and Python (for verification).

Prerequisites

  • OAuth Client Type: You need a Genesys Cloud CX environment with administrative privileges to create OAuth Clients and API Keys.
  • Terraform Version: 1.5 or higher.
  • Providers:
    • genesyscloud provider (version 1.0.0 or higher).
    • vault provider (version 4.0.0 or higher).
  • Secrets Backend: A running instance of HashiCorp Vault with KV Secrets Engine v2 enabled. Alternatively, you can adapt the pattern for AWS Secrets Manager or Azure Key Vault using their respective providers.
  • Authentication:
    • Genesys Cloud: API Key (Client ID/Secret) or OAuth Token with oauth:client:write and oauth:client:read scopes.
    • Vault: AppRole authentication or Token authentication with write permissions on the KV path.

Authentication Setup

Terraform requires authentication for both the Genesys Cloud provider and the Vault provider. You must configure these credentials securely. Never hardcode secrets in the .tf files. Use environment variables or a .tfvars file that is ignored by version control.

Genesys Cloud Provider Configuration

The Genesys Cloud provider supports multiple authentication methods. For programmatic access, using an API Key is standard.

# main.tf

terraform {
  required_providers {
    genesyscloud = {
      source  = "myntra/genesyscloud"
      version = "~> 1.0"
    }
    vault = {
      source  = "hashicorp/vault"
      version = "~> 4.0"
    }
  }
}

provider "genesyscloud" {
  # Use environment variables for credentials
  # export GENESYS_CLOUD_API_KEY="your_api_key"
  # export GENESYS_CLOUD_REGION="us-east-1"
  
  api_key = var.genesys_cloud_api_key
  region  = var.genesys_cloud_region
}

provider "vault" {
  # Use environment variables for Vault address and token
  # export VAULT_ADDR="https://vault.example.com"
  # export VAULT_TOKEN="your_vault_token"
  
  address = var.vault_address
  token   = var.vault_token
}

Vault Authentication

Ensure your Vault token has the necessary policies. For this tutorial, assume a policy that allows writing to secret/data/genesys/oauth/*.

{
  "path": "secret/data/genesys/oauth/*",
  "capabilities": ["create", "read", "update", "delete"]
}

Implementation

Step 1: Provision the OAuth Client in Genesys Cloud

First, create the OAuth Client resource. The critical step here is ensuring that the secret attribute is not stored in the Terraform state. The Genesys Cloud provider supports sensitive attributes, but Terraform state files are still encrypted only at rest (if remote backend encryption is enabled) and can be decrypted by anyone with access to the state backend. To truly remove the secret from the state, we must not store it in the Genesys Cloud resource definition directly if possible, or we must immediately export it and suppress it from state updates.

However, the Genesys Cloud API returns the secret only during creation. The Terraform provider captures this. To prevent it from appearing in the state file in a readable way, we use the sensitive flag, but this only masks it in CLI output. It does not remove it from the state file storage.

The robust solution is to create the client, extract the secret immediately using a local or data source, write that secret to Vault, and then ensure the Genesys Cloud resource does not hold the secret in a way that requires state reconciliation. Note: The Genesys Cloud provider requires the secret to be known during creation to return it. We will create the resource, capture the secret, and write it to Vault.

# resources.tf

# Variable definitions
variable "genesys_cloud_api_key" {
  type        = string
  description = "Genesys Cloud API Key"
  sensitive   = true
}

variable "genesys_cloud_region" {
  type        = string
  default     = "us-east-1"
  description = "Genesys Cloud Region"
}

variable "vault_address" {
  type        = string
  description = "Vault Address"
  sensitive   = true
}

variable "vault_token" {
  type        = string
  description = "Vault Token"
  sensitive   = true
}

variable "oauth_client_name" {
  type        = string
  default     = "TerraformManagedClient"
  description = "Name of the OAuth Client"
}

# 1. Create the OAuth Client
# The 'secret' attribute is returned by the API only at creation time.
# We mark it as sensitive to prevent accidental logging.
resource "genesyscloud_oauth_client" "this" {
  name        = var.oauth_client_name
  description = "Managed by Terraform"
  redirect_uris = ["https://myapp.example.com/callback"]
  grant_types = ["client_credentials"]
  
  # Important: The provider will store the secret in the state file.
  # We will mitigate this by immediately exporting it to Vault.
}

Step 2: Extract and Store the Secret in Vault

Now that the OAuth Client is created, we need to take the secret attribute from the genesyscloud_oauth_client.this resource and write it to HashiCorp Vault. This allows us to retrieve the secret later from Vault rather than relying on the Terraform state file.

We use the vault_kv_secret_v2 resource to store the secret. We also use a local value to reference the secret from the Genesys resource.

# secrets.tf

# Write the OAuth Secret to Vault KV v2
resource "vault_kv_secret_v2" "genesys_oauth_secret" {
  namespace    = "production" # Optional: specify namespace if used
  mount        = "secret"     # The KV secrets engine mount path
  name         = "genesys/oauth/${var.oauth_client_name}"
  
  data_json = jsonencode({
    client_id    = genesyscloud_oauth_client.this.client_id
    client_secret = genesyscloud_oauth_client.this.secret
    environment  = var.genesys_cloud_region
  })
  
  # Ensure the secret is not stored in Terraform state in plain text
  # Note: Vault provider stores the data in state if not handled carefully.
  # However, we are moving the source of truth to Vault.
}

Critical Note on State Sensitivity: Even with sensitive flags, Terraform state files contain the raw values. To completely exclude the secret from the Genesys Cloud state, we can use a workaround: create the client, read the secret, store it in Vault, and then use a null_resource with a local-exec provisioner to rotate the secret or simply acknowledge that the secret is now in Vault. However, the Genesys Cloud provider does not support “secret rotation” via Terraform without recreating the resource.

A better pattern for “zero secrets in state” is to use the vault provider to generate the secret, but Genesys Cloud requires the client to be created first. Since Genesys Cloud generates the secret, we must accept that the secret will appear in the Genesys Cloud state file unless we use a custom provider or external script.

To adhere to the “no secrets in state” requirement as strictly as possible within standard Terraform capabilities:

  1. Create the Genesys OAuth Client.
  2. Immediately write the secret to Vault.
  3. Use terraform state rm or ignore the attribute in future plans if you manage the secret rotation outside Terraform.

However, a more robust approach for new deployments is to use the external data source to handle the secret storage logic, but the core issue remains: the Genesys provider returns the secret.

To truly remove it from the state file after creation, you can use a post-provisioning step to remove the attribute from the state, but this is fragile. The industry standard best practice when using providers that return secrets is to:

  1. Mark the attribute as sensitive.
  2. Encrypt the remote state backend (S3 + KMS, etc.).
  3. Store the secret in a dedicated secret manager (Vault) for application consumption.

Let us proceed with storing it in Vault for application consumption, which is the primary goal.

Step 3: Verify the Secret in Vault (Optional but Recommended)

To ensure the secret was stored correctly, you can use a null_resource with a local-exec provisioner to print a confirmation message (without printing the secret itself).

# verification.tf

resource "null_resource" "verify_vault" {
  triggers = {
    secret_id = vault_kv_secret_v2.genesys_oauth_secret.id
  }

  provisioner "local-exec" {
    command = "echo 'OAuth Secret for ${var.oauth_client_name} has been stored in Vault.'"
  }
}

Complete Working Example

Below is the complete, copy-pasteable Terraform configuration. Save this as main.tf.

# main.tf

terraform {
  required_providers {
    genesyscloud = {
      source  = "myntra/genesyscloud"
      version = "~> 1.0"
    }
    vault = {
      source  = "hashicorp/vault"
      version = "~> 4.0"
    }
  }

  # Recommended: Use a remote backend with encryption
  backend "s3" {
    bucket         = "my-terraform-state-bucket"
    key            = "genesys-oauth/state.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-locks"
  }
}

# Variables
variable "genesys_cloud_api_key" {
  type        = string
  description = "Genesys Cloud API Key"
  sensitive   = true
}

variable "genesys_cloud_region" {
  type        = string
  default     = "us-east-1"
  description = "Genesys Cloud Region"
}

variable "vault_address" {
  type        = string
  description = "Vault Address"
  sensitive   = true
}

variable "vault_token" {
  type        = string
  description = "Vault Token"
  sensitive   = true
}

variable "oauth_client_name" {
  type        = string
  default     = "TerraformManagedClient"
  description = "Name of the OAuth Client"
}

# Providers
provider "genesyscloud" {
  api_key = var.genesys_cloud_api_key
  region  = var.genesys_cloud_region
}

provider "vault" {
  address = var.vault_address
  token   = var.vault_token
}

# Resources
resource "genesyscloud_oauth_client" "this" {
  name        = var.oauth_client_name
  description = "Managed by Terraform - Secret stored in Vault"
  redirect_uris = ["https://myapp.example.com/callback"]
  grant_types = ["client_credentials"]
  
  # The secret is generated by Genesys Cloud and returned by the API.
  # It is stored in the Terraform state file by the provider.
  # We mitigate this by immediately copying it to Vault.
}

resource "vault_kv_secret_v2" "genesys_oauth_secret" {
  mount        = "secret"
  name         = "genesys/oauth/${var.oauth_client_name}"
  
  data_json = jsonencode({
    client_id    = genesyscloud_oauth_client.this.client_id
    client_secret = genesyscloud_oauth_client.this.secret
    environment  = var.genesys_cloud_region
  })
}

# Output
output "vault_secret_path" {
  value       = vault_kv_secret_v2.genesys_oauth_secret.path
  description = "The path in Vault where the OAuth secret is stored"
}

output "genesys_client_id" {
  value       = genesyscloud_oauth_client.this.client_id
  description = "The Client ID for the Genesys OAuth Client"
}

Running the Configuration

  1. Initialize Terraform:

    terraform init

  2. Plan the changes:

    terraform plan -var="genesys_cloud_api_key=<YOUR_API_KEY>" \
               -var="vault_address=https://vault.example.com" \
               -var="vault_token=<YOUR_VAULT_TOKEN>"

  3. Apply the changes:

    terraform apply -var="genesys_cloud_api_key=<YOUR_API_KEY>" \
                -var="vault_address=https://vault.example.com" \
                -var="vault_token=<YOUR_VAULT_TOKEN>"

Common Errors & Debugging

Error: 403 Forbidden on Vault Write

Cause: The Vault token does not have the necessary permissions to write to the specified KV path.

Fix: Update the Vault policy to include create, update, and delete capabilities for the target path.

# Example Vault Policy HCL
path "secret/data/genesys/oauth/*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}

Error: Genesys Cloud OAuth Client Already Exists

Cause: You attempted to create an OAuth Client with a name that already exists in your Genesys Cloud environment. OAuth Client names must be unique.

Fix: Change the oauth_client_name variable or import the existing resource into Terraform state.

terraform import genesyscloud_oauth_client.this <oauth_client_id>

Error: State File Contains Plain Text Secret

Cause: Even with sensitive flags, the Terraform state file (state.tfstate) contains the raw secret value.

Fix:

  1. Ensure your remote backend is encrypted (e.g., S3 with KMS encryption).
  2. Restrict access to the state file storage bucket.
  3. Do not rely on local state files for production secrets.
  4. To remove the secret from the state file after creation, you can use terraform state rm on the specific resource, but this will cause Terraform to lose track of the resource. A better approach is to accept that the state file contains the secret but protect the state file rigorously, while using Vault as the source of truth for application consumption.

Error: 429 Too Many Requests

Cause: Genesys Cloud API rate limits have been exceeded.

Fix: Implement retry logic in your Terraform configuration or wait before applying. The Genesys Cloud provider has built-in retry logic for some endpoints, but not all.

provider "genesyscloud" {
  # Some providers support retry settings
  # Check the specific provider documentation for retry configuration
}

Official References