Java Platform SDK connection pool exhaustion with concurrent API calls

General Discussion
1

We are experiencing intermittent timeouts when using the Genesys Cloud Java Platform SDK to fetch routing attributes in parallel. The application initializes a single PlatformClient instance per JVM, but the underlying Apache HttpClient connection pool appears to be maxing out under load. We have configured the HttpClientConfig with a max total connections of 100, yet the logs show ConnectionPoolTimeoutException after roughly 50 concurrent requests. We are currently using PlatformClient.init() with default settings and manually setting the base URL. Is there a recommended way to configure the thread-safe HTTP client within the SDK to handle higher concurrency? We have tried increasing the pool size, but the issue persists. The error occurs specifically during the /api/v2/routing/attributes calls.

2

The Java SDK’s default pool isn’t tuned for high concurrency. You’ll want to inject a custom HttpClientBuilder with setMaxConnPerRoute and setMaxConnTotal set higher. Also, ensure you’re not blocking threads on I/O. Try wrapping your SDK calls in CompletableFuture to keep the event loop free.

3

The connection pool timeout is likely a symptom, not the root cause. The Java SDK’s default HttpClient config is fine for sequential calls, but it chokes on parallel bursts if the threads block waiting for I/O. The point above is correct about CompletableFuture, but there is a bigger issue: you are probably holding onto the PlatformClient instance longer than necessary or not closing the underlying HTTP connections properly after exceptions.

In CXone we handle this by keeping connections short-lived and stateless. In Java, you need to ensure the CloseableHttpClient is being reused correctly, but more importantly, you need to tune the ConnectionKeepAliveStrategy. If the server closes the connection faster than your client expects, you get these timeouts.

Here is a cleaner way to configure the PlatformClient with a custom HttpClientConfig that handles keep-alive and retries better. This approach avoids the pool exhaustion by being more aggressive with connection recycling.

import com.genesys.cloud.platform.client.v2.Configuration;
import com.genesys.cloud.platform.client.v2.auth.OAuth;
import com.genesys.cloud.platform.client.v2.client.ApiClient;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;

PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager();
connManager.setMaxTotal(200); // Increase total pool
connManager.setDefaultMaxPerRoute(50); // Increase per-route limit

CloseableHttpClient httpClient = HttpClients.custom()
 .setConnectionManager(connManager)
 .setKeepAliveStrategy((response, context) -> 30000L) // 30s keep-alive
 .build();

ApiClient apiClient = new ApiClient();
apiClient.setHttpClient(httpClient);
Configuration.setDefaultApiClient(apiClient);

Also, check if you are making synchronous calls inside a thread pool. If so, switch to the async methods provided by the SDK. The RoutingApi has getRoutingUserAsync which returns a Future. This keeps the HTTP client threads free while waiting for the response. Blocking threads on network I/O is what kills your pool.

4

Actually, the issue wasn’t the pool size. It was thread blocking. Wrapping the SDK calls in CompletableFuture like As noted above fixed it immediately. The connections were staying open because the threads were waiting on synchronous responses. Once we made it async, the pool cleared out and the timeouts vanished.