A Digital Asset Trading Platform Optimizes Java Services: Memory Usage Down 27.5%, EKS Nodes Down 32%+

A digital asset trading platform runs Java services on AWS EKS and already uses Karpenter to manage node autoscaling. During Java startup, class loading, framework initialization, and JIT warmup occur together, raising CPU demand. Once services reach steady state, demand falls, and configuring resources for startup peaks over the long term can lead to overprovisioning.

The platform team wanted resource configuration to follow the runtime phases of its Java services. It also wanted to optimize JVM heap settings and container memory reservations together, so improvements at the application level could free capacity across the cluster.

With CloudPilot AI, the customer reduced Java Memory Used by 27.5%, Java Memory Request by 17.8%, CPU Request by 57.8%, and EKS node count by 32%+.

Overview

About the Customer

The customer is a digital asset trading platform that runs its services on Kubernetes, with Java forming an important part of its application environment. Its platform team manages changing workloads while continuously improving cluster resource efficiency.

The customer already used Karpenter to manage node capacity. The team wanted to close the gap between application resource configuration and actual demand, turning Java optimization into greater efficiency across its EKS cluster.

Results

CloudPilot AI connected JVM memory settings, container resource requests, and cluster capacity. Reducing application memory usage and excess resource reservations gave the customer's existing node autoscaling more room to consolidate capacity.

Customer Problems: Java Resource Settings Did Not Match Actual Demand

Although the customer already used Karpenter, high application CPU and memory requests continued to occupy scheduling capacity. The gap between startup demand, everyday usage, and container reservations limited how far the cluster could consolidate nodes.

High CPU requests remained after the startup peak. Class loading, framework initialization, and JIT warmup concentrate CPU demand during startup. Once services reached steady operation, demand fell, but resource configuration did not sufficiently distinguish the two phases. Capacity reserved for startup continued to be reserved throughout normal operation.

Both JVM memory usage and container reservations needed to decrease. The Java services had heap settings that could be optimized alongside high memory requests. Heap settings influence application memory consumption, while requests determine the capacity Kubernetes reserves. Adjusting both was necessary to free actual memory and scheduling capacity together.

The platform team needed to start with its Java applications: align CPU configuration with each runtime phase, tune JVM memory and container requests to actual usage, and make the released capacity available for node consolidation.

The Solution: Manage CPU by Runtime Phase and Tune JVM and Container Resources Together

CloudPilot AI brings Java resource management into the application's runtime lifecycle. Its capabilities distinguish startup CPU from steady operation, coordinate JVM heap settings with container memory, and connect workload resource requests with cluster capacity.

Java Optimization Solution: Coordinate JVM Heap and Startup CPU

ResourceStartupBoost: Provide CPU at Startup and Restore Normal Requests After Sustained Readiness

To meet the startup needs of Java services, CloudPilot AI provides ResourceStartupBoost. During Pod startup, the platform temporarily increases CPU Request, reserving additional CPU resources for class loading, framework initialization, and warmup.

After both the minimum startup duration and the continuous Ready window are satisfied, the platform restores the normal Request through an in-place resource adjustment. Temporary resources needed for startup are therefore managed separately from everyday resource configuration, so an application does not need to reserve the same capacity over the long term for the resource peak of a single startup.

Temporarily provide additional CPU at startup, then restore normal settings after sustained readiness. Teams can manage startup and everyday resource requirements separately, reducing long-term reservations.

Java Memory Optimization: Tune the Heap and Container Together, Reducing Usage by 27.5%

CloudPilot AI's JVM heap optimization combines actual heap usage, garbage collection behavior, and non-heap memory usage to generate recommendations for Xms, Xmx, and container memory. Xms sets the initial heap size, while Xmx sets its upper limit. Accounting for garbage collection needs and non-heap memory helps reduce idle heap capacity while aligning JVM configuration with application demand.

In this customer's Java memory optimization, the team adjusted JVM parameters and container resource configuration together: new heap settings were applied when new Pods started, and Memory Request was calibrated at the same time. Bringing actual application memory usage and Kubernetes scheduling reservations down together allowed Java tuning to free more space for the cluster.

After optimization, Java Memory Used decreased by 27.5% and Java Memory Request decreased by 17.8%. Java services consumed less memory, while their container resource configuration reserved less capacity across the cluster.

Java: Actual memory usage and recommended configuration decrease in steps

The green Used line shows a clear step down, followed by operation within a lower range. The pink Recommended line also decreases. Showing usage and recommendations together lets the team see how Java memory optimization progresses.

Workload Autoscaler: Continuously Calibrate Requests, Reducing CPU Request by 57.8%

CloudPilot AI Workload Autoscaler calibrates Request based on actual application CPU and memory usage, reducing overprovisioning accumulated over time. For the platform team, this creates an ongoing connection between Java runtime needs and Kubernetes scheduling configuration.

In this customer's workload optimization, CPU Request decreased by 57.8%. More accurate requests reduced the capacity Kubernetes needed to reserve, allowing workloads to fit more closely across the cluster.

CPU: Original reservations, current requests, and actual usage

The platform view shows 3,370.17 Cores/h of original CPU reservations and a current Request of 1,420.71 Cores/h. The blue current Request line sits well below the orange original reservation line and closer to green actual usage, showing the reduction in CPU overprovisioning.

Working with the Customer's Existing Karpenter: EKS Node Count Decreases by 32%+

After workload resource configuration moved closer to actual demand, the customer's existing Karpenter setup gained more room to consolidate nodes. CloudPilot AI's application resource optimization worked with this mechanism, driving an EKS node count reduction of 32%+.

Cluster: Lower CPU requests are followed by lower allocatable capacity

The blue Requested line decreases first, followed by the orange Allocatable line, while the green Usage line remains within a similar range. The view shows how lower workload requirements create room for cluster capacity to consolidate.

Extend Java Optimization Across Applications and the Cluster

The key to Java resource management is keeping configuration aligned with actual application demand. CloudPilot AI connects startup CPU, steady-state Requests, and JVM memory settings, helping teams manage resources according to how their applications run.

With CloudPilot AI, the customer reduced actual Java memory usage, container memory reservations, and workload CPU Requests, and reduced node count by 32%+. For the platform team, the value of Java tuning extended from individual applications to the EKS cluster: resource settings moved closer to actual usage, and the existing node autoscaling could be used more fully.

About CloudPilot AI

CloudPilot AI is a cloud infrastructure company based in San Francisco, specializing in autonomous Kubernetes optimization for enterprises operating at scale.

Its mission: Autoscaling Kubernetes for the most demanding teams.

The platform continuously optimizes workload resources, adjusts node capacity, accelerates container image delivery, automates workload placement, and enables high-performance local storage. By coordinating applications and infrastructure, CloudPilot AI helps SRE and platform teams reduce cloud costs, improve application performance, and support service reliability across multiple clouds.