This guide will discuss the various options in tuning the JVM for TORO Integrate JVM.
While we would love to provide one JVM configuration that fits all use cases, unfortunately there is none. These tips are intended to assist you in optimizing the JVM but note that different usage patterns and application design will result in different optimizations for each user.
Tuning requires more than just a basic knowledge of Java. It requires the user to have a proper understanding of their production environment, JVM internal memory management, and testing methodology. It is recommended to have a thorough understanding of these topics as attempting to tune an application with no technical know-how might end up doing more harm than good.
Although JVM tuning is a broad topic, this article mainly refers to issues related to the JVM Garbage Collector. One of the major factors that contributes to JVM performance degragation is the Garbage Collector.
Garbage Collector and Java Heap Size
One of the benefits of running an application on a JVM is that under the covers there is a memory management tool called a Garbage Collector running to free up unused Java objects (Java objects that are no longer referenced at any point of the program) in the Java Heap where all live objects, dead objects, and free memory is located. However while this idea sounds appealing it comes with a cost; cleaning unused Java objects in the heap space causes all application threads to stop until the cleaning operation completes, this is called a stop-the-world event. The duration of the pause depends on the machine, JVM configuration and number of objects stored in the memory. The stop-the-world event is the major factor in degrading the overall performance of an application.
Whilst not complete, we'll discuss the two important parts of the Java Heap:
The Young Generation is where all new objects are allocated and aged. When the young generation fills up, this causes a minor garbage collection. Minor collections can be optimized assuming a high object mortality rate. A young generation full of dead objects is collected very quickly. Some surviving objects are aged and eventually moved to the old generation.
The Old Generation is used to store long surviving objects. Typically, a threshold is set for young generation object and when that age is met, the object gets moved to the old generation. Eventually the old generation needs to be collected. This event is called a major garbage collection.
Often a major collection is much slower because it involves all live objects. So for responsive applications, major garbage collections should be minimized. Also note, that the length of the stop-the-world event for a major garbage collection is affected by the kind of garbage collector that is used for the old generation space.
There are two Factors that affects garbage collection performance:
Garbadge collection performance is affected by the total available memory. Because collections occur when generations fill up, throughput is inversely proportional to the amount of memory available. One common resolution in improving the performance of an application is by giving it more memory to work with.
Garbage collection performance is affected by the proportion of the heap dedicated to the young generation. The bigger the young generation, the less often minor collections occur. However, for a bounded heap size, a larger young generation implies a smaller old generation, which will increase the frequency of major collections. The optimal choice depends on the lifetime distribution of the objects allocated by the application.
Java Heap Tuning
One of the easiest ways to tune TORO Integrate is by giving more memory however the more memory you allocate the more garbage the collector will have to clean. JVM Heap size determines how long and how often the garbadge collector spends on cleaning unused java objects. The goal in choosing the optimal heap size is to minimize the time spent on garbage collection versus the time spent on the application.
The JVM argument to set the heap size
-Xmx<size>sets the initial and minimum heap size
-Xmx<size>sets the maximum heap size
java -Xmx2g -Xms2g TOROIntegrate
-Xms size with equal values will yield higher performance throughput than setting the value
differently as this prevents JVM from spending time incrementing the allocated heap.
Garbage Collector Tuning
When increasing the JVM memory is no longer a valid resolution for your application, you can start tuning your garbage collector to improve the pause time or frequency of stop-the-world events.
The first step in the tuning your garbage collector is to choose the type of garbage collector that suits your production environment.
Choosing the right garbage collector
When does the choice of a garbage collector matter? For some applications, the answer is never. That is, the application can perform well in the presence of garbage collection with pauses of modest frequency and duration. However, this is not the case for a large class of applications, particularly those with large amounts of data (multiple gigabytes), many threads, and high transaction rates.1
There are 2 types of collectors, a throughput oriented collector and a low pause collector. Throughput is defined by the application total time that is not spent in the garbage collector. Pause-time is defined by how long the stop-the-world event takes during garbage collection.
A simple way to choose between the different types of garbage collection is to consider the requirements of your production environment. If you want TORO Integrate to perform as much work as possible and can withstand recurring longer pauses then a suitable garbage collector is a throughput oriented collector. If, however, your production environment prioritizes low latency then the low pause collector may be more suitable.
Throughput Oriented Collectors
- Serial Collector - this is used to perform all garbage collection work using a single thread. This works best on
a single processor machine as it cannot take advantage of multiprocessor hardware, although it can be useful on
multiprocessor if the dataset is less than 100 MB. This is enabled by the JVM argument
- Parallel Collector (AKA Throughput collector) - this performs minor collections in parallel, which significantly
reduces garbage collection overhead. This garbage collector is best suited if your production environment runs on a multiprocessor with
medium to large sized data sets. This is enabled by the JVM argument
XX:+UseParallelGC. This argument however only uses parallel garbage collection on minor collections, in major collection you will need to explicitly enable it with
Low Pause Collectors (Response Time Oriented Collector)
- Concurrent Mark-and-Sweep Collector - Designed to perform the cleaning work concurrent with the application.
This still requires the application to pause for certain cleaning phases. This is enabled by
`-XX:+UseConcMarkSweepGC` JVM argument
- Garbage First Collector - Another alternative for a concurrent garbage collector is the new G1. This is the default garbage collector of Java 9
and said to replace the CMS GC. This is enabled by
Tuning The Garbage Collector
After choosing the garbage collector that suits your production environment, the most reliable way to know you made the right choice is by profiling TORO Integrate. Don't just decide, configure and pray that the garbage collector you selected is the best one available for your production environment. In the world of performance tuning there is one ironclad rule, Never guess. Profile the application.
Sometimes the default configuration of garbage collection is not enough or adding more memory is not an option. One suggestion we'd make is to optimize garbage collection young generation which is considered the second most influencial factor in the performance of the garbage collector. Please refer to Oracle's quick guide.
There are three tips that can help in tuning your garbage collector:
- Perform tests in a controlled environment. This means that you have to test TORO Integrate in a dedicated environment where nothing but TORO Integrate affects the overall performance.
- Perform tests that mimics a real world scenario.
- Learn how to monitor the garbage collector's logs.
A testing procedure is a repetitive process:
Determine the production environment requirements
Determine the suitable garbage collector for the production environemnt
Configure the garbage collector suitable for the production environment
Enable garbage collector logging on JVM. This can be as simple as enabling the JVM Option
Conduct tests using a test case that mimics a real world scenario. A good example is executing REST calls. This might be a good test procedure to follow.
Analyze the garbage collect logs. You can make use of a user friendly automated GC log analyzer to do the job for you. Decide whether the performance is valid for your production environment. Two factors you can consider if you need to tune garbage collection further is to decide whether the application pause duration is too long or garbage collection occurs too frequently. If it did then repeat step 2.
To provide an example of tuning, we'll use the configuration and testing procedure of one of our performance tuning guidelines, REST Tuning.
In this test, we'll turn off both tracker and monitor and modify the test script to return a 29.127KB json response and measure the performance by their Throughput results
The test case will be showcasing the difference between two configuration
The difference between the two configuration is minimal that is because the only significant process that consumes precious memory during the test is serving and receiving REST calls. In a real world scenario TORO Integrate will most likely have more work to do than just serving and receiving a single REST endpoint.
To guide you in your journey in optimizing JVM, here's a few resource that might help