Skip to content

Overview

Concepts and Principles

Development

Overview

IDEs

API Explorer

Releases

Release Notes

TORO Integrate

Coder Studio

Coder Cloud

Bug Reports

Search

JVM Tuning

This guide will discuss the various options in tuning the JVM for TORO Integrate JVM.

While we would love to provide one JVM configuration that fits all use cases, unfortunately there is none. These tips are intended to assist you in optimizing the JVM but note that different usage patterns and application design will result in different optimizations for each user.

Performance Tuning

Tuning requires more than just a basic knowledge of Java. It requires the user to have a proper understanding of their production environment, JVM internal memory management, and testing methodology. It is recommended to have a thorough understanding of these topics as attempting to tune an application with no technical know-how might end up doing more harm than good.

Although JVM tuning is a broad topic, this article mainly refers to issues related to the JVM Garbage Collector. One of the major factors that contributes to JVM performance degragation is the Garbage Collector.

Garbage Collector and Java Heap Size

Crash course

One of the benefits of running an application on a JVM is that under the covers there is a memory management tool called a Garbage Collector running to free up unused Java objects (Java objects that are no longer referenced at any point of the program) in the Java Heap where all live objects, dead objects, and free memory is located. However while this idea sounds appealing it comes with a cost; cleaning unused Java objects in the heap space causes all application threads to stop until the cleaning operation completes, this is called a stop-the-world event. The duration of the pause depends on the machine, JVM configuration and number of objects stored in the memory. The stop-the-world event is the major factor in degrading the overall performance of an application.

Whilst not complete, we'll discuss the two important parts of the Java Heap:

java-memory

The Young Generation is where all new objects are allocated and aged. When the young generation fills up, this causes a minor garbage collection. Minor collections can be optimized assuming a high object mortality rate. A young generation full of dead objects is collected very quickly. Some surviving objects are aged and eventually moved to the old generation.

The Old Generation is used to store long surviving objects. Typically, a threshold is set for young generation object and when that age is met, the object gets moved to the old generation. Eventually the old generation needs to be collected. This event is called a major garbage collection.

Often a major collection is much slower because it involves all live objects. So for responsive applications, major garbage collections should be minimized. Also note, that the length of the stop-the-world event for a major garbage collection is affected by the kind of garbage collector that is used for the old generation space.

There are two Factors that affects garbage collection performance:

  1. Garbadge collection performance is affected by the total available memory. Because collections occur when generations fill up, throughput is inversely proportional to the amount of memory available. One common resolution in improving the performance of an application is by giving it more memory to work with.

  2. Garbage collection performance is affected by the proportion of the heap dedicated to the young generation. The bigger the young generation, the less often minor collections occur. However, for a bounded heap size, a larger young generation implies a smaller old generation, which will increase the frequency of major collections. The optimal choice depends on the lifetime distribution of the objects allocated by the application.

Java Heap Tuning

One of the easiest ways to tune TORO Integrate is by giving more memory however the more memory you allocate the more garbage the collector will have to clean. JVM Heap size determines how long and how often the garbadge collector spends on cleaning unused java objects. The goal in choosing the optimal heap size is to minimize the time spent on garbage collection versus the time spent on the application.

The JVM argument to set the heap size

  • -Xmx<size> sets the initial and minimum heap size
  • -Xmx<size> sets the maximum heap size

For example:

java -Xmx2g -Xms2g TOROIntegrate

Tip

Setting the -Xmx and -Xms size with equal values will yield higher performance throughput than setting the value differently as this prevents JVM from spending time incrementing the allocated heap.

Garbage Collector Tuning

When increasing the JVM memory is no longer a valid resolution for your application, you can start tuning your garbage collector to improve the pause time or frequency of stop-the-world events.

The first step in the tuning your garbage collector is to choose the type of garbage collector that suits your production environment.

Choosing the right garbage collector

When does the choice of a garbage collector matter? For some applications, the answer is never. That is, the application can perform well in the presence of garbage collection with pauses of modest frequency and duration. However, this is not the case for a large class of applications, particularly those with large amounts of data (multiple gigabytes), many threads, and high transaction rates.1

Available Collectors

There are 2 types of collectors, a throughput oriented collector and a low pause collector. Throughput is defined by the application total time that is not spent in the garbage collector. Pause-time is defined by how long the stop-the-world event takes during garbage collection.

A simple way to choose between the different types of garbage collection is to consider the requirements of your production environment. If you want TORO Integrate to perform as much work as possible and can withstand recurring longer pauses then a suitable garbage collector is a throughput oriented collector. If, however, your production environment prioritizes low latency then the low pause collector may be more suitable.

Throughput Oriented Collectors

  • Serial Collector - this is used to perform all garbage collection work using a single thread. This works best on a single processor machine as it cannot take advantage of multiprocessor hardware, although it can be useful on multiprocessor if the dataset is less than 100 MB. This is enabled by the JVM argument -XX:+UseSerialGC.
  • Parallel Collector (AKA Throughput collector) - this performs minor collections in parallel, which significantly reduces garbage collection overhead. This garbage collector is best suited if your production environment runs on a multiprocessor with medium to large sized data sets. This is enabled by the JVM argument XX:+UseParallelGC. This argument however only uses parallel garbage collection on minor collections, in major collection you will need to explicitly enable it with -XX:-UseParallelOldGC.

Low Pause Collectors (Response Time Oriented Collector)

  • Concurrent Mark-and-Sweep Collector - Designed to perform the cleaning work concurrent with the application. This still requires the application to pause for certain cleaning phases. This is enabled by `-XX:+UseConcMarkSweepGC ` JVM argument
  • Garbage First Collector - Another alternative for a concurrent garbage collector is the new G1. This is the default garbage collector of Java 9 and said to replace the CMS GC. This is enabled by -XX:+UseG1GC argument.
Tuning The Garbage Collector

After choosing the garbage collector that suits your production environment, the most reliable way to know you made the right choice is by profiling TORO Integrate. Don't just decide, configure and pray that the garbage collector you selected is the best one available for your production environment. In the world of performance tuning there is one ironclad rule, Never guess. Profile the application.

Sometimes the default configuration of garbage collection is not enough or adding more memory is not an option. One suggestion we'd make is to optimize garbage collection young generation which is considered the second most influencial factor in the performance of the garbage collector. Please refer to Oracle's quick guide.

Tuning Tips

There are three tips that can help in tuning your garbage collector:

  1. Perform tests in a controlled environment. This means that you have to test TORO Integrate in a dedicated environment where nothing but TORO Integrate affects the overall performance.
  2. Perform tests that mimics a real world scenario.
  3. Learn how to monitor the garbage collector's logs.

Testing Procedure

A testing procedure is a repetitive process:

  1. Determine the production environment requirements

  2. Determine the suitable garbage collector for the production environemnt

  3. Configure the garbage collector suitable for the production environment

  4. Enable garbage collector logging on JVM. This can be as simple as enabling the JVM Option -XX:+PrintGCDetails-XX:+PrintGCDateStamps-XX:+PrintGCTimeStamps -Xloggc:gclog.log

  5. Conduct tests using a test case that mimics a real world scenario. A good example is executing REST calls. This might be a good test procedure to follow.

  6. Analyze the garbage collect logs. You can make use of a user friendly automated GC log analyzer to do the job for you. Decide whether the performance is valid for your production environment. Two factors you can consider if you need to tune garbage collection further is to decide whether the application pause duration is too long or garbage collection occurs too frequently. If it did then repeat step 2.

Tuning Example

To provide an example of tuning, we'll use the configuration and testing procedure of one of our performance tuning guidelines, REST Tuning.

In this test, we'll turn off both tracker and monitor and modify the test script to return a 29.127KB json response and measure the performance by their Throughput results

Test Script

The test case will be showcasing the difference between two configuration

  1. -Xmx1g -Xms1g
  2. -Xmx2g -xms2g

Test Results

jvm-heap-test-table

jvm-heap-test-chart

The difference between the two configuration is minimal that is because the only significant process that consumes precious memory during the test is serving and receiving REST calls. In a real world scenario TORO Integrate will most likely have more work to do than just serving and receiving a single REST endpoint.

Helpful resources

To guide you in your journey in optimizing JVM, here's a few resource that might help