Java Flight Recorder (JFR)

Java flight recorder is an in-built event-based tool in the JVM for profiling Java applications. JFR collects diagnostic and profiling data about JVM and a running Java application. It is considered to be low overhead (<2%).

JFR continuously collects, saves data, and threads stacks in a buffer for the running application.

JFR Events

Events are central to the JFR. Each JFR event has a name, timestamp, and optional payload (Ex: CPU usage event). Most events also have information about the thread in which the event happened, stack trace at the time of the event, and the duration of the event.

There are primarily three types of events

· Duration Event: These events allow setting a threshold. The events longer than the threshold are recorded and logged when it completes. Ex:

· Instant Event: These happen instantly and is logged right away. Ex:

· Sample Event: These are logged at regular intervals of time. The sampling interval could be configured. Ex:

To limit the overhead, it is recommended to limit the type of recorded events to what is really needed.

JFR Data Flow

JFR collects data from JVM and Java applications. This data is stored in a small thread-local buffer, and those get flushed to a global in-memory buffer, which gets written to the repository/disk (this is different from actual JFR recording). Writing to the disk could be a bit expensive. This could be mitigated, by reducing the number of events to be recorded or by keeping only a circular in-memory buffer. In Java14, data gets written to the repository per second or when the global buffer is full.

How to enable JFR?

JFR can be enabled with -XX:+FlightRecorder JVM argument. Each recording could be started with appropriate settings through jcmd tool

Ex: jcmd 1234 JFR.start -XX:StartFlightRecording=duration=60s,filename=myrecording.jfr

Other relevant commands are JFR.check, JFR.stop, JFR.dump. maxsize, maxage, delay, and compress are some of the useful settings for recording configuration

We could also create triggers (or rules) with certain conditions (ex: heap over 1.6g) and trigger recordings only when those conditions are met.

How to Profile CPU with JFR?

CPU profiling in JFR uses Execution Sample Event. It gives an idea of where the JVM is spending the most CPU time during the execution of java code. It is cheap and has low overhead. This does not include native calls and threads. CPU time for Java code may be enough for some cases. But, for the use cases, which require profiling both Java and Native code, the async profiler is probably better suited as it uses kernel profiling data and lines it up with the information from AsyncGetCallTrace.

JFR Custom Events

JDK provides many(over 140) pre-existing events. Such as memory allocation, thread creation, GC, etc. JDK also makes it easy to create custom events to meet application-specific needs. Custom events can be created by extending the Event class, then those events can be triggered by initializing them in the appropriate code flow, then wrapping the code with begin() and commit() methods. The default behavior is to capture execution time.

import jdk.jfr.Event;
@Name(“com.sri.CoolEvent”)
@Label(“Sri Cool Event”)
public class MyCustomEvent extends Event {
}
public void testMethod(){
 MyCustomEvent event = new MyCustomEvent();
 event.begin();
 
 //actual work
 event.end(); //Optional
 event.commit();
}

Threshold annotation allows controlling when an event needs to be triggered. JDK ships with JFR binary. We could use that to read a recording. for example, to print MyCustomEvents from a recording we could run the following command

jfr print — events MyCustomEvent jfr_dump.jfr

Compare different JFR recordings

Java provides a rich set of APIs for interacting with JFR data programmatically. This way, JFR recordings could be analyzed programmatically in a headless way.

IItemCollection events = JfrLoaderToolkit.loadEvents(new File(args[0]));

Java Mission Control (JMC) and IntelliJ are excellent for viewing JFR recordings. IntelliJ makes it really easy to compare 2 different JFR recordings.

Streaming JFR events

Until Java 14, to consume JFR data, a user must start a recording, stop it, dump the contents to disk and then parse the recording file. This works well for application profiling but not for monitoring purposes. JEP349 in Java14 adds support for streaming JFR events and allows for registering callbacks. It works by allowing streaming while writing to the file. It does not support streaming directly from in-memory buffers. Events are not delivered synchronously when they are occurring in the JVM, but still, this is really helpful in profiling a live instance.

Here is a quick example of using EventStream to get some other attributes of a running Java process during specific events. Such as CPU load and garbage collections.

try (var es = EventStream.openRepository()) {
   es.onEvent("jdk.CPULoad", event -> {
     System.out.println("CPU Load " + event.getEndTime());
     System.out.println(" Machine total: " + 100 * event.getFloat("machineTotal") + "%");
     System.out.println(" JVM User: " + 100 * event.getFloat("jvmUser") + "%");
     System.out.println(" JVM System: " + 100 * event.getFloat("jvmSystem") + "%");

   });
   es.onEvent("jdk.GarbageCollection", event -> {
     System.out.println("Garbage collection: " + event.getLong("gcId"));
     System.out.println(" Cause: " + event.getString("cause"));
     System.out.println(" Total pause: " + event.getDuration("sumOfPauses"));
   });
   es.start();
 }

JFR streaming makes it easy to act on profiling data in-process, make decisions, and enable/disable additional events on the fly.

Summary

JFR is a low overhead profiling tool. JFR ships with a lot of out-of-the-box profiling events and it is easy to add a new set of events. With the addition of streaming support, it could really help in troubleshooting performance hotspots.