navigate_next

Java

Enhanced Observability with Java 21 and Spring Boot 3.2

Pratik Dwivedi

December 18, 2023

In this post, we discuss the enhanced Observability support in Spring Boot 3.2 and Java 21. By leveraging the advanced features of Java 21 and Spring Boot 3.2, developers can enhance the observability of their systems and effectively monitor various metrics and logs. Java 21 also introduces Virtual threads and Structured concurrency, which in turn make Observability easier and clearer.

What is Observability?

Observability is the ability to monitor a system just by looking at its external features/outputs. On software systems, Observability is used to determine how a running system fulfils requests, responds to events, how its interfaces/modules interact with each other, its responsiveness when handling a huge number of user requests, throughput at different loads etc. Observability traces your code's performance, responsiveness stability etc.; in all the locations or running modules at the same time, thereby enabling a bird's eye view of how your code is performing.

A core actor in Observability is a "trace"; as a request/event propagates in the software system, the actions it triggers OR the response it generates from the system is tracked via a trace. So a "trace" provides visibility into how requests behave as they move throughout the entire software system.

Benefits of Observability:-

Observability clearly shows the performance patterns that are emerging, the time taken by processes, busy or idle endpoints, and parallelly running queries/clients etc. All this is reported in clear and concise numbers or figures, e.g.

the slowest and the fastest performance of a frequently running thread in milliseconds
or how many times a service was called and the duration it was in action
if a certain utility module is experiencing an unusual workload

All this performance data is aggregated and presented visually, for discerning patterns and trends easily. This information lets you know about any possible bottlenecks, possibilities for improvements, and if the memory allocation is optimal.

Importance of benchmarking your current Java/J2EE software stack

It's necessary to benchmark your current Java setup to know how well it's performing as the sum of its parts, how individual modules and processes are performing, and the overall behaviour of your system when juxtaposed with the demands and expectations that it should deliver.

Software these days is run in complex environments; like SAAS offerings, Mobile apps and their servers, code copies running on many Virtual machines and software running on an application stack while coordinating with other software. As applications go through variable demands on their scalability, throughput, responsiveness and resilience; their monitoring and observation process is becoming more and more complex and needs to be fine-grained too.

How about an automated tool for observability that adds visual reports too?

Micrometer as an Observability Tool:-

Micrometer is a metrics instrumentation library that records/follows traces as the system responds to requests/events and further processes them. Then Micrometer can feed those traces to tracing systems that aggregate and process them, to provide useful metrics and visualisations. Micrometer traditionally supports many tracing systems like Jaeger, Zipkin, Prometheus and Datadog etc.

Further it allows you to trace your Threads via the JvmThreadMetrics binder, loaded and unloaded classes via the ClassLoaderMetrics binder, CPU's average and total load via ProcessorMetrics, garbage collector via JvmGcMetrics etc. ; each via a specialised binder.

Micrometer is a very popular tool that has a minimal code footprint and it tries to avoid java-reflection, thus it can run alongside your applications using minimal system resources.

Spring-Boot’s traditional support for Observability:-

Before Spring Boot 3, programmers needed to use Micrometer (external dependency ) for metrics , spring-cloud sleuth for distributed tracing (the external library that depended on Micrometer), where micrometer itself was a dependency on the spring-framework.

Other observability libraries were also available that worked well with Spring Boot, but all of them were external to the framework and needed significant expertise and configuration to be useful. This made using Micrometer a bit tedious and error-prone.
e.g.

For any observation to happen, one needed to register ObservationHandler objects through an ObservationRegistry.

 
@Service

public class MyObservableService {

private final ObservationRegistry observationRegistry;

public MyObservableService(ObservationRegistry observationRegistry) {

this.observationRegistry = observationRegistry;

}

....

public void tracedMethod() {

Observation.createNotStarted("tracedMethod", this.observationRegistry)

.lowCardinalityKeyValue("locale", "en-US")

.highCardinalityKeyValue("userId", "77")

.observe(() -> {

// Write business logic here, all this code will now be traced/observed by this newly created "Observation" using the observationRegistry created earlier.

});

}

}

In the example above, we use the function call highCardinalityKeyValue("nnnn", "xx") for illustration only, calling it should be avoided unless you have a specific reason to do so. Low cardinality key-value pairs can be attached with each trace for serving many purposes, like identifying the host/region/instance that generates these traces OR detecting anomalies and classifying/quantifying constituents of your application stack.

In general, the property management.observations.key-values.* can be used to automatically apply low-cardinality key-values to all observations.

With Spring Boot 3 onwards, there is no need to configure and use external libraries for observability.

Enhanced Observability support in Spring Boot 3.2 with Java 21:-

With Spring Boot3.2, all this becomes simpler, and you just need to use an annotation. You just need to add the Spring Boot AOP library in the classpath (spring-boot-starter-aop), and use

Micrometer's existing annotations like @Observed, @Counted, @Timed, @TimedSet, @NewSpan and @ContinueSpan etc. The aspects for these annotations are now auto-

configured if you have AspectJ on the classpath. e.g.

 
@Service

public class MyObservableService {

@Observed(name = "tracedMethod", lowCardinalityKeyValues = {"locale", "en-US"})

public void tracedMethod() {

// Write business logic here, Spring will internally create the necessary objects for observation.

}

}

So no need to define custom observations programmatically using the Observation and ObservationRegistery classes, you can just focus on your application's business logic,

Spring Boot 3.2 does the Observation for you inherently. Spring 3.2 has a new configuration property named spring.reactor.context-propagation, if you set this to auto, it automatically propagates observations, trace IDs, and span IDs in your reactive pipelines.

Also, Spring Boot 3.2 now has inbuilt support for Reactive Relational Database Connectivity or R2DBC, To enable it, add the io.r2dbc:r2dbc-proxy dependency to your project. If you're using Kafka, you can set the spring.kafka.template.observation-enabled property to support Micrometer observations.

Finally, you can also disable Micrometer by setting the following property: management.metrics.export.enabled=false

Additional Support for OpenTelemetry:-

If you don't want to use Micrometer for some reason OR want fine-grained control over your observations, Spring Boot’s actuator module includes basic support for OpenTelemetry. You will have to configure OpenTelemetry on your own, as no auto-configuration is provided, but that's not difficult. Spring Boot provides two major beans, one of type OpenTelemetry and another of type Resource.

The OpenTelemetry bean will automatically register any bean of type SdkLoggerProvider or SdkMeterProvider, if found in the context.

The attributes of the Resource can be configured via the management.opentelemetry.resource-attributes configuration property.

You need to add the following dependency in your POM.xml

 
<dependencies>
<dependency>

	<groupId>io.opentelemetry.instrumentation</groupId>

	<artifactId>opentelemetry-spring-boot-starter</artifactId>

	<version>1.32.0-alpha</version>
</dependency>
</dependencies>

OR gradle build

 
dependencies {

implementation( 'io.opentelemetry.instrumentation:opentelemetry-spring-boot-starter:1.32.0-alpha')

}

Now, as soon as Spring Boot detects the MeterProvider bean, it will register it on the BatchSpanProcessor.

To trace your JDBC connections, you can add the OpenTelemetry driver class and your JDBC URL(starting with JDBC:otel) to your application.properties.

spring.datasource.url=jdbc:otel:mysql:localhost:dbname

spring.datasource.driver-class-name=io.opentelemetry.instrumentation.jdbc.OpenTelemetryDriver

and add the following dependencies:-

 
<dependencies>

<dependency>

	<groupId>io.opentelemetry.instrumentation</groupId>

	<artifactId>opentelemetry-jdbc</artifactId>

	<version>1.32.0-alpha</version>

</dependency>

</dependencies>

OR, if using Gradle

 
dependencies {

implementation('io.opentelemetry.instrumentation:opentelemetry-jdbc:1.32.0-alpha')

}

Java 21’s Virtual Threads vs. OS Threads: An Observability Point of View

OS threads as the name suggests are operating system threads that the OS generates. A fraction of the available threads is given to the JVM, and the JVM distributes them among the Java processes and multi-threaded programs. So traditionally, a Java thread used to be assigned an OS thread to complete its task. Since the OS threads are limited, and some Java threads perform long-term computations that need many thread cycles whereas some other Java threads just need a minute fraction of the CPU cycle(thread) to complete their task, Java 21’s JVM creates its lightweight threads called virtual threads to manage the thread allocation optimally.

The Java 21 runtime can give the illusion of plentiful threads by mapping a large number of virtual threads to a small number of OS threads and managing them effectively. The virtual thread consumes an OS thread only while it performs calculations on the CPU and frees it when its immediate task is done, instead of holding on to it for the lifetime of the request it is handling. Virtual threads now always support thread-local variables, and need(should) not be pooled as they are short-lived by design and have a shallow call stack. Virtual threads are not faster than OS threads, rather they enable the higher concurrency needed for higher throughput.

The above properties of Virtual threads necessitate the need to observe them closely for better application design and post-production performance.

Micrometer’s low-overhead profiling and monitoring mechanism can associate events from application code (such as a web request or I/O operations) with the correct virtual thread assigned for the operation. It can step through virtual threads, show call stacks, and inspect variables in stack frames.

Embracing Java 21’s Structured Concurrency and How it Improves Concurrent Programming

Java 21 introduces Virtual threads, Scoped values, and Concurrency APIs for better concurrent programming. Imagine how the ability to create child threads from parent threads, share variables without the danger of concurrent updates or race conditions, and manage all the thread framework via Structured Concurrency APIs, can do wonders for your multi-threaded code. Structured concurrency treats groups of related tasks running in different threads as a single unit of work, thereby streamlining error handling and cancellation, improving reliability, and enhancing observability. Now you can easily visualize your threads in the form of hierarchical trees with shared variables, along with thread dumps.

Thus, error handling is improved as the API makes it very easy to do the right thing and hard to accidentally do the wrong thing: it's hard to fail to notice errors from child threads, fail to stop or wait for child threads, and fail to shut down fast in error scenarios.

Observability is improved because the parent-child thread relationship is formalized and recognized by the tooling.

E.g., consider the following code snippet:-

 
Response handle() throws ExecutionException, InterruptedException {

try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {

Supplier  user  = scope.fork(() -> findUser());

Supplier order = scope.fork(() -> fetchOrder());

Map shipmentTracking = scope.fork(() -> trackShipment(order));

scope.join()            // Join the 3 subtasks

.throwIfFailed();  // ... and propagate errors

// Here, both subtasks have succeeded, so compose their results

return new Response(user.get(), order.get(), shipmentTracking.get());

}

}

The above program:-

Depicts the task-subtask ( or parent-child) relationship
Has Logical control flow, if one of the subtasks fails, the parent task also fails as a consequence
The body of the try-with-resources statement provides a lexical scope to confine the lifetimes of all threads
Enhanced Observability as a thread dump would now clearly display the task hierarchy, with the threads running findUser(), fetchOrder(), and trackShipment() shown as children of the scope.
Provides clarity by having a well-defined structure, i.e. set up the subtasks, wait for them to either complete or be cancelled, and then decide whether to SUCCEED (and process the results of the successfully finished child tasks) or FAIL (if an error is thrown, the subtasks are already finished, so there is nothing more to clean up).

This new feature of Java 21, paired with a powerful tool like Open Telemetry or Micrometer helps improve concurrent programming with continuous feedback.

Additional Observability support in Java 21:-

In addition to the above, Java 21 provides advanced logging capabilities including structured logging and log correlation. This enables better traceability and analysis of application logs, making it easier to identify and troubleshoot issues.

Another useful tool is the improved Flight Recorder which keeps collecting diagnostic and profiling data about a running Java application, without any performance overhead. The profiling information in the Flight Recorder includes thread execution details, memory usage, and garbage collector metrics.

Combining these with other observability tools eases performance evaluation, anomaly detection, and fault correction.

Observability best practices:

1. Avoid tracking metrics that might have high cardinality, e.g. unique user_id of every logged-in user, as this could, at times, result in millions of values being tracked. This in turn could lead to a resource shortage in your computing system, causing a possible crash in either the running application or the observability tool. Moreover, high cardinality data could overshadow/clog other important observability metrics.

As a thumb rule be cautious with data like user input, emails, and random/auto-generated values.
As a rule, Low cardinality key values are added to metrics and traces, while high cardinality key values will only be added to traces.

2. Always disable/prevent observations that are not important to you, from being reported.

To prevent certain observations from being reported, you can use the management.observations.enable properties.

e.g.

To prevent/disable a group of observations, e.g. all observations with a name starting with denied prefix , just set the management.observations.enable.denied.prefix=false

If you want to prevent a specific observation, e.g. Spring Boot Security from reporting observations, set the property management.observations.enable.spring.security=false.

If you want to decide at runtime , you can register beans of type ObservationPredicate; then an Observation will only be reported if all the ObservationPredicate beans return true for that observation. e.g. to prevent all observations whose name contains "denied", you can do this:-

 
@Component

class MyCustomObservationPredicate implements ObservationPredicate {

@Override

public boolean someTest(String name, Context context) {

return !name.contains("denied");

}

}

3. Choose metrics as per the scalability and responsiveness demands of your application, not all metrics are to be tracked.

The combination of Java 21 with Spring Boot 3.2 provides effective tools and libraries to ease Observability, and how to make the best use of it.

Pratik Dwivedi

December 18, 2023