Foreign Function and Memory API

Foreign Function and Memory API comes under project Panama launched with the aim to ease the interaction between Java and foreign (non-Java) APIs, i.e., native code which can be written in C, C++ or even Assembly Language. We need this because there are many native only libraries which are not written in java. Some of these include:

1. OpenGL (for graphics processing)
2. Tensorflow, ONNX (for machine learning)
3. OpenSSL (secure communication)
4. CUDA (general-purpose computing on GPUs)

The way to achieve this, earlier was using JNI (Java Native Interface)

JNI allowed classes to have native methods. These methods would not have a body and their implementation would be written in a native language (C, C++ etc).

There were multiple downsides of this approach:

1. One of the main downsides is that we lose the “write once, run anywhere” feature of Java. If our application is made to support Windows, MacOS, Linux etc, we would need to have a new library for each combination of the platform and architecture.
2. This thereby increases the cost of building, maintaining and deploying these libraries.
3. JNI also adds a layer of communication between the code running on JVM and our native code: we need to convert the data exchanged in both ways between Java and C++ and this can be a costly process.

Overall, all these downsides led to Project panama. Project panama has multiple tools and apis to achieve its goals:

1. Foreign-Function and Memory API: Allocate and access off-heap memory and call foreign functions directly from Java code.
2. Vector API: To perform vector computations within Java on supported CPU architectures thereby providing performance improvements over scalar alternatives.
3. JExtract: a tool to mechanically derive Java bindings using a set of native headers.

In this blog, our goal is to take a look at the Foreign Function and Memory API.

In brief, FFM is an API that enables the interoperation of Java programs with code and data outside of the Java runtime by efficiently invoking foreign functions (i.e., code outside the JVM) and safely accessing foreign memory (i.e., memory not managed by the JVM), avoiding the fragility and risks associated with JNI.

‍

Foreign Memory Access:

When we create an object using a new keyword in java, it gets stored on-heap in the JVM. Garbage collector has access to this memory. Garbage collection can be costly and unpredictable and most of the performance critical libraries need the data to be stored off-heap. The native languages handle the allocation and deallocation themselves. Java has provided access to off-heap storage using ByteBufferAPI and sun.misc.Unsafe API in the past.

Both of them have their own issues, for example, in case of ByteBuffer API, the maximum size of region is limited to two gigabytes and the deallocation is controlled by the garbage collector when the buffer object is collected and not by the developer. In the case of the Unsafe API, it gives too much fine grained control to the developer making it a weak programming model, where an application can interact with multiple regions of the off-heap memory. There are also chances of having dangling pointers which can cause their own bugs. Hence we needed something more balanced to access off-heap memory. Here comes Foreign Function and Memory API:

‍

We will look at all of these briefly and then will walk through a code example that calls a method written in C using FFM in java and outputs the result.

‍

Allocation/Deallocation of Foreign Memory:

Memory Segment:

A memory segment is an abstract representation of a contiguous block of memory, which can reside either off-heap or on-heap. They ensure that the memory access operations are safe by providing both spatial (range of memory addresses associated with a segment) and temporal (lifetime of the memory region before it is deallocated) bounds. A memory segment can be:

1. A native segment, allocated in off-heap memory (like malloc).
2. A mapped segment, using a region of mapped off-heap memory (like mmap).
3. An array or buffer segment, using on-heap memory from a Java array or byte buffer.

‍Arena:

An arena defines the bounds of the memory segment it allocates. Example:

/** 
This code allocates 100 bytes of memory from b to b+99 range 
where b is the base address.
*/
MemorySegment memo = Arena.global().allocate(100); 

‍

There are multiple types of arenas. They are described in the table below:

‍

FFM API includes SegmentAllocator as an abstraction for allocation and initialization of memory segments. The Arena class implements this interface allowing them the ability to allocate native memory segments.

‍

Manipulate and access structured foreign memory:

Memory Layouts:

MemoryLayout is used to describe the content of a memory segment in a more declarative fashion. There are various types of memory layouts available to us, including:

1. ValueLayout: It is used to model basic data types. It defines various value layout constants for Java primitives and addresses. Each value layout has an associated size, alignment, byte order and a carrier (the Java type that should be used when accessing a region of memory using the value layout).
2. SequenceLayout: It represents a homogeneous repetition of zero or more occurrences of an element layout.
3. GroupLayout: Unlike SequenceLayout, it is a heterogenous representation of multiple different member layouts. It can be further of 2 types:
   1. StructLayout: member layouts are laid out one after the other.
   2. UnionLayout: member layouts are laid out at the same starting offset.
4. PaddingLayout: Its main purpose is to provide alignment to member layouts and hence its contents can be ignored.

‍

Calling Foreign Functions:

SymbolLookUp:

The very first step when it comes to supporting the calling of foreign functions is having the ability to find the address of the given symbol (function or a global variable) in the native library that needs to be called. In the FFM api this is defined as a functional interface: SymbolLookup. The SymbolLookup is created with respect to a particular library and then the find(String) method takes the name of a symbol to return its address in that library.

For example,

SymbolLookup lookup = SymbolLookup.libraryLookup(Paths.get(pathToLibraryTest, arena);
MemorySegment testSymbol = lookup.find("test")
																	.orElseThrow(() -> new RuntimeException("Symbol 'test' not found"));

‍

The above code defines a SymbolLookup for a library “Test”. It then uses the “find” function to retrieve the symbol test (a method that might have been defined in this library). It returns the address of the symbol in case the symbol is found otherwise it throws a RuntimeException.

The FFM API defines 3 kinds of SymbolLookup objects:

1. SymbolLookup::libraryLookup(String, Arena): It creates a library lookup, which locates all the symbols in a user-specified native library. It loads and associates the library with an Arena object. On closing the arena the library gets unloaded.
2. SymbolLookup::loaderLookup(): It creates a loader lookup, which locates all the symbols in all the native libraries that have been loaded by classes in the current class loader using the System::loadLibrary and System::load methods.
3. Linker::defaultLookup(): It creates a default lookup, which locates all the symbols in libraries that are commonly used on the native platform (i.e., operating system and processor) associated with the Linker instance.

Linker:

Once we have the memory address of our required native function using SymbolLookup, we need an interface that enables our java code to interoperate with the native code. The Linker interface serves this purpose. It enables both downcalls (calling native code from java code) and upcalls (calling java code from native code). Linker::nativeLinker() returns the linker for the ABI associated with the underlying native platform (combination of OS and processor where the Java runtime is currently executing). The nativeLinker is optimized for the calling conventions of multiple different platforms.

Downcalls:

For downcalls, the MethodHandle::downcallHandle comes handy. It takes the address of the foreign function obtained from SymbolLookup. We can then use the invoke method of the MethodHandle to invoke the downcall from our java code. We can even pass arguments in the MethodHandle, which will then be passed to the native function.

Upcalls:

For upcalls, FFM provides us with a MemorySegment::upcallStub. It takes a methodHandle (usually the java method), converts it into a MemorySegment Instance and passes it as a function pointer to the native code.

FunctionDescriptor:

To create the upcall stubs and downcall MethodHandles we also need a description of the signature of the foreign function. This is modelled using a FunctionDescription interface. It describes the parameter types and return type of the target foreign function. In case a C function has a return type void, then its FunctionDescriptor can be obtained using FunctionDescriptor.ofVoid(ADDRESS).

‍

Overall Example:

In this simple demonstration we will write a simple method in C that will return the result by adding two given numbers as arguments.

C Program:

Lets, define the simple C method first in a file named: addition.c:

#include <stdio.h>

int add(int a, int b) {
	return a + b;
}

‍

Now that we have our C method saved, we will compile the C code to a shared library (addition.so on Unix-like systems, addition.dll on Windows and addition.dylib on macOS)

‍

gcc -shared -o addition.dylib -fPIC addition.c

‍

Once the C code is compiled, we will move on to writing our java program that will access this foreign method, pass the arguments and print the sum.

Java Program:

Steps

1. Define the function descriptor with the return Layout and the argument Layouts: In this case the function returns an integer and takes 2 integers as arguments.
2. Define an Arena to define the memory segment bounds: Here we are using a confined arena since we want it to close once our task is complete.
3. Next we find the required “add” symbol in the library we compiled above using libraryLookup. Using the find function we get the memory Address of the “add” function.
4. Since this is a downcall, we get the MethodHandle using the downcallHandle of the nativeLinker.
5. We define 2 integer variables on heap, allocate off-heap memory to store these 2 integers and call the addFunctionHandle for these values and finally print them.
6. Note that when the try blocks end, the memory is deallocated, the native library is unloaded and the arena is closed as we used a Confined Arena.

Code:

	
import java.lang.foreign.*;
import java.lang.invoke.MethodHandle;
import java.nio.file.Paths;

public class MemoryAccessExample {
    public static void main(String[] args) throws Throwable {
        // Define the function descriptor for the foreign function
        FunctionDescriptor addDescriptor = FunctionDescriptor.of(ValueLayout.JAVA_INT, ValueLayout.JAVA_INT, ValueLayout.JAVA_INT);

        // Load the C library and get a symbol lookup
        String libraryPath = System.getProperty("user.home") + "/Downloads/addition.dylib";
        try (Arena arena = Arena.ofConfined()) {
            SymbolLookup lookup = SymbolLookup.libraryLookup(Paths.get(libraryPath), arena);

            // Find the `add` function symbol
            MemorySegment addSymbol = lookup.find("add").orElseThrow(() -> new RuntimeException("Function 'add' not found"));

            // Create a method handle for the `add` function
            Linker linker = Linker.nativeLinker();
            MethodHandle addFunctionHandle = linker.downcallHandle(
                    addSymbol,
                    addDescriptor
            );

            // Allocate on-heap memory for two integers
            int num1 = 25;
            int num2 = 10;

            // Use try-with-resources to manage the lifetime of off-heap memory
            try (Arena offHeap = Arena.ofConfined()) {
                // Allocate memory to store the integers
                MemorySegment intMemory = offHeap.allocate(ValueLayout.JAVA_INT, 2);

                // Store integers in off-heap memory
                intMemory.set(ValueLayout.JAVA_INT, 0, num1);
                intMemory.set(ValueLayout.JAVA_INT, 4, num2);

                // Invoke the foreign function
                int result = (int) addFunctionHandle.invoke(intMemory.get(ValueLayout.JAVA_INT, 0), intMemory.get(ValueLayout.JAVA_INT, 4));

                // Output the result
                System.out.println("The sum is: " + result);
            } // All off-heap memory is deallocated here
        } // The library will be unloaded here when this arena is closed
    }
}

‍

Output:

Finally, when we can run this program with –enable-preview (since it is a preview feature in java 22), we can see the output as the sum of 2 integers:

WARNING: A restricted method in java.lang.foreign.SymbolLookup has been called
WARNING: java.lang.foreign.SymbolLookup::libraryLookup has been called by MemoryAccessExample in an unnamed module
WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for callers in this module
WARNING: Restricted methods will be blocked in a future release unless native access is enabled

The sum is: 35

‍

Advantages and Performance of using FFM:

The aim of FFM is to replace the brittle machinery of native methods and the Java Native Interface (JNI) with a concise, readable, and pure-Java API. It provides foreign function and memory access with performance comparable to JNI and sun.misc.Unsafe API with a much uniform approach and a guarantee of no use-after-free bugs. Many situations that previously required using JNI can now be handled by invoking methods in the Foreign Function & Memory API, maintaining the integrity of the Java Platform. The warnings in the output ensure that the user is aware of performing unsafe operations with native code.

‍

To learn more about the FFM API, refer to the official documentation.