The journey of migrating from Java 11 to 17

Introduction

With Java 11 long-term support coming to an end in September 2023, in some of our projects in which we had it running already we've decided to upgrade to Java 17. Like version 11, Java 17 is also an LTS version of the platform, meaning that it will have extended support until September 2026 and security patches until 2029. While the upgrade has been pretty straightforward in general terms, in this article we will describe a couple of technical challenges and quirks we've found when doing the upgrade in one particular project.

Impact of changes in Security and Reflection API

Because of our usage of the Salesforce API we are required to use the PATCH HTTP method to update Objects on the SF side, because the PATCH method was standardized as part of RFC 5789 in 2010, it didn’t exist in the implementation of the HttpURLConnection class, which we use for our Salesforce integration.

While it was possible to add it as part of the HttpURLConnection class, the Oracle/OpenJDK team decided to mark the bug as WONTFIX and, instead, suggested to developers that they should use the still-to-be-released HttpClient API.

Java 11 solution

Since the HTTP protocol and its methods are essentially just text which indicate how an API must behave when contacted with a particular string of data, it was decided that it was easier to just patch the HttpURLConnection class using the Java Virtual Machine internals, namely the Reflection API.

Note that HttpURLConnection class has the following field which specifies valid HTTP methods:


private static final String[] methods = {
  "GET", "POST", "HEAD", "OPTIONS", "PUT", "DELETE", "TRACE"
};

This was implemented in the following way:


private void allowPatchingRequests() throws NoSuchFieldException, IllegalAccessException {
  Field field = HttpURLConnection.class.getDeclaredField("methods");
  field.setAccessible(true);
  Field modifierField = Field.class.getDeclaredField("modifiers");
  modifierField.setAccessible(true);
  modifierField.setInt(field, field.getModifiers() & ~Modifier.FINAL);
  field.set(null, new String[]{"GET", "POST", "HEAD", "OPTIONS", "PUT", "DELETE", "TRACE", "PATCH"});
}

In a nutshell perform these actions:

We find the ‘methods’ field inside HttpURLConnection, which is marked ‘private static final’.
We make it accessible using the reflection API
We make the Field class itself be accessible using the reflection API, this is required for us to make our “methods” field not final.
Finally, after we make “methods” not final, we are allowed to modify it, introducing the “PATCH” method at the very end of the array.

Which essentially just allows us to send ‘PATCH’ calls directly to any API we use.

The switch to Java 17

While this worked for a while, it came with a caveat. As Oracle and the OpenJDK team improved internal security machinery inside the JVM, this particular usage of reflection was eventually deprecated per JDK-8210522 which was released as part of Java 12.

As part of our migration to Java 17, we eventually came upon a compromise solution, using the Invoke API instead.

What JDK-8210522 did was make the “modifiers” field inside Field to not be accessible using reflection, remember, we need this to make “methods” in HttpURLConnection not final. However, we can instead use VarHandle and MethodHandles.privateLookupIn, which are part of the Invoke API to “force” our way in.


private static final VarHandle MODIFIERS;

 static {
   try {
     var lookup = MethodHandles.privateLookupIn(Field.class, MethodHandles.lookup());
     MODIFIERS = lookup.findVarHandle(Field.class, "modifiers", int.class);
   } catch (IllegalAccessException | NoSuchFieldException ex) {
     throw new RuntimeException(ex);
   }
 }private void allowPatchingRequestsInvoke() throws IllegalAccessException, NoSuchFieldException {
  Field field = HttpURLConnection.class.getDeclaredField("methods");
  MODIFIERS.set(field, field.getModifiers() & ~Modifier.FINAL);
  field.setAccessible(true);
  field.set(null, new String[]{"GET", "POST", "HEAD", "OPTIONS", "PUT", "DELETE", "TRACE", "PATCH"});
}

While this works, it still requires additional JVM parameters to explicitly allow reflection between java packages, namely, this requires using:


--add-opens=java.base/java.lang.reflect=ALL-UNNAMED 
--add-opens=java.base/java.net=ALL-UNNAMED

As we need to specify that we will be using the java.lang.reflect package (specifically, the .setAccessible method) to introspect java.base, and again to specify that we need to introspect the class HttpURLConnection, from the java.net package.

Java 17 and beyond

While this will work on Java 17, by Java 18 this was patched again, and won’t work anymore, which will require refactoring our entire integration to use other tools, namely, the HttpClient class introduced in Java 11.

Besides that, we didn’t find any issues with switching to JDK 17 on the code side.

Impact on CI/CD processes

We now have our project running locally, but this is a productive API, so we need to deploy it to our infrastructure in AWS. For this, we use CodeBuild to build our jar binary and store it in S3, then we run it in an ECS container with a custom, generic Docker image that fetches and runs it. So, to achieve this, first we need to update our CodeBuild project environment:

Changing the env image to standard 6.0 is important because if we don’t do so, we won’t be able to use coretto17 (Amazon flavor of Java 17) to build our app.

Finally, we updated our buildspec to tell codebuild to use coretto17 and python 3.10 (3.9 is not supported in standard 6.0)

install:
runtime-versions:
java: corretto17
python: 3.10

Also, we added the JVM params we talked about earlier in our env vars script to allow reflection:


export JVM_PARAMS="--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.net=ALL-UNNAMED"

WIth these final adjustments, we migrated successfully our project to Java 17.

Conclusion

In this post we've described the required steps that we had to perform to upgrade the Java environment from 11 to 17 in a particular project. In this case, most of the challenges have been related to particular aspects like Reflection access al related environments (CI/CD). But we want to emphasize that each project has its own quirks and challenges associated.

‍

Facundo Errobidart

Software Engineer & Solver