The Impossible Java 11
Over the past two years, I did something no one thought was possible. I updated our code from JDK 8 to JDK 16. Out of all the things I’ve accomplished at Netflix, this is the one thing I’ve had the most questions about, and the most astonishment. “How did you do it?”
If you aren’t familiar with Java, Oracle made a tough decision to modernize Java. Prior to Java 9, a major version came out once every few years, and with relatively minor breaking changes. All that changed in Java 9; major versions were going to be breaking, and coming out once every 6 months.
However, unlike Python 3000, Java’s breaking changes were extremely limited. Only JVM internals
would be restricted or removed. That’s it. All the existing language features were still
available for use. All the JVM bytecode and classes still worked. Want to use a class written in
1995? If it doesn’t depend on sun.* code, it’s good to go!
But, this is also why so many people thought upgrading was impossible. So much Java code out there
had taken liberties with the JDK internals. Want to use sun.misc.Unsafe? It’s yours for
the taking. Want to modify final (a.k.a. const) fields? It’s only a little reflection away. Want
to inject some classes into the JDK classloader? No one will stop you.
With no teeth in the API boundaries, Java 8 pretty much let application and library owners do what they wanted. Guice, Guava, Jackson, Groovy, Gradle, CGLib, Mockito, Lombok, ErrorProne, and other common Java tech played fast and loose with the guts of the JVM. Yes, their code was faster. But no, it wasn’t a long term win.
This is where that tough decision by someone who works on Java comes into play. Java could improve itself, add features, and improve the VM, but it would come at the cost of breaking the ecosystem. Many libraries, which had claimed wins, would have to give them back. Alternatives would be provided, but everyone would have to coordinate a worldwide, backwards compatible, multi-version upgrade. Every library not wanting to yield their gains would need to be rewritten. Most poignantly though, applications would need to wait till every library they depended on to make this move.
Two things stand out to me about this decision:
Having strong, enforceable API boundaries are necessary for long term improvements. Short term wins, while alluring, usually come with non-standard or unsupported baggage. The stricter it is today, the more maintainable it is in the future.
Taking away functionality, even under the auspices of standards or specifications, is going to hurt. If it used to work, and you break it, it’s your fault.
Java 11
When I joined Netflix, no one told me it was impossible to upgrade from Java 8 to 11. I just started using it. When things didn’t work (and they definitely didn’t!) on 11, I went and checked if I needed to update the library. I did this as a back-burner project, on my own machine, separate from the main repo. One by one, all the non-working libraries were updated to the working ones. When a library was not Java 11 compatible, I filed a PR on GitHub to fix it. And, plain as it sounds, when there are no more broken things, only working things are left!
It was only after I pushed these changes to production, that I found out what I did was impossible. Numerous people asked me how I made this happen. Someone, or some group, had looked at the daunting task of updating hundreds of projects and probably hundreds more libraries, and had given up hope. Like a poison placebo, people’s perception of the impossibility actually made it impossible. Socialize these obstacles and you end up with a whole company finding it insurmountable.
When I told them how I fixed it, they were usually disappointed. “Just updated the libraries?” they would ask, looking for something deeper. Java 11 was time-consuming to upgrade to, but not very hard. The only difference between my approach and theirs was a lack of despair.
I bring up this story to boost the confidence in others that using the latest and greatest is within grasp. A month ago I updated our code to Java 15, and last week to 16. It gets easier each time. Once you are close to the latest version, it’s no challenge to stay there. Since the only breaking changes were hiding JVM internals, and we’re no longer using those, it’s trivial to update. As a reward, we get all the advanced features (better JIT, GC, language features, etc.) that have been delivered over the past years.
I encourage you to take a look at updating too, since it is probably easier than you think!