Automatically align Dependencies with Platforms and Gradle Module Metadata

November 25, 2019

This blogpost has been updated to reflect that Jackson started to published Gradle Module Metadata with version 2.12.0 using the recommendations from this post.

In the previous post about dependency management with Gradle 6, we saw that growing builds can quickly end up in dependency hell. Unexpected results become particularly hard to analyze if they are introduced at the bottom of the dependency graph and propagate up through transitive dependencies. It is unfortunate that some of these issues could be avoided if the authors of libraries, which form the bottom of the dependency graph, had the means to express all the knowledge about the versioning of said libraries in the libraries’ metadata.

A typical example of such a library is the widely used JVM utility library Jackson. If several components of the library are part of the dependency graph, alignment of versions can be an issue.

The Jackson library is made up of only three core modules: jackson-annotations, jackson-core and jackson-databind. Still, it is easily possible to end up in a situation where a higher version is selected for jackson-core than for jackson-databind. This situation is illustrated in the following example, where the two modules end up having different versions: jackson-core:2.9.2 and jackson-databind:2.8.9.

Transitive dependencies resolve to jackson-core:2.9.2 but jackson-databind:2.2.2

In the example, which you can explore as a build scan, one transitive dependency (tika-parsers) upgraded jackson-core to 2.9.2 as Gradle resolves the conflict between 2.8.9 and 2.9.2 to the higher version. jackson-databind however, added via another dependency (keycloak-core), is kept on 2.8.9 as the information that it should be aligned, i.e. upgraded together, with jackson-core is missing.

BOMs are great, but we don’t use them (enough)

The situation we face in this scenario is that the Jackson modules’ versions should be aligned, but that information is not published due to limitations of the pom metadata format. What Jackson is publishing is a BOM (bill of materials), a pom.xml containing only dependency version information. The BOM contains some alignment information as this excerpt from jackson-bom-2.9.2.pom shows:

<dependencyManagement>
  <dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-core</artifactId>
    <version>2.9.2</version>
  </dependency>
  <dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.9.2</version>
  </dependency>
  ...
</dependencyManagement>
...

However, to use this alignment information, both Maven and Gradle users need to explicitly depend on the BOM in their own build.

Maven build importing the jackson-bom:

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>com.fasterxml.jackson</groupId>
      <artifactId>jackson-bom</artifactId>
      <version>2.8.9</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

Gradle 5.x build with platform dependency on the jackson-bom:

dependencies {
  // depend on a platform and enforce all version entries (Maven semantics)
  implementation(enforcedPlatform("com.fasterxml.jackson:jackson-bom:2.8.9"))

  // depend on a platform and do dependency conflict resolution with all entries
  implementation(platform("com.fasterxml.jackson:jackson-bom:2.8.9"))
}

There are several issues with this approach: knowing a BOM for Jackson exists, deciding which version of the Jackson BOM to use, and updating that version when the build evolves.

If we use Maven, the versions provided in the BOM are enforced. This means that a request for a higher version by any of the dependencies would be silently ignored. In the example above, if we chose the 2.8.9 BOM, jackson-core would be downgraded to 2.8.9. This can cause issues, as tika-parsers requires jackson-core:2.9.2 and might break with the downgrade. Consequently, the build author has to carefully choose the BOM’s version and revisit that choice when dependencies change. Without tool support, this can become unmanageable if multiple BOMs are used.

The missing link: Platform dependencies on BOMs

Gradle 5.0 introduced the ability to declare a platform dependency to a BOM. In this case, versions are not silently enforced, but the entries in the BOM participate in conflict resolution. In the example, however, this means that we are back to the initial problem: We select the 2.8.9 BOM in the build script, which recommends version 2.8.9 for jackson-core, but jackson-core is upgraded to 2.9.2 by a transitive dependency.

What we are missing is an automatic upgrade of the platform (jackson-bom) to the highest selected version (2.9.2) as well. This cannot be expressed in pom metadata, but it can with Gradle 6 using Gradle Module Metadata.

The Gradle Module Metadata answer

With Gradle Module Metadata, the Jackson team now publishes platform dependencies for each version of jackson-core with the information about which version of the platform (jackson-bom) it belongs to. For example, if jackson-core would be built with Gradle, this platform dependency could be added to its build script:

dependencies {
  // I belong to the 'jackson-bom' platform with the same version
  api(platform("com.fasterxml.jackson:jackson-bom:${project.version}"))
  ...
}

If using Gradle 6, Gradle Module Metadata is published by default and thus includes the platform dependency. In a similar fashion, the platform dependency can be added to jackson-databind and jackson-annotations. With this additional information, the dependency graph from the beginning of this post will look like this:

Transitive dependencies resolve to jackson-core:2.9.2 but jackson-databind:2.8.9

The updated graph, which you can also explore in this build scan, shows two things: First, there is a new node, jackson-bom. The edges towards the jackson-bom come from the Jackson modules. These originate from the published platform dependencies and thus the jackson-bom is automatically added without explicitly depending on it in the build. Second, each module version brings in the jackson-bom that fits its version – jackson-databind:2.8.9 brings in jackson-bom:2.8.9; jackson-core:2.9.2 brings in jackson-bom:2.9.2. Gradle then resolves the version conflict of jackson-bom to the higher version, which in turn adds the dependency constraints of all the higher module versions in the graph, eventually causing all components to align on the highest version.

With Gradle Module Metadata, platform dependencies are published for an automatic updated of the platform (jackson-bom) to the highest selected version.

Adding the missing bits to existing Jackson metadata

Starting with the Jackson 2.12.0 release, Jackson published Gradle Module Metadata as proposed in this blogpost. The following is not needed to align to 2.12.0 or higher.

Since Gradle Module Metadata is a new format, adoption of it will take time. To bridge this gap, Gradle 6 allows you to write component metadata rules to enrich published pom metadata with the missing information when it is processed by Gradle. For the Jackson example, you would add the following to your build script:

open class JacksonAlignmentRule: ComponentMetadataRule {
  @Inject open fun getObjects(): ObjectFactory = throw UnsupportedOperationException()

  override fun execute(ctx: ComponentMetadataContext) {
    if (ctx.details.id.group == "com.fasterxml.jackson.core") {
      ctx.details.allVariants {
        withDependencies {
          add("com.fasterxml.jackson:jackson-bom:${ctx.details.id.version}") {
            attributes {
              attribute(Category.CATEGORY_ATTRIBUTE,
                  getObjects().named(Category.REGULAR_PLATFORM))
            }
          }
        }
      }
    }
  }
}

dependencies {
  // apply the JacksonAlignmentRule rule defined above
  components.all<JacksonAlignmentRule>()
}

Publishing your library with alignment

Examples of libraries that use the approach described in this blog post are JUnit 5 and Jackson. If you are a library author, you can do the same!

If you use Gradle 6+, you can add alignment for the modules of your library by doing what JUnit does:

Have a java-platform project
Define platform dependencies to that project
Use Gradle’s maven-publish plugin to publish metadata with platform dependencies

If you use Maven, you can make use of this Maven plugin to add alignment for the modules of your library by doing what Jackson does:

Have a BOM project
Configure the above mentioned plugin to use your BOM for alignment in a parent POM
Activate the plugin in the POMs of all your modules

Conclusion

The Gradle Module Metadata format can be used by library authors to publish platform dependencies to align the versions of modules in their libraries. In builds that use such libraries, Gradle performs the alignment automatically. If you use a library that is not (yet) publishing this information, you can write your own rules to add the missing bits to the library’s metadata.

Version alignment is only one of many use cases that are solved with the help of Gradle Module Metadata. In the next blog post about Gradle 6 dependency management, we will explore how dependency conflicts between different modules, and variants of modules, can be detected and resolved. If you like to dive deeper right now, explore Gradle’s user manual sections about dependency management.