Smarter dependency downgrades

March 12, 2020

Louis Jacomet

New Features

Introduction

One of the biggest challenges when dealing with transitive dependencies is to keep their versions under control. Popular libraries will show up as transitive dependencies in multiple places in your dependency graph. And it is quite likely that the version information will be different on each path.

Through multiple blog posts, you have learned that Gradle offers a rich feature set for expressing complex dependency requirements. In this post, we will discuss why semantics matter when downgrading a dependency version. And you will learn about the strict version feature of Gradle 6 that provides this semantic information and effectively gives you a powerful and precise tool for dealing with this complex issue.

In this post, we will use Google’s Guava again as an illustrating example because:

• It is a very popular library, used by over 20,000 other libraries
• It has a complex history in terms of API stability which results in sometimes difficult upgrades in large dependency graphs

In the dependency view below, taken from this build scan, we can see that many Guava versions are in play:

The dependency declaration for this build is however quite simple:

dependencies {
    implementation("org.optaplanner:optaplanner-core:7.24.0.Final")
    implementation("com.spotify:folsom:1.5.0")
}

With only two direct dependencies, we already have a conflict between four Guava versions. It will always resolve to 25.0-jre in this example. This is the result of Gradle accounting for all versions in a dependency graph and optimistically choosing the highest matching the constraints, 25.0-jre in our example.

If we compare this Gradle project with a matching Maven project, we get a somewhat different result. With Maven, the first of the shortest path to a dependency will be used to determine the version. This means that, in our example, the Guava version is effectively order dependent since both libraries have a direct dependency on Guava. If com.spotify:folsom:1.5.0 is declared first in a Maven POM file, Guava will be resolved at version 24.1-jre. However if org.optaplanner:optaplanner-core:7.24.0.Final comes first, Guava will be resolved at version 25.0-jre.

What if that version upgrade is a problem? #

Let’s pretend that bumping Guava to 25.0-jre is an issue for Folsom because it relies on the Files.fileTreeTraverser() API from 24.1-jre, removed in 25.0.

Prior to Gradle 6, the most frequently used solutions for dealing with this problem were:

• Add an exclude on Guava on org.optaplanner:optaplanner-core:7.24.0.Final
• Add a resolutionStrategy.force on Guava to version 24.1-jre

Unfortunately, none of these solutions gave clear reasons for the version downgrade. When excluding the dependency, it is not clear whether you meant that your usage of org.optaplanner:optaplanner-core:7.24.0.Final does not require Guava or if you only did it for its side effects on the resolved version. If instead, you chose to force a particular version of the dependency, the information would not be available to the consumers of your library, which can be as simple as a different project in your multi project build, who would then be exposed to the same incompatibility you resolved.

The Maven situation is quite similar:

• An exclude has the same lack of semantics.
• The use of dependencyManagement for transitive dependencies, like a Gradle force, is not applicable to consumers of your library.

A meaningful downgrade #

With Gradle 6, rich version provides an enhanced strict version declaration. This version declaration has the following semantics:

1. A strict version effectively takes precedence over all other dependency versions contributed by the subgraph of the project declaring the strict version.
2. A strict version effectively rejects all incompatible versions.

So in our example, we would combine that version declaration with a dependency constraint to select Guava 24.1-jre:

dependencies {
    constraints {
        implementation("com.google.guava:guava") {
            version {
                strictly("24.1-jre")
            }
            because("Guava 25.0-jre removed APIs used by Folsom")
        }
    }
    implementation("org.optaplanner:optaplanner-core:7.24.0.Final")
    implementation("com.spotify:folsom:1.5.0")
}

Why do the semantics matter? #

In the example we took, simply combining two dependencies resulted in broken code. We had to figure out which combination of dependency versions works in the context of our own library. Then, we recorded our decision by adding a dependency constraint with a strict version. If the library we are building is reused, it is important that this decision is preserved for future consumers.

With the pre Gradle 6 solutions, this information was lost. Neither the exclude nor the force carried enough information to our consumers, which would most likely be bitten by the problem again.

The Maven solutions have the same downsides, with the same potential consequences.

With the strict version definition of Gradle 6, your consumers will know about the choice you made. If any other of their dependencies causes a Guava update, the build will fail, indicating that your library has a strict requirement on Guava 24.1-jre. With the additional information provided by the because clause, these developers will know about the problem and already be on a path to find a solution of their own. They can respect your choice or overrule it by defining their own strict version.

Best practices for strict versions #

Because of their semantics, strict versions should be added with the following best practices in mind:

• For reusable software libraries:
  • It is recommended to use a version range in the strictly part of the version when possible. This gives more freedom to consumers of the library and enables them to find a solution without relying on another strict version definition.
  • Provide a prefer version which can be used when consumers do not care.
• For applications, a fixed version in the strictly part is the simplest and most direct option.
• In all cases, make sure to document your decision with a because.

Conclusion #

A consequence of libraries reuse is that version conflicts are unavoidable, especially for popular ones. Sometimes it is necessary to make an explicit version choice for a specific combination of libraries. The strict version concept of Gradle 6 allows you to make this choice and to preserve it for your consumers.

While the Maven resolution mechanism sounds simpler at first, we showed that it has semantic issues as the dependency graph grows. A solution that makes sense for your library loses all meaning when consumed.

On the other hand, Gradle, by associating semantics with version declaration, has a consistent resolution model, which allows developers to clearly express the choices they made and document their reasoning.

These choices are then available when a library is consumed by others, giving the opportunity to consumers to better respect, or overrule and document, the choices made by the library developers.

Discuss