Introducing Configuration Caching
Table of Contents
Introduction
This is the second installment in a series of blog posts about incremental development — the part of the software development process where you make frequent small changes. We will be discussing upcoming Gradle build tool features that significantly improve feedback time around this use case. In the previous post, we introduced file system watching for Gradle 6.5.
In Gradle 6.6 we are introducing an experimental feature called the configuration cache that significantly improves build performance by caching the result of the configuration phase and reusing this for subsequent builds. Using the configuration cache, Gradle can skip the configuration phase entirely when nothing that affects the build configuration, such as build scripts, has changed.
On top of that, when reusing the configuration cache, more work is run in parallel by default and dependency resolution is cached. The isolation of the configuration and execution phases, and the isolation of tasks, make these optimizations possible.
Note that configuration caching is different from the build cache, which caches outputs produced by the build. The configuration cache captures only the state of the configuration phase. It’s also separate from IDE sync and import processes that do not currently benefit from configuration caching.
In order to cache the configuration result, Gradle applies some strong requirements that plugins and build scripts need to follow. Many plugins, including some core Gradle plugins, do not meet these requirements yet. Moreover, support for configuration cache in some Gradle features is not yet implemented. Therefore, your build and the plugins you depend on will likely require changes to fulfil the requirements. Gradle will report problems found with your build logic to assist you in making your build work with the configuration cache.
The configuration cache is currently highly experimental and not enabled by default. We release it early in order to collect feedback from the community while we work on stabilizing the new feature.
That being said, we are committed to making the configuration cache production-ready with the ultimate goal to enable it by default. You can expect that it will get significantly better in the next Gradle releases.
Configuration caching in action #
It is recommended to get started with the simplest task invocation possible. Running help with the configuration cache enabled is a good first step:
Here it is in action from Android Studio, deploying the middle sized Santa Tracker Android application to an Android emulator, making changes to how the snowflakes move on the screen and applying the changes to the emulator:
Build time improvements #
The practical impact of the feature depends on a number of factors, but in general it should result in a significant reduction of build time. We’ve seen drastic improvements on large real world builds, let’s have a look at some of them.
Java builds #
On a large Java enterprise build with ~500 subprojects and complex build logic, running :help went from 8 seconds down to 0.5 seconds. That’s 16 times faster. Of course, running :help isn’t that useful but it gives an idea of the saved time for the configuration phase. On the same build, running assemble after changing some implementation code went from ~40 seconds down to ~13 seconds, that’s ~3 times faster.
Now let’s look at the gradle/gradle build. It has a hundred subprojects and a fairly complex build logic. You can use it to reproduce these results. Running a test after making an implementation change goes from 16.4 seconds down to 13.8 seconds, skipping the ~2 seconds configuration phase:
In blue you can see the configuration phase, in green the execution phase. On the left, without the configuration cache enabled, configuration phase takes more than 2 seconds and goes down to 214 milliseconds with the configuration cache on the right.
You can also see that the execution phase benefits from the configuration cache but is dominated by compiling and running the tests in that case.
Android builds #
Another notable example is a very large real world Android build with ~2500 subprojects. On that build, running :help went from ~25 seconds down to ~0.5 seconds, that’s 50 times faster! Running a more useful build such as assembling the APK after changing some implementation, goes from ~50 seconds down to ~20 seconds, almost 3 times faster.
In the Santa Tracker Android project, we’ve seen the following improvements in the build time for a small implementation change:
The configuration phase is cut in half, from 129 milliseconds down to 63.5 milliseconds. You can also see that the execution phase is accelerated by the configuration cache due to more task parallelisation and caching of dependency resolution.
If you want to reproduce with the above builds or measure your own builds you can use the Gradle Profiler by following the instructions in this repository. Note that the Gradle Profiler will show a slightly different picture, closer to the experience from IDEs, because both use the Gradle Tooling API. This skips the fixed cost of starting the Gradle client JVM that happens when you use the command line.
How does it work? #
When the configuration cache is enabled and you run Gradle for a particular set of tasks, for example by running ./gradlew check, Gradle checks whether a configuration cache entry is available for the requested set of tasks. If available, Gradle uses this entry instead of running the configuration phase. The cache entry contains information about the set of tasks to run, along with their configuration and dependency information.
The first time you run a particular set of tasks, there will be no entry in the configuration cache for these tasks and so Gradle will run the configuration phase as normal:
- Run init scripts.
- Run the settings script for the build, applying any requested settings plugins.
- Configure and build the buildSrc project, if present.
- Run the builds scripts for the build, applying any requested project plugins.
- Calculate the task graph for the requested tasks, running any deferred configuration actions.
Following the configuration phase, Gradle writes the state of the task graph to the configuration cache, taking a snapshot for later Gradle invocations. The execution phase then runs as normal. This means you will not see any build performance improvement the first time you run a particular set of tasks.
When you subsequently run Gradle with this same set of tasks, for example by running ./gradlew check again, Gradle will load the tasks and their configuration directly from the configuration cache, skip the configuration phase entirely and run all tasks in parallel. Before using a configuration cache entry, Gradle checks that none of the “build configuration inputs”, such as build scripts, for the entry has changed. If a build configuration input has changed, Gradle will not use the entry and will run the configuration phase again as above, saving the result for later reuse.
Requirements and limitations #
In order to capture the state of the task graph into the configuration cache and reload it again in a later build, Gradle applies certain requirements to tasks and other build logic. Each of these requirements is treated as a configuration cache “problem” and by default causes the build to fail if violations are present.
If any problem is found caching or reusing the configuration, an HTML report is generated to help you diagnose and fix the issues.
If you encounter such problems, your build or the Gradle plugins in use probably need to be adjusted. See the Troubleshooting section of the documentation for more information about how to use this report.
You can find the set of supported core plugins and the set of not yet implemented Gradle features in the configuration cache documentation.
The latest Android Gradle Plugin preview, 4.2.0-alpha07 at the time of writing, works with the configuration cache.
The latest Kotlin Gradle Plugin, 1.4.0-RC at the time of writing, works on simple JVM projects emitting some problems.
Kotlin 1.4.20 is the current target for a fully compliant plugin.
This information can be found at gradle/gradle#13490 alongside the status of the most used community plugins.
Try out configuration caching #
If you would like to see how your project benefits from configuration caching, here is how you can try it out.
First, make sure you run Gradle 6.6 or later. In order to enable configuration caching, you need to pass --configuration-cache on the command line. Alternatively, add
org.gradle.unsafe.configuration-cache=true
to the gradle.properties file in the project directory or in the Gradle user home, so you don’t need to pass the command-line option on every build. That’s it: the next build will run with configuration caching enabled.
Keep in mind that you will only see performance improvements when subsequent builds with the same requested tasks have the feature enabled. If you want to benchmark your build, you can do it easily with Gradle Profiler by following the instructions in this repository.
If you run into any problems, check out the supported core plugins or community plugins, learn how to troubleshoot in the user manual. You can also read our recommended adoption steps.
If you still have problems open a Gradle issue if you think the problem is with Gradle, or check the supported community plugins issue at gradle/gradle#13490. You can also get help in the #configuration-cache channel in the Gradle community Slack.