Introducing Incremental Build Support
Table of Contents
Task inputs, outputs, and dependencies #
Built-in tasks, like JavaCompile declare a set of inputs (Java source files) and a set of outputs (class files). Gradle uses this information to determine if a task is up-to-date and needs to perform any work. If none of the inputs or outputs have changed, Gradle can skip that task. Altogether, we call this behavior Gradle’s incremental build support.
To take advantage of incremental build support, you need to provide Gradle with information about your tasks’ inputs and outputs. It is possible to configure a task to only have outputs. Before executing the task, Gradle checks the outputs and will skip execution of the task if the outputs have not changed. In real builds, a task usually has inputs as well—including source files, resources, and properties. Gradle checks that neither the inputs nor outputs have changed before executing a task.
Often a task’s outputs will serve as the inputs to another task. It is important to get the ordering between these tasks correct, or the tasks will run in the wrong order or not at all. Gradle does not rely on the order that tasks are defined in the build script. New tasks are unordered, therefore execution order can change from build to build. You can explicitly tell Gradle about the order between two tasks by declaring a dependency between one task another, for example consumer.dependsOn producer.
Declaring explicit task dependencies #
Let’s take a look at an example project that contains a common pattern. For this project, we need to create a zip file that contains the output from a generator task. The manner in which the generator task creates files is not interesting—it produces files that contain an incrementing number.
build.gradle #
apply plugin: 'base'
task generator() {
doLast {
def generatedFileDir = file("$buildDir/generated")
generatedFileDir.mkdirs()
for (int i=0; i<10; i++) {
new File(generatedFileDir, "${i}.txt").text = i
}
}
}
task zip(type: Zip) {
dependsOn generator
from "$buildDir/generated"
}
The build works, but the build script has some issues. The output directory for the generator task is repeated in the zip task, and dependencies of the zip task are explicitly set with dependsOn. Gradle appears to execute the generator task each time, but not the zip task. This is a good time to point out that Gradle’s up-to-date checking is different from other tools, such as Make. Gradle compares the checksum of the inputs and outputs instead of only the timestamp of the files. Even though the generator task runs each time and overwrites all of its output files, the content does not change and the zip task does not need to run again. The checksum of the zip task’s inputs have not changed. Skipping up-to-date tasks lets Gradle avoid unnecessary work and speeds up the development feedback loop.
Declaring task inputs and outputs #
Now, let’s understand why the generator task seems to run every time. If we take a look at Gradle’s info-level logging output by running the build with --info, we will see the reason:
Executing task ':generator' (up-to-date check took 0.0 secs) due to:
Task has not declared any outputs.
We can see that Gradle does not know that the task produces any output. By default, if a task does not have any outputs, it must be considered out-of-date. Outputs are declared with the TaskOutputs. Task outputs can be files or directories. Note the use of outputs below:
build.gradle #
task generator() {
def generatedFileDir = file("$buildDir/generated")
outputs.dir generatedFileDir
doLast {
generatedFileDir.mkdirs()
for (int i=0; i<10; i++) {
new File(generatedFileDir, "${i}.txt").text = i
}
}
}
If we run the build two more times, we will see that the generator task says it is up-to-date after the first run. We can confirm this if we look at the --info output again:
Skipping task ':generator' as it is up-to-date (took 0.007 secs).
But we have introduced a new problem. If we increase the number of files generated (say, from 10 to 20), the generator task does not re-run. We could work around this by doing a clean build each time we need to change that parameter, but this workaround is error-prone.
We can tell Gradle what can impact the generator task and require it to re-execute. We can use TaskInputs to declare certain properties as inputs to the task as well as input files. If any of these inputs change, Gradle will know to execute the task. Note the use of inputs below:
build.gradle #
task generator() {
def fileCount = 10
inputs.property "fileCount", fileCount
def generatedFileDir = file("$buildDir/generated")
outputs.dir generatedFileDir
doLast {
generatedFileDir.mkdirs()
for (int i=0; i<fileCount; i++) {
new File(generatedFileDir, "${i}.txt").text = i
}
}
}
We can check this by examining the --info output after we change the value of the fileCount property:
Executing task ':generator' (up-to-date check took 0.007 secs) due to:
Value of input property 'fileCount' has changed for task ':generator'
Inferring task dependencies #
So far, we have only worked on the generator task, but we have not reduced any of the repetition we have in the build script. We have an explicit task dependency and a duplicated output directory path. Let’s try removing the task dependencies by relying on how CopySpec#from evaluates arguments with Project#files. Gradle can automatically add task dependencies for us. This also adds the output of the generator task as inputs to the zip task.
build.gradle #
task zip(type: Zip) {
from generator
}
Inferred task dependencies can be easier to maintain than explicit task dependencies when there is a strong producer-consumer relationship between tasks. When you only need some of the output from another task, explicit task dependencies will usually be cleaner. There is nothing wrong with using both explicit task dependencies and inferred dependencies, if that is easier to understand.
Simplifying with a custom task #
We call tasks like generator ad-hoc tasks. They do not have well-defined properties nor predefined actions to perform. It is okay to use ad-hoc tasks to perform simple actions, but a better practice is to move ad-hoc tasks into custom task classes. Custom tasks let you remove a lot of boilerplate and standardize common actions within your build.
Gradle makes it really easy to add new task types. You can start playing around with custom task types directly in your build file. When using annotations like @OutputDirectory, Gradle will create output directories before your task executes, so you do not have to worry about making the directories yourself. Other annotations, like @Input and @InputFiles, have the same effect as manually configuring a task’s TaskInputs.
Try creating a custom task class named Generate that produces the same output as the generator task above. Your build file should look like the following:
build.gradle #
task generator(type: Generate) {
fileCount = 20
}
task zip(type: Zip) {
from generator
}
Here is our solution:
build.gradle #
class Generate extends DefaultTask {
@Input
int fileCount = 10
@OutputDirectory
File generatedFileDir = project.file("${project.buildDir}/generated")
@TaskAction
void perform() {
for (int i=0; i<fileCount; i++) {
new File(generatedFileDir, "${i}.txt").text = i
}
}
}
Notice that we no longer need to create the output directory manually. The annotation on generatedFileDir takes care of this for us. The annotation on fileCount tells Gradle that this property should be considered an input in the same way we used inputs.property before. Finally, the annotation on perform() defines the action for Generate tasks.
Final notes about incremental builds #
When developing your own build scripts, plugins and custom tasks, declaring task inputs and outputs is an important technique to keep in your toolbox. All of the core Gradle tasks use this to great effect. If you would like to learn about other ways to make your tasks incremental at a lower level, take a look at the incubating incremental task support. Incremental tasks provide a fine-grained way of building only what has changed when a task needs to execute.