Declarative Gradle
Part of our vision for Gradle Build Tool is to deliver an elegant and extensible declarative build language that allows developers to describe any kind of software in a clear and understandable way.
Gradle’s build language is already extensible in the most fundamental ways, which results in a high degree of flexibility. This is one of the main reasons Gradle is the build system of choice for many companies, including large enterprises, and the default build system for Android and Kotlin Multiplatform.
However, our build language is currently not always fully declarative, clear, and understandable. While it has long been a best practice that build scripts should be declarative, and the build logic should be kept in plugins, this is not the reality for many projects. We’ve seen projects in the wild that mix declarative and imperative code and make build scripts that are long and complex. Gradle-specific concepts used in build scripts are not always familiar to software developers. This can make Gradle less approachable for developers unfamiliar with Gradle. At the same time, it makes it difficult for the IDEs to offer reliable support for editing build scripts.
This blog post explains the Gradle team’s perspective for what we call a developer-first software definition, or Declarative Gradle for short. This post outlines our plans for making the “elegant and declarative build language” part of our vision a reality.
Software Developers and Build Engineers
Let’s first clarify the distinction between the two largest groups of Gradle users: software developers and build engineers.
The majority of Gradle Build Tool users are software developers. Their job is to improve their software product by shipping features, fixing bugs, etc. Software developers don’t start their day with the intention of running builds. While they rely on the build system to complete their tasks, it’s not their primary focus.
The second group of users are build engineers. Unlike software developers, their responsibility is to maintain the build itself. This group has historically appreciated Gradle’s flexibility to address even the most complex build automation requirements.
In practice, there’s often overlap between the two roles. In smaller teams, a few engineers familiar with Gradle tend to own the build. In larger teams, there are often dedicated build engineers.
Software Definition vs Build Logic
Next, let’s clarify the distinction between software definition and build logic.
Software definition is what needs to be built. It defines the structure of the software being produced and includes information such as:
- What kind of software is being produced?
- Which languages is the software implemented in?
- Which platforms does the software target?
- What are the dependencies of the software?
- Which toolchains should be used to compile/link/instrument/bundle/document the software?
- What quality checks need to be done before the software can be released?
Build logic is how the software will be built. It is the code that adds new capabilities to Gradle, integrates different tools, and supplies conventions to the software definition.
The software definition is meant to be read and modified by software developers. Build logic is intended to be read and modified by build engineers.
In Gradle, the software definition lives in Groovy DSL and Kotlin DSL build scripts and settings. It is recommended that build logic is kept within plugins. Nevertheless, users have the flexibility to include build logic within build scripts alongside software definitions. This has resulted in a blurring of the boundaries between the concerns and requirements of software developers and build engineers.
What Is Developer-first Software Definition
Our strategic goal is to make the software definition clearer and simpler for developers.
We see two key steps to achieve this:
Separate the software definition and build logic with restricted DSL
As mentioned above, mixing software definition and build logic is not recommended.
We plan to provide a restricted DSL that separates the software definition and build logic so that the build language is fully declarative. This will effectively enforce existing best practices.
The restricted DSL will allow only a limited set of constructs, such as nesting blocks, assigning values, and selected method invocations. Generic control flow and calls to arbitrary methods will be disallowed. You will be able to write your build logic in any JVM language, such as Java, Kotlin, or Groovy, but that logic will reside in plugins (either local or published).
This will provide a better separation between the concerns of software developers and build engineers, as well as a simpler view of the build without limiting the tool’s flexibility.
Match the software definition to the software domain
While the restricted DSL is a step in the right direction, it is only a partial solution to make the software definition truly developer-first.
The other problem is that the software definition currently does not use concepts that reflect the underlying software domain in all cases. To work with Gradle, developers typically need to understand Gradle-specific building blocks, such as dependency configurations and tasks. These concepts are specific to build logic and should therefore only concern build engineers. In most cases, software developers should be able to configure everything they need by using concepts they are already familiar with, such as libraries, applications, versions, and test suites.
The Android Gradle Plugin is a good example of a more declarative software definition DSL based on Gradle. While the broader JVM ecosystem hasn’t reached our desired state, recent features, such as test suites and toolchains, were a notable step in this direction.
One of our goals is to provide a broader set of such high-level constructs that can be shared across ecosystems. This will make existing ecosystems more consistent, such as Java, Android, and KMP. It will also make it much easier to implement support for new ecosystems, such as iOS or .NET, in the future.
IDE Experience
The crucial aspect of the developer-first software definition is that it makes providing an excellent IDE experience easier.
The software definition is traditionally edited by hand. With Groovy DSL, only rudimentary IDE assistance was possible because of the dynamic nature of the Groovy language. With Kotlin DSL, excellent support is available in supported IDEs such as IntelliJ IDEA and Android Studio. This includes auto-completion, smart content assist, quick access to documentation, navigation to source, and context-aware refactoring. With Kotlin DSL, you can edit your build logic with the same editing experience you are used to when working with your production code.
However, it is difficult for IDE vendors to automatically change the software definition or create a UI editor when arbitrary code is possible in build scripts. By making the software definition fully declarative, support will be much easier. We know that IDE vendors are very interested in providing such capability. With that, software developers can choose the mode of editing (UI or by hand) based on their preferences and level of expertise.
There are also performance considerations. For example, the IDE sync process involves a series of steps, including complex calculations such as dependency resolution, to provide a complete view of the project in the IDE. With a fully declarative software definition that can be parsed swiftly and reliably, we can promptly provide an initial view of the project in the IDE, even for large projects.
Collaboration Between Gradle, Google and JetBrains
Gradle, Google, and JetBrains have a long and fruitful history of collaboration. Our last major joint effort resulted in Kotlin DSL Becoming the Default for New Gradle Builds. This triggered an explosion in the popularity of Kotlin DSL and enabled many users to benefit from the full IDE assistance for Gradle build scripts in the supported IDEs.
When we met in person this October with representatives from all three companies, we put most of our attention into declarative software definition. We all recognize the existing constraints to user experience resulting from the current mixture of software definition with build logic and lower-level constructs in many projects. Gradle, Google, and JetBrains are collaborating on delivering a solution to those problems. We are working closely to ensure the solution will work well for Android and Kotlin Multiplatform ecosystems with excellent IDE support.
Gradle Team drives this project and collaborates closely with JetBrains and Google to ensure excellent IDE support in IntelliJ IDEA and Android Studio. This project is one of the top priorities for Gradle. We are already intensively working on an initial prototype. Early next year, we plan to share an initial solution for simple projects with early adopters, starting with Android.
At the same time, JetBrains is running an experiment with similar goals called Amper. One of our objectives is to leverage this experiment to benefit the development of Declarative Gradle and provide a great user experience for Kotlin and Kotlin Multiplatform. Therefore, in addition to the dedicated group focused on Declarative Gradle, we are working closely with the Amper team at JetBrains.
We hope this collaboration will improve user experience across multiple ecosystems, including Java, Android, and Kotlin Multiplatform.
Transition
We are excited about the developer-first software definition and strongly believe it will make both software developers and build engineers using Gradle happier. At the same time, we are mindful of billions of lines of code that are currently not compatible with this approach.
One of our goals is to make it easy for teams to migrate to the new software definition incrementally once it’s available. We are currently investigating using a restricted DSL syntax that would be compatible with existing declarative Kotlin DSL scripts. The restricted DSL will be an additional option next to Groovy and Kotlin DSL. The existing DSL will continue to be fully supported, and we are exploring ways to allow incremental migration of build files to restricted DSL. We are also looking into tooling that could assist with the transition.
Feedback
We’d love to know how you feel about our vision and plans for developer-first software definition. As always, let us know if you have any questions or feedback on our forums or Community Slack.