I've spent a fair amount of time writing Gradle plugins, and if there's one thing I've learned, it's that testing them is a completely different beast than testing a standard Kotlin library. You can't just instantiate a class, call a method, and assert a result. Why? Because your plugin doesn't live in a vacuum—it lives inside a Gradle Project object, which is an absolute monster of a class to mock.

The "Just Mock It" Trap

When I first started with this, I tried to use Mockito. I thought, "I'll just mock the Project and the Task objects." It looked like this:

val project = mock(Project::class.java)
val plugin = MyReportPlugin()
plugin.apply(project)
verify(project).extension(...) 

It worked for the simplest things, but the moment I tried to actually run a task, everything collapsed. Gradle's internal state is deeply intertwined. Mocking a Project doesn't give you a working project; it gives you a hollow shell that throws NullPointerException the second Gradle tries to resolve a dependency or look up a file system path. I realized quickly that I needed a real Gradle environment, but I couldn't possibly create a whole new physical folder and run ./gradlew manually for every single test case.

Entering the Sandbox

That's where the gradle-test-kit comes in. It provides a tool called GradleRunner. Instead of mocking, the idea is to create a "functional" test: we create a temporary directory on disk, write a real build.gradle.kts file into it, and tell Gradle to execute it.

I started by setting up a temporary folder. Since I'm using JUnit 5, the @TempDir annotation is my best friend here. I'll try to run a simple plugin that creates a file called version.txt in the project root.

@TempDir
lateinit var testProjectDir: File

@Test
fun `plugin creates version file`() {
    testProjectDir.resolve("settings.gradle.kts").writeText("")
    testProjectDir.resolve("build.gradle.kts").writeText("""
        plugins {
            id("com.example.version-plugin")
        }
    """.trimIndent())

    GradleRunner.create()
        .withProjectDir(testProjectDir)
        .withArguments("createVersionFile")
        .withPluginClasspath() // This is the magic bit
        .build()
}

I ran this, and... it failed immediately. Plugin [id: 'com.example.version-plugin'] was not found. This is the most frustrating part of plugin testing: the "fake" project has no idea that the plugin code is currently running in the same JVM as the test.

Connecting the Plugin to the Runner

The .withPluginClasspath() method is supposed to handle this, but it only works if the plugin is actually on the classpath of the test worker. I discovered that I needed to make sure my build.gradle.kts for the plugin project itself included the kotlin-dsl plugin and that the test dependencies were correctly aligned.

Once I fixed the classpath, I hit another snag. The build "succeeded," but I hadn't actually checked if the file existed. I was just trusting the exit code. That's a dangerous habit. I added a check to verify the file system state after GradleRunner finished:

val result = GradleRunner.create()
    .withProjectDir(testProjectDir)
    .withArguments("createVersionFile")
    .withPluginClasspath()
    .build()

assertTrue(result.output.contains("Version file created"))
assertTrue(testProjectDir.resolve("version.txt").exists())

Now we're getting somewhere. I can see the console output and verify the side effects on the disk. This is the "Gold Standard" for plugin testing: you treat the plugin as a black box, feed it a project configuration, and observe the result.

Testing for Failure

But what happens when things go wrong? A good plugin should fail gracefully with a helpful message. I wanted to test that my plugin throws an error if a certain property is missing. If I just call .build(), the test fails because the Gradle build failed. That's not what I want; I want the test to pass because the build failed as expected.

I swapped .build() for .buildAndFail(). This tells the TestKit, "I expect this build to crash."

val result = GradleRunner.create()
    .withProjectDir(testProjectDir)
    .withArguments("createVersionFile")
    .withPluginClasspath()
    .buildAndFail()

assertTrue(result.output.contains("Error: version property is missing"))

It's a simple change, but it transforms the test suite from a "happy path" check into a robust safety net. By simulating real project directories and using GradleRunner, we've moved away from fragile mocks and toward tests that actually prove the plugin works in the wild.

📋 Practical Task

Build Failure Validation: The "Missing Manifest" Plugin

Your task is to write a functional test for a Gradle plugin. You don't need to write the plugin logic itself—assume a plugin exists called com.example.manifest-checker that adds a task called checkManifest. This task should fail the build if a file named manifest.txt is missing from the project root.

Create a test class using GradleRunner and @TempDir that performs the following:

1. Sets up a temporary project directory with a settings.gradle.kts and a build.gradle.kts that applies the com.example.manifest-checker plugin.
2. Executes the checkManifest task without creating the manifest.txt file.
3. Uses buildAndFail() to assert that the build fails.
4. Asserts that the output contains the specific error message: "Manifest file is missing! Please create manifest.txt".