how to configure for incremental builds and task avoidance

[radopas]

Hi,

I have a big multi-module project and spotbugs is the slowest part of the build.
Is it possible to configure the spotbugs tasks so they are used incrementally only on changed sources and even skip the task execution if there is no changes in the sources?

At the moment the spotbugs tasks are executed on every module every time no matter if there are changes or not.

I'm using the Gradle 8.0.2 and 'com.github.spotbugs.snom:spotbugs-gradle-plugin:5.0.12'

[radopas]

Here is what i've managed to fabricate, but i'm new to gradle and i'm sure it can be done in a better way.
Note: I've attached the execution to "classes" as our company has strong policy about code conforming to spotbugs so i wanted to execute the spotbug check as soon as possible. And spotbugsTest is disabled because we have no policy about tests, but i guess a similar configuration could be implemented for tests too.

PS: The spotbugs invocation can be further "incremented" by using Incremental inputs, but i think it would require some changes in the code of the plugin.

def spotbugsReportFile = file("${buildDir}/spotbugs/spotbugs.html")

tasks.named("spotbugsMain"){
    enabled = false

    reports {
        html {
            required = true
            outputLocation = spotbugsReportFile
        }
    }
}

tasks.register("spotbugsCaching"){
    description "declares inputs and outputs so the gradle incremental rules apply"
    //https://docs.gradle.org/current/userguide/incremental_build.html#incremental_build

    dependsOn "compileJava"

    outputs.file(spotbugsReportFile)

    finalizedBy "spotbugsMain"

    doLast{
        tasks.spotbugsMain.enabled(true)
    }
}

tasks.register("spotbugsChecking"){
    description "check spotbugs results, when the cached task is skipped because of incremental builds"

    doLast{
        if(!tasks.spotbugsCaching.didWork) {
            //TODO: somehow store and use findBugs2.getErrorCount() or errorCountingBugReporter.getErrorCount(), instead of relying on the report and some text inside.
            String marker = "warningtable";
            if (spotbugsReportFile.exists()) {
                if (spotbugsReportFile.getText("UTF-8").indexOf(marker) != -1) {
                    throw new GradleException("Spotbugs errors! See the report at: ${spotbugsReportFile}")
                }
            }
        }
    }
}

tasks.named("classes"){
    dependsOn("spotbugsCaching")
    dependsOn("spotbugsChecking")
}

[Vampire]

That's actually a big ball of mixed together bad practices that you have there and that most likely also does not really work as intended.
Just remove all those lines and you should have the intended result.

There is no incremental analysis I'm aware of in SpotBugs.
But that the task is not re-executed if nothing was changed should be the case.
If it is not, look at the --info output or a build --scan to find out why the task was re-executed.

[radopas]

But in vanilla configuration spotbugs is skipped only if there were no errors in the previous build.
The goal is to skip it also when there were spotbugs errors in the previous build and to fail the build without executing the whole analysis again.

I've cleared the first version to this one. I'm still quite new to gradle and there probably are still lots wrong things, so i'm grateful for any improvements.

def spotbugsReportFile = file("${buildDir}/spotbugs/spotbugs.html")

tasks.named("spotbugsMain"){
  //will be enabled by spotbugs if need to be run
  enabled = false

  reports {
    html {
      required = true
      outputLocation = spotbugsReportFile
    }
  }
}

tasks.named("compileJava"){
  doLast{
    //clean the spotbugs report file, so the spotbugs is not cached
    delete(spotbugsReportFile);
  }
}

tasks.register("spotbugs"){
  description = "Use this task to invoke spotbugs checks"
  group = JavaBasePlugin.VERIFICATION_GROUP

  dependsOn "compileJava"
  //initially is disabled - will be enabled if needed
  finalizedBy "spotbugsMain"

  doLast{

    //if spotbugs was cached - check the results file, and fail if there is error
    if (spotbugsReportFile.exists()) {
      String marker = "warningtable";
      if (spotbugsReportFile.getText("UTF-8").indexOf(marker) != -1) {
        throw new GradleException("Spotbugs errors! See the report at: ${spotbugsReportFile}")
      }
    }
    else{
      //the spotbugs error status is unknown so enable execution of spotbugsMain to recalculate it
      tasks.spotbugsMain.enabled(true)
    }
  }
}

[Vampire]

Ah, yeah, a failing task is of course not up-to-date or cached, that's correct and expected and you really should not try to.
Especially changing the configuration of the spotbugsMain task from the execution phase of the spotbugs task is not compatible to configuration cache, so you can never use it while executing spotbugs.

You should probably instead set the ignoreFailures property of the SpotBugsTask to true, then iirc the task will not fail, no matter whether there are findings or not.

And then you can have another task that finalizes the SpotBugsTask and consumes its XML report to then fail if there were findings.
I think that should achieve what you want in a clean way.

Btw. deleting the output file of the spotbugs task will not make it not be cached. Only not up-to-date, but the result can still be taken from the cache.

[Vampire]

Here some code that should do exactly what you intend and is fully configuration cache compatible:

tasks.withType<SpotBugsTask>().configureEach {
    ignoreFailures = true

    reports {
        create("text")
        create("html")
    }
}

sourceSets.configureEach {
    val spotbugsTaskName = getTaskName("spotbugs", null)

    val spotbugsTask = tasks.named<SpotBugsTask>(spotbugsTaskName)

    val spotbugsVerify = tasks.register("${spotbugsTaskName}Verify") {
        val textReport = spotbugsTask.flatMap { it.reports["text"].outputLocation }
        val htmlReport = spotbugsTask.flatMap { it.reports["html"].outputLocation }
        doLast {
            check(textReport.get().asFile.readBytes().isEmpty()) {
                "$spotbugsTaskName failed with errors see HTML report at ${htmlReport.get().asFile.toURI().toURL()}"
            }
        }
    }

    spotbugsTask {
        finalizedBy(spotbugsVerify)
    }
}

[radopas]

Thank you. The flag ignoreFailures=true was the key. I've removed all the trickery from my script and now it seems to be caching both failures and successes. Here is my current version that i use:

def spotbugsReportFile = file("${buildDir}/spotbugs/spotbugs.html")

tasks.named("spotbugsMain"){
  ignoreFailures = true

  reports {
    html {
      required = true
      outputLocation = spotbugsReportFile
    }
  }
}


tasks.register("spotbugs"){
  description = "Use this task to invoke spotbugs checks"
  group = JavaBasePlugin.VERIFICATION_GROUP

  dependsOn "spotbugsMain"

  doLast{

    //if spotbugs was executed before - check the results file, and fail if there is error
    if (spotbugsReportFile.exists()) {
      String marker = "warningtable";
      if (spotbugsReportFile.getText("UTF-8").indexOf(marker) != -1) {
        throw new GradleException("Spotbugs errors! See the report at: ${spotbugsReportFile}")
      }
    }
    
  }
}