Refactored buildSrc Gradle-based tests around GradleFixture and reduce resource use. #11370
Refactored buildSrc Gradle-based tests around GradleFixture and reduce resource use. #11370AlexeyKuznetsov-DD wants to merge 6 commits into
buildSrc Gradle-based tests around GradleFixture and reduce resource use. #11370Conversation
bric3
left a comment
There was a problem hiding this comment.
This change is nice, however I'd rather not switch back to groovy dsl.
Since it looks ai generated for some parts, I'd rather convert the remaining fixture to kotlin dsl.
| file(projectDir, "build.gradle.kts").writeText( | ||
| writeRootProject( |
There was a problem hiding this comment.
This switched to groovy dsl.
There was a problem hiding this comment.
@bric3 thanks for pointing to this!
I switched all scripts to Kotlin DSL. And ./gradlew -PrunBuildSrcTests=true :buildSrc:test started to take ~20min. I investigated and after couple of rounds of refactoring I was able to do more-or less elegant way to reuse Gradle daemons! Now this test suite passed in ~1.5 mins locally and do not consume insane amount of RAM. But the change is quite huge and non trivial, please do one more round of review. With daemons reuse results are very impressive.
There was a problem hiding this comment.
Cool thanks for this ! Indeed that's the way to go with Gradle : "daemon reuse". But it may be possible some tests need to offer a way for non reusable daemons for some tests 🤔 , maybe not yet though.
I'll take a second review
The Groovy→Kotlin DSL conversion of test scripts pushed :buildSrc:test from 6 min to ~11 min because every test method killed its TestKit daemon, forcing kotlinc to recompile every .gradle.kts in the next test. Move testKitDir into a JVM-wide companion-object lazy val, remove the per-test stopDaemons() call, and reap daemons once from the shutdown hook. Shared daemons cut :buildSrc:test from 10 min 40 s down to 1 min 36 s on Kotlin DSL (and ~3.8× faster than the original Groovy DSL runtime). Fix MuzzleMavenRepoUtils.MUZZLE_REPOS, which cached MAVEN_REPOSITORY_PROXY in a daemon-static `by lazy` val. With per-test daemons the cache was harmless; with daemon reuse it leaked a stale repo URL into the next test's resolution and caused version-range queries to fail. Replace it with defaultMuzzleRepos(), a function that reads the env on each call. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
What Does This Do
Shares the TestKit directory used by
GradleFixtureso the Gradle daemons spawned bybuildSrcplugin tests can be reused across test methods. Several of these tests run real Gradle builds, including Kotlin Gradle DSL script compilation, and paying for a fresh TestKit daemon/cache on every test was expensive incheck_build_src.The fixture keeps the shared TestKit directory outside each JUnit
@TempDirso temp-dir cleanup does not race daemon file locks, and it stops the TestKit daemons from a JVM shutdown hook. Because shared daemons can also share plugin classloader state,MuzzleMavenRepoUtilsno longer caches theMAVEN_REPOSITORY_PROXY-derived repository list in a daemon-static lazy value; it now builds the default muzzle repository list on demand.The duplicated
GradleRunnersetup in individual tests has also been consolidated intoGradleFixture. Test classes now extend the fixture directly and use focused helpers such aswriteFile,writeRootProject,writeSettings,writeGradleProperties,writeJavaSource,addSubproject,dir,file, andbuildFilethat read more like a DSL than boilerplate.Motivation
Reduce
check_build_srcoverhead. Reusing the TestKit daemon/cache amortizes Gradle and Kotlin DSL setup across the buildSrc plugin tests.Keep daemon reuse correct. Shared daemons make daemon-static state visible across fixture builds, so environment-derived muzzle repository configuration must be recomputed instead of cached.
One way to write a fixture test. Centralizing
GradleRunnersetup and project-file helpers makes the tests shorter and keeps build fixture behavior uniform.Additional Notes
Stats from GitLab