[Android] Add --experimental_test_content_key for test execution avoidance#38
Closed
azad-uber-2 wants to merge 1 commit into
Closed
Conversation
…n avoidance Summary: Adds --experimental_test_content_key, a new Bazel flag that makes remote test cache hits insensitive to unused dependency changes. Problem Today, when a transitive dependency changes (even one whose classes the test never uses), the TestRunner action's Merkle tree changes and the test re-executes. This causes unnecessary test runs in CI, especially for large targets with many transitive deps — a compile-time ABI change to an unused library invalidates every test that transitively depends on it, even though the test's behavior is completely unchanged. Solution When --experimental_test_content_key is enabled, two things happen atomically for each TestRunner spawn: 1. Content key injection — RemoteExecutionService reads a .content_key file produced by a build aspect (see android-content-key-remote-cache repo). This file contains a SHA-256 hash of only the class bytecodes the test transitively uses (derived from .jdeps files). The hash is appended to the cache salt, becoming the sole discriminator for what the test actually depends on at the class level. 2. Jar omission — SpawnScrubber.withJarOmission() is applied programmatically to exclude first-party .jar inputs from the Merkle tree. External/maven jars (under .../bin/external/) are intentionally kept in the Merkle tree so version bumps are caught naturally. This keeps jar-omission in sync with key injection without requiring an xplat.cfg entry. Atomic gate: both steps are gated on the content key file being present on disk. If the file is absent (aspect not run, --experimental_remote_download_regex not set, stale files from a disabled flag, etc.), jar omission is suppressed and the full Merkle tree acts as the discriminator — no false cache hits. ┌──────────────────────────────────────┬────────────────────────┬────────────┬──────────────────────────────────────┐ │ Scenario │ Jar omission │ CK in salt │ Result │ ├──────────────────────────────────────┼────────────────────────┼────────────┼──────────────────────────────────────┤ │ Flag OFF │ No │ No │ Standard Bazel key (unchanged) │ ├──────────────────────────────────────┼────────────────────────┼────────────┼──────────────────────────────────────┤ │ Flag ON, CK absent │ No │ No │ Full jar Merkle = safe discriminator │ ├──────────────────────────────────────┼────────────────────────┼────────────┼──────────────────────────────────────┤ │ Flag ON, CK present │ Yes (first-party only) │ Yes │ CK-based cache key │ ├──────────────────────────────────────┼────────────────────────┼────────────┼──────────────────────────────────────┤ │ Flag OFF, stale .content_key on disk │ No │ No │ Stale files ignored — safe │ └──────────────────────────────────────┴────────────────────────┴────────────┴──────────────────────────────────────┘
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cherry-pick of 67d336c from
uber/android/7.6.1touber/android/8.1.1with conflict resolution for thebuildSalt()static method signature (Bazel 8 keeps it static; addedcontentKeyparameter alongside existingmnemonicCacheSalts).Adds
--experimental_test_content_key, a flag that makes remote test cache hits insensitive to unused dependency changes. When enabled and a.content_keyfile is present (produced by a build aspect), the content key hash replaces first-party jar inputs in the cache discriminator. When the file is absent, standard Bazel caching behavior is used (safe fallback).Test plan
//experimental/sample/simple/...successfully with the flag enabled.content_keyfile exists.content_keyfiles, confirmed changing the content key causes cache miss and re-execution, same key gets cache hit🤖 Generated with Claude Code
Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com