refactor: replace download-source with mount-based API by azchin · Pull Request #133 · ossf/oss-crs

azchin · 2026-03-19T13:21:30Z

Summary

Replaces the old download-source target/repo commands with a new mount-based API (download-source fuzz-proj/target-source), eliminating the complex fallback resolution chain in libCRS.

Three-phase refactor:

Mount Infrastructure — Add volume mounts for /OSS_CRS_FUZZ_PROJ (read-only) and /OSS_CRS_TARGET_SOURCE to both build-target and run-crs-compose templates. Add corresponding env vars in env_policy.py.
WORKDIR Extraction — When --target-source-path is not provided, automatically extract source from the built Docker image's WORKDIR via docker create/docker cp/docker rm. Make /OSS_CRS_TARGET_SOURCE unconditional — CRSes always find the mount present in both build and run phases.
libCRS API Migration — Replace SourceType.TARGET/REPO with SourceType.FUZZ_PROJ/TARGET_SOURCE. Rewrite download_source() as a simple 10-line mount-path copy (was ~170 lines of fallback resolution). Delete all old helper methods. Update docs.

Key changes:

libCRS download-source fuzz-proj <dest> copies from /OSS_CRS_FUZZ_PROJ
libCRS download-source target-source <dest> copies from /OSS_CRS_TARGET_SOURCE
Old target/repo subcommands removed — argparse rejects them naturally
download_source() return type changed from Path to None (no nested repo resolution needed)
~140 lines of old fallback resolution logic deleted from local.py
16 old tests replaced with 6 new mount-based tests

User Impact

User-facing change
Internal-only change

Breaking change for CRS developers:

libCRS download-source target <dest> → libCRS download-source fuzz-proj <dest>
libCRS download-source repo <dest> → libCRS download-source target-source <dest>
download_source() Python API now returns None instead of Path

CRSes using the old SourceType.TARGET/SourceType.REPO values need to update to SourceType.FUZZ_PROJ/SourceType.TARGET_SOURCE. Clean break — no deprecation period (CRSes are internal).

Release Note / Changelog

I updated CHANGELOG.md ([Unreleased]) for user-facing changes
No changelog entry needed (internal-only refactor/test/chore)

Validation

uv run pytest oss_crs/tests/unit/test_libcrs_download_source.py — 6/6 pass (new mount-based tests)
uv run pytest oss_crs/tests/unit/test_template_rendering.py — 6/6 pass (mount infrastructure)
uv run pytest oss_crs/tests/unit/test_workdir_target_source.py — 4/4 pass (WorkDir path construction)
uv run pytest oss_crs/tests/unit/test_target_workdir_extraction.py — 7/7 pass (WORKDIR extraction)
uv run pytest oss_crs/tests/unit/test_env_policy.py — pass (unconditional env vars)
uv run pytest oss_crs/tests/unit/test_crs_compose_snapshot_sanitizer.py — 6/6 pass (mock regression fixed)
E2E: ran oss-crs build-target + oss-crs run with crs-codex against sanity-mock-c-delta-01 — download-source commands work, agent receives correct source at /work/src

Checklist

I followed Conventional Commits
I updated docs for behavior/config/CLI changes
I added/updated tests for behavior changes
I considered backward compatibility and migration impact

azchin · 2026-03-19T13:30:41Z

Hm note to self: double check whether these APIs are available to builder as well (or assess whether that's necessary)

- Create TestMountInfrastructure class in test_template_rendering.py - Add unit tests for fuzz_proj_path mount in build-target template - Add unit tests for target_source_path conditional mount in build-target template - Add unit tests verifying :ro flag on both mount types - Add unit test for fuzz_proj_path mount in run-crs-compose template - Create integration test stubs in test_mount_infrastructure.py (skipped) Signed-off-by: Andrew Chin <achin34@gatech.edu>

- Add fuzz_proj_path:/OSS_CRS_FUZZ_PROJ:ro mount to build-target template - Add conditional target_source_path:/OSS_CRS_TARGET_SOURCE:ro mount to build-target template - Add same volume mounts to run-crs-compose template per-CRS service section - Update render_build_target_docker_compose() to populate fuzz_proj_path and target_source_path context - Update render_run_crs_compose_docker_compose() to populate fuzz_proj_path and target_source_path context - Fix test_renderer_run_compose.py mock target to include proj_path and _has_repo attributes Signed-off-by: Andrew Chin <achin34@gatech.edu>

…o env_policy - Add OSS_CRS_FUZZ_PROJ=/OSS_CRS_FUZZ_PROJ to build_target_builder_env() system_env - Add OSS_CRS_FUZZ_PROJ=/OSS_CRS_FUZZ_PROJ to build_run_service_env() system_env - Add include_target_source parameter to build_target_builder_env() (default False) - Add include_target_source parameter to build_run_service_env() (default False) - Conditionally add OSS_CRS_TARGET_SOURCE=/OSS_CRS_TARGET_SOURCE when include_target_source=True - Pass include_target_source=target._has_repo from both renderer callers Signed-off-by: Andrew Chin <achin34@gatech.edu>

…rget extraction methods - Test WorkDir.get_target_source_dir() path structure and create flag behavior - Test Target.extract_workdir_to_host() docker create/cp/rm lifecycle - Test Target._resolve_effective_workdir_with_inspect_fallback() fallback logic Signed-off-by: Andrew Chin <achin34@gatech.edu>

…ction methods - WorkDir.get_target_source_dir() returns path <sanitizer>/builds/<build_id>/targets/<target_key>/target-source/ - Target._resolve_effective_workdir_with_inspect_fallback() prefers non-/src Dockerfile values, falls back to docker inspect for /src ambiguity - Target.extract_workdir_to_host() orchestrates docker create/cp/rm lifecycle with finally-block cleanup on failure Signed-off-by: Andrew Chin <achin34@gatech.edu>

… templates - Remove include_target_source parameter from build_target_builder_env and build_run_service_env - Add OSS_CRS_TARGET_SOURCE unconditionally to system_env in both functions - Remove {%- if target_source_path %} guards from both compose templates - Add test_build_target_env_always_includes_target_source and test_run_env_always_includes_target_source - Rename test_target_source_mount_omitted_when_none to test_target_source_mount_always_present - Add test_target_source_unconditional_in_run_crs and get_target_source_dir to _MockWorkDir Signed-off-by: Andrew Chin <achin34@gatech.edu>

- Add resolved_source_path block in CRSCompose.build_target() after image build - Call target.extract_workdir_to_host() when target._has_repo is False - Pass target_source_path=resolved_source_path through crs.build_target lambda - Add target_source_path parameter to CRS.build_target() and __build_target_one() - Pass target_source_path to renderer.render_build_target_docker_compose - Add target_source_path parameter to render_build_target_docker_compose - Remove include_target_source from both renderer call sites - Use work_dir.get_target_source_dir for non-repo targets in run compose renderer - Add get_target_source_dir to work_dir mock in test_renderer_run_compose.py Signed-off-by: Andrew Chin <achin34@gatech.edu>

_DummyTarget and _DummyWorkDir in test_crs_compose_snapshot_sanitizer.py were missing attributes added in Phase 2, causing 6 test failures. Signed-off-by: Andrew Chin <achin34@gatech.edu>

…ount-based implementation - Replace TARGET/REPO enum values with FUZZ_PROJ/TARGET_SOURCE in base.py - Update abstract download_source signature to return None instead of Path - Add _FUZZ_PROJ_MOUNT and _TARGET_SOURCE_MOUNT module-level constants in local.py - Rewrite download_source as 10-line mount-path implementation - Delete all 5 old resolution helper methods (_resolve_repo_source_path, _normalize_repo_source_path, _translate_repo_hint_to_build_output, _resolve_downloaded_repo_path, _relative_repo_hint) - Remove unused os import from local.py - Update CLI help text for download-source parser (fuzz-proj, target-source) - Replace 16 old unit tests with 6 new mount-based tests (all passing) Signed-off-by: Andrew Chin <achin34@gatech.edu>

- Add OSS_CRS_FUZZ_PROJ and OSS_CRS_TARGET_SOURCE to env vars table - Rewrite download-source CLI section: fuzz-proj/target-source args with mount path details - Update implementation status table: Copies from /OSS_CRS_FUZZ_PROJ or /OSS_CRS_TARGET_SOURCE mounts - Remove old target/repo command references and OSS_CRS_PROJ_PATH/OSS_CRS_REPO_PATH in download-source context - Preserve builder sidecar /OSS_CRS_PROJ_PATH/test.sh reference (different context) Signed-off-by: Andrew Chin <achin34@gatech.edu>

…ew download-source API - Add download-source quick reference line with fuzz-proj/target-source types - Add OSS_CRS_FUZZ_PROJ and OSS_CRS_TARGET_SOURCE to env vars table - Add download-source to libCRS/README.md Build output command category - Preserve builder sidecar /OSS_CRS_PROJ_PATH/test.sh reference (different context) Signed-off-by: Andrew Chin <achin34@gatech.edu>

Signed-off-by: Andrew Chin <achin34@gatech.edu>

0xdkay · 2026-03-25T15:47:22Z

I can review this from Friday. If anyone available, they can do that.

…s context conflict) Signed-off-by: Dongkwan Kim <0xdkay@gmail.com>

…source Signed-off-by: Dongkwan Kim <0xdkay@gmail.com>

0xdkay

LGTM!

As we discussed previously, we will have the backward-compatibility: submit-build-output and download-build-output, to keep the directory structures from the Dockerfile if a user runs oss-fuzz projects.

azchin force-pushed the refactor/124-download-source branch 2 times, most recently from 60f840d to cfe6318 Compare March 19, 2026 13:25

azchin force-pushed the refactor/124-download-source branch 3 times, most recently from 4056c1d to 59c377f Compare March 23, 2026 20:55

This was referenced Mar 23, 2026

Refactor/download source Team-Atlanta/crs-copilot-cli#1

Open

Refactor/download source Team-Atlanta/crs-gemini-cli#1

Open

Refactor/download source Team-Atlanta/crs-claude-code#1

Open

refactor: new download-source api Team-Atlanta/crs-codex#1

Open

azchin requested a review from 0xdkay March 23, 2026 22:26

azchin added 15 commits March 25, 2026 11:31

fix(125): add _has_repo and extract_workdir_to_host to test mocks

5bddd77

_DummyTarget and _DummyWorkDir in test_crs_compose_snapshot_sanitizer.py were missing attributes added in Phase 2, causing 6 test failures. Signed-off-by: Andrew Chin <achin34@gatech.edu>

chore: gitignore

4d928e5

Signed-off-by: Andrew Chin <achin34@gatech.edu>

chore: temp bump crs-codex for new download-source

eb9914c

Signed-off-by: Andrew Chin <achin34@gatech.edu>

docs: changelog

bd74086

Signed-off-by: Andrew Chin <achin34@gatech.edu>

chore: linter

cbaa019

Signed-off-by: Andrew Chin <achin34@gatech.edu>

azchin force-pushed the refactor/124-download-source branch from bc9baad to cbaa019 Compare March 25, 2026 15:32

0xdkay added 2 commits April 3, 2026 17:33

merge main into refactor/124-download-source (resolve litellm/postgre…

ff8f0d9

…s context conflict) Signed-off-by: Dongkwan Kim <0xdkay@gmail.com>

test(133): add edge case and cross-layer contract tests for download-…

66a6d3f

…source Signed-off-by: Dongkwan Kim <0xdkay@gmail.com>

0xdkay approved these changes Apr 4, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: replace download-source with mount-based API#133

refactor: replace download-source with mount-based API#133
azchin wants to merge 17 commits intomainfrom
refactor/124-download-source

azchin commented Mar 19, 2026 •

edited

Loading

Uh oh!

azchin commented Mar 19, 2026

Uh oh!

0xdkay commented Mar 25, 2026 •

edited

Loading

Uh oh!

0xdkay left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

azchin commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

User Impact

Release Note / Changelog

Validation

Checklist

Uh oh!

azchin commented Mar 19, 2026

Uh oh!

0xdkay commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

0xdkay left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

azchin commented Mar 19, 2026 •

edited

Loading

0xdkay commented Mar 25, 2026 •

edited

Loading