Skip to content

Switch to ldk-fuzzing-corpus repo rather than CI cache#4588

Merged
valentinewallace merged 1 commit into
lightningdevkit:mainfrom
TheBlueMatt:2026-05-ci-corpus-repo
May 4, 2026
Merged

Switch to ldk-fuzzing-corpus repo rather than CI cache#4588
valentinewallace merged 1 commit into
lightningdevkit:mainfrom
TheBlueMatt:2026-05-ci-corpus-repo

Conversation

@TheBlueMatt
Copy link
Copy Markdown
Collaborator

Rather than storing our fuzzing corpus in the CI cache, move it to a new repo which anyone can use for their own local fuzzing and can be updated outside of CI with additional seeds.

@ldk-reviews-bot
Copy link
Copy Markdown

ldk-reviews-bot commented May 4, 2026

I've assigned @valentinewallace as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

Comment thread .github/workflows/build.yml Outdated
Comment thread .github/workflows/build.yml Outdated
@ldk-claude-review-bot
Copy link
Copy Markdown
Collaborator

ldk-claude-review-bot commented May 4, 2026

I've thoroughly re-examined every hunk in the diff, cross-referencing with ci-fuzz.sh and generate_fuzz_coverage.sh. After careful analysis, I have no new issues to report beyond the prior review pass.

Review Summary

No new issues found in this review pass.

Corrections to prior review

Upon re-examination, two of my prior inline comments on the coverage job (.github/workflows/build.yml:87-95) were false positives and should be disregarded:

  1. "Coverage job cp -r uses wrong relative path with ../../" — Incorrect. The coverage job uses cp -r "ldk-fuzzing-corpus/rust-lightning/${NAME}" "hfuzz_workspace/${NAME}_target/input", which is the correct relative path from CWD (fuzz/). The ../../ prefix only appears in the fuzz job's ln -sfn, where it is correct (symlink targets resolve relative to the symlink's location, not CWD).

  2. "Coverage job gets zero corpus files due to find not following symlinks" — Incorrect. The coverage job uses cp -r to create real directories, not symlinks. The find in generate_fuzz_coverage.sh:98 works correctly on real directories.

Prior comments that remain valid (not repeated)

  • .github/workflows/build.yml:274-284 — SIG* crash files staged to wrong path (new-corpus/$NAME vs new-corpus/rust-lightning/$NAME)
  • .github/workflows/build.yml:273 — Clone failure causes staging step to crash with git error
  • .github/workflows/push-fuzz-corpus.yml:92--base master may not match the corpus repo's default branch
  • .github/workflows/push-fuzz-corpus.yml:10 — Comments reference master but rust-lightning's default branch is main

Minor observation (not worth an inline comment)

  • Coverage job step name at line 87 says "Symlink corpus into hfuzz_workspace" but actually uses cp -r (copy-paste from the fuzz job without updating the name).

Comment thread .github/workflows/build.yml Outdated
@codecov
Copy link
Copy Markdown

codecov Bot commented May 4, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.85%. Comparing base (8712856) to head (efa95b4).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4588      +/-   ##
==========================================
- Coverage   87.17%   86.85%   -0.32%     
==========================================
  Files         161      161              
  Lines      109260   109260              
  Branches   109260   109260              
==========================================
- Hits        95244    94899     -345     
- Misses      11544    11781     +237     
- Partials     2472     2580     +108     
Flag Coverage Δ
fuzzing-fake-hashes 26.82% <ø> (-4.38%) ⬇️
fuzzing-real-hashes 20.62% <ø> (-2.32%) ⬇️
tests 86.23% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@TheBlueMatt TheBlueMatt force-pushed the 2026-05-ci-corpus-repo branch from af96cf5 to 2ec72ad Compare May 4, 2026 13:53
@TheBlueMatt
Copy link
Copy Markdown
Collaborator Author

Okay, updated to run on PRs properly, but sadly we'll have to actually merge this to test it.

Comment thread .github/workflows/push-fuzz-corpus.yml Fixed
Comment thread .github/workflows/build.yml
@TheBlueMatt TheBlueMatt force-pushed the 2026-05-ci-corpus-repo branch 2 times, most recently from aa96a76 to ae92c0e Compare May 4, 2026 14:03
Comment thread .github/workflows/build.yml
@TheBlueMatt TheBlueMatt force-pushed the 2026-05-ci-corpus-repo branch 2 times, most recently from 6837950 to 6176a55 Compare May 4, 2026 14:05
Comment thread .github/workflows/build.yml Outdated
Comment thread .github/workflows/push-fuzz-corpus.yml Outdated
Rather than storing our fuzzing corpus in the CI cache, move it to
a new repo which anyone can use for their own local fuzzing and
can be updated outside of CI with additional seeds.
@TheBlueMatt TheBlueMatt force-pushed the 2026-05-ci-corpus-repo branch from 6176a55 to efa95b4 Compare May 4, 2026 14:21
Copy link
Copy Markdown
Contributor

@valentinewallace valentinewallace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Landing to see if it works

#
# Caveat: GitHub only fires `workflow_run` for workflow files that live on
# the default branch, so this workflow does nothing until it's merged to
# `master`.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: main

@valentinewallace valentinewallace merged commit 1a26867 into lightningdevkit:main May 4, 2026
24 of 25 checks passed
@TheBlueMatt
Copy link
Copy Markdown
Collaborator Author

Seems to work: lightningdevkit/ldk-fuzzing-corpus#3 I have a feeling that repo's git history is going to get...bad

@joostjager
Copy link
Copy Markdown
Contributor

Wanted to add a few of my strings in lightningdevkit/ldk-fuzzing-corpus#4, but it complains

@TheBlueMatt
Copy link
Copy Markdown
Collaborator Author

Complains how? If you open a PR that is add-files-only it should get auto-merged daily.

@joostjager
Copy link
Copy Markdown
Contributor

It says "Merging is blocked. Cannot update this protected ref.". Or is that expected?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants