Skip to content

repository: read-only check by hashing pack/index objects#9794

Open
mr-raj12 wants to merge 2 commits into
borgbackup:masterfrom
mr-raj12:pack-files-step9-check-repo-readonly-storehash
Open

repository: read-only check by hashing pack/index objects#9794
mr-raj12 wants to merge 2 commits into
borgbackup:masterfrom
mr-raj12:pack-files-step9-check-repo-readonly-storehash

Conversation

@mr-raj12

@mr-raj12 mr-raj12 commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Makes the repository part of borg check work with packs (N>1).

packs/ and index/ objects are named by the sha256 of their content, so check just verifies store.hash(name) == name for each one. It doesn't parse pack headers at all, so a pack with many objects is checked exactly like a pack with one. On REST the hash is done server-side, so nothing is downloaded there; other backends still download the object to hash it.

The chunk index is not rebuilt anymore. A read-only check shouldn't write, and rebuilding from pack headers is unsafe: one corrupt header would write wrong offsets for every later object in that pack. Repository repair isn't done yet, so for now it runs the same read-only check and reports problems without dropping any pack (dropping it would lose the pack's other, good objects too).

Adds a progress indicator over packs/ and index/.

The archives check rebuilds its index from pack headers read-only (disable_caches) so it matches the actual packs and can still spot missing chunks. Note this is on purpose and not what #8476 suggests for the repository check: the repository check above does reuse the existing index objects, but the archives check needs an index that reflects the real packs, otherwise a stale cached entry would hide a chunk whose pack is gone. The scan reads headers only and writes nothing back.

Tests cover corruption in a later object, index corruption, an intact multi-object pack, and the progress wiring. check_cmd_test.py: 24 passed, 27 skipped.

refs #8476 #8572

@codecov

codecov Bot commented Jun 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.02326% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.48%. Comparing base (0ece418) to head (dc5aa66).
⚠️ Report is 3 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/borg/repository.py 92.85% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #9794      +/-   ##
==========================================
- Coverage   84.70%   84.48%   -0.23%     
==========================================
  Files          92       92              
  Lines       15172    15142      -30     
  Branches     2279     2271       -8     
==========================================
- Hits        12852    12792      -60     
- Misses       1617     1648      +31     
+ Partials      703      702       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@ThomasWaldmann ThomasWaldmann left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i only looked at archive and repository code, not tests.

Comment thread src/borg/repository.py Outdated
Comment thread src/borg/repository.py Outdated
Comment thread src/borg/repository.py Outdated
Comment thread src/borg/repository.py Outdated
Comment thread src/borg/repository.py Outdated
Comment thread src/borg/repository.py Outdated
Comment thread src/borg/repository.py Outdated
Comment thread src/borg/repository.py Outdated
Comment thread src/borg/repository.py Outdated
Comment thread src/borg/archive.py Outdated
per-kind counters and clearer messages; skip 4 check tests pending repair, refs borgbackup#8476 borgbackup#8572
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants