fix(dedupe): stable content-based tiebreaker for import/reimport ordering

[Maffooch]

[sc-13656]

Problem

Deduplication chose the "original" among findings that collide on the dedupe key by lowest DB id — i.e. whichever the scanner listed first in the file. Scanners don't guarantee a stable finding order between scans, so the original/duplicate assignment flips back and forth between scans even though the underlying vulnerabilities never change. Reported via a customer support ticket; observed with Fortify but not tool-specific (any parser without a stable export order can trigger it). Discovered via reimport, but the same class of bug exists on the import/deduplication path.

Fix

• Add deduplication_ordering_key() — a stable key derived from finding content (hash_code, unique_id_from_tool, file_path, line, title, then id as final tiebreak).
• Make _is_candidate_older() batch-aware: findings that predate the current import batch keep id ordering (so an already-established original never flips), while ties within a single import/report are broken by the content key instead of parser emission order.
• Order candidate lists and the batch itself by this key across all four algorithms (hash_code, unique_id_from_tool, uid_or_hash, legacy) and in the reimport matcher.

Backward compatibility

• No migration; no change to existing data. Dedup only runs during import/reimport.
• Pre-existing findings always keep the original role (covered by a dedicated test).
• Only the previously-arbitrary intra-batch / intra-report winner changes, and only for findings that collide on the dedupe key but differ in content.

Tests

• Content-order unit tests (hash_code path) asserting the content-canonical finding wins regardless of creation order, plus a pre-existing-stays-original guard.
• End-to-end Fortify fixtures: two .fpr files whose findings share a unique_id_from_tool but differ in content, in opposite document order; the surviving original must be identical across both orders. Fails on the pre-fix code (surviving finding flips), passes with the fix.
• Note for reviewers: the Fortify test forces unique_id_from_tool dedup via override_settings (open source defaults Fortify to legacy, where different-content findings would not collide at all). This mirrors the realistic customer configuration.

Related

Pairs with the corresponding Dojo Pro change, which mirrors this fix in Pro's dedupe engine and depends on the batch-aware _is_candidate_older and the exported ordering helpers here. They should merge together.

🤖 Generated with Claude Code