Skip to content

Conversation

@SuaYoo
Copy link
Member

@SuaYoo SuaYoo commented Dec 9, 2025

Resolves #2874
WIP for #2864 and #2863 (see follow-ups)

Changes

  • Displays whether dedupe is enabled in workflow, archived item, and collection list views.
  • Displays warning notice in crawls with dependencies.
  • Adds new "Dependencies" tab to crawls.
  • Adds new "Deduplication" tab to collections.
  • Adds new "Deduplication" tab to org settings.
  • Dexex: Adds MSW for mock API testing in Storybook.

Manual testing

  1. Log in as crawler
  2. Enable deduplication in a workflow and run crawls. Verify that dedupe state is intuitively displayed in workflows, crawls, and collections.

Screenshots

Page Image/video
Workflows Screenshot 2025-12-09 at 9 08 19 AM
Workflow Screenshot 2025-12-09 at 8 16 12 AM
Workflow / Latest Crawl Screenshot 2025-12-09 at 10 00 34 AM
Archived Items Screenshot 2025-12-09 at 10 19 05 AM
Archived Item Screenshot 2025-12-09 at 9 29 33 AM
Archived Item / QA Screenshot 2025-12-09 at 9 46 12 AM
Archived Item / Files Screenshot 2025-12-09 at 10 00 21 AM
Archived Item / Dependencies Screenshot 2025-12-09 at 11 24 26 AM
Collections Screenshot 2025-12-09 at 10 19 16 AM
Collection / Deduplication (not enabled) Screenshot 2025-12-09 at 11 23 34 AM
Collection / Deduplication Screenshot 2025-12-09 at 6 12 08 PM

Follow-ups

Dedupe stats and index management is not included in this PR. The collection stats UI and action menu in org settings is commented out while #2861 is in progress. See remaining tasks in #2864 and #2863

Dependencies are not currently paginated.

@socket-security

This comment was marked as resolved.

@socket-security

This comment was marked as off-topic.

@SuaYoo SuaYoo requested review from emma-sg, ikreymer and tw4l December 9, 2025 02:34
@SuaYoo SuaYoo marked this pull request as ready for review December 9, 2025 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants