Skip to content

fix(kb): auto-refresh documents after connector sync and remove redundant invalidations#3601

Merged
waleedlatif1 merged 1 commit intostagingfrom
waleedlatif1/fix-kb-connector-refresh
Mar 15, 2026
Merged

fix(kb): auto-refresh documents after connector sync and remove redundant invalidations#3601
waleedlatif1 merged 1 commit intostagingfrom
waleedlatif1/fix-kb-connector-refresh

Conversation

@waleedlatif1
Copy link
Collaborator

Summary

  • Fix KB documents not appearing after connector sync without manual page refresh
  • Add document query invalidation to useCreateConnector mutation hook
  • Poll document list while connectors are syncing (5s interval)
  • Refresh documents when connector sync completes
  • Remove redundant refreshDocuments()/refreshTagDefinitions() calls from component callbacks — mutation hooks already handle cache invalidation via TanStack Query prefix matching

Type of Change

  • Bug fix

Testing

Tested manually

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@cursor
Copy link

cursor bot commented Mar 15, 2026

PR Summary

Medium Risk
Changes React Query invalidation and polling behavior for knowledge base documents/connectors; risk is moderate due to potential missed/extra refetches or stale UI if query keys/refetch intervals are wrong.

Overview
Fixes knowledge base document lists not updating after connector sync by polling documents while any connector is syncing and explicitly refreshing the documents/KB detail when syncing transitions to complete.

Simplifies cache management by removing several component-level refreshDocuments/invalidateQueries calls (including document tags updates) and adjusting connector mutation hooks to invalidate broader knowledgeKeys.detail(...) / connectorKeys.all(...) instead of only connector list/detail keys; bulk chunk enable/disable now only updates local chunk state when the operation fully succeeds.

Written by Cursor Bugbot for commit efe4816. Configure here.

@vercel
Copy link

vercel bot commented Mar 15, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Mar 15, 2026 11:15am

Request Review

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 15, 2026

Greptile Summary

This PR fixes KB documents not appearing after connector sync by adding document list polling during sync (5s interval via refetchInterval) and an explicit refresh when sync transitions from syncing to active. It also broadens the useCreateConnector cache invalidation to cover the full knowledgeKeys.detail scope (which includes documents), ensuring new connector syncs populate documents without a manual page refresh.

As a secondary clean-up, all manual refreshDocuments(), refreshTagDefinitions(), and refreshChunks() calls in component onSuccess callbacks are removed. This is safe because:

  • useBulkChunkOperation already invalidates knowledgeKeys.document(knowledgeBaseId, documentId), which covers chunk queries via prefix matching.
  • useUpdateDocumentTags already invalidates both knowledgeKeys.detail and knowledgeKeys.document, making the removed handleDocumentTagsUpdate callback in document.tsx redundant.
  • useCreateTagDefinition and useDeleteTagDefinition both invalidate knowledgeKeys.tagDefinitions, so the removed refreshTagDefinitions() calls in base-tags-modal.tsx were redundant.

The hasSyncingConnectorsRef ref is intentional — it gives the refetchInterval callback (a stale closure in useKnowledgeBaseDocuments) access to the latest connector-syncing state without re-creating the query options on every render.

Confidence Score: 5/5

  • This PR is safe to merge — all manual refresh removals are backed by verified mutation-hook cache invalidations.
  • All four changed files have been cross-checked against their underlying mutation hooks in knowledge.ts. Every removed refreshX() call is genuinely superseded by existing invalidateQueries calls in the corresponding mutation hooks, and the new polling/refresh logic in base.tsx correctly models the sync lifecycle. No regressions or logical gaps were found.
  • No files require special attention.

Important Files Changed

Filename Overview
apps/sim/hooks/queries/kb/connectors.ts Consolidates query invalidations across all connector mutations to use knowledgeKeys.detail (or connectorKeys.all for update), eliminating the previously redundant connectorKeys.list invalidations; adds document query coverage to useCreateConnector.
apps/sim/app/workspace/[workspaceId]/knowledge/[id]/base.tsx Moves useConnectorList before useKnowledgeBaseDocuments to enable connector-status-aware polling; adds 5s document poll interval during sync and an explicit refreshDocuments() on sync completion; removes redundant manual refresh calls from mutation onSuccess callbacks.
apps/sim/app/workspace/[workspaceId]/knowledge/[id]/[documentId]/document.tsx Removes handleDocumentTagsUpdate / onDocumentUpdate (superseded by useUpdateDocumentTags own invalidation); inverts bulk-chunk-operation condition so that delete and error paths no longer need a manual refreshChunks() call, relying on useBulkChunkOperation cache invalidation instead.
apps/sim/app/workspace/[workspaceId]/knowledge/[id]/components/base-tags-modal/base-tags-modal.tsx Removes refreshTagDefinitions() from after create/delete tag mutation calls; useCreateTagDefinition and useDeleteTagDefinition already invalidate knowledgeKeys.tagDefinitions, so this was redundant.

Sequence Diagram

sequenceDiagram
    participant User
    participant KnowledgeBase as KnowledgeBase (base.tsx)
    participant ConnectorList as useConnectorList (3s poll)
    participant DocList as useKnowledgeBaseDocuments
    participant TQ as TanStack Query Cache

    User->>KnowledgeBase: Trigger Sync
    KnowledgeBase->>TQ: useTriggerSync → invalidate knowledgeKeys.detail
    TQ-->>DocList: Background refetch (documents)
    TQ-->>ConnectorList: Background refetch (connector status = syncing)

    loop While hasSyncingConnectors = true
        ConnectorList->>TQ: Poll every 3s (connector status)
        DocList->>TQ: Poll every 5s (document list)
        Note over KnowledgeBase: hasSyncingConnectorsRef.current = true
    end

    ConnectorList->>TQ: Poll → connector status = active
    Note over KnowledgeBase: hasSyncingConnectors: true → false
    KnowledgeBase->>TQ: prevHadSyncingRef effect: refreshKnowledgeBase() + refreshDocuments()
    TQ-->>KnowledgeBase: Fresh documents & KB data rendered
    Note over DocList: refetchInterval returns false (stop polling)
Loading

Last reviewed commit: efe4816

@waleedlatif1 waleedlatif1 force-pushed the waleedlatif1/fix-kb-connector-refresh branch from 25b9b51 to efe4816 Compare March 15, 2026 11:15
@waleedlatif1
Copy link
Collaborator Author

@greptile

@waleedlatif1
Copy link
Collaborator Author

@cursor review

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

@waleedlatif1 waleedlatif1 merged commit aa1f605 into staging Mar 15, 2026
12 checks passed
@waleedlatif1 waleedlatif1 deleted the waleedlatif1/fix-kb-connector-refresh branch March 15, 2026 11:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant